293 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Fable 5.1 838638a692 release: v2.0.0 -- ax plane, HarnessX removed, S3 memory on RustFS, AI decision engine (in development)
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 15:16:10 +02:00
Marius CiepluchandClaude Fable 5.1 c06190bbfd merge(learn): advisory-fix harvest and slice 2 -- six families out-of-repo
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 15:11:00 +02:00
Marius CiepluchandClaude Opus 5.5 97afa1d220 bench(learn): slice 2 -- six families out-of-repo
Decision-engine task 6.7 (partial). After the advisory-fix harvest (memory 724 -> 904 static examples),
one compile per evaluable family and an outer grouped 5-fold evaluation of that family's own candidates,
input profile v1, priors off, k = 5, cross-fitted Platt, nested operating points (repository-cluster
bootstrap 95%):

  family                 n   AUC(s)              ADVISORY R / P   'vulnerable' P / R   ECE    canary
  injection            110   0.77 [0.67, 0.88]   0.88 / 0.57      0.84 / 0.36          0.093  0.77
  untrusted_dest.       62   0.85 [0.76, 0.94]   0.84 / 0.73      0.96 / 0.66          0.071  0.87
  path                  35   0.89 [0.79, 0.97]   0.86 / 0.69      0.89 / 0.76          0.119  0.90
  output_encoding       44   0.91 [0.82, 0.97]   0.86 / 0.81      0.94 / 0.55          0.045  0.90
  deserialization       46   0.90 [0.82, 0.97]   0.83 / 0.76      0.93 / 0.83          0.095  0.93
  access_control        37   0.96 [0.90, 1.00]   0.91 / 0.91      1.00 / 0.61          0.076  0.83

BLOCK offered for none: calibration fails everywhere except output_encoding, where no fold reaches a
Wilson lower bound of 0.95. Within paired groups (a function and its own fix) AUC is lower, 0.72-0.93:
every single-label group in the new families holds positives only. Injection learning-curve point,
paired (same program, candidates, folds): memory 724 -> 904, AUC 0.76 -> 0.77 (no injection example
was added). $2.88 metered DeepSeek (ceiling $8; balance 17.73 -> 16.73), $0.0009 embeddings.

Instrument fix: compiled demonstrations now follow the near-duplicate rule (cos >= 0.98) that retrieval
and the pre-call assertion apply; the assertion had stopped the untrusted_destination evaluation.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 11:33:16 +02:00
Marius CiepluchandClaude Fable 5.1 6d601c1464 docs: token ergonomics -- runtime routing/budgets/caches and the development-side tooling
One page, two halves with a Mermaid diagram each. A: the plane's token economy (one Model per Task with a metered usd/calls budget, attribution table, credentials only in the start request, repo-facts and content-addressed reuse, replay at $0, triage-hunt-tie-break order, prefix-cache prompt order) with every cost quoted from plane-increment-1/2.json, the injection-slice and harvest records, and an honest not-yet list (no token budget in ax, triage applied not run, k and contrast fixed outside experiments). B: the development-side measurement (tool I/O 85.5%, previously 90%) and the benchmarks/dev remedies: read_guard hook, bulk_read portal, token_report, plus the practices. Linked from the nav, index, memory-and-detection and the README docs list.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 10:42:17 +02:00
Marius CiepluchandClaude Fable 5.1 39b0c0a9d1 docs(memory): the fixed queryable-field schema is in place, not in progress
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 10:16:32 +02:00
Marius CiepluchandClaude Fable 5.1 51d064e381 refactor(memory): S3-generic store on boto3; RustFS is the tested server (no MinIO naming)
The plane's memory store no longer names one vendor. `MinioStore`/`SdkClient`/
`minio_settings`/`MINIO_KEYS` become `S3Store`/`S3Client`/`s3_settings`/`S3_KEYS`;
the URL scheme is `s3://<bucket>[/<prefix>]` (`s3://` alone uses `S3_BUCKET`); the
settings are `S3_ENDPOINT` (a full URL), `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`,
optional `AWS_SESSION_TOKEN` and `S3_REGION`. The client is boto3 (extra `s3`,
imported lazily): SigV4, path-style addressing, SDK checksum trailers off, paginated
listing, versions and delete markers listed and deleted by version (`purge`, used by
the contract tests' teardown so the bucket keeps nothing of a run).

Behaviour is unchanged: the store never configures its bucket and verifies
versioning, the `runs/` expiry rule, S3 Select and tag reads at startup; no local
fallback; every row carries every queryable field with "" placeholders. The admin
hints read `--endpoint-url "$S3_ENDPOINT"`. The fake client still reproduces RustFS's
two Select failures (unbound leading-row field, null-typed column).

Docs: rustfs.md stays the tested server's setup page (policy JSON unchanged, env
names updated, any S3-compatible store with versioning, lifecycle, tags and Select
works); memory.md, ops/ax/README.md, README.md, steering and the two current spec
designs use the S3 naming. Retired specs and committed measurements are untouched.

Real server (RustFS, OUSAST_MEMORY_TEST_S3=1): test_plane_memory + test_learn_compile
75 passed, 1 skipped. Store copy of the live local store to a temporary prefix: every
count matched (776 row objects, 2982 rows, 22 facts, 788 excerpts, 2601 responses,
2 programs, 1793 index entries), 4190 versions purged afterwards.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 10:15:19 +02:00
Marius CiepluchandClaude Opus 5.5 d9608b9a63 bench(learn): harvest the advisory-fix pairs
The 97 advisory-fix pairs (194 sides) harvested for the decision engine: verify a/b + agree on the 34
untrusted_destination sides and model roles on every side as plane Runs on ax ($0.38 metered, task
budgets $1.49, ceiling $5; DeepSeek balance 17.88 -> 17.73), engine features on the host (frozen
7b17eac, one container, --memory 3g: 54 ran, 58 asked no question, 82 no language model), feature
records for all 194 sides, static memory rebuilt with the merged harvest units: 724 -> 904 examples.
No unit was skipped for the 20 KB inline limit.

Instrument fix: SideReader read only the first pair-kind source, so the first rebuild left all 194
advisory sides `unread` (0 examples). It now reads every pair-kind source; test added.

Leak audit: the static profile flags only engine signals and facts.function_lines (withheld by v1);
the plane profile newly flags verify.flag_b and ms.operation (plane-only, not in any v1 prompt).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 09:40:00 +02:00
Marius CiepluchandClaude Fable 5.1 285b6a367f bench(independent): OpenCVE case licence checked (BUSL-1.1); published by maintainer decision
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 09:12:22 +02:00
Marius CiepluchandClaude Fable 5.1 f0110f4429 bench(independent): verify the licences of five v2 cases before publishing
Read-only check at the pinned vulnerable commits via the GitHub API
(LICENSE, license.txt, plugin header License: line). The five license
fields now record the SPDX id with the evidence; nothing else in the
frozen population changes. protocol-v2.md notes the verification and
that the OpenCVE case in v1 is BSL-1.1 (not OSI; non-production use
permitted by the Additional Use Grant; change date 2030-08-14 to
Apache-2.0).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 09:02:58 +02:00
Marius CiepluchandClaude Fable 5.1 5b96199322 merge: legacy cleanup -- audit by real imports, dead modules removed, unread config retired, archives, dockerignore, comments
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:43:32 +02:00
Marius CiepluchandClaude Opus 5.5 82c8a4d341 docs(src): refresh comments and docstrings
Comment-only changes outside learn/, plane/ and cli.py:

- model/pipeline.py: the `judge` its docstring contrasts with is the deleted
  model/judge.py, now named as such.
- fusion.py: the panels are deterministic and only review and vote; the
  "critique and revise" step belonged to the retired LLM panels.
- mcp.py: hunter tools stay inside the scan's stages, not "harness stages".
- fusion, mcp, gate, redaction, skills: their "Phase N" tags now say which
  plan they cite (the retired openrouter-sast-harness).

Requirement citations to retired specs stay: they explain where the code
came from. No behaviour change; the byte-identical benchmark golden passes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:36:31 +02:00
Marius CiepluchandClaude Fable 5.1 e0afebf736 merge(docs): MkDocs site with flow diagrams; RustFS permissions
14 Mermaid diagrams over 8 pages (overview, scanning, plane, memory, RustFS
setup with the complete agent policy, memory-and-detection, evaluation, ops);
strict build passes; uv.lock carries the docs extra.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:30:33 +02:00
Marius CiepluchandClaude Fable 5.1 709320be7d test(plane): task-timeout test leaves room for apply and resume under host load
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:30:00 +02:00
Marius CiepluchandClaude Opus 5.5 80eb0f2a36 chore: .dockerignore; drop the committed baseline .sarif
- Root .dockerignore for the root Dockerfile's context: local state (.git,
  .claude worktrees, .venv, .env, caches, reports, *.sarif) and the
  benchmarks/measurements archive (~20 MB of evidence). Kept: everything the
  Dockerfile COPYs, the two 2026-09-13 records ops/smoke_report.py reads
  from /app inside the image, and the harnessx-removal baseline pairs.json
  that learn/sources.toml names as a label source. plane/Dockerfile.runner
  keeps its own ignore file.
- Remove 2026-09-30-harnessx-removal-baseline/scan/report.sarif: tracked
  despite .gitignore's *.sarif, with no counterpart on the candidate side.
  The directory's README says why.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:28:57 +02:00
Marius CiepluchandClaude Fable 5.1 73f9725103 merge(memory): every row carries its kind's queryable fields
Placeholders are empty strings, not null: RustFS Select breaks on a column that
is null in an object's first 1000 rows. Real-server contract tests 49/49.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:26:51 +02:00
Marius CiepluchandClaude Opus 5.5 c8d2fefbb8 docs(kiro): archive the superseded roadmap; status notes on specs
A4: roadmap-2026-09-04.md moves to .kiro/archive/steering/ so it no longer
loads as steering (roadmap.md and safety-net.md replaced it); roadmap.md's
pointer follows it. overview.md's spec table is recounted from tasks.md and
spec.json (adds ai-service-plane, harnessx-removal, learned-decision-engine,
pre-push-safety-net; drops the stale "active one"), and the module line now
cites the regenerated audit: 150 modules, 145 load-bearing, 5 standalone,
0 orphaned; 1,964 tests collected.

A5: a dated status note at the top of the requirements (or brief) of 16
specs: retired, superseded, dormant or complete, and for
model-grounded-detection the removal of judge/calibrate/execution (closing
the Req 10 execution-tier option). spec.json phase corrected to
implementation-complete where every task is checked (model-grounded-
detection, propose-adjudicate-prove, tree-sitter-overlay-extra,
corpus-seeded-mechanisms, pair-corpus-honesty); openrouter-sast-harness gets
a spec.json with status retired. Nothing deleted, no spec moved.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:19:48 +02:00
Marius CiepluchandClaude Opus 5.5 95a252b89b docs: MkDocs site with flow diagrams; RustFS permissions
Material for MkDocs site (mkdocs.yml, optional extra `docs`) with Mermaid via
the superfences custom fence. New pages: overview, scanning (modes, quick
rules, Joern engine, priors, pre-push delta), the plane on ax (one task as a
sequence, golden snapshot, Run DAGs), memory (row kinds, write/read paths,
FileStore vs S3, verify at startup), memory and detection, and RustFS setup
(admin steps, complete agent policy, .env, refusal messages, Select limit).
The ops READMEs are included at build time; evaluation gains the population
lifecycle and the fold layout. tests/test_docs_site.py runs the strict build
when mkdocs is importable and checks Mermaid blocks and the RustFS page.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:17:40 +02:00
Marius CiepluchandClaude Opus 5.5 067adb9845 chore(bench): archive reference-only scripts
A1-A3 of the legacy inventory, moved with git mv to benchmarks/archive/:
the stage-2 verifier chain (verify_batched, score_batched, verify_sinks,
model_sinks, sink_candidates), the v2 tool hunter (hunt.py) and the
harnessx-removal equality proof. The plane's verify/agree/roles tasks
replaced the first six; the harnessx-removal spec is closed.

- The six independent scripts keep HERE = benchmarks/independent (population,
  evaluate.py, results) and add their own directory for archived siblings.
- harnessx_removal_equality's REPO root is one level deeper.
- tests/test_plane_scoring_parity.py loads score_batched from the archive
  (ran here, not skipped); test_harnessx_removal_equality points at the new path.
- benchmarks/archive/README.md: what each script was, which committed
  records cite it, the commit that last ran it. paths.md maps old -> new;
  committed records are not rewritten.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:07:35 +02:00
Marius CiepluchandClaude Opus 5.5 5ab2c25a43 test(plane): resume-timeout test leaves room for two fake-ax calls under host load
Each fake ax call is a Python subprocess (0.3-1 s on a loaded host); a 0.3 s
resume timeout fit only one, so the test failed whenever the host was busy.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:05:34 +02:00
Marius CiepluchandClaude Opus 5.5 132d3ef45c feat(memory): every row carries its kind's queryable fields
RustFS's S3 Select infers an object's schema from its first 1000 rows: a where
field absent there is unbound (EvaluatorBindingDoesNotExist), and a column null
in all of them but set later fails the whole object (JSONParsingError, measured:
999 leading nulls answer, 1000 fail). plane/memory.py now declares per kind the
fields a where may name (QUERY_FIELDS: facts candidates_digest, verdict
candidate/final, example profile, plus the ROW_FIELDS every row has) and writes
their union on every row at write time as strings, "" where one does not apply
(null measured to break). rows() refuses an unknown kind, an undeclared where
field and where = null before any read. `ousast plane memory-normalise
[--store URL] [--dry-run]` rewrites pre-rule rows once, idempotently, with
counts. The fake object client mimics both RustFS failures.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 08:03:39 +02:00
Marius CiepluchandClaude Opus 5.5 bd74ba63b2 chore(config): retire unread keys [dynamic], [evidence], [variants], [models] patcher
Nothing in src reads config.dynamic, config.evidence or config.variants, and
[models] patcher was only echoed into harness.json. They now go through the
existing RETIRED_KEYS mechanism and fail with RetiredConfigError naming why
("retired 2026-10-02 because nothing ever read it: ..."); RETIRED_KEYS now
carries the full reason per key so the HarnessX entries keep their wording.

- DynamicConfig, EvidenceConfig, VariantsConfig and their loaders removed;
  ModelConfig.patcher removed. resolved_config.json loses those sections.
- harness.json keeps its shape: model_roles.patcher is always null.
- RELEASE_NOTES "Unreleased" lists the keys and the output shape change.
- Delete .opencode/skills/openultrasast-kiro-impl: it pointed agents at the
  retired openrouter-sast-harness tasks.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 07:51:38 +02:00
Marius CiepluchandClaude Opus 5.5 30cde99e2d merge(memory): the S3 store verifies its bucket and never falls back
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 07:33:44 +02:00
Marius CiepluchandClaude Opus 5.5 e26fe4b608 fix(memory): the S3 store verifies its bucket and never falls back
Maintainer decisions 2026-10-02: "it has the featurs. fallback not desired" and
"Admin sets it once; agent verifies".

- MinioStore never configures its bucket: configure()/enable_versioning()/
  expire() are gone; the client protocol only reads versioning and lifecycle.
- verify_bucket() runs when a MinioStore is opened (once per process per
  store): versioning Enabled, an enabled lifecycle rule expiring <prefix>/runs/
  (none expiring the whole store), S3 Select answering a probe, object tags
  readable. Every missing piece is listed in one MemoryStoreError with the
  aws-cli admin command or the permission to grant.
- No S3 Select fallback: a server without Select is refused at open, and a
  failing Select on one object raises (naming RustFS's leading-row schema
  inference when the field is unbound) instead of filtering locally.
- MINIO_REGION is passed through when set.
- Tests: fakes for versioning off/unreadable, lifecycle missing/disabled/
  tag-narrowed/whole-store, Select missing, tags unreadable, and the pass case;
  the real-server contract uses OUSAST_MEMORY_TEST_BUCKET (else MINIO_BUCKET)
  and never writes bucket configuration.
- ops/ax/README.md: RustFS support, the admin one-time setup, the agent policy.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 07:26:01 +02:00
Marius CiepluchandClaude Opus 5.5 3091ed42ea chore: remove dead modules (judge, execution, calibrate, stage_processors, slot_contract)
The reachability-first audit (previous commit) found these five with no
import path from any entry point:

- model/judge.py: replaced by model/pipeline.py, which never imported it.
- model/execution.py: the MGD Req 10 execution seam, never built or called.
- model/calibrate.py: no CLI, no importer; learn/calibrate.py is the live one.
- stage_processors.py + slot_contract.py: kept only so test_gate could pin
  them; the scan runs findings.quick_scan_findings directly.

Their tests go with them, test_gate's zero-dependency list drops the two
entries, the audit lists drop their rows, and the manifest is regenerated:
150 modules, 145 load_bearing, 5 standalone, 0 orphaned.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 07:22:35 +02:00
Marius CiepluchandClaude Opus 5.5 b7f1f3d05b fix(audit): the module audit checks real imports before the spec-owned list
model/audit.py classified anything named in SPEC_OWNED or STANDALONE before
looking at the import graph, so five modules nothing imports were reported
load_bearing and the committed manifest said 0 orphans.

- Reachability from the gates, the CLI and every module with a __main__ block
  (python -m, the plane runner) now decides the class; the lists only supply
  reason text, and an unreachable listed module is orphaned with its listing.
- `from .pkg import sub` now records the submodule edge (plane.engine_alerts
  was only reachable by its SPEC_OWNED entry).
- `python -m openultrasast.model.audit` regenerates the manifest.
- Manifest regenerated: 145 load_bearing, 5 standalone, 5 orphaned
  (model.calibrate, model.execution, model.judge, slot_contract,
  stage_processors). The zero-orphan test is strict-xfail until they go.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 07:17:26 +02:00
Marius CiepluchandClaude Opus 5.5 78c31ba722 docs: refresh merged; v3 has 15 cases; .env.example names DeepSeek, OpenRouter embeddings and the S3 store
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 07:11:50 +02:00
Marius CiepluchandClaude Opus 5.5 32a2cb8bcf docs: describe the system as of 2026-10-02 (ax plane, AI decision engine, PHP, v3)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-02 07:08:44 +02:00
Marius CiepluchandClaude Opus 5.5 7b17eacd44 bench(learn): injection first slice -- compiled AI classifier, out-of-repo evaluation (decision-engine 6.2, 6.6)
6.2: the 724 static examples' 616 distinct excerpts embedded with openai/text-embedding-3-small via
OpenRouter, 198,819 tokens, $0.0040 metered (ceiling $1).

6.6, family injection only, memory = every family (at most 2 contrast examples per prompt), input
profile v1: smoke fixed k = 5 (repeat prefix-cache hit share 0.96); one compile (program c1237a49,
proposed instruction 5, balanced Brier 0.128 on 13 C_val candidates vs 0.199 baseline); outer grouped
5-fold, 110 out-of-repository candidates in 29 groups: ADVISORY recall 0.90 [0.82, 0.97], precision
0.54 [0.44, 0.65], AUC 0.74; BLOCK not offered (ECE 0.064 > 0.05; no fold reached a Wilson lower bound
of 0.95); canary re-ask agreement 0.83. $0.91 metered at list price; DeepSeek balance $18.20 -> $17.90.

Instrument fixes found on the way, each with a test:
- memory build: pair sides join the record stored under their harvest unit pin (--units), and only the
  built profile's records are read (static and plane shared a key; 85 -> 724 examples);
- retrieval: stage 1 per side (family / contrast) and at most 2 contrast examples, never padded;
- program: the JSON answer contract is part of the signature, appended to proposed instructions
  (DeepSeek JSON mode refused a prompt without "json");
- compile: candidate_families for a one-family compile; compile.toml k = 5 from the smoke rule.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 20:40:41 +02:00
Marius CiepluchandClaude Opus 5.5 68294e9db3 fix(learn): v1 also withholds function length (a leak by construction)
The leak audit found facts.function_lines separating fixed from vulnerable pair
sides (a fix guard lengthens the function: 0.80 untrusted_destination, 0.75
config_secrets). Input profiles may now name single features; v1 withholds the
engine and the function length from prompts, retrieval and the proposer summary.
A withheld feature keeps its instrument block in the distance (state still counts).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 19:51:49 +02:00
Marius CiepluchandClaude Opus 5.5 9e3c83cfdc fix(learn): withhold label-tracking signals from the v1 classifier; leak audit
Maintainer decision "Neutralise both for v1".

1. Phantom candidates. A pair side's candidates are now only the functions that
   side declares (plane/harvest.py `declares`, the excerpt builder's
   function_bounds); a labelled function absent from a side is listed under
   `absent_on_side` and never asked, and a side declaring none of its verify
   family's functions is no verify unit. The recorded harvest units held 58
   phantom verify candidates (53 fixed, 5 vulnerable sides). build_features.py
   skips label rows whose side does not declare the function and drops their
   earlier rows (MemoryStore.drop_rows); rebuilt the pair records from the
   recorded signals, no model call: 126 label rows absent_on_side (117 fixed,
   9 vulnerable) -- access_control 13, config_secrets 4, deserialization 20,
   injection 28, memory 4, output_encoding 16, path 15, untrusted_destination
   26 -- so 252 feature rows dropped (1712 -> 1460); no other row changed.

2. Engine withheld. Input profiles in learn/schema.py (`v1` withholds the
   engine instrument, `full` withholds nothing); ProgramSpec.inputs defaults to
   v1 and compile.toml records it. Neither the engine's state nor its values
   reach the candidate, examples, demonstrations or the instruction proposer's
   error summary, and the Gower retrieval distance leaves the engine block out.
   Stored records are unchanged. design.md Risks records the decision and that
   the engine returns only through a pre-registered A/B on assumed-benign and
   benign-control negatives.

3. `ousast learn audit-leaks` (learn/leaks.py): per feature and per instrument
   state, separation = (pairs the signal alone puts on the right side - on the
   wrong side) / pairs, over all pairs and per family; flagged at >= 0.20 over
   >= 10 pairs. Counts under
   benchmarks/measurements/2026-10-01-decision-engine-leak-audit/ (before and
   after the rebuild). After: facts.function_lines (0.80 untrusted_destination,
   0.75 config_secrets, 0.26 injection; higher on fixed), verify.turns_mean
   0.67 config_secrets, eng.* 0.35-0.40 output_encoding, verify.flag_a/votes/
   site_offset and agree.final 0.20-0.30 on 10-12 pair families. Before the
   rebuild the phantom fixed sides drove state.source/state.roles/ms.flagged
   to 1.0 and state.engine to 0.95 on deserialization.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 19:40:58 +02:00
Marius CiepluchandClaude Opus 5.5 4c96a59f8f merge(learn): harvest signals for the decision engine (decision-engine task 5)
Verify a/b + agree on 150 labelled units ($2.54 attributed), model roles on 799
files ($1.40), engine features for 682 of 686 pair sides; 1,712 feature records.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 19:13:20 +02:00
Marius CiepluchandClaude Opus 5.5 10c97c1b8d bench(learn): record the finished engine harvest (decision-engine task 5)
The pair-corpus engine job finished: 682 of 686 sides (199 ran, 475 read their input
and asked no question, 8 no engine model; the 4 sides of the two family-unknown
pairs were not run). Feature records rebuilt (1712); counts-only record updated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 16:54:27 +02:00
Marius CiepluchandClaude Opus 5.5 34da916030 bench(learn): harvest signals for the decision engine (decision-engine task 5)
`ousast plane harvest` (plane/harvest.py) writes two Runs over a label snapshot:
verify a/b + agree on the verify-family units (both sides of each labelled pair as
an opaque files-only Workspace, candidates outside the hunt's root, no declared
sites; v1 and dev-php pins with repo-facts first) and model roles on the candidate
files (pair excerpts grouped, one Task per population pin). Budgets sum under the
ceiling by construction. New roles task template; the generator re-pins it.

Spent by attribution: verify $2.54 (1961 calls, 150 units, 182 hunts per pass),
roles $1.40 (1034 calls, 799 files, 0 unclassified chunks); DeepSeek balance
19.63 -> 18.23. Engine on the pair corpus (benchmarks/learn/engine_pairs.py,
frozen e6ed785, one 3g container) is PARTIAL: 384 of 686 sides at commit, still
running; unfinished sides stay missing. benchmarks/learn/build_features.py built
1712 static+plane records into the memory store; counts-only record under
benchmarks/measurements/2026-10-01-decision-engine-harvest/.

source/roles parts are `none`, not `failed`, for a language without a declaration
pattern (C). ops/ax/README: Substrate golden-actor disk leak and the inline
Workspace limit (~20 KB).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 14:04:28 +02:00
Marius CiepluchandClaude Opus 5.5 53c9a7cc65 bench(pairs): real fix-commit pairs for the short families
97 advisory-fix pairs, one repository each (309 advisories examined, 212
rejected with reasons in research-2026-10-01.md): a public fix commit
distinct from its parent, the advisory-named function declared on both
sides with a changed body. Pointer pairs: materialize.py reads one file
per side from a shallow blobless fetch into the pair cache and verifies
them with the label builder's matcher (verification.json, bytes read).

learn/sources.toml lists the catalog as the advisory-fixes source. Label
snapshot: negative repository groups untrusted_destination 8->25, path
3->21, output_encoding 1->22, deserialization 0->21, access_control 0->20;
all five reach the per-family floor. Five pairs on reserved repositories
were dropped after the reserved-population guard flagged them.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 11:51:28 +02:00
Marius CiepluchandClaude Opus 5.5 658f2d5e39 fix(learn): fixed-side negatives only where the fixed side declares the function
A pair now gives a fixed-side negative only when the fixed side's source declares the
labelled function (the excerpt builder's matcher; a name on a code line where the matcher
resolves neither side), or when a deterministic rename at a distinct fix commit replaces it
(one new function, same parameter list, changed body: provenance fixed_side_moved). Anything
else is counted under pair_fixed_side.absent_fixed_side, not labelled.

108 pair labels lose their negative: 80 seeded pointer pairs whose fixed side is a guard
helper of the same commit, and 28 Real-Vuln-Benchmark traps whose fixed side is a sibling
handler of the same snapshot (4 of those handlers are labelled positives themselves). The
memory build no longer substitutes the catalog's fix_function on the fixed side. The pair
cache is an allowed label-source tree so pointer sides are read through the guard.

Snapshot: benchmarks/measurements/2026-10-01-decision-engine-labels (counts only).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 11:03:54 +02:00
Marius CiepluchandClaude Opus 5.5 7f6bbacb6e feat(learn): compile, calibrate, evaluate (decision-engine 6.5)
learn/compile.toml: the pre-registered spec -- balanced Brier score of the
raw score on C_val (cells by family x label, repositories capped at 5,
unsure = 0.5, ties within 0.005 to fewer prompt tokens), seed, the
compile split (25%, boot 60/40, compile-split-only memory), bootstrap
(150 candidates, confidence >= 0.7, 4 sets of 3), search (6 proposals +
baseline, 20 errors shown, 60 validation candidates), k = 1, the
retrieval settings, and the $3 / 1,200-call ceiling.

learn/compile.py: bootstrap few-shot keeps only right, confident
rationales that cite a line and name no repository, path or CVE/GHSA id;
seeded demo sets (positive, negative, any; families mixed) with
other-framework alternates; one proposal call; 7 instructions scored with
set 1, the best with sets 2-4. The artifact is canonical JSON stored as
programs/<program_id>.json, program_id = sha256 of the artifact without
its id; it records the spec sha, memory/compile-groups/folds digests and
the response-cache digest, with spend priced from cached usage, so a
rerun from the cache is byte-identical at $0 (tested on FileStore and
MinioStore through the fake client; real MinIO gated as today). A ceiling
ends the compile `unfinished` with no artifact. packageable() keeps only
permissively licensed demonstrations (MIT, BSD, Apache-2.0, ISC, zlib),
swapping in a permissive alternate or dropping, under a new program id.

learn/calibrate.py: Platt scaling on logit(s), pure Python Newton, per
family / pooled with family intercepts / insufficient data by the floors;
cross-fitted (fold j's map is fitted on the other folds only); the check
(5-bin reliability, slope and intercept with repository-bootstrap CIs,
ECE <= 0.05), ADVISORY only when it fails.

learn/evaluate.py: predictions under each fold's boundary, families below
the floor never asked; operating points read from the calibrated,
cross-fitted predictions of the other folds (BLOCK: Wilson lower bound
>= 0.95, never on a majority unsure; ADVISORY: recall >= 0.90);
repository-cluster bootstrap intervals; strata with "insufficient data";
>5% parse failures is an instrument failure; explanations keep only
signals the record holds; version mismatch names the fallback reason;
the memory-size learning curve.

`ousast learn compile|evaluate|curve`: paid runs need --budget-usd (exit
3 at the ceiling), --replay-only runs from the cache. Nothing was spent:
every test and the dry run used scripted clients.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 10:36:34 +02:00
Marius CiepluchandClaude Opus 5.5 ea61dc1bb3 feat(learn): the classifier program and its response cache (decision-engine 6.4)
learn/program.py: the declared signature (code, signals, roles, family in;
verdict, family, confidence, rationale with cited lines out), the
baseline instruction, rendering as [system: instruction + demonstrations]
-- byte-identical for every candidate, so the provider's prefix cache can
hit -- then [user: retrieved examples with their labels, the candidate
without one]. Signals pass validate_x first (no EXCLUDED_FIELDS, missing
instruments in words). Answers are one JSON object; one that does not
parse or cites no line inside the excerpt is asked once more at
temperature 0 and otherwise becomes unsure/parse_failed, never
not_vulnerable. Classify(k): sample 1 at 0.0, the rest at 0.7; score s =
mean p_j; majority verdict, ties unsure; a majority unsure cannot BLOCK.
Caller caches each response as responses/<sha>.json keyed by (model,
Model parameters digest, messages, temperature, sample, json_object) and
replays at $0; spend is priced from the recorded usage once per distinct
request, so a replay reports the same usd. Program.decide asserts the
evaluation boundary on the rendered ids before any call.

MeteredClient.complete passes temperature/logprobs through (only when
set); DeepSeekChatClient sends temperature beside thinking=disabled.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 10:13:31 +02:00
Marius CiepluchandClaude Opus 5.5 f5180a14ec feat(learn): retrieval and the evaluation boundary (decision-engine 6.3)
learn/retrieve.py: stage 1 is a per-instrument Gower distance over the
profile's allow-listed features with explicit missing (both ran: mean
feature distance; differing states 1; same non-ran state 0; null in one 1;
priors dropped with priors=off), n1 = 40 with seeded tie-breaks; stage 2
re-ranks by r = lam(1 - D) + (1 - lam)cos, signals_only without the
target's vector; the balance rule takes k_ret = 6 (<= k/2 per label, <= 2
per group, <= 2 other families; `short` recorded). eligible() is the only
way into a prompt: not the target's group, not an evaluated group, not the
held-out source or framework, and outside deployment not a near-duplicate
(cos >= 0.98). assert_boundary repeats the check on rendered ids;
demonstrations of a held-out framework swap to their alternates;
deployment_fold excludes a scanned repository that is a corpus group;
seed_memory never hands a case its own group.

learn/folds.py: compile split (25% of groups per primary family, 60/40
boot/val), outer grouped 5-fold stratified by family and label mix,
leave-one-source-out, leave-one-framework-out (insufficient data below 10
groups), assert_disjoint and a folds digest; all seeded.

Tests: test_learn_retrieve.py (hand-computed Gower, lam 0/0.5, balance,
near-duplicates, test_boundary over outer/LOSO/LOFO folds with
demonstrations), test_learn_folds.py.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 10:05:19 +02:00
Marius CiepluchandClaude Opus 5.5 98fa4ec8cd feat(learn): metered embeddings and the cache (decision-engine 6.2)
plane/models/openrouter-embedding.yaml: openai/text-embedding-3-small via
OpenRouter (embeddings only), input_per_m 0.02, cache/output 0, egress
openrouter.ai; `openrouter` joins the provider extensions.

plane/budget.py MeteredEmbeddingClient: prompt_tokens priced from the
Model manifest, usd/calls ceilings stop the next call, a priced call that
reports no tokens raises UnmeteredUsage (a silent zero is an instrument
failure). OpenRouterEmbeddingClient keeps one usage row per call
(parse_embedding_payload); its api key is out of repr.

learn/embeddings.py + `ousast learn memory embed`: excerpts embedded in
batches into embeddings/<model-slug>/<excerpt_sha>.json (float32 base64),
cached by content hash so a rerun makes no call; the declared dimension is
checked; --dry-run estimates tokens and cost without a call; a paid run
needs --budget-usd and exits 3 (unfinished) at its ceiling. Not run against
the provider: the store embedding waits for its budget go-ahead.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 09:56:52 +02:00
Marius CiepluchandClaude Opus 5.5 7b81c9ac4e feat(learn): excerpts and the example store (decision-engine 6.1)
learn/excerpt.py: the labelled function at its pin as numbered lines, bounded
(80 lines / 4,000 chars for candidates, 40 / 2,000 for memory examples),
centred on focus lines; advisory ids anywhere and comments with security
wording (labels.security_reason, multilingual) become <redacted>; owner,
owner/name, path and commit ids are replaced; a delta unit appends the
base->head diff (<= 40 lines). C/C++ and other brace languages without a
repo-facts declaration pattern use a brace matcher (346 c_cpp pair rows
had no excerpt without it).

learn/examples.py + `ousast learn memory build`: label + feature record +
excerpt -> `example` rows (no rationale), excerpts as content-addressed
blobs; refuses origin: user and sources outside sources.toml (exit 2);
conditional labels skipped; exit 2 when >10% of a source's recorded
candidates have no excerpt. A fixed side that was read but no longer
declares the labelled function is counted apart (absent_fixed_side): a
label defect, not an unread clone.

plane/memory.py: `example` in KINDS; put_blob/get_blob/blob_names on the
MemoryStore base, so FileStore and MinioStore share them (tested on both,
MinIO through the fake object client).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 09:45:17 +02:00
Marius CiepluchandClaude Opus 5.5 0f9318f58b spec(learned-decision-engine): revised design approved; permissive-only demonstrations
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 09:23:35 +02:00
Marius CiepluchandClaude Opus 5.5 b6a49fa4dc spec(learned-decision-engine): design and tasks revised for the AI classifier
In-house DSPy-style program (retrieve -> classify(k)) on the metered client;
example store with signal-profile Gower neighbours re-ranked by code embeddings;
balanced Brier metric, bootstrap few-shot and instruction search on a compile
split; Platt calibration on other folds; cached replay; tasks 6.1-6.7 with paid
steps flagged for go-ahead.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 09:22:29 +02:00
Marius CiepluchandClaude Opus 5.5 2d69a8b677 spec(learned-decision-engine): Req 3 -- an AI classifier with local memory, DSPy-style
Maintainer: data will not scale; the decision engine is an AI classifier, not ML,
with local memory, DSPy style. In-house on the metered client; retrieval by signal
profile re-ranked by code embeddings; compiled (bootstrap few-shot, instruction
search) per training fold; retrieval never crosses the evaluation boundary;
confidence calibrated on held-out repositories. Design and tasks 6+ to revise.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-10-01 09:12:24 +02:00
Marius CiepluchandClaude Opus 5.5 e6ed7854bb feat(learn): labels from ground truth with provenance (decision-engine task 4)
learn/sources.toml is the fail-closed source list: every read goes through one guard
that refuses an unlisted path or an [[excluded]] file name before opening it, and a
source whose evaluation record is missing cannot label. Population v3 is named only
as excluded, by file name, and is never opened (tested with a v3-named fixture and
open/read_text tracked).

learn/labels.py and `ousast learn labels` apply the design's rules: v2 declared
sites and v1 fix-range functions positive, the same function on the fixed pin
negative only when the fix changed it, introduce/repair delta units, benign control
pushes as conditional negatives; pairs vulnerable/fixed (unscorable and title tier
out, title as a sensitivity arm); dev-php recipe known functions; recorded
adjudications with their reasons. v2 rows are labels marked design_informed, as the
design says. Repository groups: normalised owner/name, merged by shared CVE/GHSA,
shared fix commits and [[alias]] rows.

assumed_benign (maintainer decision): ordinary commits of the population
repositories, >= 90 days from any fix, no later security commit touching their
files, weight 0.5, conditional, reported separately and never merged with verified
negatives. The security filter is multilingual (CJK, Korean, Russian, European and
other languages) plus CVE/GHSA/CWE references and dependency bumps; widened after
reading accepted pushes (validation, passwords, guards).

First snapshot, counts only (benchmarks/measurements/2026-09-30-decision-engine-
labels/labels.json): verified pin labels in positive/negative repository groups
injection 44/41, untrusted_destination 35/34, deserialization 20/20 (per-family),
output_encoding 15/15, path 15/14, access_control 12/12 (pooled), config_secrets 5/5,
memory 5/5 (insufficient). assumed_benign: 969 pushes in 27 groups from 36,000
commits scanned.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 19:54:26 +02:00
Marius CiepluchandClaude Opus 5.5 884092e62c feat(learn): per-repository role inference, framework-free (decision-engine task 3)
Wrapper inference (learn/roles.py, no model): a function whose parameter reaches a
call already in the role set becomes a derived sink (or a sanitizer when it returns
the cleaned value) of the same operation, at depth + 1; a function returning a
source's value becomes a derived source. The role set starts from language-level
entries only (priors off by default, optional), grows to depth 3, and reads
vendored dependency source when present, so a vendored database class reaching
mysqli makes `->query` a sink without a framework table. A name the repository
declares more than once becomes a role only when every definition derives it.

Model roles (plane/tasks/roles.py): the model_sinks.py classification moved into
the package and extended to sink/source/sanitizer/guard/dispatch, per file chunk,
metered and resumable, client injected; a failed chunk is unclassified, never
"no roles". vocabulary_overlay merges inferred and model roles over the
language-level facts in the shape taint_specs takes, origin in the entry id.
Features: the roles (static) and model_sinks (plane) instruments are filled.

On the recorded validation-46 facts (15 cases, 14,368 product files, 87 MB read):
1,999 wrapper roles (1,753 sinks, 157 sources, 89 sanitizers; depth 1/2/3 =
973/651/375), 14 of 15 cases with at least one; the WordPress gopay plugin has
none with priors off, which is what model roles are for.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 19:16:54 +02:00
Marius CiepluchandClaude Opus 5.5 1fcc31c857 merge(learn): features schema and framework priors (decision-engine tasks 1-2)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:53:15 +02:00
Marius CiepluchandClaude Opus 5.5 af81bccc70 feat(ruleset): tag framework-specific knowledge as optional priors (decision-engine task 2)
ruleset/frameworks.toml lists 19 framework and library ids (wordpress, flask, django, express,
spring, jakarta-servlet, react, vue and 11 libraries), each with its languages, dependency
package names and a symbol lexicon used only by the lint and for stratification.

Every framework/library entry of section 2 is tagged `framework`/`library`, and mixed entries
are split so a tag covers a whole entry (split pieces keep the entry's id and flags, in place):
semantic facts 38 tagged entries (wordpress 6, django 6, flask 4, express 4, jakarta-servlet 3,
pyyaml 2, one each for spring, sqlalchemy, requests, httpx, lodash, node-http-clients,
escape-html, sanitize-html, dompurify, esapi, commons-text, react, vue); the dispatch fact
behind the hook-as-source path is wordpress; the PHP layout row is renamed php_layout and stays
untagged (a product filter). Quick rules: 9 tagged (wordpress 6, flask, pyyaml,
node-http-clients); php-unserialize is split into php-unserialize and php-maybe-unserialize;
the lint also caught WordPress escapers in exclusions and axios/yaml.load in mixed rules, so
those keep their ids as tagged rules and gain a `fallback_for` language-level rule (6) that is
active only when the prior is off.

load_facts and load_ruleset take priors ("all" by default -- today's scan --, "off", or a set
of ids). tests/test_semantic_priors.py: the lint (no untagged entry contains a lexicon symbol
of its language), "off" removes exactly the tagged entries and activates the fallbacks, split
entries keep their language-level calls. Rule and layout counts in three existing tests follow the split and the rename.

Unchanged defaults: test_benchmark_output_is_byte_identical_to_golden_baseline passes, and the
quick findings of all 15 benchmark manifests (full StaticFinding records) are byte-identical
before and after. priors=off on the same manifests: 61 -> 57 findings; the 4 lost are js-ssrf
(axios), php-wpdb-sql-composition, python-ssti and python yaml.load; 5 PHP and 1 Python finding
move to their fallback rule at the same location. Semantic calls with priors off: php 75/92,
python 32/77, javascript 42/70, java 24/41, c 27/27 (the engine-side effect needs Joern and was
not measured).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:49:05 +02:00
Marius CiepluchandClaude Opus 5.5 d1896cb8e7 feat(learn): candidate feature schema and builders (decision-engine task 1)
learn/schema.py is the closed, typed allow-list of candidate features: 66 features over
11 instruments (language one-hot, quick rules with 23 mechanism buckets and a prior count,
engine, repo-facts, source span, entry-point distance, roles, delta, verify, agree, model
sinks), two profiles (static 57; plane adds verify, agree and model sinks for 66), schema_version and a feature-set digest.
No repository, path, file, function, rule or commit identity is a feature; site_match,
in_fix_range, fixed-pin alerts, case.json fields and costs are refused. Missing is not zero:
each record carries per-instrument state (ran | none | failed | not_applicable), and every
feature of an instrument that did not run is null.

learn/features.py builds records on the host (build_for_scan, whole repository or base..head
delta); plane/tasks/features.py builds them per case in a Run from the delivered artifacts,
dropping site_match and in_fix_range on read and ignoring fixed-pin alerts. The generator
adds a model-free `features` entry per case before `remember`, which stores the records as
`features` rows (re-validated). memory KINDS gains features, label, decision, experiment,
arm_outcome and experiment_result. The committed validation-46 manifests are regenerated
(entries appended; the first Run's entries are unchanged). The runner image must be rebuilt
and re-pinned before a Run executes the new task.

Proof on real artifacts (scratch store, not committed): the recorded validation-46 run gives
75 feature records over 15 cases (43 verify candidates plus vulnerable-pin alerts), each
case's vulnerable pin exported and read (0.5k-9k files); remember stored 75 features rows
beside 43 verdict, 84 alert, 61 unit_cost, 15 facts and 8 coverage rows.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:38:04 +02:00
Marius CiepluchandClaude Opus 5.5 3bc79274a9 bench(independent): protocol v3 budget and shadow-rule choices confirmed by the maintainer
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:23:09 +02:00
Marius CiepluchandClaude Opus 5.5 556717ab53 bench(independent): protocol v3 pre-registered before any scan
Three instruments scored separately at one analyzer commit (quick PHP rules
with a fixed rule-to-family table, the Joern engine, the plane pipeline
triage -> verify a/b -> tie-break -> 2-of-3), protocol v2's six pins, one
matching rule for all six families (declared site or fix range, same family;
evaluate.matches_v2 unchanged), per-family meaning of a match, the M4 gates
copied from protocol v2 and applied per instrument, a $60 total / $6 per case
model-spend ceiling, and the retune rule.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:21:59 +02:00
Marius CiepluchandClaude Opus 5.5 0a04dbfada bench(independent): freeze population v3 (PHP)
freeze.py, extended for v3 and still reading only git history, file lists and
fix diffs:
- a file the advisory names without a function resolves to the function(s)
  the fix's diff changes at the vulnerable pin (maintainer's decision 2),
  recorded with its basis (advisory-named / resolved-from-fix-diff); a new
  helper inserted at file scope adds no site;
- an inferred fix must change a declared site function, else the case drops;
- a single-language population gets a strict benign proposal (PHP change,
  merges judged with all they bring in, pre-vulnerable history as fallback).

All 15 cases verified, none dropped (pagible and podlove fixes confirmed).
Nine sites in eight cases resolved from diffs; five benign proposals
replaced by hand.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:15:52 +02:00
Marius CiepluchandClaude Opus 5.5 edcf145b00 spec(learned-decision-engine): tasks approved; tasks 1-4 first, harvest waits for budget
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:14:53 +02:00
Marius CiepluchandClaude Opus 5.5 d7a572498d spec(learned-decision-engine): design approved; implementation plan for review
Assumed-benign pushes mined from non-security commits as a separate label source
(maintainer decision). Ten tasks in the design's commit order; the harvest,
exp-001 and the v3 check wait for budget go-ahead.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:14:31 +02:00
Marius CiepluchandClaude Opus 5.5 f6f9591cd0 spec(learned-decision-engine): design for review
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:13:27 +02:00
Marius CiepluchandClaude Opus 5.5 ca1449a0aa spec(learned-decision-engine): Req 8 -- no framework knowledge as a prerequisite
Maintainer: hard-coded application specifications are problematic; WordPress is
not the only PHP application, same for Python and JS. Language-level knowledge
stays; framework roles are inferred per repository; existing framework entries
become tagged optional priors kept only by A/B evidence; leave-one-framework-out
evaluation.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:03:13 +02:00
Marius CiepluchandClaude Opus 5.5 d86ba281ec bench(independent): reserve population v3 (PHP, 16 cases) by rule
Maintainer's selection 2026-09-30, "Recommended 16, minus v2 mentions": the
research's rule-based 16 with Winter, elFinder and Easy!Appointments replaced
by the next candidate in their family by the same rule (named fix first, then
newest first), v2-mentioned alternates (Grav, Backpack CRUD, Koel) skipped.
Winter -> mantisbt-tokenizer-eval-rce; elFinder -> pagible (inferred fix, to
be confirmed at freeze); Easy!Appointments -> empty slot (no remaining
untrusted_destination candidate), so 15 cases are reserved.

The reservation guard now covers v3: all six families, a declared-language
population, status "reserved".

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:03:07 +02:00
Marius CiepluchandClaude Opus 5.5 1b291e81b7 spec(learned-decision-engine): Req 7 -- judged on unseen repositories as a repo safety net
Maintainer: the tool will never run against the test cases it is trained with;
its purpose is a repo safety net. Repository-grouped folds, repository-agnostic
features, BLOCK/ADVISORY operating points for changed code, out-of-repo
calibration, adaptation kept separate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:01:06 +02:00
Marius CiepluchandClaude Opus 5.5 0fad93fcc5 spec(learned-decision-engine): requirements approved
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 17:59:40 +02:00
Marius CiepluchandClaude Opus 5.5 afde23d3cf spec(learned-decision-engine): brief and requirements for review
Maintainer direction: A/B testing to extrapolate detection instead of hard-coding
it, and a trained decision engine. Instruments become signal producers; labels
from ground truth with provenance; a simple calibrated engine with population
folds; experiments as plane Runs; extrapolation with intervals, checked once on
frozen v3; the engine is what scan/pre-push/plane report.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 17:59:06 +02:00
Marius CiepluchandClaude Opus 5.5 ff51bb517d chore(plane): runner image with the PHP quick rules; re-pin task manifests
The in-sandbox alerts scan now sees PHP (php/rules.toml confirmed inside the
image); the engine still runs alongside for PHP.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 15:12:13 +02:00
Marius CiepluchandClaude Opus 5.5 7927d6e539 feat(ruleset): PHP quick-mode rules from the engine's vocabulary, measured on the development corpus
Quick mode had no PHP rules, so a PHP quick scan read the files and reported a plausible zero.
ruleset/php/rules.toml adds 13 line/statement rules built on semantic/php.toml's sinks, sources and
sanitizers (sql/wpdb, command, code_eval, unserialize, echo, file_inclusion, filesystem,
outbound_request, redirect). 9 are enabled and 4 are shadow (sql-call-composition, code-eval,
unserialize, xss-echo-input). A rule is shadow when it fired on the corpus but never at a known site.
precision_estimate is the measured lower bound.

Measured in-sample on the development corpus only (pmpro introduce/repair/benign, population-v1 PHP
cases; population-v2 was not read). Known site hit on the vulnerable pin in 4 of 5 cases (LearnPress
missed). The hit disappears on the fixed pin for pmpro and YesWiki. Numbers, per-pin files/bytes read
and the engine comparison are in benchmarks/measurements/2026-09-30-php-quick-rules/.

- fixtures php-vulnerable / php-benign plus manifests: ousast benchmark covers PHP
  (9/14 reported, 13/14 including shadow; the benign twin records 2 known false alerts)
- PHP `#` comments are comments; `#[...]` attributes are not
- the alerts task now reports PHP as quick coverage; alerts-engine keeps running the engine for PHP
  next to the quick rules (ENGINE_ALONGSIDE_QUICK)
- other languages' benchmark findings and metrics are unchanged

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 15:05:47 +02:00
Marius CiepluchandClaude Opus 5.5 45d02ecf20 feat(plane): honest coverage in alerts; PHP alerts from the engine on the host
Quick mode has rules only for c/cpp, groovy, java, javascript/typescript and
python, so every PHP case's "0 alerts" read as clean. The `alerts` task now
records per language of the files read whether quick mode covers it
(derived from the bundled ruleset, not a list): coverage quick | none, with
file counts per pin, and `uncovered`. `remember` turns that into `coverage`
memory rows (a new kind; the Run passes the alerts summary.json as input),
loop-measure reports coverage per language, and improve/memory.py's M1/M2
skip a repository+pin whose coverage for the rule's language is only
`none` (counted under `coverage_none` in the proposal's excluded).

`ousast plane alerts-engine <Run.yaml> --source <frozen export>`
(plane/engine_alerts.py) does the alerts task on the host for cases whose
files include a language the engine covers but quick mode does not (PHP):
exports both pins from the case cache, runs the quick scan and the
openultrasast:dev engine container (finding_dump.py, --network none,
--memory 3g, one at a time, per-pin deadline), writes engine findings in
the alerts.jsonl row shape (rule_id engine:<family>, source engine,
in_fix_range from case.json) with a summary (files read, questions,
completed, seconds, degradations per pin), and marks the task done via
reconciler.mark_done. No result, no files read, or questions == 0 fails
loudly and leaves the run untouched. Refuses a git checkout as source.

Runner image rebuilt and re-pinned (sha256:ebb40a9a...); validation-46
regenerated (digest only). Live, recorded in
benchmarks/measurements/2026-09-30-php-engine-alerts/: froxlor 366 files,
130/130 questions per pin, 6 alerts per pin, 0 in the fix range; wp-gopay
19 files, 101/101 questions, 0 alerts. The loop: 32 run rows (coverage 8,
php=engine), 0 proposals, gate no_proposals, $0.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 13:44:32 +02:00
Marius CiepluchandClaude Opus 5.5 f4bd984fda docs(harnessx-removal): the plane replaces HarnessX in README, docs and specs (task 8)
README: "Configuring the HarnessX agentic plane" becomes "The agentic
plane (ax)" (ousast plane run|status|doctor|remember, per-task Model,
budget and egress, pointer to ops/ax/README.md, one tagged retirement
line); "The HarnessX self-improving cycle" becomes "The self-improving
cycle", and its MetaAgent paragraph is replaced by the memory proposer
(`ousast improve --memory`, provenance, train-on-test guard). The fusion
panel_model/decider_model lines and the [harnessx] spend caps are gone.
The README reference test now passes without its strict xfail.

docs/examples.md section 6 runs the plane; docs/threat-model.md describes
the per-task EgressPolicy, the usd/calls budgets and the start-request
credential path. RELEASE_NOTES.md gets an Unreleased entry: HarnessX
removed, the retired keys and their errors, the silent loss of the
second hunter for users who set [models] hunter, and the plane as the
agentic path.

Dated retirement notes (2026-09-30) head the requirements, design and
brief of harnessx-self-improving-rulesets, learning-harness,
three-stage-scan and openrouter-sast-harness, and follow the spec table
in steering/overview.md; history is unchanged. All tasks are checked, so
spec.json moves to implementation-complete.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 12:45:29 +02:00
Marius CiepluchandClaude Opus 5.5 800204de87 bench(harnessx-removal): deterministic outputs byte-identical after the removal (task 7)
Reran benchmarks/harnessx_removal_equality.py record from a git archive
export of b9afb66 (tasks 1-6 landed; same venv freeze hash as the
baseline, harnessx not importable, no API keys, empty config) and
compared it against 2026-09-30-harnessx-removal-baseline:

  equal=True differences=0

All 13 output files (pairs.json, benchmark, standard scan, improve
--dry-run) are raw-byte identical to the baseline. The suite map
(suite-map.json) accounts for 29 deleted ids, 2 renamed ids and 83
added ids; each deleted or renamed id cites the design's Tests table and
commit cbadc38 or b9afb66. Every surviving test id keeps its outcome
(1740 passed, 38 skipped incl. the README xfail).

Instrument note: a first record failed test_push_hook twice because the
recording shell's PATH lacked .venv/bin (the hook execs `ousast`); the
same two tests fail at b9afb66 without it and pass with it. The
committed record has .venv/bin on PATH.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 12:40:57 +02:00
Marius CiepluchandClaude Opus 5.5 b9afb66250 chore(harnessx-removal): delete HarnessX code and packaging (task 6)
Deletes harness_ext.py and hunter_harness.py with their HarnessX-only
tests (the judge path, verify_judge.py, went in task 5). The harnessx
extra, its mypy override and its uv.lock entries are removed (uv lock
drops only packages; no remaining version changes). The conftest
fixture becomes assert_cold_import; test_gate's two never-import checks
become one allow-list (stdlib, openultrasast, yaml) over the governance
planes' module-level imports, and the skipped live-extra tolerance test
is deleted. Audit rows for the two modules are removed, stage_processors
is reworded, and the module-audit manifest is regenerated (no orphan).

test_removed_plane_references implements Req 2.4 over src, tests and
pyproject.toml: a match must carry a `retired 2026-..` tag (the spec's
own slug is not counted). The README part is a strict xfail until task 8
rewrites the README's plane sections.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 12:18:42 +02:00
Marius CiepluchandClaude Opus 5.5 cbadc38d5e refactor(harnessx-removal): retire the HarnessX scan paths (task 5)
The standard scan now runs run_hunter_pool, verify_findings and the
deterministic fuse_findings directly; the HarnessX hunter orchestrator,
the LLM judge dispatcher (verify_judge.py, deleted with its test) and the
LLM fusion panels are gone, and so are their extra-absent degradations.
host_under_harnessx is dropped from stage_processors.

config: [models] verifier and [fusion] panel_model/decider_model raise
RetiredConfigError naming the plane replacement (the CLI prints it and
exits 2); a [harnessx] section loads with one warning and is otherwise
ignored. harness.json keeps "verifier": null. Tests that asserted the
fallback are renamed to the behaviour they assert; the module-audit
manifest drops verify_judge. Nothing outside the two modules deleted in
task 6 imports harness_ext or hunter_harness.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 12:13:26 +02:00
Marius CiepluchandClaude Opus 5.5 b68fe27c74 feat(plane): the improvement loop as a plane Run (harnessx-removal task 4)
`ousast plane workspaces --validation-set ... --loop` appends the loop to a Run:
per case `alerts` (plane/tasks/alerts.py: the quick-mode scan on the vulnerable
and the fixed pin, a fixed-pin Workspace per case, alerts attributed to their
enclosing function by the repo-facts patterns, `in_fix_range` from the fix's
old-side and new-side ranges, failing on an unread tree) feeding `remember`,
then `memory-snapshot` -> `loop-measure` -> `loop-propose` -> `loop-improve`
(plane/tasks/loop.py). Every new task binds no Model and has budget
{usd: 0, calls: 0}. The snapshot is the store's rows after the train-on-test
guard, written on the host by memory.seed (a sandboxed run of it fails loudly);
propose runs M1/M2 against this repository at the commit under test; improve
runs one unchanged evolve.run_round and delivers gate.json, journal.json, the
sidecar, rule_policy.json only when accepted, and proposal_outcome rows that
the host ingests. Adoption stays a maintainer commit.

`remember` is now a Run entry per case (the image contains it), with the facts
row keyed by the image the facts were computed on (repo-facts records it in
its summary; unknown -> the row is omitted, never guessed). Alert rows carry
their own pin; improve/memory.py counts a repeated alert once.

Runner image rebuilt (+0.39 MB: only the new modules; the quick scan needs no
extra and the rulesets were already in the package), pushed, plane/ re-pinned
and regenerated (validation-46: 105 entries). Fixture test runs the whole loop
graph through the fake ax, whose new `exec` mode runs the real task modules.
Live: one loop Run over three validation-46 cases on the local ax, recorded in
benchmarks/measurements/2026-09-30-harnessx-removal-loop/ (all done, $0 new
spend, 0 proposals, gate verdict no_proposals).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 11:59:50 +02:00
Marius CiepluchandClaude Opus 5.5 d104166697 feat(improve): proposals from plane memory through the unchanged gate (harnessx-removal task 3)
improve/memory.py: rules M1 (enabled->shadow on repeated false alerts across
>=2 repositories, no agreed or site_match hit) and M2 (shadow->enabled on
>=2 agreed declared sites no enabled rule catches); M3 disputed families are
advisory (signals.json) and never reach the validator. The train-on-test
guard drops rows of --qualify-population, gated benchmark cases and holdout
pair repositories before any rule sees a row. run_round(proposals=...) adds
memory edits after the benchmark's through the unchanged validator and gate;
provenance (row ids, object keys and versions, counts, exclusions, index
digest) goes to memory_proposals.jsonl and the journal's evidence field;
proposal_outcome rows are written back after a non-dry run.
ousast improve --memory is opt-in; default improve output is byte-identical
to the task-1 baseline.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 11:25:52 +02:00
Marius CiepluchandClaude Opus 5.5 52d28b5121 feat(plane): memory store (file and MinIO) and the remember task (harnessx-removal task 2)
plane/memory.py: the MemoryStore interface with FileStore (index.jsonl, facts by
content hash, repos/<host>__<owner>__<name>/<pin>.jsonl, idempotent ingest,
deterministic row ids, refusal below 1 GiB free naming the path) and MinioStore
(object metadata and tags, S3 Select pushdown probed once per process with a
logged local fallback, object version ids on every row read, versioning and
runs/ lifecycle helper, presigned URLs). OUSAST_MEMORY selects the backend;
MINIO_* settings come from .env; the minio SDK is the optional extra `minio`.

plane/tasks/remember.py: model-free rows (facts, verdict, unit_cost, alert)
from a case's artifacts; remember_run computes them on the host for a recorded
run without writing into it. reconciler.mark_done (477 lines total), seed
before and ingest after `ousast plane run`, `ousast plane remember <run>`.
The generator annotates each repo-facts Task with openultrasast.io/memory-key
(repo, pin, candidates digest, image digest) and the Run with its population
and split; plane/ regenerated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 11:10:28 +02:00
Marius CiepluchandClaude Opus 5.5 d64ab2817e merge(harnessx-removal): deterministic baseline without the extra (task 1)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:43:27 +02:00
Marius CiepluchandClaude Opus 5.5 ca5caf735f bench(harnessx-removal): deterministic baseline without the extra (task 1)
Equality script (design section 3) records suite outcomes, `ousast pairs --json`,
the python-vulnerable quick benchmark, one standard scan and `improve --dry-run`
from a frozen export with the harnessx extra absent and no model reachable.
Two independent records are byte-identical (determinism.json).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:42:57 +02:00
Marius CiepluchandClaude Opus 5.5 b3561b5a20 spec(harnessx-removal): memory store on MinIO -- metadata, S3 Select JSON queries, versioned provenance
Maintainer: use MinIO features including its JSON support; a server exists.
One MemoryStore interface with FileStore (laptop, tests) and MinioStore (the
store of record for the separate Kubernetes deployment): tags for indexing,
S3 Select pushed-down queries probed with a local fallback, versioning for
proposal provenance, lifecycle for raw outputs, presigned URLs and bucket
notifications kept for the follow-on that removes the host receiver.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:37:14 +02:00
Marius CiepluchandClaude Opus 5.5 e19687a379 ops(ax): list the host-specific assumptions to change for a separate Kubernetes cluster
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:25:56 +02:00
Marius CiepluchandClaude Opus 5.5 2ed6d9a036 spec(harnessx-removal): tasks approved; implementation phase
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:23:51 +02:00
Marius CiepluchandClaude Opus 5.5 5240a02de5 spec(harnessx-removal): implementation plan for review; design approved
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:23:20 +02:00
Marius CiepluchandClaude Opus 5.5 647e84cdba spec(harnessx-removal): design for review
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:22:39 +02:00
Marius CiepluchandClaude Opus 5.5 f0bc372769 spec(harnessx-removal): requirements approved
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:13:10 +02:00
Marius CiepluchandClaude Opus 5.5 6eee8e6b1d spec(harnessx-removal): remove completely; the memory layer carries evolution and the loop
Maintainer amendment: no HarnessX path stays, optional or not; the plane memory
layer replaces the HarnessX evolution proposer and carries loop engineering
(Requirement 6: persistent run memory, proposals from it through the existing
validator and gate, the loop as a plane Run, provenance per proposal).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:12:55 +02:00
Marius CiepluchandClaude Opus 5.5 095eeee12e spec(harnessx-removal): brief lists the comment-only mentions
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:10:41 +02:00
Marius CiepluchandClaude Opus 5.5 9391a69f25 spec(harnessx-removal): requirements for review
The ai-service-plane increment met its gate on ax (plane-increment-2.json), the
condition for removing HarnessX. Requirements: an explicit fate per capability
(moved to a plane task or retired), code/packaging/tests removed, deterministic
outputs proven byte-identical against a recorded baseline, docs and specs
retold, one reviewable sequence. Brief inventory re-verified.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 10:10:33 +02:00
Marius CiepluchandClaude Opus 5.5 df1dfa3b7a docs(plane): handover -- ops README points at the ax runbook; replaced scripts marked reference-only (task 9)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 08:39:24 +02:00
Marius CiepluchandClaude Opus 5.5 67699e8737 bench(plane): increment gate met after the tie-break -- 16/20 agreed at $0.0205/candidate
A third verify pass on the 10 disputed candidates, 2-of-3 agreement (Req 6.4),
run on ax by extending the same Run: 16/20 declared sites agreed (reference 16),
$0.0205 per candidate with recorded triage (reference $0.022). Caveats recorded:
a design change on the same set; 17 vs 19 sites flagged in any pass; three
non-declared pgadmin candidates promoted.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 08:35:41 +02:00
Marius CiepluchandClaude Opus 5.5 34d11c7af0 chore(plane): re-pin task manifests to the rebuilt runner image
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 08:26:17 +02:00
Marius CiepluchandClaude Opus 5.5 b797605eda merge(plane): tie-break pass on disputed candidates; 2-of-3 agreement
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 08:20:57 +02:00
Marius CiepluchandClaude Opus 5.5 2f1b852966 bench(plane): correct the increment reading -- 17/20 flagged in either pass, not 19
agree's declared_sites_matched counts candidate coverage, not detection. Recounted
from agreed.json rows, plane passes flagged 17 of 20 declared sites against the
reference's 19 (budibase read and fuxa getDaqValue missed by both passes). The
earlier "detection unchanged" reading was wrong.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 08:20:50 +02:00
Marius CiepluchandClaude Opus 5.5 6e6bcfc40c feat(plane): tie-break pass on disputed candidates; 2-of-3 agreement
The design revisit after plane-increment-1.json (14/20 agreed, 19/20 in either
pass): a third verify pass on the candidates passes a and b dispute, and agree
decides 2-of-3 where pass c asked (the a/b rule elsewhere; c alone never decides).

- verify: optional OUSAST_INPUT_ONLY restricts the hunt to the named candidates
  (agree's disputed.json); an empty list is done with 0 units and no model client.
- agree: optional OUSAST_INPUT_PASS_C; metrics keep every key and add tiebreak
  {asked, flipped_to_agreed, flipped_to_rejected, rule}; usd_total, hunts and
  declared-site metrics are on the final decision.
- generate: per case <case>-vc and <case>-final appended to validation-46; the
  first 60 entries stay byte-identical, so a rerun keeps their done state.
- Offline replay over the recorded outputs with an empty pass c reproduces the
  recorded agree summaries exactly (14/20 agreed, 23 agreed, 10 disputed).
- Spec: Requirement 6.4, design Measurement, task 8.1.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 08:19:46 +02:00
Marius CiepluchandClaude Opus 5.5 c5e6ed3221 bench(plane): first increment measured on ax -- gate not met (14/20 agreed vs 16)
validation-46 ran done on google/ax: 60 tasks, 357 DeepSeek calls, $0.83 with
recorded triage ($0.0194 per candidate, reference $0.022: cost met). Declared
sites agreed 14/20 against the reference's 16/20: recall criterion not met.
Both runs find 19/20 sites in at least one pass; the two-pass agreement
flicker moved three sites. Design revisit before task 9.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 22:19:38 +02:00
Marius CiepluchandClaude Opus 5.5 c97e216d45 fix(plane): ax task names leave room for the template suffix
ax names a task's ActorTemplate <task>-tmpl-<8 hex> and Substrate caps names at
63 bytes; validation-46's first task (51 characters) fell back to a missing
default template on the live cluster. Task names are capped at 49 with a
stable hash suffix when longer.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 21:51:51 +02:00
Marius CiepluchandClaude Opus 5.5 46f0492eed chore(plane): re-pin task manifests to the rebuilt runner image
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 21:40:39 +02:00
Marius CiepluchandClaude Opus 5.5 89d000c704 fix(plane): fetch task-produced inputs from the receiver after the start
A producer's artifact rendered as spec.files of a generated <task>-inputs
Workspace ends up in ax's AX_WORKSPACES_YAML env value; Agent Substrate
rejects values over 32768 characters, so a 182 KB facts.json broke ax apply
on the live cluster. The reconciler now passes OUSAST_INPUTS instead, the
receiver serves a running task its declared inputs over GET /inputs/..., and
the runner fetches them after the start (failed fetch -> failed summary).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 21:35:28 +02:00
Marius CiepluchandClaude Opus 5.5 554f00e2f7 chore(plane): re-pin task manifests to the rebuilt runner image
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 21:23:50 +02:00
Marius CiepluchandClaude Opus 5.5 3d0d83c585 fix(plane): prepare workspaces only after the start request
On the live cluster the case clone ran at boot: the golden boot had no egress
policy, and the real actor was still being restored, so both TLS handshakes to
GitHub were cut. After the start the actor is RUNNING under its policy (the
delivery the same minute went through). Adds the one-case check Run (a subset
of validation-46) and re-pins the task manifests to the current runner digest.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 21:23:25 +02:00
Marius CiepluchandClaude Opus 5.5 344df61fc1 ops(ax): README with measured footprint and live lessons; tasks 0-7 done
The end-to-end smoke Run finished done on the local ax: submit, egress policy,
resume, router start, repo-facts in gVisor, artifacts delivered through the
egress gateway, attribution table printed, task and policy cleaned up.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 20:56:14 +02:00
Marius CiepluchandClaude Opus 5.5 137c6c8ed5 feat(plane): per-task egress policy; receiver by name through the gateway
Agent Substrate's egress gateway denies an actor without an EgressPolicy, so
no task could deliver its artifacts or reach a model. egress.py grants each
task exactly: http/80 to ousast-receiver.ax-system.svc.cluster.local, and
TLS passthrough/443 to the bound Workspaces' https Git hosts and the bound
Model's openultrasast.io/egress-hosts (api.deepseek.com for deepseek-flash).
No IPs, no wildcards, no https (MITM) rules.

The reconciler writes the policy with kubectl-ate between ax apply and
resume, reads it back, waits OUSAST_EGRESS_SETTLE (11 s, the gateway's
cached deny), and after ax delete task confirms the policy went with its
actor (kubectl-ate has no delete verb; the store cascades).

The receiver is a selector-less Service (ops/ax/receiver-service.yaml.tmpl,
applied by up.sh together with the no-build agentgateway egress gateway)
on port 18090 of the host. Measured live: the gateway checks the policy on
Host but connects to the address the actor dialled, so the runner dials the
Service's ClusterIP (OUSAST_ARTIFACT_DIAL) with the receiver's name as Host.
The runner now labels its delivery with the Run's task name (OUSAST_TASK),
which is what the receiver accepts; the ax Task name was refused with 403.
The Receiver moved to egress.py to keep reconciler.py at 460 lines.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 20:51:21 +02:00
Marius CiepluchandClaude Opus 5.5 0744a157bf merge(plane): scoring parity with the reference -- sites, ranges, triage cost
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 20:30:45 +02:00
Marius CiepluchandClaude Opus 5.5 30e4778b11 fix(plane): apply a Task after the Workspaces it binds; end-to-end smoke Run
On the live cluster ax created the Task before its Workspace in the same apply
and the runner saw an empty AX_WORKSPACES_YAML. Tests now find rendered
documents by kind and name, not position. ops/ax/smoke-run.sh renders a
model-free repo-facts Run with the current runner digest and executes it
through the reconciler.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 20:26:30 +02:00
Marius CiepluchandClaude Opus 5.5 56b1717246 feat(plane): scoring parity with the reference -- sites, ranges, triage cost
agree matched declared sites through benchmarks/independent/evaluate.py,
which the runner image does not carry, so site matching would have been
"not assessed". The generator now resolves the fix's old-side ranges
(evaluate.hunks, line for line) into case.json, with the declared sites in
the set and the reference's cost, and agree.matches_declared reproduces
matches_v2; a parity test runs both over the recorded outputs of the 15
validation cases (200 probes, identical verdicts) and agree on those outputs
returns the reference's 16 of 20 and $0.022.

agree reports cost_per_candidate (plane spend) and
cost_per_candidate_with_recorded_triage; Requirement 6.2 is judged on the
latter (design.md). --runner-image re-pins the templates from ops/ax/up.sh's
pin file; the committed manifests are regenerated at the current digest.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 19:49:28 +02:00
Marius CiepluchandClaude Opus 5.5 14ea48b1ea feat(plane): router-addressed start signal; credentials only in the start request
Agent Substrate's golden-snapshot boot runs the same image with the same task env,
so the runner no longer starts its command by itself: after the workspaces are
ready it waits for POST /ousast/v1/start (run/task checked, once per boot, refused
after a delivered run). The reconciler sends it through svc/atenet-router with
ate-target-actor, which never addresses the golden actor (plane/router.py), and the
bound Model's secretKey variable travels only in that body, into the child's env.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 19:41:35 +02:00
Marius CiepluchandClaude Opus 5.5 ac296ee3a4 spec(ai-service-plane): start signal through the router; credentials only in the start request
The live cluster ran the task command twice: in the actor and in Substrate's
golden-snapshot boot, which gets the same image and task env. Requirement 4.5
gates the command on a router-addressed start request; 4.6 carries provider
credentials only in that request.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 19:19:11 +02:00
Marius CiepluchandClaude Opus 5.5 2e7e0550b2 fix(plane): runner drops the fields ax's server sets before validating
On the live cluster ax hands AX_TASK_YAML with metadata.creationTimestamp and a
status block; the strict authoring schema rejected it, readiness never turned
green and every resume timed out. Other unknown fields still fail. up.sh retries
the first resume while the golden snapshot builds.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 19:17:19 +02:00
Marius CiepluchandClaude Opus 5.5 8e6485e577 merge(plane): ax task lifecycle -- resume, stay-up runner, delivery as completion
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 19:07:52 +02:00
Marius CiepluchandClaude Opus 5.5 bb5c534fb2 fix(plane): ax task lifecycle -- resume, stay-up runner, delivery as completion
The first live run on the local ax cluster and ax's source (docs/runner.md,
docs/sandbox.md, internal/controller/reconciler.go, internal/server/server.go)
showed a lifecycle the task-3/4 code did not model:

- `ax apply` creates a Task Suspended (Ready=False, TaskSuspended); it runs only
  after `ax resume task`. The first resume of a new image can fail with
  DeadlineExceeded while Agent Substrate builds the golden snapshot, and ax's
  server then stores phase Failed; a later resume succeeds.
- ax has no Completed phase and never reads a command's exit back.
- The runner is PID 1 and must keep serving after the command exits; stop and
  suspend deliver SIGTERM, to be forwarded to the command's process group.
  Our runner imported the task in-process and exited, which stopped the
  container ("Container guest is stopped") and made every resume time out.
- ax's controller probes /readyz?check=workspace; the query string made our
  handler answer 404.

Runner (Req 4.1-4.4): spec.command runs as a child
`python -m openultrasast.plane.tasks.<name>` in its own process group, cwd the
first workspace, AX_METADATA_URL set; ax's metadata paths are served. After the
child exits the runner reads summary.json (a crash without one becomes failed
with the exit code and stderr tail), delivers the output directory, writes
/workspace/.ousast-state/<task>.done.json and serves until SIGTERM (forwarded
to the group, SIGKILL after OUSAST_TERM_GRACE, exit 0). A boot with the marker
never reruns the command; a failed delivery is retried alone. Workspaces are
prepared on the first boot only. AX_RUNNER_EXIT_AFTER_COMMAND=1 is the
unit-test escape.

Reconciler (Req 2.2, 2.3, 3.1-3.3): `ax resume task` after apply, retrying
DeadlineExceeded/Unavailable with backoff up to OUSAST_RESUME_TIMEOUT (900 s);
completion is the artifact delivery; Failed after a resume went through, a
vanished task, or no delivery within OUSAST_TASK_TIMEOUT (7200 s) fail the
task with ax's condition message; `ax delete task` follows. Ax gains resume()
and phase() returns the condition message. doctor moves to plane/doctor.py
(tasks.md task 0 allows the sibling) to keep reconciler.py under 500 lines.

Tests: the fake ax follows the real lifecycle (Suspended, a failing first
resume that leaves Failed, Running forever, delivery only after a resume went
through), plus Failed-with-message, vanish, resume-timeout and task-timeout
scenarios; runner tests cover the child's cwd/group/metadata URL, stay-up and
SIGTERM in a real subprocess, SIGKILL of a SIGTERM-ignoring group, and the
marker. The opt-in live test now checks resume, Running, suspend and a second
resume instead of a Completed phase. design.md and tasks.md state the
lifecycle; the module audit manifest counts plane.doctor.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 19:04:52 +02:00
Marius CiepluchandClaude Opus 5.5 56755f4fd6 merge(plane): increment manifests for the 46-candidate validation run (task 7)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:58:32 +02:00
Marius CiepluchandClaude Opus 5.5 a35397c0ec feat(plane): increment manifests for the 46-candidate validation run (task 7)
Model deepseek-flash (declared provider extension, endpoint prices in
parameters, secretKey naming the DEEPSEEK_API_KEY secret), digest-pinned
Task templates repo-facts / verify / agree, and a generator
(plane/generate.py, `ousast plane workspaces --validation-set ...`) that
emits per case the vulnerable-pin Workspace, a files-only inputs Workspace
(recorded candidates after the recorded triage, declared sites, triage),
per-case Tasks and one Run validation-46: facts -> verify a, verify b ->
agree for the 15 cases, 46 candidates (43 kept), budgets 1.5x the recorded
per-pass share. A Run entry cannot add a binding or env, and Run.inputs
names only task artifacts, so the case is fixed by per-case Tasks.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:55:38 +02:00
Marius CiepluchandClaude Opus 5.5 1a69e50a68 merge(plane): ax field names verbatim; pins and bindings as stripped annotations
Live test made opt-in (OUSAST_AX_LIVE=1) against the digest-pinned manifest
up.sh renders, so the host gate no longer depends on cluster state.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:40:19 +02:00
Marius CiepluchandClaude Opus 5.5 d579b724b3 ops(ax): worker pool, digest-pinned runner image, CLI from the deployed checkout
Substrate installs no WorkerPool (resume failed: no free workers) and rejects
images not pinned by digest; ax's default runner image on gcr.io needs
credentials, so the smoke task runs our runner (repo-facts over a one-file
workspace). A release ax CLI skewed from the server (UpdateTask unimplemented).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:37:25 +02:00
Marius CiepluchandClaude Opus 5.5 18b67cf942 fix(plane): ax field names verbatim; pins and bindings as stripped annotations
Requirement 1.1 (Task/Workspace/Model use only fields ax documents) and
Requirement 1.4 (no ax field repurposed) were violated: manifests accepted
metadata.namespace and a Workspace git `commit`, and the reconciler dumped
every git field into what `ax apply` receives.

Facts from ax's source (pkg/apis/v1alpha1/ax.proto, internal/workspace/setup.go):
- ObjectMeta is `name` and `atespace` (plus server-set creation_timestamp);
  no `namespace`, no `annotations`.
- GitRepo is `name`, `repo`, `branch`, `dir`, `depth`; no `commit`.
- The API server decodes with protojson.UnmarshalOptions{}: an unknown field
  is an error at `ax apply` (confirmed on the live cluster: a rendered
  Workspace+Model applied cleanly; the same Workspace with `commit` was
  refused with `unknown field "commit"`).
- ValidateModel checks only metadata; any spec.provider string is accepted.

Changes:
- manifests: metadata takes `name`, `atespace`, `annotations` (our extension,
  never sent to ax); git entries take exactly ax's five fields (depth int);
  `commit` in spec.git is rejected with a pointer to the new Workspace
  annotation `openultrasast.io/git-commits: "<git name>=<40-hex>,..."`,
  validated for unknown git names and non-hex shas.
- reconciler: rendered docs carry only ax fields; each bound Workspace's pins
  reach the Task as env OUSAST_GIT_PINS = {"<workspace>/<git name>": "<sha>"}.
  A test walks every rendered doc against ax's field lists.
- runner: pins come from OUSAST_GIT_PINS; git is fetched as setup.go does
  (init/remote/fetch with --depth when depth > 0/checkout FETCH_HEAD, default
  branch main, cloneDestination's dir and placeholder-name rules), then the
  pin is checked out. A pinned entry without a branch fetches the commit.
- workspaces: emits the annotation, `dir: repo` and `depth: 1` instead of
  `commit`, so generated case workspaces keep <path>/repo at depth 1.
- design.md: ax field names verbatim; pins and model binding as stripped
  openultrasast.io/* annotations carried to the task by env.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:36:29 +02:00
Marius CiepluchandClaude Opus 5.5 e6d5124ffc fix(plane): doctor no longer reads "No resources found" as a ready pod
kubectl prints the sentence as one whitespace-split row, which passed the
ready check and reported an absent ax controller as "1 pods ready". Rows must
carry a READY column; an empty namespace fails. Regression test added.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:21:32 +02:00
Marius CiepluchandClaude Opus 5.5 ea9d7c7b7d merge(plane): runner contract with artifact delivery (task 3)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:18:12 +02:00
Marius CiepluchandClaude Opus 5.5 c250c5738c fix(audit): drop the duplicate plane.tasks row left by the task 6 merge
Merges ee44d3a (task 6, verify and agree) and b850a86 (task 4, ax-backed
reconciler) were committed before the gate; this commit is gated.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:15:59 +02:00
Marius CiepluchandClaude Fable 5.1 86e7a61f83 feat(plane): runner contract with artifact delivery (task 3)
ai-service-plane task 3, Requirements 4.1-4.4 (with 2.2 and 2.3 honoured by the
exit-code contract):

- src/openultrasast/plane/runner.py: the ax-task-runner entrypoint. Reads the Task
  from AX_TASK_YAML and the bound Workspaces from AX_WORKSPACES_YAML (task-1 loaders),
  serves /healthz and /readyz on port 80 (AX_RUNNER_PORT; 0 picks a free port,
  AX_RUNNER_HTTP=0 disables it for unit tests), materialises each workspace at its
  bound path (spec.files verbatim; spec.git at <path>/<name>, exported from the
  OUSAST_CASE_CACHE checkout whose origin matches, else a shallow clone at the pin;
  a repo already present on a resumed volume is kept) and only then answers 200 on
  /readyz. Runs spec.command by importing openultrasast.plane.tasks.<name> with the
  remaining elements as argv; sets OUSAST_WORKSPACE_DIR from the first binding when
  the Task's env leaves it unset. A crash writes summary.json with status failed and
  the traceback. When OUSAST_ARTIFACT_URL is set, the whole OUSAST_OUTPUT_DIR is
  POSTed as an uncompressed tar (Content-Type application/x-tar, X-Ousast-Task,
  X-Ousast-Run) with three attempts and backoff; a failed delivery rewrites
  summary.json to failed and exits 2. Exit codes: done 0, failed 2, unfinished 3.
- plane/Dockerfile.runner: python:3.12-slim + git, `pip install .` (no extras), a
  shell wrapper at /usr/local/bin/ax-task-runner exec'ing the module; built and
  loaded into kind by ops/ax/up.sh, not by the tests.
- tests/test_plane_runner.py (+ tests/plane_stub_task.py registered as the task
  module): env contract on a files workspace, readyz 503 then 200, delivery to a
  local receiver with the headers and tar checked, delivery failure -> failed and
  exit 2 after three attempts, crash -> failed with traceback, exit codes per
  status, case-cache export at the pin, shallow-clone fallback, resume keeps the
  checkout.
- model/audit.py: SPEC_OWNED row for plane.runner; module-audit measurement
  regenerated (122 modules, no orphans, note kept).

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:11:24 +02:00
Marius Ciepluch b850a86e2a Merge branch 'worktree-agent-ad03a357c98ed488d'
# Conflicts:
#	benchmarks/measurements/2026-09-08-module-audit.json
#	src/openultrasast/model/audit.py
2026-09-29 18:11:05 +02:00
Marius Ciepluch ee44d3a1f6 Merge branch 'worktree-agent-a75a91d22b62da581'
# Conflicts:
#	benchmarks/measurements/2026-09-08-module-audit.json
#	src/openultrasast/model/audit.py
2026-09-29 18:11:04 +02:00
Marius CiepluchandClaude Opus 5.5 a6998173c1 ops(ax): log the failing line and exit code of the bring-up
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:10:51 +02:00
Marius CiepluchandClaude Opus 5.5 6dfa5daa76 merge(plane): repo-facts memory task (task 5)
Audit registry conflict resolved by keeping every plane row; manifest regenerated from the merged tree.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:10:51 +02:00
Marius CiepluchandClaude Fable 5.1 4866665e5e feat(plane): ax-backed reconciler, status attribution and doctor (task 4)
Requirements 3.1, 3.2, 3.3, 3.4, 3.5, 2.2, 7.1, 7.3 of ai-service-plane.

- plane/reconciler.py (492 lines, test-counted): a Run's tasks submitted to ax
  in dependency order through the injectable `ax` CLI (apply / get task /
  delete task), serialize labels never overlapping, --workers, state.json with
  pending|running|suspended|done|failed|unfinished, a PID lock per run, done
  tasks skipped on rerun, unfinished tasks holding their dependants, a failed
  task halting the run. Budgets, inputs, output dir, artifact URL and the bound
  Model (annotation openultrasast.io/model; identity and priced parameters) go
  to the task as env in the rendered Task; consumed artifacts travel as files
  of a generated Workspace bound under /workspace. A threaded HTTP receiver
  takes each task's tar delivery, validating run/task and rejecting traversal.
  Outcome comes from the delivered summary.json (done only when every unit
  finished), never from ax's phase alone. status prints the token attribution
  table per task and Model from the summaries' usage fields (n/a, never an
  estimate) and writes attribution.json; doctor runs the four host checks.
- plane/workspaces.py: one Workspace manifest per case pin of a population.
- CLI: ousast plane run|status|doctor|workspaces.
- tests/test_plane_reconciler.py: a fake ax executable drives the DAG;
  tests/test_plane_ax_live.py (marker ax) applies ops/ax/smoke-task.yaml on a
  live cluster and skips unless doctor passes.
- Module audit rows and regenerated manifest.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:10:09 +02:00
Marius CiepluchandClaude Fable 5.1 0cbdc5d2ac feat(plane): verify and agree tasks with metered hunts (task 6)
Requirements 5.3, 2.1, 2.2, 2.3, 7.2 of ai-service-plane.

- plane/tasks/verify.py: the batched hunt of verify_batched.py moved into
  the package (system prompt, one hunt per file, up to 6 candidates, 6
  steps, candidate-anchored sites). New against the reference: each
  candidate's known callers from facts.json in the prompt (capped at 8),
  tool turns and usage per hunt in units.jsonl, per-unit resume (rows
  with "error" are re-asked), pass label from env, MeteredClient wrapper:
  BudgetExhausted -> unfinished (exit 3), AccountError -> failed with the
  provider message (exit 2); summary.json is written last, names the bound
  Model and the summed usage, and is done only when every unit finished.
- plane/tasks/agree.py: verify_batched.agreement over two pass outputs,
  agreed.json / disputed.json, per-candidate cost and turns, declared-site
  match via benchmarks/independent/evaluate.matches_v2 loaded by file;
  metrics block in summary.json; no model calls.
- tests/test_plane_verify.py: a scripted usage-recording hunter client;
  callers line, resume, 402, budget stop and resume, model and summed
  usage, unpriced usd None, both main() env contracts, agreement metrics.
- model/audit.py + module-audit manifest: SPEC_OWNED rows for plane.tasks,
  plane.tasks.verify, plane.tasks.agree; manifest regenerated.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 18:05:08 +02:00
Marius CiepluchandClaude Fable 5.1 2a490eed59 feat(plane): repo-facts memory task (task 5)
plane/tasks/repo_facts.py computes, from source text only and without a
model or git, the product files of a workspace (tests, vendored trees,
node_modules, dist/build, minified bundles and dot-directories excluded via
the layout facts plus the model_sinks.py filter), the functions each file
declares (Python, PHP, JavaScript/TypeScript, Java, Go, Ruby declaration
patterns extended from sink_candidates.py) and, per candidate function, the
call sites in other product files with path, 1-based line and enclosing
function, capped at 40 and sorted (Requirement 5.1). Output is byte-identical
across runs over the same tree (Requirement 5.2).

Under the task contract the unit is one file: a row goes to units.jsonl as
each file finishes and a restart reuses the rows computed for the same
candidate set (Requirement 2.2); facts.json is assembled from the rows and
summary.json is written last, refusing `done` unless every unit finished.
The entrypoint reads OUSAST_WORKSPACE_DIR, OUSAST_OUTPUT_DIR and optional
OUSAST_INPUT_CANDIDATES only (Requirement 2.1) and exits 0 on done, 2 on
failed with the traceback recorded in summary.json.

tests/test_plane_repo_facts.py: a three-file tree with a tests/ file, a
vendored file, a node_modules bundle and a dot-directory; cross-file callers
with line and enclosing function, same-file and declaration lines excluded,
the 40 cap, byte-identical second run, resume that recomputes only the
missing unit, a crash that leaves `failed` with the unit counts and resumes
to `done`, and the declaration patterns per language.

The two new modules are registered as spec-owned in the module audit (their
importer, the runner, lands in task 3) and the audit manifest is regenerated.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:55:34 +02:00
Marius CiepluchandClaude Fable 5.1 6fed361387 ops(ax): bring-up, tear-down and smoke Task for the local ax (task 0, in progress)
kind cluster + registry, Agent Substrate, ax control plane via ko, one files-only
smoke Task; helpers run from their own checkout because they resolve paths from
the git toplevel. Footprint and README follow once the cluster is up.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:50:28 +02:00
Marius CiepluchandClaude Fable 5.1 488e63951e merge(plane): manifest schemas and validation (task 1)
Audit registry conflict resolved: plane, plane.budget and plane.manifests rows kept; manifest regenerated from the merged tree.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:47:45 +02:00
Marius CiepluchandClaude Fable 5.1 c728808316 feat(plane): manifest schemas and validation (task 1)
Add openultrasast.plane.manifests: frozen dataclasses and loaders for ax's
Task, Workspace and Model kinds (ax.io/v1alpha1, ax's fields only) and the
Run kind (openultrasast.io/v1alpha1: tasks, dependsOn, inputs, outputs,
budget, serialize). Every violation raises ManifestError naming the kind,
metadata.name and field path: unknown fields at any depth, wrong apiVersion
or kind, a provider outside google/anthropic unless deepseek is declared via
the openultrasast.io/provider-extension annotation, a Run with a dependency
cycle, an input naming no task, and a Run task naming no loaded Task.
load_manifests reads multi-document YAML files and indexes by kind and name.

PyYAML becomes the one declared core dependency; the two zero-dependency
guards now pin exactly that, and the module audit registers the plane
package under the ai-service-plane spec.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:44:31 +02:00
Marius CiepluchandClaude Fable 5.1 3acd6c8822 merge(plane): budget metering (task 2)
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:42:48 +02:00
Marius CiepluchandClaude Fable 5.1 6884bc8c2c spec(ai-service-plane): switch to ax completely; routing and token attribution requirements
Maintainer decisions 2026-09-29: ax (kind + Agent Substrate + ax controller on
this host) is the only executor; the Run layer carries dependencies, artifacts,
budgets and attribution until ax has them. Requirement 7 declares routing per
Task (one Model per Task, no runtime switching) and a token attribution table
from provider usage fields. Task 0 (bring-up, doctor), runner artifact delivery,
ax-backed reconciler with a fake-ax test seam. harnessx-removal brief records
the HarnessX inventory; removal is gated on the increment's measurement.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:42:39 +02:00
Marius CiepluchandClaude Fable 5.1 f460735a87 feat(plane): budget metering (task 2)
MeteredClient wraps a ChatClient, sums the provider's usage rows per call,
prices them from a Model manifest's parameters (cache-hit, input, output per
million), and raises BudgetExhausted before a call once the usd or calls
ceiling is reached. A 401/402 or "Insufficient Balance" failure becomes
AccountError with the provider's message. An unpriced model reports usd None,
never zero, and refuses a usd budget; summary() gives the spend block of a
task's summary.json.

The two plane modules are registered as spec-owned in the module audit (their
consumers land in tasks 3-4) and the audit manifest is regenerated.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:41:09 +02:00
Marius CiepluchandClaude Fable 5.1 152728c052 spec(ai-service-plane): tasks approved; implementation phase
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:29:10 +02:00
Marius CiepluchandClaude Fable 5.1 c8ec677814 spec(ai-service-plane): implementation plan for review; design approved
Nine tasks in dependency order: manifests, budget metering, runner contract,
reconciler and CLI, repo-facts, verify and agree, the increment's manifests, the
measurement against the recorded batched check, handover.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:28:13 +02:00
Marius CiepluchandClaude Fable 5.1 d8f4443702 spec(ai-service-plane): design for review; requirements approved
Package src/openultrasast/plane/: manifests (ax kinds + Run), a reconciler under
500 lines with persisted status and resume, an ax-contract runner, budget metering,
and task entrypoints. First increment: repo-facts (source-only callers, no model),
verify (the batched hunt with callers in the prompt, per-unit resume), agree.
Measurement on the 46-candidate set against the recorded batched check.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:27:04 +02:00
Marius CiepluchandClaude Fable 5.1 dcc51d8a8d spec(ai-service-plane): requirements for review
Six requirements from the brief: ax kinds plus a Run kind; isolated, idempotent,
resumable tasks with metered budgets; a local reconciler under 500 lines with no
pipeline logic; ax's runner contract; repo-facts as the memory layer; and a
measured first increment (repo-facts + verify on the 46-candidate set: at least 16
of 20 declared sites, under $0.022 per candidate for two passes).

Phase: requirements, not yet approved.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:24:49 +02:00
Marius CiepluchandClaude Fable 5.1 27812499b5 chore(dev): token-lean workflow tools -- read guard, bulk reader, transcript report
Measured on the main session: 90% of development tokens were tool I/O. Three
tools from the Portal pattern:
- read_guard.py: PreToolUse hook that denies a whole-file Read over 350 lines and
  points to a ranged read or the bulk reader (pipe-tested: deny, range, small).
- bulk_read.py: answers a question about files with the cheap chat model, citing
  path:line, under a spend ceiling.
- token_report.py: where a session's tokens go, from its transcript.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:23:30 +02:00
Marius CiepluchandClaude Fable 5.1 38af0a86c2 spec(ai-service-plane): brief -- Kubernetes-style service design for the model pipeline
Adopt the design of google/ax (Task / Workspace / Model manifests, a reconciler,
isolated resumable execution, a runner contract), not yet its software: it needs
Kubernetes and Agent Substrate, lists only Google and Anthropic providers, and has
no pipelines, artifacts or budgets. We add a thin Run layer for those.

Each pipeline concern (classify, triage, repo-facts, verify, judge, recheck, score)
becomes a declared task with its own inputs, outputs and budget. That bounds the
tokens per task and lets each be developed alone. Decisions listed for the
maintainer.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 17:21:52 +02:00
Marius CiepluchandClaude Fable 5.1 ec7a5ed90d docs(pre-push-safety-net): record the model-driven pipeline result on v2 and stop
Exploratory, 7 of 17 cases, stopped when the account emptied:
- source-only scope: a model reading every product file puts the vulnerable
  function among the candidates in 15/17 (ranker 1/17, vocabulary 2/17);
- per-file triage + per-file 6-step hunts + two-pass agreement: 12/15 candidate
  cases on the validation set; 4/7 on the full run, two more in one pass of two;
- precision: 5 pgAdmin agreed findings read against source, 3 true (privileged),
  2 false with self-contradicting rationales; agreement cannot catch an error
  both passes make;
- cost: $0.009-0.022 per candidate for two passes.

Four instrument defects fixed on the way (model id, provider routing, JSON-mode
final turn, unbounded tool output) void the earlier hunter number. No gate is
met; qualification needs population v3 and a protocol that states the judge
stage, the pass rule and a config-family question.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 11:29:06 +02:00
Marius CiepluchandClaude Fable 5.1 6acd354311 feat(independent): fixed-pin recheck and an exploratory scorer for the batched verifier
- verify_batched.py --fixed asks the agreed candidates again at the fixed pin
  (no triage); a function the fix removed is recorded as removed.
- score_batched.py applies protocol v2's matching to the batched output
  (vulnerable pin, and the fixed pin where rechecked) and draws the precision
  sample.
- A finding's site is anchored to the candidate operation asked about; the
  model's own reported location is kept in `reported_at`. ReactPress's confirmed
  findAll was reported at the controller file and failed to match its declared
  site. The scorer anchors older records the same way.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 10:25:08 +02:00
Marius CiepluchandClaude Fable 5.1 072a102b20 measure(independent): triage + per-file batching + two-pass agreement -- 16/20 sites, flicker cut 26 -> 9, 34% cheaper
verify_batched.py replaces the per-candidate verifier's design:
- triage per file (no tools): where does the operation's data originate;
  internal/constant candidates are not verified;
- one 6-step hunt per file for up to 6 candidates, a verdict per operation;
- two independent passes; only agreement is reported, the rest is "disputed".

On the 46-candidate validation set ($1.01):
- declared sites: 16 of 20 agreed, 12 of 15 cases (the per-candidate design
  confirmed 11). Budibase and Froxlor are new; Unleash dropped to one pass of two.
  winml is triaged out as "constant": a CORS literal, which the reachability
  question does not fit.
- the 26 pgAdmin candidates that flickered between single runs: 2 triaged out,
  9 agreed in both passes, 4 disputed, 13 quiet.
- $0.022 per candidate for two passes vs $0.033 under the old design; on a file
  with several candidates, 2.3x cheaper per pass.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 10:22:11 +02:00
Marius CiepluchandClaude Fable 5.1 bad8c7ada4 measure(independent): known-answer check of the verifier -- 11/15 sites, unstable verdicts, escalation costs more
Declared vulnerable sites: 11 of the 15 that were candidates are confirmed (8 at 3
steps, 3 more at 6). Missed: winml (a CORS config; the reachability question does
not fit), FUXA, Budibase, Froxlor.

Restart-based escalation is strictly worse than a straight 6-step budget: 88% of
pgAdmin candidates used every step, so they paid 3 + 6 ($0.0201 per candidate vs
$0.0139 baseline). Defaults are now 6 steps, no escalation. The cache order change
did not lower the per-call cost ($0.002): the conversation prefix was already
cached; 56% of prompt tokens are served from cache.

The larger finding: on 180 pgAdmin candidates verified twice, 15 and 13 were
flagged with only 2 in common. Verdicts on borderline candidates flip run to run,
while the declared sites were found consistently. Precision needs agreement
across runs or a second judgement, not a single pass.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 10:10:33 +02:00
Marius CiepluchandClaude Fable 5.1 6df74cc3ae fix(independent): verifier checkouts live under the run's own results directory
Two runs exporting the same pin into one path deleted each other's tree.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 09:56:46 +02:00
Marius CiepluchandClaude Fable 5.1 7be1f18778 chore(independent): verifier flags for a known-answer check into a separate results directory
--declared-sites-only, --candidates-from <jsonl> and --results-suffix, so the
efficiency change can be validated on the 180 already-verified pgAdmin candidates
and the 15 declared sites without touching the main results.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 09:52:50 +02:00
Marius CiepluchandClaude Fable 5.1 041d87199a perf(hunter): cacheable message order; adaptive step budget in the sink verifier
Token efficiency, measured on 180 verified pgAdmin candidates: stage 2 costs
$0.0139 per candidate, 90% of the spend, because the conversation is re-sent on
every turn and grows with each tool result.

- Context files now precede the question in the hunter's first message. DeepSeek
  bills a cached prefix at 1/30 of a fresh one, by prefix; with the question first,
  every candidate asked about the same file (about 5 per file) broke the prefix on
  its first line and paid for the file again. A test pins the order.
- The verifier runs each candidate at 3 steps and repeats it once at 6 only when
  the hunt spent every step on tool calls without a verdict. On the seven declared
  v2 sites, 3 steps found 5 and 8 steps found 6; most candidates conclude in 3.
- Per-candidate token usage (prompt, cached, completion, calls) and escalation
  are recorded, and each case reports its cache share, so the next efficiency
  decision is made on numbers.

Validation on the 180 pgAdmin candidates plus the 22 known sites (about $1)
waits for account credit: DeepSeek returns 402 at the moment.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 09:52:23 +02:00
Marius CiepluchandClaude Opus 5.5 66fa08cdd0 fix(independent): the sink verifier stops on an account error and retries unverified candidates
The rerun hit DeepSeek HTTP 402 ("Insufficient Balance") after 180 pgAdmin
candidates. Every later call failed at $0, and the runner wrote sixteen cases as
complete with nothing asked, because an unverified candidate counted toward
completion. That is a clean-looking zero from an instrument that could not call
the model.

Now:
- 401/402 or "Insufficient Balance" stops the run at once, exit 2, nothing
  recorded;
- only verified candidates count as done, so unverified ones are asked again on
  resume;
- a case is complete only when every candidate was verified.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-29 07:00:59 +02:00
Marius CiepluchandClaude Opus 5.5 8794884a64 fix(hunter): bound every tool result; show the provider's reason for an HTTP error
FUXA failed every hunt with HTTP 400, and neither run could say why. The client
discarded the response body. With the body shown, the reason was "This model's
maximum context length...": FUXA ships minified bundles whose single lines run to
megabytes, and grep_repo returned a matched line whole.

- grep_repo and find_refs quote a matched line (240 characters), never the whole
  line.
- Every tool result is capped at 16 x max_chars, with a truncation marker.
- OpenRouterChatClient errors carry the provider's response body (first 500
  characters). That would also have exposed the rejected model id in one call
  instead of eighteen days.

After the fix, FUXA candidates verify normally at $0.004-0.013 each. A test covers
the megabyte-line case.

verify_sinks.py: skip futures cancelled at the spend ceiling instead of reading
them (the run crashed with CancelledError).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 20:15:14 +02:00
Marius CiepluchandClaude Opus 5.5 3ffb8b4c54 fix(hunter): ask for the final answer in plain text; DeepSeek's JSON mode lost it
The final-answer turn requested the provider's JSON mode with the tools still
attached. DeepSeek (deepseek-flash) then answered with the format itself
(`{"type": "json_object"}`), wrapped the array (`{"type": "json_object",
"value": [...]}`), or emitted its tool-call markup as text. A hunt that had found
the vulnerability reported nothing.

Measured on the v2 sink verifier with the declared vulnerable functions: 0 of 7
confirmed in JSON mode, 5 of 7 asked plainly (6 of 7 at 8 steps). The model named
Medplum's actual redirect-URI prefix-match bug, LMDeploy's redirect bypass of
_is_safe_url, and ContextForge's unsandboxed Jinja2.

The final turn is now plain text without tools. The parser also reads a
`{"value": [...]}` wrapper. Two tests that pinned JSON mode now pin the plain
turn, and a new test reproduces the wrapped answer.

Invalidates as measurements: the hunter experiment on v2 (c4a0bd8) and the first
sink-verifier run. Both used this turn.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 20:10:57 +02:00
Marius CiepluchandClaude Opus 5.5 267bca5232 perf(hunter): read a repository's source once per state, not once per tool call
Each grep_repo and find_refs call re-enumerated the tree (walk, ignore rules,
vendored checks) and re-read every file. The v2 sink verifier spent almost three
hours on its first case (pgAdmin, about 1,800 files) at 108% CPU with one open
connection. It was CPU-bound in these tools, not waiting on the model.

Source lines are now memoised per root, keyed by path and modification time. A hit
is re-validated file by file, because an edit inside a subdirectory does not touch
the root's mtime; a test covers one enumeration and a detected edit.

verify_sinks.py now runs candidates in worker processes, not threads, and appends
each candidate's result to a per-case log, so a stopped run resumes where it left
off. GoPay: 12 candidates in 14 s, the real site found.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 18:30:09 +02:00
Marius CiepluchandClaude Opus 5.5 023c24535f measure(independent): stage-2 verifier for model-classified sink candidates
Per candidate function, one bounded tool-hunter pass (custom prompt, the file as
context, 3 steps) asks whether attacker-controlled data reaches the operation
without a sufficient guard. It reports only with a stated source-to-sink path.
Hard spend ceiling, parallel workers; one run per pin; the fixed pin is rechecked
only for detected sites. Exploratory on v2.

Pilot: on GoPay, the one real site was reported in 3 of 4 runs and nothing else
out of 12 candidates.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 13:24:22 +02:00
Marius CiepluchandClaude Opus 5.5 32eb46f297 measure(independent): model-classified sinks put the vulnerable function in scope in 15/17
Source-only selection on the v2 vulnerable pins, compared on the same cases:
- ranker top-20 entry points: the vulnerable file was in scope in 1/17;
- shipped-vocabulary sink-first: the declared site was a candidate in 2/17;
- model-classified sinks (deepseek-flash, no vocabulary): the declared site was a
  candidate in 15/17, and a fix or site file in 16/17. Every chunk was classified;
  $14.42.
Misses: User Registration (file in scope, not the function) and Sulu (JCR-SQL2
query builder). The cost of this reach is volume: 3-1037 candidate functions per
repository, 4,762 in total. The next stage must decide which candidates are fed by
attacker data without a guard. v2 is spent for tuning; this qualifies nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 13:21:41 +02:00
Marius CiepluchandClaude Opus 5.5 05d56993fc measure(independent): model-classified sinks diagnostic (source only, no vocabulary)
This asks DeepSeek, per product source file (in chunks of 400 numbered lines),
which functions perform the family's dangerous operation on non-constant data. It
is scored like sink_candidates.py: is the declared vulnerable function among the
candidates? A failed chunk counts as unclassified, never as "no sinks". The spend
ceiling is enforced across worker threads.

Pilot: GoPay found its declared site for $0.03. Estimated full v2 run: ~$14-18.
The maintainer raised the ceiling to $20.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 12:43:03 +02:00
Marius CiepluchandClaude Opus 5.5 83cca9894f measure(independent): sink-first selection from source -- the vocabulary misses most real sinks
This is a source-only diagnostic: no engine and no model. At each v2 vulnerable
pin, every call matching the family's shipped sink vocabulary becomes a candidate,
attributed to its enclosing function.
- The declared vulnerable function is a candidate in 2/17 cases (Medplum, GoPay);
  the ranker's top 20 managed 1/17.
- A fix-changed or site file is among the candidates in 8/17.
- Candidates per repository range from 2 to 904 functions.

The misses are vocabulary, not direction. Examples: wp_redirect, download_url,
spawn(shell:true), ky(), requests.Session().get, sympy parse_expr, Jinja
from_string().render, Mongo collection.find, JCR-SQL2 createQuery, TypeORM
andWhere template strings, and CORSMiddleware(allow_origins=["*"]) are not named.
v2 is spent for tuning, so this informs design and qualifies nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 12:36:43 +02:00
Marius CiepluchandClaude Opus 5.5 c4a0bd8bd9 measure(independent): the unchanged tool hunter on population v2 -- recall 1/17, and why
The run follows hunter-protocol-v2.md: deepseek-flash, the ranker's top 20
regions per pin, protocol v2 scoring, $8.88 spent.
- Recall: 1/17 (Wilson 0.01-0.27).
- Precision: 2/14 sampled (both true positives need a privileged role, one of
  them GoPay's CVE).
- Fixed-side alerts: 1. Benign alerts: 16.
- Stability: 73% of findings differ between two identical runs.

The deciding observation: the ranker's top 20 contained the vulnerable file in 1
of 17 cases, and in that case (GoPay) the hunter detected it. With 4 tool steps
per hunt it cannot search its way to code it is not handed. The result therefore
says less about the model than about scope selection, which sits upstream of the
engine too.

Instrument failures, reported as failures: every FUXA pin and three Directorist
pins (DeepSeek HTTP 400 on every hunt; neither offered a region of its family).

Scorer fix: protocol v2 defines benign alerts and the precision pool over the
CASE FAMILY, but score_v2 pooled every family. The engine's dump emits only the
case family, so its recorded result (21adc40) is unchanged by this; rescoring
reproduces 3 benign alerts and 49 pooled findings.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 11:36:32 +02:00
Marius CiepluchandClaude Opus 5.5 6bfb05f952 spec(pre-push-safety-net): pre-register the hunter experiment on population v2
This is the deciding experiment of the strategy review, fixed before any v2 pin is
hunted.
- Hunter: the unchanged tool hunter (default prompt, tools and 4 steps) on
  deepseek-flash.
- Scope: the ranker's top 20 regions for the case family, in four hunts. The model
  never sees the fix diff or the declared sites.
- Families: a fixed keyword table maps the model's family to the protocol's three.
- Scoring: exactly protocol v2's scorer.
- Cost: a hard $10 ceiling.

hunt.py runs it and scores it (--score). A pilot on a v1 development case
(mongo-express) found the vulnerability at lib/bson.js:60, at $0.031 per pin,
alongside a lot of speculative noise.

Transport fix found by the pilot: the DeepSeek endpoint the hunter builds was
unpriced, so the cost meter read $0.00 for any number of calls. It is now priced.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 09:10:02 +02:00
Marius CiepluchandClaude Opus 5.5 cf579956d5 fix(model): the chat model is deepseek-flash; the hunter reaches DeepSeek
Every model call has failed since 2026-09-10 (83ab533):
- The default detector and judge model was `deepseek-v4.1-flash`, the product name.
  The API rejects it with HTTP 400 ("The supported API model names are
  deepseek-flash, deepseek-v4-pro"). The judge and pipeline record an endpoint
  failure as a degradation, never a verdict, so nothing failed loudly.
- `tool_hunter.resolve_hunter_client` resolved OpenRouter only. In this project
  DeepSeek is the chat provider and the OpenRouter key is scoped to embeddings, so
  the hunter could not reach a chat model at all; the call came back 401.

The defaults are now `deepseek-flash` (priced). The hunter resolves DeepSeek first
when its key is set, and OpenRouter stays as the explicit or fallback route.
Verified live: plain completion and a tool call through both the endpoint resolver
and the hunter client.

Not affected: the independent evaluations (finding_dump runs with
max_model_calls=0) and the 2026-09-06 hunter measurement, which predates the id.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 09:01:14 +02:00
Marius CiepluchandClaude Opus 5.5 5835239808 docs(pre-push-safety-net): strategy review after the two independent evaluations
v1 (1/11) and v2 (0/17 recall, 0/17 sampled precision, 12% completion) show that
fixes learned on one population do not transfer to the next. The causes are
breadth: framework-shaped entry points, module boundaries the frontends do not
resolve, name-matched sinks colliding, and scale.

Options:
- A: continue engine-first;
- B: the model proposes, the engine proves;
- C: narrow the product to one ecosystem.
Recommendation: first run the unchanged tool hunter on the v2 pins as the deciding
experiment.

For maintainer decision; no approved spec changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 08:55:07 +02:00
Marius CiepluchandClaude Opus 5.5 21adc408f1 measure(independent): evaluation of population v2 under protocol v2 -- fails every M4 gate
The analyzer was frozen at a75b9f6. 17 cases, 102 scans (vulnerable and fixed pins
twice each), scored exactly as pre-registered.
- Recall: 0/17 (Wilson 95% 0-0.18). No unstable detections.
- Precision: 0/17 sampled (0-0.18). Every sampled finding is false: regex
  match/exec read as sinks, int casts, ORM statements, client-SDK code, the
  process's own argv, a generated database name, a registered-URI check.
- Completion: 13206/108374 questions (12%).
- Fixed-side alerts: 0, trivially, since nothing was detected.
- Benign alerts: 3.
- Stability: 2 of 100 findings differ between runs.
The zero is not an instrument artefact: FUXA, GoPay, cct-studio, ReactPress, User
Registration and Directorist completed questions in the sink file itself and still
found nothing.
Instrument failures, reported as failures: LMDeploy (every taint batch failed on all
six pins) and Froxlor's two benign scans.

This record is committed before any analyzer change informed by v2.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-28 08:52:53 +02:00
Marius CiepluchandClaude Opus 5.5 a75b9f62f9 spec(pre-push-safety-net): pre-register evaluation protocol v2 and its runner
Committed before any v2 case is scanned. It has two changes from v1, both chosen by
the maintainer from lessons of the v1 run:
- declared sites: a finding at a `path::function` the advisory names is credited,
  as well as one on the fix's changed lines;
- two runs of the vulnerable and fixed pins: detection needs the finding in both
  runs, a fixed-side alert in either counts, and the difference between runs is
  reported as stability.

evaluate.py takes --population. v2 scans six pins per case and scores with
matches_v2 / score_v2. v1's scoring is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-26 18:02:03 +02:00
Marius CiepluchandClaude Opus 5.5 21634ccf58 spec(pre-push-safety-net): freeze the independent population v2
The analyzer was not run. freeze.py read each case's source at both pins:
- both pins are commits in the repository and in the right order;
- the declared sink site's file is present at the vulnerable pin, and its declared
  functions appear in it;
- the fix touches a declared site file (Budibase's sites reordered so the changed
  file, queries.ts, comes first).

Benign controls:
- 12 are reviewed automatic proposals.
- 3 automatic proposals were rejected as not benign: qwed's merged a ReDoS fix,
  ReactPress's corrected a SQL column whitelist, and Sulu's was a branch merge that
  can carry the fix.
- Those three, plus cct-studio and Directorist (no automatic proposal), were chosen
  by hand. Directorist's comes from before the vulnerable pin, because only 3
  commits follow its fix.
freeze.py now accepts a recorded benign control instead of reporting that none was
proposed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-26 18:02:03 +02:00
Marius CiepluchandClaude Opus 5.5 c64e37edbd spec(pre-push-safety-net): reserve the independent population v2 (17 cases, unscanned)
v1 was evaluated and then used to retune the analyzer. A retune can only be
qualified on a population that did not inform it.

The candidates came from web research alone (research-2026-09-26.md):
- advisories were taken newest-first per language under fixed criteria, never by
  how easy they look;
- 261 advisories were examined and 224 rejected, each with a reason;
- every SHA pair was verified through the GitHub API;
- no clone and no scan.
The maintainer selected the rule-based "balanced" set: 5 Python, 6 JS/TS, and 6 PHP
of which 4 are WordPress plugins; 10 injection, 6 destination and 1 config.
Each case declares the advisory's sink sites before any scan, for protocol v2.

freeze.py takes --population and writes freeze-vN.json. The reservation tests now
cover every population-v*.toml.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-26 17:53:51 +02:00
Marius CiepluchandClaude Opus 5.5 d14fd5e960 Revert "feat(taint): project sink wrappers, one level and confirmed by dataflow"
This reverts commit cc885f0. It failed the acceptance bar set before the run (each
recall pin within +2 findings, completion unchanged):
- WP Statistics: 16 -> 31.
- PMPro: 16 -> 26, and completion 378/382 -> 103/382. Six prior findings were lost
  to the budget; CVE line 936 survived.
- YesWiki: completion 2798 -> 1968, still not detected.

Two causes:
- Cost: wrapper confirmation runs a flow query per parameter of every sink-bearing
  method, repeated at each engine start of a portioned scan.
- Precision: a helper whose parameter reaches a sink under its own escaping (for
  example `getMemberOrderByPayPalToken(sanitize_text_field(...))`) is not a sink
  for its caller.
Doing this properly needs summaries that carry the wrapper's internal
sanitisation, computed once per scan, not per request batch.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-26 15:36:25 +02:00
Marius CiepluchandClaude Opus 5.5 cc885f025c feat(taint): project sink wrappers, one level and confirmed by dataflow
This is the second attempt at the YesWiki shape: SQL run through
`$this->dbService->query($query)`, whose body is the `mysqli_query` the facts name,
through a receiver the frontend cannot type. The first attempt was reverted in
c7e3a05.

A method is now a wrapper for a parameter only when the engine's own dataflow
inside the method carries that parameter to a fact sink's argument, on a plausible
path that no cleansing or guard discharges. The guard rails:
- one level only;
- nothing under a bundled library tree (lib/, vendor/, node_modules/, ...);
- a name counts only if every project method with that name is a wrapper;
- wrappers are asked in the flow pass only, never in the evidence pass, so no
  region's selection or pruning changes;
- at a wrapper call, only the confirmed argument is asked.

Three Joern-gated tests cover the direct shape (reported), a literal, an
intval-casting helper and a level-2 feeder (not reported), a harmless same-named
method, and a wrapper inside lib/. Each fails when its rule is removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-26 13:05:12 +02:00
Marius CiepluchandClaude Opus 5.5 c7e3a05ba0 Revert "feat(taint): project-local sink wrappers are sinks where they are called"
This reverts commit 54c9dfc. The recall-pin check (p16) showed a large precision
and completion regression, although every recall target survived:
- MW WP Form: 3 -> 60 findings. Form-field renderers became sinks.
- WP Statistics: 16 -> 40.
- PMPro: 16 -> 65, 20 of them inside bundled Braintree/Stripe code. Completion fell
  to 390 of 1189 questions, because wrappers made many more regions look
  sink-bearing.
- YesWiki: completion fell from 2798/2807 to 216/2807.

The structural summary was too loose. Any identifier in a sink argument counted as
fed by a parameter if any assignment anywhere in the method mentioned it, with no
order or dominance. It also ignored vendored library code and cascaded three
levels by name. A redesign needs dataflow-confirmed, level-limited summaries that
leave the question count alone.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-26 10:14:30 +02:00
Marius CiepluchandClaude Opus 5.5 54c9dfcd44 feat(taint): project-local sink wrappers are sinks where they are called
Real code rarely calls the driver where it builds the query. YesWiki builds its
INSERT in `FormManager::create($data)` and runs it through
`$this->dbService->query($query)`, whose body is the `mysqli_query` the facts name.
The only sink the facts knew sat in a helper that no request reached directly.

A method whose parameter feeds a family sink is now a sink for that argument,
reported at the call site. The summary is structural, to stay cheap and
deterministic. It is not a flow query per parameter, which would repeat at every
engine start of a portioned scan. The parameter must reach the sink call's
argument directly or through at most two local assignments, with at least one
occurrence outside every cleansing call. It iterates up to three levels, so a
wrapper of a wrapper counts.

Calls resolve by name, because an injected receiver has no type the frontend can
see. A name therefore counts only if every project method with that name is a
wrapper. At a wrapper call, only the summarised arguments are asked.

Two Joern-gated tests cover:
- a request value into a second-level wrapper, reported at the controller's call;
- direct use, a literal, and an `intval`-casting helper, which are not wrappers;
- a harmless same-named method, which disqualifies the name.
Both fail with their part removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-26 07:55:24 +02:00
Marius CiepluchandClaude Opus 5.5 64ef488ce4 test(cpg-budget): a process vanishing mid-read is the success case too
Reading /proc/<pid>/stat for a process that is reaped during the read raises
ProcessLookupError (ESRCH), not FileNotFoundError. The previous race fix caught only
the latter, so the hanging-process-group test failed twice today under full-suite
load. It passed alone. The disappearance is exactly what the test asserts.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-26 07:44:33 +02:00
Marius CiepluchandClaude Opus 5.5 63460f8af4 fix(taint): a guard validates only the value it tests
p14 showed PMPro's functions.php:906 discharged by `elseif (in_array('E', $levels))`.
That checks whether a constant is in `$levels`; it says nothing about `$levels`. The
guard counted a carried variable in any argument. Only argument 1, the tested value
(`in_array`'s needle, the URL given to `url_has_allowed_host_and_scheme`), is
validated now. The allowlist test gains the haystack shape as a reported control.
The test fails when any argument counts.

The other p14 removals were right: WP Statistics export_data x2
(`if (!in_array($table, $allow_tbl)) $table = false;`) and PMPro memberorder.php:1633.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 18:34:15 +02:00
Marius CiepluchandClaude Opus 5.5 69d4a8626f fix(taint): a value stored while its guard held stays guarded
Ultimate Member's fixed pin kept one alert. The fix writes
`$this->sql_order = " ORDER BY u.{$sortby} "` inside
`elseif (in_array($sortby, $allowed, true))`, and after the if/elseif chain it
passes the field through `apply_filters(...)` into a second assignment. The guard
was judged only at the flow's final node, which is outside the branch.

A guard is now also judged at every point where the path stores the value (an
assignment target). A value stored while the check held carries the check forward,
because the rest of the path derives from that store. A mere read inside a branch
is not a candidate: the engine lists uses in branches the flow does not depend on.

The Joern-gated test gains the shape and its control: the same store in the
unguarded else branch, through the same filter, is still reported. The test fails
when only the end is judged.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 17:24:49 +02:00
Marius CiepluchandClaude Opus 5.5 03f2ce7162 feat(taint): guards -- a validator or allowlist check makes a value safe where it held
Three of the four fixed-side alerts left in the independent population had one
cause: the fix was a check, and checks were not modelled.
- wger's fix is `url_has_allowed_host_and_scheme`: `if not ...(next_url):
  next_url = reverse(..)` and `if next_url and ...(next_url): return redirect`.
- Ultimate Member's fix is an allowlist: `in_array(strtoupper($order), [...],
  true) ? $order : 'ASC'` and `elseif (in_array($sortby, $allowed, true))`.
(The fourth, Signal K, is `assertAllowedHost`, a project-local function. It is not
covered.)

Sanitizer facts can now say `guard = true`. A guard is a check, never a cleansing
call on a path. A flow is discharged only when a guard tests a variable the path
carries, and the flow is in one of these places:
- inside the branch whose condition requires the check. The guard may be combined
  with `&&` only;
- past a failure test, a single `!` combined with `||` only, whose branch exits
  (return, exit, die, wp_die, throw) or overwrites the value, and whose condition
  dominates the flow's end;
- through a ternary whose condition requires the check, where the other arm cannot
  carry the value. The engine can enter through the condition's own copy of the
  value, which never becomes the result.

Facts: `in_array` (PHP) and Django's `url_has_allowed_host_and_scheme` /
`is_safe_url` (Python). A row now consults guards even when a request lists no
sanitizers.

Two Joern-gated tests cover the discharged shapes and, as controls, an unguarded
redirect, a guard on a different variable, a failure that is only logged, the
unguarded else branch, and the ternary arm the check does not select. Both tests
fail without the guard.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 16:56:15 +02:00
Marius CiepluchandClaude Opus 5.5 dfc5e71730 fix(preprocess): read .gitignore with git's semantics; never ignore a tracked file
The enumeration dropped every `!` line and matched with fnmatch, whose `*` crosses
`/`. YesWiki's `tools/*` followed by `!tools/bazar` therefore removed all 583 PHP
files under tools/, the vulnerable FormManager.php among them. The scan read 243 of
803 files, completed 1128 of 1128 questions and looked clean: a zero from an unread
tree.

Rules now follow git:
- the last matching rule wins, and `!` re-includes;
- `*` and `?` stay inside one path component, and `**` crosses components;
- a rule with an inner `/` is anchored to the root;
- a trailing `/` matches directories only;
- nothing under an ignored directory can be re-included.
In a work tree, a tracked file is never ignored, because `.gitignore` describes what
the repository does not hold.

The test uses `git check-ignore` as the oracle on 16 paths covering negation,
anchoring, depth, directory-only and `**`. It also covers a git-archive export with
no .git, and a tracked file under an ignored directory. All three tests fail on the
old reader.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 15:29:43 +02:00
Marius CiepluchandClaude Opus 5.5 3358631448 fix(scan,taint): TypeScript's limit is owned by TypeScript; a source pattern is a token
Two defects found by the cause-B rescan of the independent population (exploratory,
a retune after protocol-v1):

- `typescript_property_support_unvalidated` still reached the whole repository.
  YesWiki is a PHP application that ships a few `.ts` files, and all 1,128 of its PHP
  questions were unresolved. The limit is now owned by the TypeScript partition. A
  TypeScript project (Mockoon, Signal K) is unchanged.
- `isSourceCall`, and the method-level "reads a source" heuristics, matched source
  patterns as substrings. Flask's `request.get` therefore claimed Django's
  `request.get_host()`. wger's fix passes that host to
  `url_has_allowed_host_and_scheme`, and both fixed functions were reported as open
  redirects from it. Patterns are now matched as bounded tokens, like sinks and
  sanitizers. wger's vulnerable-pin detections rest on `request.GET`, which still
  matches.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 15:13:30 +02:00
Marius CiepluchandClaude Opus 5.5 65dcb46b5e fix(scan): symlink, unsupported-language and unparsed-file gaps are owned by what they name
This extends dbcb230 to the remaining graph-integrity gaps. In the independent
evaluation these gaps were repository-wide:
- alerta: one symlink, `alerta/app.wsgi`, which is not even source, made 261 of 261
  questions unresolved.
- wger and Signal K: a language no frontend reads did the same.
- YesWiki: unparsed files did the same.

Now:
- the partition build records the paths a symlink boundary skipped (a directory
  ends in `/`) and the languages no frontend reads;
- `files_unparsed` carries its full paths;
- the scan demotes only questions about an owned path or under an owned directory,
  and only questions in an owned language.
Every gap stays in the report as a boundary. A gap that names nothing keeps its
partition-wide or repository-wide reach. Tier-0 pruning stays off, as before.

The tests that pinned repository-wide reach now assert ownership. The one about a
vendor exclusion masking an integrity failure keeps its intent, using a census gap
that names no file.

Retune after protocol-v1; exploratory only.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 14:30:36 +02:00
Marius CiepluchandClaude Opus 5.5 949977c7a6 fix(taint): an SQL escape protects only a quoted value; splice the engine's object detours
`esc_sql` and the mysql/pg `*_escape_string` functions only backslash quotes. A value
they escape is safe inside a quoted literal and nowhere else. Ultimate Member
CVE-2024-1071 is `$sortby = esc_sql(...)` built into `" ORDER BY u.{$sortby} "`, and
it read as sanitized. Sanitizer facts can now say `quoted_only`. The escape keeps its
credit unless the first string the value is built into (concat, interpolation,
addition, f-string) places it after literal text that opens no quote. With no string
built downstream, the credit stays as before. pg_escape_literal adds its own quotes
and stays a full cleanser.

Two further defects hid the same CVE:
- The engine is field-insensitive on the object. Its one path from `$this->sql_order`
  went `$this` -> `$this->having` -> `esc_sql($this->having)` -> ... -> the query.
  absorbedByObject cut it, and it credited an unrelated field's escape. When a path
  leaves a field through the object for another field and reads the first field
  again, the detour is spliced out. The later read may be on the path or inside the
  sink argument where the path ends.
- The field join judged a field clean if ANY sanitizer name was on its path. It now
  uses the same quote-aware check as a row. This matters because the engine's own
  path to the query changed between runs: 2 of 4 identical runs chose a path via
  `$wpdb->prepare`. The field join is the deterministic route.

Exploratory (retune after protocol-v1): the Ultimate Member vulnerable pin is detected
at class-member-directory-meta.php:936. The fixed pin still alerts at :945. Its fix is
an allowlist (`in_array(...) ? $v : 'ASC'`, and an `in_array` guard), which is not
modelled yet.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 13:35:45 +02:00
Marius CiepluchandClaude Opus 5.5 562b670a43 feat(rules): Django request sources; a destination with a fixed origin is not untrusted
python.toml's sources were Flask-only, so no flow could start anywhere on a Django
project. OpenCVE's SSRF reads its URL from `form.cleaned_data`, and the scan answered
the sink file and found nothing. Added: request.GET/POST/body/COOKIES/FILES/META,
DRF's query_params, and cleaned_data.

With those sources known, OpenCVE reported three redirects that cannot leave the
site: `reverse("cves") + "?" + request.GET.urlencode()` (twice) and
`reverse(route, kwargs=...)`. Destination sink facts can now set
`prefix_fixes_origin` and `origin_anchors`. A path is cut when it enters a
concatenation, f-string or `+=` through anything other than its leftmost operand,
and that operand fixes the origin. Origin-fixing operands:
- a literal `/x`, `?` or `#`;
- a literal of the form scheme://host/;
- an anchor call;
- a variable whose every assignment fixes the origin.
A path is also cut when it passes through an anchor. `"/" + x` (//evil) and
`"https://" + host` stay reported. A Joern-gated test covers all seven shapes.

Exploratory (retune after protocol-v1): OpenCVE vulnerable and fixed pins are both
0 findings, down from 2 and 3. The SSRF itself is still missed, because the flow
crosses a registry dispatch (`REGISTRY.get(t)(extras).run()`) that the engine does
not follow.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 12:37:22 +02:00
Marius CiepluchandClaude Opus 5.5 ec3d1457fb fix(rules): Node vm is an eval sink; a package name is not a sink token
mongo-express CVE-2019-10758 passes the request body to vm.runInNewContext. No
injection sink named that call, so the flow was answered and missed. runInNewContext,
runInContext, runInThisContext and compileFunction are now CWE-95 sinks.

Once the file's questions completed, the fixed pin showed a second defect. Every call
on `require('mongodb-query-parser')` has that package name as its full name. The
bounded-token match treats `-` as a boundary, so the name matched the SQL sink
`query`. In a full name, `-` belongs to a package name, so it no longer separates
tokens.

Exploratory (retune after protocol-v1): mongo-express vulnerable pin now detected at
lib/bson.js:60.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 11:47:15 +02:00
Marius CiepluchandClaude Opus 5.5 dbcb230990 fix(scan): a census gap that names its missing files is owned by those files
The first independent evaluation (results-v1.json) completed no question on eight of
eleven projects. On the cases traced, the cause was a census gap: `.eslintrc.js` or
`webpack.config.js` missing from a JavaScript graph, or `contrib/apache/api.wsgi` from a
Python graph, and every question in that language was then marked unresolved. Now only
questions about a named missing file are demoted. The gap is still reported as a
boundary. A gap that names no files still reaches its whole partition.

This is a retune after the evaluation. Per protocol-v1 it cannot be qualified on the
same population.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 11:16:45 +02:00
Marius CiepluchandClaude Opus 5.5 98e8dcdfb2 measure(independent): first evaluation on the frozen population -- fails every M4 gate
Scored exactly per the pre-registered protocol-v1.md, analyzer ebcc2cf,
44 scans on eleven untouched cases.

Recall 1/11 (95% CI 0.016-0.377): only Signal K's SSRF sink is detected.
Precision 1/12 on the full pooled sample: the other eleven findings are
guarded redirects (isSafeRelativeUrl, getSafeDestination), Host-header
redirects, constant SET NAMES queries, a config-file eval and an installer.
Four fixed-side false alerts (mongo-express bson.js, both wger sinks after
their guard was added) and one benign.

The dominant cause is coverage, not precision: at the vulnerable pin eight
of eleven cases completed ZERO questions and Saltcorn asked none, so most
targets were never decided. Recorded before any investigation or change.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 09:56:26 +02:00
Marius CiepluchandClaude Opus 5.5 ebcc2cf7fd fix(independent): lint in the evaluation runner (committed past a failing check)
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 16:45:36 +02:00
Marius CiepluchandClaude Opus 5.5 13180a11db feat(independent): the evaluation runner for protocol v1 -- run (four scans per case, resumable) and score
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 16:45:24 +02:00
Marius CiepluchandClaude Opus 5.5 e4de54610a spec(pre-push-safety-net): pre-register the evaluation protocol for the independent population
Committed before any case is scanned: what is under test (the analyzer at the
runner's commit, per-case family only, four scans per case), how a finding
MATCHES a case (a fix-changed file, the case family, the named function or
within 10 lines of a fix hunk), detection / fixed-side / benign false alerts,
unanswered and instrument failures, the precision sampling rule (every third
pooled vulnerable-pin finding, or all if 15 or fewer), what is reported
against the M4 gates, and that no retune may be informed by the results.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 16:44:32 +02:00
Marius CiepluchandClaude Opus 5.5 d1dc0d1131 spec(pre-push-safety-net): freeze the independent population v1 (15.1)
Every reserved case verified with git and file reads only -- nothing
scanned: pins present and ordered, the sink present at the vulnerable pin,
the fix touching the sink (or LearnPress's declared source-side fix site),
and the sink file hashed at both pins (benchmarks/independent/freeze.py,
record freeze-v1.json).

Each case now carries a benign control from its own history, reviewed by
diff. The mechanical keyword filter was not enough: it proposed an access
restriction, a filter check, API-token scopes, a login change and a
TLS-verification option, all rejected; Ultimate Member and alerta got
hand-picked controls (a WPCS reformat; a dependency change with both sides
still vulnerable). The guard test requires a benign pair per case once
frozen.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 16:43:32 +02:00
Marius CiepluchandClaude Opus 5.5 451d4e4be7 spec(pre-push-safety-net): reserve the independent population v1 (15.1), unscanned
Eleven cases chosen by the maintainer from fifteen researched by advisory
and fix diff only -- nothing cloned, nothing scanned, none among the 262
repositories the benchmarks reference -- plus the earlier Ghost
reservation. Four are PHP repositories that ship JavaScript (two WordPress
plugins), satisfying the multi-language requirement; families are limited
to the v1 scope (injection, untrusted destination, permissive config).

The development subjects shaped every rule of the last week and can no
longer measure whether those rules generalise; this population can, until
something scans it. A guard test fails the moment a reserved repository is
referenced by a recipe, catalog, manifest or measurement outside
benchmarks/independent/. Freezing (pins and sinks verified by reading
source, a benign change per case) is the next step.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 16:12:22 +02:00
Marius CiepluchandClaude Opus 5.5 d5277c60d0 fix(taint): a field the method itself just overwrote is not object state
The largest remaining class in the third PMPro sample (2 of 4 false
positives): every MemberOrder method builds its statement in
$this->sqlQuery and runs it at once. One method writes request data there,
the field join keyed the field file-wide, and deleteMe() -- which had just
set it from the internal $this->id -- and updateTimestamp() (no caller in
the plugin) were reported as SQL injections.

A field read is now a field-join source only when it reads state another
method left: not the target of a write, and not a read that the method's
own write to the same field dominates. If that own write carries request
data, the ordinary flow from the source still finds it. A real-Joern
control covers all three: the method whose write carries $_POST (found),
the method that overwrote it with an internal id (quiet), and a method
reading what another left (found). It fails on the old query.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 15:53:57 +02:00
Marius CiepluchandClaude Opus 5.5 107a80d41d measure(sqli): PMPro 1-in-5 sample after the fifth fix -- 1/5, recall pins intact
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 13:36:36 +02:00
Marius CiepluchandClaude Opus 5.5 6c03b4a38c fix(taint): a sanitizer the value passes through counts; only an access is a source
Found on PMPro's re-added class-pmpro-levels.php:87: every REST handler
casts the level id, `isset($p["id"]) ? intval($p["id"]) : null`, and the
flow was reported unsanitized. Traced, the engine steps from intval's
argument straight to the conditional, so `intval` is never a path element
and the element-wise sanitizer check never saw it.

- sanitizedBetween: where a path element contains the previous one, the
  syntax between them is what the value was wrapped in; a sanitizer there
  sanitizes the flow. A sanitizer elsewhere in the expression (the other
  branch of a conditional) does not count. Used by the sink query and the
  hook join; the field join already matched sanitizer text.
- The test exposed the conditional itself as a source: isSourceCall matched
  any operator by its full text. Of the operators only an access
  (field/index) names its own value; a conditional or concatenation built
  from `$_GET` is not a source.

A real-Joern control covers the cast inside a conditional (sanitized) and
a sanitizer in the other branch (not); 20 query tests pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 12:39:23 +02:00
Marius CiepluchandClaude Opus 5.5 d57966a6b6 fix(taint): the field join links methods only through the object itself
profile.php:424 survived the hook-argument fix through the field join: in
another function of the same file, `$user = new stdClass; $user->ID =
$_POST['user_id']`, and the join keys a field by its TEXT across the whole
file, so the WP_User parameter's `$user->ID` became a field source.

The join exists to carry object state between methods ($this->attachments
written in the constructor, read in _delete_files). That reading of a
field's text holds only when the base IS the object ($this, this, self).
Any other base is a variable, matched within its own method only, where
the engine's dataflow already sees it. A real-Joern control covers both
(a local variable's field in another function stays quiet; $this state
across methods is traced) and fails on the old query.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 11:20:30 +02:00
Marius CiepluchandClaude Opus 5.5 53d20691c3 fix(mapping): record every callback of a register_rest_route call as a route
The mapper took the FIRST 'callback' in a 12-line window. One registration
commonly carries an array of endpoints; PMPro's /discount_code has a GET
and a POST handler, so the writer, pmpro_rest_api_set_discount_code, was an
ordinary `function` entry. Since da0ee57 such an entry's parameters are
not attacker input, so its $request stopped being a source and the REST
route that injects SQL through the discount-code save() had no source at
all.

The argument list of the call is now read to its balancing parenthesis,
every callback in it is a route, and each keeps the permission callback of
its own endpoint (the first after it before the next callback, else the
last before it). PMPro: 15 -> 18 REST routes.

This restores the source, not the finding: traced unfiltered, the engine
does not carry $discount_code->code = $code (written in the handler) into
$this->code inside save() once other methods have been called on the
object; the one path it reports detours through a prepared query's result.
Recorded as the next recall gap: a cross-file field join keyed by class.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 11:08:22 +02:00
Marius CiepluchandClaude Opus 5.5 ece4473c2b fix(scan): a hook callback's parameters are the registry's arguments, not the request's
The core-hook-argument class from the second PMPro adjudication:
pmpro_membership_history_profile_fields($user) on `edit_user_profile` was
a route entry, so its parameters were attacker input -- and $user is the
WP_User WordPress loaded; its ID became an SQL injection.

A callback registered in a string-keyed registry is handed its arguments by
the matching apply call, not by the request. The mapper now records the
registration call on the entry (`registered_by`), regions carry it, and the
dispatch fact says which registrations DO hand a callback request-borne
arguments (`request_arguments`; WordPress: add_shortcode, whose $atts are
post content). Every other registered callback keeps its call depth but
its parameters stop being sources; request data that reaches an apply
call's arguments is still followed by the hook half of the taint query.
REST routes are not registry entries and keep the parameter rule.

Controls: the fact field is validated against `register`; a hook callback
is not input, a shortcode and a plain route are.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 09:04:42 +02:00
Marius CiepluchandClaude Opus 5.5 7b8ff3cf44 fix(taint): a filter returns its value argument; a source is the access, not a call around it
The apply_filters class from the second PMPro adjudication (2 of 3
remaining false positives): `apply_filters('pmpro_random_code', $code,
$this)` returned whatever $this carried, and a report's `$type` reached SQL
only as an extra filter argument. Two causes, both fixed generally:

- The dispatch fact now says what an apply call RETURNS: `returns` names
  the calls whose result is the dispatched value and `value_arg` its
  position (WordPress: apply_filters/apply_filters_ref_array return
  argument 2; do_action returns nothing). A path through such a call must
  ENTER by that argument -- the first of the run of the call's arguments
  before it, because the engine hops between sibling arguments
  ($_GET["c"] -> md5("x") -> apply_filters). Loaded and validated as data;
  the query knows no framework.
- A call was a source when the pattern appeared anywhere in its text, so
  `apply_filters(..., $_GET["c"])` and `sanitize_text_field($_POST[..])`
  were sources themselves -- the "a token matches the thing it names"
  invariant. A call is now a source when the pattern names its callee,
  receiver or the access it is; the inner access stays a source.

Controls: dispatch facts load/validate; a real-Joern case with request
data as filter context (quiet) and as the filtered value (found). 1,453
host and 18 query tests pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 08:36:28 +02:00
Marius CiepluchandClaude Opus 5.5 f1e43d3646 measure(sqli): PMPro precision sample after the path rules -- 2/5, six of seven sampled false positives gone
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 03:29:57 +02:00
Marius CiepluchandClaude Opus 5.5 7bc7acbf52 fix(scan): keep the field join's parameter seeds when a region's own parameters are not input
da0ee57 turned parameter sources off for the mapper's catch-all `function`
entries, and with them the field join's seeds, which read the same flag:
MW WP Form's CVE-2023-6559 ($this->Data, filled by a constructor) was lost
(10 -> 3). The field join now has its own request field,
`fieldParameterSources`, set by the old entry rule; a region's own
parameters follow the new one. Both known CVEs are found again (MW WP Form
10 -> 4, WP Statistics 39 -> 18, 91/91 and 774/774 completed). The dropped
findings sampled against their call sites all take internal values:
ExistTable('useronline'), wp_statistics_visit('today'), remove() on the
plugin's own temp directory, _render('text', ...).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 02:40:00 +02:00
Marius CiepluchandClaude Opus 5.5 da0ee576f8 fix(scan): a catch-all function's parameters are not attacker input; field reads need the same field
Two classes the traced PMPro survivors named after 1f91c0b:

- The mapper records every named PHP function as an entry of kind
  `function` (3,507 of PMPro's 4,412), and every entry's parameters were
  sources, so internal helpers became injections: sendCancelAdminEmail's
  $user, a gateway hook callback's $user_id/$morder fired by do_action.
  Regions now carry the entry kind; parameters are input only for request
  entries (route, parser, privileged surface). Call depth is unchanged, so
  such a region is still followed into its callees. JavaScript and Python
  emit no `function` entries.
- The object rule is general: a field read continues a flow through the
  bare object only if the taint came through the SAME field, or was
  assigned to the variable or passed in as a parameter. It now also cuts
  $this->Gateway -> this -> $this->user_id (memberorder.php:705).

plausible() memoises the sink test per node and skips the object walk for
paths with no field access (the filter had cost one outlier 101 s -> 160 s).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 02:26:07 +02:00
Marius CiepluchandClaude Opus 5.5 1f91c0b089 fix(taint): drop paths through a query's result or through a whole object
Traced element by element (new opt-in `trace` on taint requests), every
false PMPro SQL injection in the adjudicated sample took one of two steps
no real injection needs, both over-approximations of Joern's dataflow:

A. Through a query: `$id = $wpdb->get_var("...'" . $token . "'")` is the
   injection; the engine carried $token on through the query's RESULT
   into later queries (getMemberOrderByID:705, profile.php:515).
B. Through the object: request data written to $order->ProfileStartDate,
   or passed as an argument beside $wpdb, tainted the whole object and a
   DIFFERENT field read from it came out tainted
   (memberorder.php:1122/1633, pmproemail.php:262).

A path is now dropped when it passes through a call to a sink of its own
family, or when a field read is reached through the bare object after the
taint entered it by writing another field or without being assigned to
it. A member of a value that IS tainted (the result of a read, a return,
an assignment) and a field read back after being written are kept. Applied
to the sink query, the field join and the hook join. A real-Joern control
covers both cuts and both keeps, and fails on the old query; all 17 query
tests pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 00:54:17 +02:00
Marius CiepluchandClaude Opus 5.5 bdb5a866d6 revert(taint): the caller-fed field rule removed a true positive and missed the false ones
Reverts 6ebbc89 and e0c260c (keeping finding_dump's completed-regions
record). Measured on PMPro, comparing only questions both runs answered
(157): 37 -> 31 findings, but the six removed included the adjudicated TRUE
positive discount-codes.php:154 (REST-reachable), and only one of the six
sampled object-field false positives went away; five are still reported.
It also cut completed questions 295 -> 158. MW WP Form and WP Statistics
were unchanged either way.

The diagnosis was wrong: the object-field false positives do not come from
setter parameters. The witnesses name one field (e.g.
$update_order->BillingFrequency) and the query interpolates another
($this->membership_id), which points at field-insensitive propagation
through the object instead. To be confirmed on a flow path before any fix.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 00:34:04 +02:00
Marius CiepluchandClaude Opus 5.5 e0c260ca5f perf(taint): ask the caller check only of a parameter that starts a flow into the field
Asked of every parameter up front, 6ebbc89's caller check ran a repository-
wide flow query per parameter of every method in a file and cut PMPro's
completed injection questions from 272 to 67 in the same budget -- so its
48 -> 26 findings could not be read as precision. The flow into the field
is asked once as before; only a parameter at the head of an unsanitized
flow is checked against its callers. finding_dump now records completed
regions so runs compare only where both answered.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 22:49:44 +02:00
Marius CiepluchandClaude Opus 5.5 6ebbc89d64 fix(taint): a field is attacker input only when a caller feeds it request data
The field join (task 5.11) seeded every parameter of every method in the
file, so every setter made its field a source. Adjudicated on PMPro, six of
eight false SQL injections in a 1-in-5 sample were object fields holding
database or internal values ($this->membership_id, $morder->user_id).

A parameter now seeds the join only where some call site passes it request
data without a sanitizer on the way (memoised per method and index); a
request source read in the file still seeds as before. A real-Joern control
covers both directions -- a setter only given a constant stays quiet, a
constructor handed $_GET (the CVE-2023-6559 shape) is still found -- and
fails on the old query with exactly the PMPro false positive.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 21:47:41 +02:00
Marius CiepluchandClaude Opus 5.5 faf4f643df measure(sqli): adjudicate a 1-in-5 sample of PMPro SQL injection findings
48 findings, every 5th reviewed against source: 2 true (both privileged),
8 false; 6 of the 8 share one cause, object fields treated as attacker
input when they are loaded from the database. benchmarks/push/finding_dump.py
writes every finding of chosen families for review.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 20:57:02 +02:00
Marius CiepluchandClaude Opus 5.5 c82ce7fa44 spec(pre-push-safety-net): correct the 13.3 budget note -- portioning bounds arbitration
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 19:45:20 +02:00
Marius CiepluchandClaude Opus 5.5 ea9006c1c5 spec(pre-push-safety-net): record how the 13.3 loop was broken and the budget finding
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 19:38:15 +02:00
Marius CiepluchandClaude Opus 5.5 6b85a60ff1 perf(push): find the Python-side scale walls all at once, and remove them
The loop the maintainer called out: four scale defects in the PHP transfer
replay, found one per two-hour Joern run, all Python bookkeeping between
engine calls. benchmarks/push/python_overhead.py ends that: the real
replay() on the real PMPro pins, with a JoernBackend whose graphs are stubs
and whose queries answer instantly with rows recorded from the real engine.
Engine time is zero, so everything measured is ours, in about a minute,
under the sampling profiler. Its first pass named two more walls:

- Region discovery re-parsed the ruleset for every region: 89,127 TOML
  parses, 275 s of a 339 s discovery, twice per push. The fact, obligation
  and taxonomy loaders now keep the parsed (immutable) result keyed by each
  file's name, size and mtime -- an edit in place or an added file reloads.
  Discovery 339 s -> 7 s; output digest identical on PMPro.
- affected_context decoded every span's and every line anchor's path for
  every context row -- a changed file has an anchor per unchanged line --
  78% of a stubbed push. Decoded once per call, anchors indexed by line and
  bisected, original order kept. A randomised control compares it against
  the row-by-row matching over 200 contexts (>100 attachments).

Result: the whole stubbed PMPro push, 27,542 answered requests and 2,688
outcomes, runs in 58.5 s of Python. What is left is the engine.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 19:36:32 +02:00
Marius CiepluchandClaude Opus 5.5 c629fea89a fix(cli): import the sampler in a form the module audit resolves
b4b9308 was committed with two failing tests: the module audit read
`from . import profiling` as an import of the package, called profiling
orphaned, and the committed audit manifest drifted. My check read tail's
exit status, not pytest's. Now a from-import; manifest regenerated (109
load-bearing, 0 orphaned); 1,444 pass with pytest's own exit checked.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 19:10:39 +02:00
Marius CiepluchandClaude Opus 5.5 b4b9308af8 perf(push): drop per-entry cache fsync; let long runs profile themselves
The cache fsynced every payload and manifest -- two per entry, ~18 ms, and a
push publishes one entry per question (~22,000 on a WordPress plugin). A
read already rejects any entry whose manifest receipt, payload size or
payload SHA-256 does not check, so a torn write is a miss, never a wrong
answer; the fsyncs bought nothing the verification does not.

The broader problem: three quadratic walls in this replay were found one
per two-hour run, because all were Python bookkeeping between engine calls
and the only record was the engine log. `OUSAST_SAMPLE_PROFILE=<path>`
starts an in-process sampler that rewrites the hottest stacks every 30 s,
so a run killed at its deadline still names where its time went. The host
cannot ptrace the container, hence in-process. The replay harness turns it
on per case.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 19:05:08 +02:00
Marius CiepluchandClaude Opus 5.5 e7e2941e37 fix(scan): attach change relationships in linear time
The third wall under the 13.3 replay, found by the streamed case log: the
evidence and context passes finished at 14:53, then nothing -- no engine
call, no log line -- for 82 minutes, until the case was killed 60 s past
its 7,200 s deadline. `affected_context` tested membership with
`relation not in relationships` on a list: 7,998,000 comparisons for 4,000
relationships (profiled, 83% of the time), and a push over a WordPress
plugin attaches tens of thousands. The deadline is checked once per
question, so a single late question could run for minutes, which is also
why the process overran its cancellation allowance.

Membership now goes through a set beside the ordered list, and each
distinct context path is resolved once instead of once per row. Measured:
4,000 relationships 7.4 s -> 0.44 s; 400,000 in 16 s. The control counts
comparisons rather than timing them, and fails on the old code with
7,998,000.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 18:33:26 +02:00
Marius CiepluchandClaude Opus 5.5 982cb14115 fix(replay): stream each case's output and survive a deadline overrun
The 13.3 replay on 7cf256b crashed on case 1: `pre-push` ran past its
7,200 s deadline plus the harness's 60 s grace, `subprocess.run` raised
TimeoutExpired, the harness died without running cases 2 and 3, and the
case's captured output died with it. Each case now streams stdout and
stderr to files with INFO logging on; an overrun kills the case's process
group, is recorded as `deadline_overrun` with its size -- the product
failing its cancellation contract -- and the next case still runs.
`--only` reruns selected cases. CLI log lines now carry a timestamp, so a
log can say where the time went.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 16:15:01 +02:00
Marius CiepluchandClaude Opus 5.5 d409c630cc spec(pre-push-safety-net): correct the 13.3 diagnosis -- the wall was the reuse cache
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 14:13:18 +02:00
Marius CiepluchandClaude Opus 5.5 7cf256b2a7 fix(push): stop rescanning the whole cache on every publication
The real cause of the 13.3 replay wall. Every cache publication called
`_make_room`, which listed and stat'ed every entry to total the cache size,
and a push publishes one answer per question -- 22,125 on a WordPress
plugin. Measured on the live replay cache: one scan 0.68 s at 12,238
entries, ~70 minutes spent scanning by then, ~2.3 hours to finish the
evidence pass. The engine was idle while the Python process sat at 83% CPU;
the 13/31/51-minute portions were the quadratic growing.

The scan is now remembered, keyed by the cache root's modification stamp:
our own publications and evictions update the tally in place, and any
change another process makes moves the stamp and forces a rescan, so the
size limit stays exact under concurrent pushes. Lock stripes are created up
front so their first use never invalidates the tally. Eviction order and
lease handling are unchanged.

Controls: 61 publications (one a same-key replacement) take exactly one
scan and the tally equals a fresh scan; a second writer on the same root
forces a rescan and the limit holds.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 14:12:47 +02:00
Marius CiepluchandClaude Opus 5.5 9b47e2c345 test(cpg): read /proc state without a check-then-read race
`test_hanging_process_group_killed_inside_shared_allowance` failed about one
run in thirty under load with FileNotFoundError: it checked that
/proc/<pid>/stat existed, the killed process was reaped, and the read then
failed. A reaped process is the outcome the test wants, so it now reads and
treats a missing entry as gone. The engine-stage test had the same race.
150 of 150 runs pass under engine load.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 14:03:48 +02:00
Marius CiepluchandClaude Opus 5.5 0bcaabfbed spec(pre-push-safety-net): record the 13.3 context-output wall and its fix
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 12:20:13 +02:00
Marius CiepluchandClaude Opus 5.5 3c195c9369 fix(scan): itemise change context only in the files a change touched
The PHP transfer replay (pre-push 13.3) spent its whole 7,200 s case in the
head side's evidence pass. A probe on one PMPro graph named the cause: not
compute -- context adds ~20% engine time -- but OUTPUT. Each evidence request
listed every method its scope reached, up to 1,845 rows; 200 requests into
the bundled Braintree SDK returned 52,831 rows, 9.4 MB, against 211 rows
without context, over 22,125 requests in a 3 GB container.

`affected_context` only ever attaches a location in a changed file, and
every other row was checked (real file under the root, usable extent) and
discarded. The scan now sends the changed files; the engine itemises those
and folds the rest into one `context_elsewhere` row carrying exactly what
that check reads. With no safe file list the engine itemises everything,
as before.

Equivalence: unit controls over valid, missing-file, no-extent and
outside-root cases, with and without a changed-file row; a real-Joern
control; and on PMPro itself, 2,440 relationships and zero gaps decided
identically from both forms, 52,831 rows down to 3,241 with the most
referenced file as the change. Probe: benchmarks/push/context_cost_probe.py.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 12:19:35 +02:00
Marius CiepluchandClaude Opus 5.5 cda679cb3b measure(contributor-scan): 7.9 measured portioning completes 281 of 406, up from 153
Same 500-region PHP run on frozen source: completion 281/406 against 153
(hard cap) and 131 (7.8), no portion killed, per-family findings never
lower, declared target still entailed. Records the 7.9 task, its limits,
and corrects the 7.8 reuse note, which was stated backwards.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 08:51:43 +02:00
Marius CiepluchandClaude Opus 5.5 d930f5bc69 fix(scan): report requests the reserve held back as unasked, not failed
The measured-cost run left 99 taint requests unasked to keep time for
arbitration and reported them as `query_failed` -- an engine that could not
answer -- because they were simply missing from the answer. They are now
`query_budget_reserved`, both as a degradation and as each question's outcome
reason, and are no longer double counted when nothing answered.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 08:51:12 +02:00
Marius CiepluchandClaude Opus 5.5 1b28465a61 feat(scan): size portions from measured per-answer cost and isolate the kill culprit
The 7.8 trace showed the completion loss was a sizer death spiral, not
expensive families: after two kills the budget fell to 3 sink visits and
never recovered, because portions finishing in 25-30 s sat under an assumed
70 s fixed start and read as "no information", so the budget could only
shrink.

- taint.sc streams each answer's own cost (`ms`); the backend carries it as a
  reserved `__timing__` list through shards, partitions and reuse.
- The sizer budgets predicted seconds from per-family seconds-per-visit rates
  learned from those costs, and measures the fixed start as the wall time the
  answers do not explain (median), instead of assuming it.
- A kill names its culprit: the first unanswered id in request order. The ids
  behind it never started and are re-queued at once; the culprit is asked
  alone after the rest. One that still fails alone after more than a
  portion's target is reported as `query_too_expensive`, apart from
  `query_failed`.
- Portions grow id by id from the queue head, and a single-portion scan now
  goes through the queue too, so a kill there re-asks its remainder.
- Fix: the reuse wrapper wrapped only `run_batch`, so since 7.8 every
  portioned query bypassed the graph-reuse cache and its source-root
  normalization. `run_batch_once` is wrapped too.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 07:55:33 +02:00
Marius CiepluchandClaude Opus 4.8 c1a689f4b3 measure(contributor-scan): 7.8 portioning matches the cap on findings, not on completion
Final end-to-end on the 500-region PHP subject: findings identical at 125 either way, completion
slightly lower under portioning (131 of 406 vs the hard-cap-plus-reserve 153), wall time higher
(2,982 vs 2,338 s). Told straight: portioning does not beat the cap on throughput here; it matches
it on what it finds, and what it buys is kill-safety -- a killed batch keeps its streamed answers
instead of losing them. Reaching the same 125 findings is the evidence that streaming loses nothing
the cap kept.

Completion did not rise because the requeue re-asks a killed portion remainder, and on this subject
those are the expensive output-encoding questions that keep hitting the ceiling in smaller pieces,
spending budget without completing. Telling a transient kill from a too-expensive family is the next
lever, not more portioning. Neither approach is release-ready anyway: 32-38% completion against a
95% gate on development data that qualifies nothing.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 00:54:13 +02:00
Marius CiepluchandClaude Opus 4.8 18c74bc571 spec(contributor-scan): record the honest end-to-end tuning of 7.8
Four interacting defects surfaced in sequence when the portioner was measured end to end, each
hidden by the one before: run_batch timing polluted by its own retry, a single-attempt path missing
on the partitioned graph every plugin uses, and killed remainders dropped by a fixed chunk list. All
fixed and unit-tested; the end-to-end completion number is re-measuring on a frozen tree.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 00:50:38 +02:00
Marius CiepluchandClaude Opus 4.8 6942446710 feat(scan): re-ask a killed portion remainder instead of dropping it
The portioner iterated a fixed pre-split list, so a killed portion left its unanswered ids behind:
on the plugin the list ran out at 1,263 s of a 3,600 s budget with those remainders lost, and
completion sat far below what the budget could buy. It is now a work queue. A portion that comes back
short pushes its unanswered ids to the front, to be re-asked at the budget the kill just halved;
`fit` splits an over-budget head, so heads shrink toward single requests, and a single request a
whole ceiling could not answer is dropped rather than looped on forever. A control asserts both the
re-ask and that termination.

This is the last piece of portioning-over-cap: streaming keeps what a kill finished, single-attempt
timing keeps the sizer honest, coverage tells a too-big portion from a slow one, and the queue keeps
working while budget and work remain.

1,425 tests pass, nineteen skipped.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 00:00:53 +02:00
Marius CiepluchandClaude Opus 4.8 03011b8fe6 fix(scan): give the PARTITIONED graph a single-attempt path too
run_batch_once was wired on the disposable and sharded paths but not the partitioned one -- and a
multi-language repository, which every WordPress plugin is, builds through the partitioned graph. So
the portioner fell back to run_batch there, whose internal split-retry folded into the time it
measured: a portion read 956 s and collapsed the sizer, exactly as before the single-attempt fix.
The fix had no effect on the one subject it was written for.

PartitionGraph now has run_batch_once, dispatching to each sub-graph the same way run_batch does --
the routing is factored into one _dispatch method that takes the per-graph method name. Sub-graphs
are built through the disposable path and already carry run_batch_once.

The lesson, again: a capability added at one construction site is not added at the others. There are
three that build a queryable graph -- disposable, sharded, partitioned -- and the one that mattered
for the measurement was the one missed.

1,424 tests pass, nineteen skipped.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 23:33:23 +02:00
Marius CiepluchandClaude Opus 4.8 0fef399869 fix(scan): the portioner asks single attempts and reads coverage, not wall time
End-to-end, the adaptive portioner regressed completion -- 64 of 406 against 7.7 hard-cap 153 --
while finding the same defects. The cause: the portioner timed each portion with run_batch, which
splits on failure internally, so one portion folded its own eight-way retry into its number and read
953 s. The sizer fitted a per-visit rate from that and collapsed its budget to 14 visits, spending
the back half of the deadline on tiny portions.

Two portioning layers were fighting. The portioner is now the sole retry layer: it asks
run_batch_once, a single invocation whose kill returns the answers it streamed, so its wall time is
the cost of one attempt. And it observes COVERAGE, not emptiness -- a portion that came back short
of the ids it asked halves the budget, because it was too big for the ceiling; only a whole answer
gives a clean per-visit sample to fit from.

run_batch_once is new on CpgResult, wired on the disposable and sharded paths; a backend without it
falls back to run_batch unchanged. Host controls cover the single-attempt path and the
halve-on-short-answer rule.

1,424 tests pass, nineteen skipped. The end-to-end plugin re-measure on a frozen tree is the open
item; the mechanism (a kill keeps what finished) and the sizer logic are unit-verified.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 23:00:14 +02:00
Marius CiepluchandClaude Opus 4.8 2e96ccb8ea test(push): freeze_source mints its own path so it can never clobber a live mount
The helper took a destination from the caller and rm -rf it. Re-running it into the same path while
a container had that path mounted deleted taint.sc out from under a live scan, and every request
after printed no such cpg query -- the live-mount hazard the helper exists to prevent, in a new
form. It now mints base/frozen-<commit>-<pid>-<n> itself and prints it, so reuse is impossible by
construction and the caller captures the printed path.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 22:22:30 +02:00
Marius CiepluchandClaude Fable 5.1 1aa7f175e9 fix(scan): a portion sizer that never extrapolates from a sample with no work in it
Measured on the plugin, first run of the adaptive sizer. A 600-visit portion answered in 184 s and
the budget was correctly cut to 262. The next portion, 199 visits, answered in 73 s -- three seconds
of work above the 70 s fixed start -- and 3 s over 199 visits read as 0.015 s each, so the budget
grew thirteen-fold to 3,404 visits. The ceiling killed that portion. The harvest kept the 13
answers it had streamed, which is the mechanism working; the sizing was not.

Two rules. A sample with less than ten seconds of work above the fixed start is not extrapolated
from, because it carries no information about cost per visit. And a budget may double per portion,
never more: the sizing may still be wrong, but by at most one factor of two, which a kill undoes.

1,423 tests pass, nineteen skipped; the sizer control asserts both rules.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 22:17:30 +02:00
Marius CiepluchandClaude Fable 5.1 e3edf11cf8 fix(cpg): count requests, not the census key, when reporting a split retry
The warning once read 22126 of 22125 requests answered in parts, because the census rides the
answer under its own key and was counted as a request. An instrument that miscounts by one casts
doubt on every number beside it.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 22:02:38 +02:00
Marius CiepluchandClaude Fable 5.1 a084dce780 test(push): a verified source freeze for long measurement runs
Every long run mounts src into a container, and mounting the working tree is a live mount: edits
made during a multi-hour run are picked up mid-run, and a measurement whose code changed underneath
it is not a measurement of anything. A five-hour PHP transfer replay was lost that way today.

The export has to be checked too. git archive piped into tar once produced a tree missing the whole
ontology -- every tracked file after groovy alphabetically -- and reported success, because tar
status in a pipeline was never read. Every scan on that tree died at its first fact load and looked
like a slow run for fourteen minutes.

So: archive to a file, extract, count against git, and refuse an export without the ontology.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 21:56:20 +02:00
Marius CiepluchandClaude Fable 5.1 cfd1493d69 spec(contributor-scan): record 7.8 verification and one bypass
The joern-gated check passes on the real engine: the taint query streams one line per request, and
the comment-sink and literal-destination controls hold under the streamed form. Two defects the work
exposed are recorded as fixed -- the unportioned evidence pass, and a harvest that read a timed-out
pipe before killing its writer -- and one bypass is recorded as a limit rather than fixed: the reuse
layer re-query of missing ids calls the backend batch directly around the sink-weighted portioning.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 21:45:09 +02:00
Marius CiepluchandClaude Fable 5.1 a4042fdb27 fix(scan): portion the evidence pass so a single kill is never the whole pass
The evidence pass bypasses the sink-weighted portioning by design: it is the pass that PRODUCES the
weights, and per request it is cheap, 17,700 answered in 44 s. It was still one all-or-nothing batch
protected only by the ceiling. On a plugin push transaction that batch is 22,125 requests, and under
the CPU and memory contention of concurrent test runs it exceeded the ceiling and cost the whole case.

It now runs in portions of 4,000 requests, roughly ten seconds of work behind one JVM start, with
answers merged and the census kept from the first portion that carried it. Streaming keeps what a
killed portion finished; portioning keeps that kill from being the entire pass.

Task 13.3 records the attempt that exposed this honestly: the case deadline was exhausted with no
artifact, and the failure was mine twice over -- the container mounts src live and the backend was
edited mid-run, and the evidence pass was unportioned. Neither is the algorithm. The rerun uses a
frozen export of the committed source with per-invocation logging.

1,423 tests pass, nineteen skipped; mypy clean.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 21:44:45 +02:00
Marius CiepluchandClaude Fable 5.1 a8b226a5c8 feat(cpg): portion the work instead of capping it -- a kill keeps what finished
A hard per-invocation ceiling is all-or-nothing. A batch killed at 301 s lost every answer it had
already computed, and the retry paid a fresh JVM start per part to ask them all again: measured, 116
requests killed at 300 s, then eight restarts carrying the same everywhere-sinks in every part.

Three coordinated changes make portioning, not the cap, the thing that governs cost.

  The taint query STREAMS. The census first, then one line per request as it is answered, flushed as
  it goes, inside the unchanged fence. The other queries do not stream and need not; the driver
  reads both forms, and a legacy single document parses exactly as before.

  The driver HARVESTS a kill. The process group is terminated first -- a first cut read the pipe
  before killing and a pipe whose writer is alive never reaches end-of-file; the suite own
  hanging-engine control sat on that read for twenty-eight minutes -- then the pipe is drained and
  every complete line becomes an answer, marked partial. The retry asks only the remainder.

  Portion sizing ADAPTS. After each portion the observed seconds per sink visit, less the fixed
  start, resizes the rest toward a 120 s target; a portion that did not answer halves the budget
  rather than being fitted, because its true cost is unknown and above the ceiling. A portion cut
  down to fit puts its remainder back at the front of the queue.

A census LINE carries nothing but the census. A first cut misread a legacy document that carried the
census beside its answers as a census line and dropped every answer it sat beside; a control caught
it. Task 7.8 in contributor-scan.

Verified: 1,423 host tests pass, nineteen skipped, including new controls for the assembler in both
forms and under a kill, for a retry that asks only the remainder, and for the sizer. Not yet
verified: the joern-gated streaming check and the plugin measurement, which wait for the transfer
replay to free the machine and are stated as open in the task.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 21:03:04 +02:00
Marius CiepluchandClaude Opus 5 1c9d7f501f docs(status): record that PHP reaches its declared target
Detection is not transfer, and the status says so: the comparison run is what decides M3. But the
target that task 13.2 could not reach for months is now found, and the reason is worth keeping where
the next reader will see it -- nothing about detection changed, the population became affordable.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 19:38:23 +02:00
Marius CiepluchandClaude Opus 5 d4ce173592 measure(m3): the declared PHP target is detected, and the replay budget is sized
classes/class.memberorder.php:936:getMemberOrderByCode, injection. A value concatenated into SQL with
no prepare, which is the declared M3 target for PHP, found for the first time.

Nothing about detection changed. The population became affordable: tier-zero pruning scoped to its
partition, batches budgeted by the sink work they imply, a per-query timeout no longer cancelling the
scan, and a reserve kept for arbitration. Before those this population completed zero questions at
every budget tried.

At 500 regions with a 3,600 s deadline: 271 of 406 questions completed, 213 findings, 2,950 s, and
eight findings in the target file including the target itself.

The earlier bounded attempt failed for a reason no budget increase could fix -- 25 regions of 4,461
is 0.6 percent of the population and the target was never asked about -- so this sizes the replay
before spending hours on it. One scan at this budget is 2,950 s, a push transaction is two scans plus
discovery and preparation, so the three declared comparisons are roughly five and a half hours.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 19:34:49 +02:00
Marius CiepluchandClaude Opus 5 f09d788568 test(census): record which regions a budget actually reached
A census that reports what it found cannot say whether a defect it missed was examined and cleared or
never looked at, and that is the difference between a negative and a silence. It is also how a
bounded run budget gets sized without guessing: task 13.3 needs to know at what population the
declared PHP target enters scope, and asking the record is cheaper than binary-searching with
forty-minute scans.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 18:52:51 +02:00
Marius CiepluchandClaude Opus 5 8874145874 measure(contributor-scan): confirm 7.7 costs the small population nothing
PMPro at 25 regions, which completed before this work and still does: 27 questions completed and 19
findings before and after, injection 1 and output encoding 18 in both. The batching changes what a
large population can afford and nothing about what a small one answers.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 13:56:00 +02:00
Marius CiepluchandClaude Opus 5 7ad3decf9a measure(contributor-scan): 7.7 makes a large PHP population affordable, 0 findings to 125
On the 500-region population where every earlier run produced nothing: 153 of 406 questions completed
and 125 findings, inside the deadline. 380 requests carrying 5,807 sink visits split into 11 batches,
each answering in 80 to 197 s, none near the 300 s ceiling. Every family that can speak on this
subject now does, injection included, which is the one holding a true positive here.

Four defects, each hidden by the one before it, and three of them one rule.

  Batches sized by request count, when cost is per matched sink.
  Packing heaviest-first, which starved injection of all 114 of its questions.
  Any per-query timeout cancelling the whole scan, abandoning batches with 33 minutes left.
  No budget kept for arbitration, which removing that cancellation exposed: rows are not findings.

The reserve had to be checked per BATCH. Checked between query kinds it changed nothing at all,
because one kind is a single loop iteration that runs every one of its batches inside it.

What it does not claim: 253 of 406 questions still went unanswered and one batch still records a
timeout; the raw finding count is dominated by output encoding, the family already measured to be
imprecise on PHP; and the reserve share of a fifth is a first value, not a tuned one.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 13:55:44 +02:00
Marius CiepluchandClaude Opus 5 d0026f4ea5 fix(scan): check the arbitration reserve per batch, not per query kind
The reserve was computed once and checked between kinds, and one kind is a single iteration that runs
every one of its batches inside it. So taint spent the whole budget and the reserve only stopped the
cheap kinds queued behind it: 0 of 406 questions completed, 2,435 s of a 2,700 s deadline in queries,
identical to having no reserve at all.

The granularity of a budget check has to match the granularity of the work it guards. It is now
checked before each batch, and the batches left unasked are logged rather than silently dropped.

1,420 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 13:12:21 +02:00
Marius CiepluchandClaude Opus 5 8cf1b383d4 fix(scan): keep budget back for arbitration, because rows are not findings
Querying fills rows. Arbitration turns them into findings and the outcomes are assembled from that,
and neither costs the engine anything -- so a scan that spends its whole budget on queries reports
nothing at all, however many of them answered.

Found by accident, which is the honest way to tell it. A per-query timeout used to cancel the whole
scan. Removing that cancellation was right on its own terms and made the measured result worse: the
query phase then ran the full 2,700 s deadline to exhaustion and completed 0 of 406 questions, where
the cascade had stopped it at 958 s and left time to arbitrate 78 findings. The cascade had been an
accidental budget guard. This is the deliberate one.

The query loop now stops issuing batches with a reserve in hand: a fifth of what was left when
querying started, a sixty-second floor, and never more than half. The cap is not decoration -- held
back without it the floor swallowed a whole small budget and the query loop never ran at all, which a
control caught. A scan with a fifth of a second to spend must still spend it.

What the loop skipped is recorded as query_budget_reserved with the kind and the request count, not
left to look like a query that failed.

1,420 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 12:21:12 +02:00
Marius CiepluchandClaude Opus 5 18d0319436 fix(cpg): a query that exceeds its own ceiling no longer cancels the scan
Two different timeouts reached the same handler and got the same consequence. If the SCAN deadline is
gone, cancellation is right: stop everything promptly, which is what the push two-second gate
measures. If a single query merely exceeded its own per-query ceiling, the scan may have most of its
budget left and every other batch is independent and cheaper -- and cancelling took them all.

Measured on Paid Memberships Pro: one batch of 116 requests hit a 300 s ceiling 734 s into a 2,700 s
scan, and the four batches behind it returned in 0.0 s unanswered with thirty-three minutes to
spare. The handler already distinguished the two cases in the reason it recorded, deadline_exhausted
against process_timeout, and then set the cancel deadline in both.

Fourth instance this week of one rule, now written in the design: a failure is owned by the narrowest
scope that caused it.

1,420 tests pass, eighteen skipped, including the cancellation gate.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 11:26:54 +02:00
Marius CiepluchandClaude Opus 5 6365c49849 feat(scan): size a taint batch by the sink work it implies, in the ranker order
A taint batch costs what its matched sink NODES cost, because reachableByFlows is asked per sink.
Measured at a fixed 100 requests on one plugin graph: SQL injection 147 s with six declared sinks,
path 194 s with sixteen, output encoding 704 s with seven. The declared count does not predict cost
and the request count does not predict cost.

So batches are budgeted in sink visits rather than requests, using the count the evidence pass
already produced: a family whose sinks are rare rides in one batch, and a family whose sinks are
everywhere is split until it fits. A request over the budget on its own gets its own batch rather
than being dropped, because the scan job is to report that it is expensive, not to decide it is
unaskable.

Measured on Paid Memberships Pro at 500 regions, where every earlier run produced nothing at all:
380 arbitration requests carrying 5,807 sink visits split into 12 batches, every one answered,
heaviest 103 s and none near the 300 s ceiling. 144 findings where there were none.

The order is the ranker and must stay the ranker. A first attempt packed heaviest-first, which packs
better and analyses worse: the expensive output-encoding batches ran first, spent the deadline, and
left injection with 114 questions asked and none answered -- the one family holding a true positive
on that subject. A deadline always cuts a tail, and the tail it should cut is the one the ranker put
last.

1,420 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 10:52:38 +02:00
Marius CiepluchandClaude Opus 5 9910dd18b9 measure(php): the query cost is the matched sink count, and the blocker is identified
Open since September and re-measured nine times. The cost of a taint batch is the number of CALL
NODES a family sink names match in the repository. Not the request count, not the declared sink
count, not any request parameter. reachableByFlows is asked per matched sink, and PHP
soutput-encoding sinks are echo and print, which a WordPress plugin writes thousands of times.

Three controlled sweeps on one graph, built once.

  Batch size exonerated: 400 requests answer in 170 s, and cost per request FALLS with size because
  roughly 70 s is fixed JVM start and graph load.

  Request parameters exonerated: at 200 requests the full production shape, call depth 3 with
  parameter sources, costs 221 s against a 150 s baseline.

  The family is the variable: at the same 100 requests, SQL injection 147 s with six declared sinks,
  path 194 s with sixteen, output encoding 704 s with seven.

And the driver named its own expensive step once it was asked to: 2,435 evidence requests in 45.5 s,
then 315 arbitration requests unanswered at 1,200 s, then a split abandoned for want of time.

It survived since September because every reproduction outside the driver carried the SQL sinks --
the family the declared target belongs to, and the one family that is cheap on this subject. Nine
hypotheses were eliminated by re-measuring the cheap case.

Splitting never rescued it because a failed batch is split by REQUEST COUNT and each part carries the
same everywhere-sinks. contributor-scan 7.7 carries the fix: budget a batch by the sink nodes it
implies, using the count the evidence pass already computes, and report a family whose sinks are
everywhere as expensive rather than as failed.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 09:53:52 +02:00
Marius CiepluchandClaude Opus 5 4a5bf0aaf9 test(cpg): log what each engine invocation costs and whether it answered
Nine hypotheses about the PHP query cost have now been eliminated by rebuilding the request outside
the driver: batch size, call depth, parameter sources, hook table, argument length, sharding,
dataflow cost, the malformed-CPG bug and the engine version. Every one of them answered in under
four minutes while the driver spent whatever ceiling it was given.

The reason the guessing continued is that a scan issues several invocations and the record could not
say which of them burned the budget. It now logs the query kind, the request count, the elapsed time
and whether it answered, for the whole batch and for each part of a split retry, including the part
where a split is abandoned for want of time.

This is task 16 applied to the backend rather than to a harness: an instrument that cannot name its
own expensive step turns a measurement into a guess, and this one has been a guess since September.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 09:05:47 +02:00
Marius CiepluchandClaude Opus 5 617fab157c measure(php): batch size is not the cost, and the curve is sublinear
Four census runs on this subject spent whatever query ceiling they were given: 363 s of 300, 1,255 s
of 1,200, 1,874 s of 1,800, at question counts of 2,526 and 406. Cost that tracks the budget rather
than the work is not per-question cost, and tier-zero pruning cutting the count six-fold changed
nothing. A census runs one batch, so no census could answer whether the batch itself scales.

One graph, built once, five batch sizes, every one answered:

     25 requests   71.4 s   2.86 s each
     50 requests   76.0 s   1.52 s each
    100 requests  155.4 s   1.55 s each
    200 requests  166.7 s   0.83 s each
    400 requests  169.7 s   0.42 s each

Sublinear, and roughly 70 s of it is fixed JVM start and graph load. Four hundred dataflow requests
answer in 170 seconds on the graph production cannot finish 406 on.

So the difference is in the request PARAMETERS. The probe sent callDepth=0 and no parameter sources;
production sends callDepth=3 with parameter sources on every entry point, plus a hook table, a
dispatch list and context evidence. Task 13.2 recorded in September that every reproduction outside
the push path answered in under a minute and the difference was not known. Size is now removed as a
candidate and a four-variant probe is running to isolate the rest.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 08:46:23 +02:00
Marius CiepluchandClaude Opus 5 3b6e642433 fix(scan): prune a tier-zero question unless ITS partition is the incomplete one
The mechanism already existed. A question whose evidence vector says no sink of its family is in
reach cannot produce a flow, and tier-zero pruning defers it. That was switched off for the whole
scan whenever any of twelve degradation reasons appeared, and a multi-language repository always
carries two of them, so the measured plugin arbitrated every question it could ask. Third instance
this week of one blast radius, and the invariant written into the design on Monday named it.

A prune blocker that names a partition now withholds pruning from that partition questions and from
nobody else. cross_partition_semantics_unresolved leaves the blocker set entirely, for the reason it
left the integrity gaps: a PHP family sink is PHP, so a second language cannot be why its vector saw
none.

Measured on Paid Memberships Pro. At 25 regions arbitration falls from 127 questions to 34 with 93
recorded deferred as tier_zero, and 19 findings before and after, identical family by family. At 500
it falls from 2,526 to 406 with 2,120 deferred.

It is not sufficient, and that is the useful half. The 500-region run still fails in the same way at
the same cost, because 406 questions at roughly one and a half seconds each still exceed the
300-second per-batch ceiling. This lever moved the volume; the ceiling is a separate limit.

The census now records what a scan skipped and why, per family, so a cheaper scan and a narrower one
stop looking identical: a family whose every question was pruned reports no sink of this family in
reach rather than region offered, no question asked.

1,420 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 08:06:15 +02:00
Marius CiepluchandClaude Opus 5 b0500ecfa0 measure(contributor-scan): first ontology coverage run, and it found a family we do not have
NodeGoat models 18 of 175 distinct call names, covering 12 percent of 1,185 calls. That share is not
a score: most of what a program calls is not security-relevant, and a high number would mean the
tables were too loose. It is a baseline to move deliberately.

The top product candidate is res.render, 23 calls with five in a method that reads untrusted input.
Task 14.3 reasoned its way to that same gap by hand and declined to model it as a flow sink; the
instrument found it in one run, which is the validation that matters. It also surfaced Math.random in
the user data-access module, and weak randomness is a weakness class this ontology has no family for
at all -- so no family census could have found it, because the family list is itself the blind spot.
That is the kind of gap only a coverage instrument finds.

It had to be taught what is not an API. The first run ranked _tmp_3.push and __ecma.Array.factory
first, which is how the frontend lowers a destructuring and an array literal, and a Cypress suite and
a Gruntfile held four of the next five. Both are separated now, because noise at the top of a
worklist is how a worklist stops being read.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 17:19:59 +02:00
Marius CiepluchandClaude Opus 5 0beae46ba9 feat(contributor-scan): ask a repository what it calls that the ontology does not model
Five of seven families established nothing in the family census, and every one failed for an ontology
reason with the flow engine ready and nothing to aim at. Each was found by investigating one family
at a time over days. vocabulary.sc asks it once: census the calls, subtract everything any fact table
names, rank the remainder by how often it is called and whether it sits in a method that reads
untrusted input. It needs no dataflow, because the structural pass already walks every call.

The harness reads its token set from the SPECS rather than from the TOML, so it measures what the
analyser actually consults: a fact the loader drops is not part of the ontology however neatly it is
written. Ontology sizes today are 152 tokens for JavaScript, 167 for Python, 84 for PHP.

Both carry the constraint the maintainer set: the output is a candidate list, never an ontology. A
name becomes a fact only when someone resolves it to a library API, states the weakness class and
records provenance, because adding a name because a subject called it is how a general-purpose tool
becomes a fixture of its own benchmark.

Also records what the bounded PHP transfer attempt found. At 25 regions the first case now runs to
completion and writes a full artifact in 1,706 s, where at 100 it had exhausted a 2,400 s deadline
with nothing. But 25 of 4,461 regions is 0.6 percent of the population and the declared target was
not among them, so the question was never asked. Task 13.3 therefore depends on 7.3, skipping
arbitration where the evidence says there is nothing to find: raising the budget is already measured
to fail, and making most of the budget unnecessary has not been tried.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 17:12:24 +02:00
Marius CiepluchandClaude Opus 5 032efc926c spec(pre-push-safety-net): a harness must report the cause of its own failure
Named by a week in which three separate causes looked identical as one count of unresolved. The
census reported 19 findings and 0 completed questions side by side without complaint, a stale image
measured the engine a bug had been diagnosed against, and a partition fix looked applied while an
unscoped copy of the same gap kept demoting everything.

Every harness record should carry the counts that disambiguate its failure modes, the degradations
whole rather than by reason, and the engine and ruleset identities actually used; and each harness
should assert its own consistency, because findings above zero with completion at zero is a
contradiction an instrument can notice faster than a person.

This was missed from the previous commit by an escaping error in the edit, not by intent.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 16:29:39 +02:00
Marius CiepluchandClaude Opus 5 e245040855 spec: name the fact tables an ontology, and commit to a general-purpose hook
The source, sink, sanitizer, operation, discharger and layout rows are not a pattern list. They are an
ontology of program elements and their security relations, and nearly all of this tool accuracy lives
there: the family census found five of seven families establishing nothing, every one for an ontology
reason with the flow engine ready and nothing to aim at.

The hook is general-purpose, not application-specific. The ontology describes libraries, frameworks
and language constructs; it does not describe repositories. An entry names an API and is justified by
what that API does, carries provenance and a weakness class, and may not encode a path or identifier
peculiar to one project -- a plugin bundled library tree was measured and declined on exactly that
ground. A measured gap is a candidate, never an entry: accretion by subject is how a general tool
becomes a fixture of its benchmark.

contributor-scan owns the ontology, so its design carries that statement and a new group 7 makes
growth measurable rather than anecdotal: report what a repository calls that nothing models, key
sanitizers to the weakness they prevent, skip arbitration where the evidence says there is nothing to
find, share one matcher across the queries, require a degradation to declare its scope, and commit
golden findings so a silent change cannot pass.

pre-push-safety-net carries the general-purpose test for its own reviews, and a task for harnesses
that report the cause of their own failure -- named by a week in which three separate causes looked
identical as one count.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 16:29:23 +02:00
Marius CiepluchandClaude Opus 5 3a18d025f4 spec(m4): the independent population must hold a multi-language repository
Task 13.4 found that a second language in a repository cost every question its completion, and so
cost every finding its admission, on the declared v0.1 target: a WordPress plugin is PHP plus its
admin JavaScript. Neither subject that passes today could reveal it, because both hold exactly one
language. A population of single-language repositories would qualify a capability that fails on the
product own target.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 16:22:25 +02:00
Marius CiepluchandClaude Opus 5 a487d1dfb5 fix(push): a gap from another language partition no longer blocks admission
_blocking_degradations already exempted declared exclusions. It now also ignores a gap another
language partition raised, because a JavaScript file missing from the JavaScript graph cannot make a
PHP answer wrong. Treating it as though it could left a WordPress plugin -- PHP plus its admin
JavaScript, which is every plugin -- unable to admit a finding it had already established.

A gap that names no partition still reaches everything, and so does every gap when the claim own
language is unknown, because unknown scope is not evidence of independence. The control asserts all
four directions.

With this and the scan fixes, task 13.4 is verified on the subject that motivated it: PMPro completes
120 of 127 questions where it completed none, its 19 findings unchanged, and the seven still
unresolved are the JavaScript partition own questions, which is the correct answer.

1,420 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 13:50:38 +02:00
Marius CiepluchandClaude Opus 5 d105d2f84a fix(scan): carry the partition in both census passes, and PHP completes 120 of 127
The scoping fix looked applied and was not. The same census gap is raised in TWO passes, arbitration
and evidence, and only the first carried the partition it came from, so an unscoped copy kept
demoting every question in the repository. One key in one dict.

What found it was the record, not the code: the census now prints each outcome reason rather than
only its status, and three separate causes had been looking identical as a bare count of unresolved.

Measured on Paid Memberships Pro at 25 regions: questions completed goes from 0 to 120 of 127, with
its 19 findings unchanged. The seven still unresolved are the JavaScript partition own questions,
which is correct, because the four missing files are its files and its answers are the ones that
could be wrong.

Still open, and recorded rather than implied: the push policy _blocking_degradations exempts declared
exclusions but does not yet respect a partition, and a post-hoc diagnostic copy carries no language,
so a JavaScript gap would still block admission of a PHP finding.

1,419 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 13:45:00 +02:00
Marius CiepluchandClaude Opus 5 29cef32bac fix(scan): scope a partition census gap to its own partition, and write the rule down
A missing JavaScript file cannot make a PHP answer wrong. A census gap now carries the partition it
was raised for, and reaches only that partition questions. On the measured plugin four JavaScript
files -- a webpack build output, a minified select2, a build config and one source file -- had taken
the admissibility of 4,425 PHP regions with them.

Two controls exercise the real boundaries rather than injecting a reason, and both were confirmed to
FAIL with the old classification restored, so they guard what they claim to.

And the rule is now in the design, because this was the fifth instance found and fixed one at a time.
Two invariants, eleven instances between them:

  A degradation is owned by the narrowest scope that caused it. Vendor exclusion as a global veto,
  line ambiguity inherited by every question, an empty scope blocking a sibling comparison, a language
  boundary demoting every answer, a census gap demoting every answer. The fourth and fifth cost the
  declared v0.1 target its entire admissibility, and neither passing subject could reveal it because
  both hold one language.

  A token matches the thing it names, not text that contains it. gets inside fgets, resolve inside
  resolveUrl, user inside users, a sink word inside a comment, a dotted setting on its last segment,
  a flow query reading the receiver.

Each carries a review test: a new degradation must state the scope it reaches and justify demoting
that far, and a text match must be bounded AND confined to the part of the node that names the callee.

1,419 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 13:21:27 +02:00
Marius CiepluchandClaude Opus 5 c93669c34f fix(scan): a language boundary is a declared limit, not a graph-integrity failure
Membership of GRAPH_INTEGRITY_GAPS should turn on whether a gap can make a reported answer WRONG,
not on whether it cost coverage. An unparsed file may hold the sanitizer that would have cleared a
flow this scan reports, so it belongs there. A flow this tool cannot follow from one language into
another can hide a finding and never invent one, because a PHP sink is sanitised in PHP.

cross_partition_semantics_unresolved therefore moves to DECLARED_EXCLUSION_GAPS. It touches exactly
two places, both of which already honour the distinction: the outcome demotion in scan.py, and
_blocking_degradations in the push policy, whose docstring already says a declared exclusion is not
one of them. The reason stays published in the scope unresolved boundaries either way, so the limit
is still reported; the class decides only whether every answer in the repository is demoted.

This corrects the task note written earlier today, which said not to reclassify on the grounds that
it would claim cross-language flows are out of scope rather than unresolved. That was a misreading
of the class.

PMPro still completes none of its questions, and the measurement now names why: the JavaScript
partition expected 36 files and holds 32, so partition_file_census_incomplete fires and is still an
integrity gap. The four absent files are js/blocks/blocks.build.js, js/jsapi.js, js/select2.min.js
and webpack.config.js -- a build output, a source file, a minified third-party library and a build
config. A bundled select2 costs 4,425 PHP regions their admissibility. Same over-broad blast
radius, third instance of one principle in this thread, and the fix direction is recorded.

The census harness now records each degradation whole rather than only its reason, because a
partition census that reports files missing carries WHICH files and without them the gap is a label
rather than a lead.

1,417 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 12:04:28 +02:00
Marius CiepluchandClaude Opus 5 73e9c1fed6 measure(m3): two languages in a repository defeat completion, and so defeat admission
No PHP question is ever marked completed, and PHP was never the variable. partitions.py raises
cross_partition_semantics_unresolved whenever more than one language partition exists, that reason
is a member of GRAPH_INTEGRITY_GAPS, and a graph-integrity gap demotes every completed outcome in
the scan. Admission requires a completed outcome.

The evidence is one column wide. NodeGoat is JavaScript alone and completes 299 of 299. VAmPI is
Python alone and completes 35 of 35. PMPro is PHP with 36 regions of bundled JavaScript and
completes 0 of 127 and 0 of 520 while establishing 19 and 43 findings. NodeGoat own degradation,
vendor_semantics_unresolved, is a DECLARED exclusion rather than an integrity gap, which is exactly
why it does not demote.

This lands on the declared v0.1 target. A WordPress plugin is PHP plus its admin JavaScript, so
every one of them is multi-language, so on the target this tool was built for it can detect defects
and never admit one. Neither subject that currently passes could reveal it, which is why a census
across a third subject was worth running.

Recording the boundary is right: a flow from PHP into JavaScript is semantics the partitioned graph
cannot follow. Demoting every question is not, including one wholly inside a single PHP file. Task
12.3 already fixed this bug class one level down by making a gap question-owned so it could not
block another question comparison; the same principle applies to a partition boundary. Reclassifying
the reason as a declared exclusion is explicitly the wrong fix, because that would claim
cross-language flows are out of scope when they are unresolved.

Task 13.4 rewritten with the diagnosis, and the milestone amendment now ranks it with M2 rather
than behind it: a narrow JavaScript-only release can proceed without it, the WordPress target
cannot proceed at all.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 10:57:19 +02:00
Marius CiepluchandClaude Opus 5 6bd6e3531e test(m3): let the transfer replay take a region budget
The replay hardcoded 500 regions, which on a 673-file PHP plugin asks 2,526 questions and fits no
ceiling this project sets. The budget is now an argument, defaulting to 500 so every earlier
measurement keeps its shape.

It bounds the RUN, never the repository: the artifact census still reports what was left
unexamined, and a bounded population establishes transfer rather than coverage. Task 13.3.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 10:55:22 +02:00
Marius CiepluchandClaude Opus 5 5b7288aa58 spec(pre-push-safety-net): amend the sequence and declare the shippable scope
Four decisions, recorded after the family detection census and the PHP correction. They change
sequence and declared scope; they change no acceptance threshold and no owner boundary.

Close M3 at a scale whose questions complete, before anything else. The not-established PHP record
came from two instrument defects rather than from the analyser, and bounded to 25 and 100 regions
PMPro completes with no timeout and establishes a true-positive SQL injection. Tasks 13.3 and 13.4
carry the replay and the question-completion gap, the second because admission reads outcomes and
M4 measures completion at 95 percent.

Reserve the untouched M4 population now, in parallel. M4 already permits preparation before the
frozen candidate exists and it is the longest lead-time item on the path. Task 15.1.

M2 is the only engineering item left and cannot be scoped away: a changed-code transaction spends
its whole budget in the build, and meeting the latency gate by analysing less is explicitly not
the same result.

Declare the v1 advisory scope as the families whose claims can be ADMITTED, which requires the
entailed rung: injection, untrusted destination and configuration. Access control is excluded
because on an application that splits route from data access the obligation is carried and a
carried obligation corroborates by design, so the family would ship as silence. Task 15.2.

The risk the amendment does not remove is stated with it: every number so far comes from subjects
used to build the analyser, one week of examining two of them surfaced seven false positives, and
the first independent evaluation should be expected to fail.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-21 10:41:25 +02:00
Marius CiepluchandClaude Opus 5 1f51c82b51 fix(census): stop hiding findings behind a question-outcome count, and correct the PHP record
PHP is not silent. Paid Memberships Pro establishes findings, including a true positive, and
this project's own instrument is why it looked otherwise.

The census harness ordered its verdict on completion before findings, so a family with eighteen
findings and no completed question reported "asked but no question completed". Findings are now
checked first: a question-outcome count is bookkeeping and cannot hide the thing being censused.
NodeGoat and VAmPI were never affected, because they complete every question and their verdicts
reached the findings check. The scan summary now also carries the total findings and the outcome
status counts, so the two numbers can be seen disagreeing instead of one silently winning.

The second mistake was mine rather than the harness's: only the 500-region runs had been looked
at, which are the ones that time out.

What PHP actually detects on PMPro. At 25 regions, 19 findings in seven minutes with no timeout;
at 100 regions, 43 findings in twelve minutes with no timeout. Among them adminpages/
discountcodes.php:749, where sanitize_text_field($_REQUEST["s"]) reaches
$wpdb->get_results($sqlQuery, OBJECT) through `$sqlQuery .= "WHERE code LIKE '%$s%' "` with no
prepare. That is the class of CVE-2023-23488, which php.toml's own comment names as the reason
those WordPress sanitizers are kept out of the sanitizer list. The prediction and the measurement
agree.

The precision cost beside it is measured too: output encoding reports eighteen findings at 25
regions and the three sampled are all properly escaped, two with esc_attr and one coerced with
intval. Those escapers are absent from a sanitizer list that is flat across families, because
none of them stops SQL injection. Keeping the injection finding costs this family its precision.

The scale limit is roughly one to two seconds per question. 25 and 100 regions complete cleanly;
500 regions asks 2,526 questions, needs more than the 1,800-second ceiling and is killed.

Four things are measured NOT to be the problem: the malformed-CPG bug (fixed in 4.0.625, verified
on the real trigger file), the dataflow overlay (42 s on the full 637-file graph), the engine
version (4.0.625 and 4.0.631 indistinguishable), and the bundled Stripe, Braintree and Twocheckout
SDKs under includes/lib that the vendored list does not cover -- that was my hypothesis for the
cost and the overlay measures 42 s with them and 37 s without, so it is refuted.

What remains is bookkeeping, not detection: questions_completed is zero in every PHP run including
the ones that find things, with cross_partition_semantics_unresolved and
partition_file_census_incomplete present throughout. The admission policy reads those outcomes.

1,417 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-20 19:22:04 +02:00
Marius CiepluchandClaude Opus 5 99abe12062 measure(php): quantify what remains once the malformed-CPG bug is excluded
The graph bug is fixed and was never what these runs were hitting. joernio/joern#6281, this
project's own report, is a PHP `global` inside a closure giving a node two AST parents, which made
the dataflow overlay throw and cost the whole graph. Filed against 4.0.623, fixed by #6270 in
v4.0.625, which the image has pinned since before any measurement recorded under task 13.2.
Verified today on the real trigger rather than a transcription of the repro, which does not
reproduce: PMPro's includes/fields.php builds and applies the default overlays cleanly on 4.0.625
and on 4.0.631, files=2 methods=158, no ReachingDefPass assertion.

What remains is runtime. A taint batch of 2,526 questions over the 673-file checkout does not
complete in thirty minutes. Raising the query ceiling six-fold, from 300 to 1,800 seconds, changed
nothing but how long the failure took: query time went from 363 s to 1,874 s, and zero questions
completed in both. The eight-way split retry did not rescue it either, finishing its parts in
about three minutes against a deadline with 3,300 seconds still on it, which is worth its own
look.

The engine pin stays at 4.0.625. An image on 4.0.631, today's release, was built and measured and
is indistinguishable on this subject: 627.7 s against 619.1 s, 363.4 s against 367.3 s of query
time, zero completed in both. There is no demonstrated benefit to weigh against re-verifying every
JavaScript and Python measurement on a new engine. Should that change, the bump is three values
rather than a port: the upstream AstGenRunner.scala is byte-identical in 4.0.631 and its patch
anchor still appears exactly once, so only the jar path, the jar checksum and the source URL tag
move. Both are recorded.

Also closes task 14.5. Neither prototype pollution nor deserialization has a subject that can
answer, each for its own reason. Deserialization's mechanism exists on exactly one available
subject and that subject cannot be queried at all. Prototype pollution has no available subject
containing the mechanism, so its ability to answer is established on a control instead.

1,417 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-20 18:13:25 +02:00
Marius CiepluchandClaude Opus 5 2c5fba55ef fix(semantic-facts): reach the deserialization form WordPress actually writes
Across four WordPress plugins and 1,164 PHP files there is exactly one deserialization site, and
it is spelled maybe_unserialize. The bare token never reached it, and correctly so: the sink
matcher is word-bounded and the character before unserialize inside maybe_unserialize is a word
character. A token a repository never spells is a family that can never answer, which is what the
2026-09-19 census recorded for this one. maybe_unserialize is a real entry point rather than a
safe wrapper: it calls unserialize whenever the string looks serialized.

Also adds the prototype control. That family had never established anything on any subject, and
its only census report was the word `set` inside a comment. No repository available to this
project contains the mechanism: the only first-party matches on the Node subject are
app.set("view engine", "html") and its sibling, Express settings whose arguments are literals. So
the control demonstrates what the family can do, on a merge of the request body into an object,
and asserts the Express setting is not reported alongside it.

Corrects the PHP record under task 13.2. joernio/joern#6281, this project's own report, is a PHP
`global` inside a closure producing a node with two AST parents, which made the dataflow overlay
throw and cost the whole graph. It was filed against 4.0.623 and fixed by #6270 in v4.0.625,
which the image has pinned since before any measurement recorded here. Verified today on the real
trigger rather than on a transcription of the repro: PMPro's includes/fields.php, the file the
original bisection found, builds and applies the default overlays cleanly on 4.0.625 and on
4.0.631 alike. What remains under that task is a runtime failure and must not be re-diagnosed as
the graph bug.

1,417 tests pass, eighteen skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-20 18:09:57 +02:00
Marius CiepluchandClaude Opus 5 3bd844055c fix(config): report every permissive setting in a file, not the first one
NodeGoat's server.js configures a permissive session at line 78 and disables template
auto-escaping at line 135, both inside the same function. The arbiter returned one verdict per
question, so the escaping defect that task 14.3 added was detected on every run and reported on
none: which of two real defects a contributor saw depended on where in the file it sat.

Configuration now returns one verdict per setting whose value the model could read. The pipeline
already emitted one finding per entailed answer, so this arbiter was the only place still folding
them to one.

Three boundaries are drawn rather than assumed.

- A setting whose value is COMPUTED stays one verdict per region. That band is a question for the
  judge rather than a finding, and returning every undecided setting would multiply model calls
  for claims the graph did not establish.
- A call carrying two permissive literals is one finding. cors({origin: true, credentials: true})
  is one misconfiguration, not two.
- Config verdicts now carry a path:line:function location. Without one they shared the region's
  identity and the deduplicator kept a single finding, so the plural answer collapsed straight
  back to one. That is the same gap dominance carried until its findings stopped arriving with no
  line at all.

Dominance is left answering once per region, and that is settled rather than pending: its claim is
that a region contradicts itself, so the region is the unit and a second verdict would restate the
same inconsistency. The pipeline now says so where it used to call this a known gap.

Measured: NodeGoat's config_secrets goes from one finding to two, server.js:78 for the session and
server.js:135 for the disabled escaping, with the applied marked.setOptions fix between them
staying silent. VAmPI is unchanged and no other family moved.

1,417 tests pass, seventeen skipped; thirteen Joern-gated query tests pass in the image.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-20 16:30:04 +02:00
Marius CiepluchandClaude Opus 5 e6bdac46e2 fix(access-control): decide the sibling scope, and see identity a destructuring spelled away
Two of NodeGoat's three corroborated claims were correct code. `const { userId } = req.session`
lowers to a temporary and then a field read, so nothing in the graph spells session.userId by the
time the value reaches dao.getByUserId(userId, ...), and the constraint the handler really applies
was invisible.

The arbiter now walks back three hops over the method's own assignments from the names the
operation is GIVEN, and req.session joins the identity sources. The token is only safe because of
that requirement: what discharges an object-level obligation is that the value passed to the
operation comes from the authenticated context, not that a session is mentioned nearby. A handler
that reads the session for logging and keys its query on a path parameter is the bug this family
exists for, and it stays unguarded.

Two exclusions the measurement forced, each caught by a page whose whole purpose is the leak it
demonstrates.

- A callback is not a constraint. Taking identifiers from the whole argument subtree credited
  NodeGoat's memo listing with a discharge, because memosDAO.getAllMemos((err, docs) => ... userId
  ...) reads every memo in the collection and only RENDERS the session's user. Method references
  are excluded from the argument names.
- An assignment is not a dominating guard. The session read dominates the query, so that clause
  credited it instead and the memo listing went quiet again. An assignment is a read of the
  authenticated context, not a check applied to it, so operator calls are excluded there -- the
  same exclusion the sink matcher needed. The leak stays reported.

On NodeGoat displayContributions and displayProfile go silent, the allocations insecure direct
object reference, both benefits pages and the memo listing stay, and the login and signup handlers
become honestly unguarded rather than accidentally cleared by a callback's text. Six corroborated
where there were seven, and a better six. VAmPI is unchanged at two entailed findings.

The sibling scope is decided by measurement and kept at the file. On NodeGoat the region's own
file entails two handlers, both public routes and both false, and no true defect at all. Every
handler in the application entails six: four true, and the same two public routes. Both scopes
carry the same false positives, because neither can tell a public route from a protected one --
a semantic question that is already the corroborated band's residual. With the carried rung capped
the scope changes no verdict, only the witness, so widening it would buy a repository-wide walk
per question and no precision. A public-route discriminator would change that; the file scope is
not what holds this family back.

1,415 tests pass, sixteen skipped; thirteen Joern-gated query tests pass in the image.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-20 15:16:03 +02:00
Marius CiepluchandClaude Opus 5 c0427e6ceb feat(access-control): model the collection read, by argument shape rather than by name
NodeGoat's documented insecure direct object reference reads with
allocationsCol.find(searchCriteria()), and `find` was absent from an operation table holding
findOne, findById, findAll and findByPk. The obligation was missing at the commonest read there
is, so that defect raised nothing at any point in the pipeline.

It was absent for a good reason, and the reason is the argument, not the name. Measured on
wp-statistics, a WordPress plugin that ships jQuery: its assets hold 88 calls named find, of
which 70 pass a jQuery selector, 13 pass a predicate to Array.prototype.find and 5 pass a
selector built at run time by a template literal or a concatenation. None is a collection read.

So the obligation is claimed only where the first argument has the shape of a query document.
The operation fact states which argument with query_document_arg, and the arbiter that can see
argument shapes decides it: no arguments at all, an object literal, or a builder call are
claimed; a method reference, a literal, an operator call and a bare identifier are not. Each
exclusion is one of the measured classes above rather than a guess.

Zero of those 88 calls became an obligated operation, and the zero is measured rather than
empty: the same run's census reports 60 files and 4,552 methods, and the files holding those
calls are in it.

On NodeGoat all three find calls are collection reads and all three are claimed, which raises
the obligation at four more handlers -- the allocations handler that takes userId from req.params
on an authenticated route, the benefits pages whose isAdmin guard is commented out in the route
table, and the memo listing. Seven of fifteen handlers now reach a verdict, all corroborated,
none reported without a judge, and the census is unchanged on both subjects.

A bare identifier is deliberately not claimed: col.find(query) and arr.find(pred) are the same
shape and neither repository measured holds one, so this is a stated false negative rather than
an unmeasured false-positive risk.

1,415 tests pass, sixteen skipped; twelve Joern-gated query tests pass in the image.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-20 12:10:27 +02:00
Marius CiepluchandClaude Opus 5 102a21fc4e feat(access-control): carry the obligation across the route-and-DAO split
The census measured this family completing 35 questions on NodeGoat with no obligation raised
anywhere: every modelled data operation lives in app/data/*-dao.js, none in a route handler
file, and operations were scoped to the region's own file. The family had nothing to be right or
wrong about.

The obligation now follows the one named call the handler writes, out of the region's file and
no further. How it is followed is the point. jssrc2cpg does not resolve
allocationsDAO.getByUserIdAndThreshold(...) to the DAO body: the callee is an external stub with
no code. That stub's fullName records where the member was written, so the implementation is
reached by the graph's own provenance rather than by a repository-wide name guess, which is the
false positive the file scope was closed against. A control asserts it: a third module defining
a member of the same name that nothing imports is not attributed.

A row is attributed to the region's DECLARED function, which is what lets a question about
displayAllocations match a graph that holds a <lambda>. A carried row is also checked for a guard
at the CALL, because that is where a handler establishes who is asking. The method holding a
carried operation joins the question's context scope, so the module holding the query cannot
change under a claim that depends on it without the change being attributed. The witness names
both sites and the finding is located at the call, which is the line a contributor changed.

A carried obligation corroborates, it does not entail, and that was measured rather than
assumed. Entailing it reported handleSignup, which looks a user up by name to check the name is
free during registration and is public by design, because two authenticated siblings in the same
file pass req.session.userId into the same module. The entailed rung's justification is that one
FILE contradicts itself, same file and same trust context; carried, the compared operations live
in a data-access module that handlers of different trust contexts share. The corroborated band
already asks the right residual: does this operation require a check, or is the route
legitimately public?

Measured on NodeGoat: 15 handlers asked, obligations raised at zero before and six now, three
reaching a verdict, all corroborated, none reported without a judge. VAmPI unchanged at two
entailed findings, because it keeps handler and query in one file and carries nothing.

What this does not reach is recorded, not implied. NodeGoat's documented insecure direct object
reference still raises nothing, because its module reads with allocationsCol.find(...) and find
is not a modelled operation: Array.prototype.find and jQuery's .find are everywhere, so it needs
an argument-shape discriminator and its own measurement (task 14.8). Two of the three corroborated
claims are correct code whose identity read a destructuring spelled away, and the cross-file
contradiction that would entail the real defect needs a sibling-scope decision (task 14.7).

1,415 tests pass, fifteen skipped; eleven Joern-gated query tests pass in the image.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-20 08:57:32 +02:00
Marius CiepluchandClaude Opus 5 f31ef862a7 fix(output-encoding): ask about the server half, and say what the family cannot decide
The census recorded this family completing 44 questions on a Node application and finding
nothing, which is what a client-only vocabulary must do there: all six declared sinks were DOM
sinks. NodeGoat's cross-site scripting is server-side.

Decision: the family asks about a value reaching a rendering call, and states in its own
declaration that it cannot decide template-mediated escaping. Whether res.render is safe depends
on the engine's autoescape setting and on the template's markup, neither of which is at the
call, so res.render is not a sink and the family's limits field says so. A new CWE-79 sink
covers res.write, res.end, res.send and the response.* spellings; res.json stays out because an
application/json body is not a document.

Template auto-escaping is claimed where its evidence is, as a configuration defect:
swig.setDefaults, nunjucks.configure and marked.setOptions are security settings now.

It establishes a finding: app/routes/research.js:25, model_entailed,
`req.query.url + req.query.symbol -> res.write(body)`. The handler fetches a URL the caller
chooses and writes the response body into a text/html response, so the document served from
this origin is the attacker's. The record states how coarse the step is: the path from url to
body runs through the client's callback parameter, which the graph treats as reachable from the
call's arguments, so the conclusion rests on the call's semantics rather than on what the graph
established. Five of seven families now establish findings.

Four defects were found on the way, three of them false positives this change would otherwise
have shipped.

- The permissive-value set was shared across every setting, while a boolean's polarity belongs
  to the setting: true leaves the door open in cors({origin: true}) and closes it in
  swig.setDefaults({autoescape: true}). Adding the template engines would have reported the fix
  as the defect on the line that offers it. ConfigSpec now carries per-setting values, and
  Jinja2's Environment moved out of Python's shared list for the same latent reason.
- config.sc matched a dotted setting on its trailing segment alone, so marked.setOptions would
  have claimed an editor's setOptions. dominance.sc already required the qualifier.
- taint.sc asked its flow query of every argument including the receiver at index 0. On a
  response sink the receiver is the response object, so res.end() -- which takes no data -- was
  reported exactly like the res.write(body) two lines above it. Flows now use the real
  arguments, as arity already did.
- The arbiter sorted a file's settings by line number as TEXT, so line 126 preceded line 78 and
  which setting a file reported depended on the spelling of its line numbers.

Template escaping disabled at server.js:135 is detected, measured model_entailed against the
real query and the real arbiter, and still not reported: the arbiter returns one verdict per
question and that file has an earlier permissive setting. Task 14.6 carries it rather than
hiding it.

1,415 tests pass, thirteen skipped; nine Joern-gated query tests pass in the image.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-19 20:38:39 +02:00
Marius CiepluchandClaude Opus 5 57f5a3f0dc fix(semantic-facts): give untrusted destination the clients a Node server calls
The family census recorded this family completing 44 questions on NodeGoat, finding the open
redirect, and having no way to ask about the repository's documented SSRF: needle.get on a
query-string URL at app/routes/research.js:16. A thin vocabulary is a silent false negative.

One data change, no query change. The declared set now covers axios, got, needle, request,
superagent, unirest, undici and fetch as bare module tokens, plus http.get, http.request,
https.get and https.request dotted. Bare is the documented call form of each module and covers
its verb methods too, since a token matches word-bounded against the callee and axios is
bounded inside axios.get; axios.get and axios.post are dropped as subsumed rather than
unwanted. http and https stay dotted because their namespace also holds createServer, and a
bare token would have claimed every server a repository starts.

The request is now reported at app/routes/research.js:16, model_entailed, witness
`req.query.url + req.query.symbol -> needle.get(url, ...)`. This family goes from one finding
to two on NodeGoat, every other family is unchanged, and 299 of 299 questions still complete.
Measured on that graph, needle matches exactly one call and the eleven other clients match
none in first-party or vendored code, so NodeGoat shows the gap closing and cannot show those
clients' noise.

A control asserts both halves. A handler calling axios.get with a literal and needle.get with
req.query.url reports the tainted client alone, and it reads the shipped vocabulary rather than
a copy, so removing the client fails it. The CWE-601 redirect entry is untouched: a
Location-header redirect is value-shaped and stays outside what this family asks.

1,411 tests pass, eleven skipped; seven Joern-gated query tests pass in the image.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-19 17:55:31 +02:00
Marius CiepluchandClaude Opus 5 4e2619e248 feat(pre-push-safety-net): census the declared families and fix a sink matched in a comment
Seven vulnerability families are declared and two had been proven. The census asks each of
them, on whole checkouts of NodeGoat and VAmPI, whether it establishes anything at all, and
reports a family that answered and found nothing separately from one that was never asked.

Four establish findings: injection, access_control, config_secrets and untrusted_destination.
Output encoding, path, prototype and deserialization establish nothing on either subject. All
299 NodeGoat and 35 VAmPI questions completed, so the silences are answers.

Three of those silences are diagnosed gaps, not clean code. NodeGoat's documented SSRF calls
needle.get, which the sink vocabulary does not hold. Its XSS is server-rendered, and output
encoding declares only DOM sinks, so no question it can ask could have found it. Its two
access-control defects raise no obligation at all: measured with the real dominance query,
every modelled data operation lives in app/data/*-dao.js and none in a route handler file,
while the family scopes operations to the region's own file. Tasks 14.2 to 14.5 carry each
gap; none is fixed here.

The census also found a false positive and it is fixed. NodeGoat's only prototype report was
the word `set` inside `// set these up in case we have an error case`: the matched node is the
<operator>.assignment holding the entire 1,000-character signup handler, and sinkMatches
searched a node's whole text. It now confines the text clause to the callee and excludes
operator nodes. Fourth instance of this bug class after fgets/gets, resolve/resolveUrl and
user/users. Measured on NodeGoat's non-vendor calls, injection sinks fall 28 to 3, prototype
13 to 2 and untrusted destination 26 to 7, with no true positive lost; req.query had been an
injection sink because `query` is word-bounded inside it. Witnesses now name the sink call
rather than the assignment that contained it.

1,411 tests pass, ten skipped; the six Joern-gated query tests pass in the image, including a
new one that fails on the shape this fixed.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-19 17:39:29 +02:00
Marius CiepluchandClaude Opus 5 770bae0e9e test(m3): record the PHP blocker as open, six hypotheses eliminated
Paused at the maintainer's direction to return to vulnerability work, with the
state recorded rather than left implied.

Measured and eliminated: batch size, since 17,700 requests answer in 44 s with a
2.6 MB response; dataflow cost, 50 execution requests in 24 s; call depth 3 with
parameter sources, 24.7 s for 20 requests; argument length, a 16 KB hook table
against a 128 KB single-argument cap; hook callback resolution, one second of
difference with 233 hooks; and build or sharding, 79 s with no shards. Preparation
measures 273.5 s, which is real but a small share of a 3,600-second case.

Every reproduction of the failing 22,125-request batch outside the push path answers
in under a minute. The difference from the production invocation is not identified,
and the exception itself has still never been read: the backend logged a stack
trace's tail, which is frames, and the corrected extractor reports its head, which
on this engine is JVM startup noise.

Four instrument defects were found and fixed along the way, all of the same shape:
a failure detail that discarded the cause, stages that reported no cost, logging
that was never configured so the new stage lines could not emit, and a harness that
buffers stderr until a case ends.

Node and Python transfer stand. PHP stays unestablished.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-19 13:24:26 +02:00
Marius CiepluchandClaude Opus 5 6089c67bf6 fix(cpg): report the line that names an engine failure, not its frames
Every PHP failure logged `cpg batch taint failed: dure1.java:10)`, which is the
middle of a stack frame and says nothing. A trace puts its exception on the FIRST
line and frames after it, and the backend logged the last 400 characters, so the
cause was discarded and the noise kept. Four investigations were spent on that.

Failures now report the leading non-frame lines, which is where the exception and
its message are, with the frames dropped. A control asserts an OutOfMemoryError
survives while a frame does not, and that a timeout and an empty exit stay
distinguishable.

1,411 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-18 17:58:42 +02:00
Marius CiepluchandClaude Opus 5 dfc29238b5 fix(cli): make the stage log reachable, nothing configured logging (M3)
The stage costs added for PHP diagnosis never appeared: nothing in this project
configures logging, so the default root level of WARNING dropped every
informational line and a PMPro case ran for an hour producing none.

That is the same failure shape the project keeps relearning. A run that reports
nothing is indistinguishable from a run with nothing to report, and it is how four
PHP investigations were spent reading a later stage's timeout as the cause.

Logging is now configured from OUSAST_LOG_LEVEL, quiet by default and sent to
stderr when asked, so a hook stays silent while a diagnostic run says where its
budget went. Verified end to end: six stage lines with durations on a real replay.

1,410 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-18 15:50:08 +02:00
Marius CiepluchandClaude Opus 5 45f9d6562e feat(push): report what each stage cost as it finishes (M3 diagnosis)
Three PHP investigations were spent reading `cpg batch taint failed: timeout` as a
slow query. The query is not slow: 4,425 evidence requests answer in 30 s, 50
dataflow requests in 24 s, and 20 entry-point requests at call depth 3 with
parameter sources in 24.7 s. The production PHP build takes 79 s and does not
shard. None of that explains a case running 26 minutes without finishing.

The reason it could not be diagnosed is that nothing reported where the budget
went. A run that is killed, or that blames a later stage's timeout, left no account
of what had already consumed the transaction, and a subprocess bounded by the
remaining time reports its own timeout whatever spent it.

Each stage now records and logs what it cost as it finishes, so a killed run still
says where it was. A control asserts the stages report and that each line carries
its duration.

1,410 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-18 14:32:03 +02:00
Marius CiepluchandClaude Opus 5 537483657d test(m3): PHP still not established; blocker is the dataflow execution pass
Retried the PMPro transfer with batch splitting, a 4500-second deadline and a
900-second query ceiling. Stopped after 26 minutes with no case completed and no
verdict, so PHP transfer remains unestablished.

A probe narrowed the cause and corrected my earlier reading of it. The evidence
pass answers all 4,425 PMPro PHP regions in ONE batch in 29.7 seconds with a 681 KB
response, exit 0, and its cost is fixed rather than per request: one request took
40.2 s and 4,425 took 29.7 s. Neither request count nor response size is anywhere
near a limit, so batch size and serialization are not the cause as I first supposed
from a single extrapolated data point.

What fails is the execution pass, which runs reachableByFlows per candidate.

The probe's own scope is recorded with it: it sent evidenceOnly=true and therefore
measured the structural pass, not the failing one. Those numbers would otherwise
read as evidence that PMPro's taint work is cheap, which they are not.

Splitting stays, because it is correct on its own terms and recovers a batch that
fails whole. It is worth noting it also makes an unrecoverable batch slower to
abandon, since each of the eight parts burns its own share of the query ceiling.

M3 remains two of three: Node established in M1b, Python established, PHP blocked
on dataflow cost for a 1,274-file repository.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-18 13:22:05 +02:00
Marius CiepluchandClaude Opus 5 aa66429a82 fix(cpg): answer a failed batch in parts instead of losing it whole (M3)
PMPro's 1,274 PHP files produced a response the engine could not serialize: it
failed inside ujson while building its own answer array, and every question in that
batch was lost with it. Three budgets from 900 to 3000 seconds made no difference,
because the size of the answer was the problem rather than the time available.

One invocation stays the first thing tried, so a working scan costs exactly what it
did. Each request in a batch is independent, so a batch that fails as a whole can
still answer in parts: on failure the requests are split and retried, and a part
that fails leaves its own requests unanswered rather than discarding every answer
beside them. Only a batch where no part answered is reported as unavailable, which
keeps "could not ask" distinct from "no rows".

Controls cover all three shapes: a whole batch that fails is answered in parts with
the split recorded, a failing part leaves only its own requests unanswered while the
rest are kept, and a batch no part can answer is still unavailable rather than
empty.

Whether this is enough for PMPro is a separate question the measurement will
answer: a single taint request on that tree already takes 40 seconds and returns
36 KB, so the batch may be slow as well as large.

1,410 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-18 12:47:50 +02:00
Marius CiepluchandClaude Opus 5 d64abc2d43 test(m3): PHP transfer not established; the taint batch cannot answer PMPro
All three PMPro comparisons exhausted a 3000-second deadline with a 1200-second
query ceiling and produced no artifact, so there is no target evidence at all. The
engine reports `cpg batch taint failed` twice per case on the 1,274-file tree:
first an exception whose trace runs through ujson array transformation, then a
timeout on the retry.

Budgets tried: 900 s, 1200 s and 3000 s deadlines against 300 s and 1200 s query
ceilings. The pins, source receipts and declared target are all correct, so the
question was asked properly and the engine could not answer it. This is a runtime
blocker on a large repository, not evidence about whether the design transfers to
PHP.

Two harness defects were found on the way, and both had made an earlier run look
like a transfer failure rather than an instrument problem. The manifest labels a
case by what it is, vulnerable or fixed or benign, while the id says what the
comparison does, introduce or repair or benign; filtering on one while keying
expectations by the other ran a single case and reported an all-false verdict. And
the per-query ceiling is taken as the minimum with the remaining transaction time,
so raising only the deadline could never have helped, and twice did not.

M3 is not met. Its exit asks for Node, PHP and Python together: Python is
established, Node was established in M1b, PHP is blocked.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 21:01:51 +02:00
Marius CiepluchandClaude Opus 5 bb72fcd413 test(m3): Python transfer detects all three declared targets (M3 13.1)
Generalized the replay instrument so the case family is an argument rather than a
hard-coded target, since a harness that only knows one language cannot say whether
the design transfers. Targets, pins and expectations come from the committed inputs,
so Python is asked exactly the way Node was, through the same core, ranker,
comparison and admission.

  VAmPI-SQLI        models/user_model.py:73:get_user        detected, answered
  VAmPI-BOLA        api_views/users.py:187:update_password  detected, answered
  VAmPI-BOLA-BOOKS  api_views/books.py:51:get_by_title      detected, answered

35 of 35 questions complete on every case, nothing reported new, zero admitted
defects.

Two results stand out. The SQL injection is the evidence the milestone spec names
as previously unresolved. And two of the three targets are access control, the
family that could not complete a question on any repository before group 12, so
this is the first time broken object-level authorization has been asked on Python
at all, and it answered on both endpoints.

VAmPI has the same revision on both sides, so the regression mode establishes
detection and completion, never novelty. It asserts that directly: with nothing
changed, a target reported new would be a comparison defect, and none was. That
tests the novelty logic from the opposite side to the NodeGoat case.

Identical revisions cannot show novelty, and VAmPI is inspected teaching data that
cannot qualify production Python support. No capability enabled, no latency claim.

1,407 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 18:17:31 +02:00
Marius CiepluchandClaude Opus 5 fc5dc453b6 test(ranker): every question now completes on the real case (M1b 12.4)
Measured after scoping a registration-only region to its file.

  M1a, before group 12        0 of 436 head questions completed
  after per-family context  298 of 436
  after region attribution  299 of 299, and 299 of 299 on the base

The count falls from 436 to 299 because the mis-attributed function-scoped regions
in a route-registration module are gone, replaced by file-scoped questions the graph
can answer. Nothing is unresolved on either revision.

Detections are unchanged: the vulnerable revision is still new with
operation_absent_from_comparable_base, the fixed twin analyzed and quiet, the
unchanged revision not a regression.

What remains is recorded limits rather than failures: 210 bounded-reachability
boundaries, each owned by the question that reported it, and one vendor exclusion.
Neither blocks a comparison.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 14:17:51 +02:00
Marius CiepluchandClaude Opus 5 573b764bbb fix(ranker): scope a region to the file that defines its handler (M1b 12.4)
A route-registration module registers handlers that other modules define.
app/routes/index.js requires seven handler modules and defines none, yet regions
there were scoped to handleContributionsUpdate and its siblings. The graph rightly
held no such method, so every question in those regions was unresolvable: 138
across six families on that one repository.

The entry record still names the handler. The authorization question is which
middleware guards which handler, and that is cross-file by nature. It now also
records whether the file defines what it names, and a region built from an entry
that does not is scoped to the file instead of to a method the file lacks. The
handler's own module still gets its function-scoped region, so nothing is lost.

My first attempt changed the mapper to stop naming cross-file handlers at all. An
existing control caught it by asserting that a multi-line registration names the
handler its middleware guards, which is exactly the attribution the authorization
family needs. The evidence was against me and the fix belonged one layer up, in
region construction.

1,407 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 14:04:39 +02:00
Marius CiepluchandClaude Opus 5 ca2c3a8b5f test(contributor-scan): record the group 12 end state (M1b 12.1-12.3)
Final verification of the three security capability fixes on the pinned revisions.

  head questions completed     205 -> 298 of 436
  base questions completed             436 of 436
  access control completed       0 -> 52
  context_projection_unavailable 116 -> 0
  context_scope_empty            138 -> 0

Every remaining unresolved question now reports context_function_unresolved and
names the function its file does not define, so nothing is unresolved for a reason
the tool could not explain. That is a region-attribution question for the ranker,
not a limit of what the engine can establish.

Detections are unchanged: the vulnerable revision is still new with
operation_absent_from_comparable_base and every verdict entry holds.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 13:44:20 +02:00
Marius CiepluchandClaude Opus 5 b95a371b7e feat(contributor-scan): name the function a question cannot locate (M1b 12.3)
After per-family context, every remaining unresolved question reported
context_scope_empty, all in three files. The cause is not a missing producer:
app/routes/index.js requires seven handler modules and defines none of them, so a
question naming handleContributionsUpdate there names a function that file
genuinely does not hold, and the empty scope is the correct answer.

Reporting that as "scope empty" hid which of two different things had happened. A
file the frontend produced no methods for is one problem; a question naming a
function defined in another module is another, and it is the common one. Taint and
configuration now separate them, reporting context_function_unresolved with the
name in the first case and context_scope_empty only when the file has no methods at
all. The driver no longer stacks its generic gap on top of a query that already
explained itself, and the new gap is question-owned so it cannot block another
question's comparison.

Completion is deliberately unchanged. The tempting fix is to widen such a question
to its whole file, which would raise the number by answering a question nobody
asked; a question whose function the graph does not hold stays unresolved, with the
name now recoverable from the question that asked it.

Group 12 is complete: admission no longer rejects genuine findings on a
mis-extracted operation symbol, both families that could never complete their change
context now can, and the remaining unresolved questions say precisely why.

1,406 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 13:28:40 +02:00
Marius CiepluchandClaude Opus 5 3e4ab8d219 test(contributor-scan): measure the per-family context effect (M1b 12.2)
Rerunning the pinned revisions with each family producing its own change context.

  head questions completed          205 -> 298 of 436
  context_projection_unavailable    116 -> 0
  access_control completed            0 -> 52

Access control is the headline: the family could not complete a question on any
repository before, and now completes every question it is asked. That is a
vulnerability class the tool could not reason about at all.

Detections are unchanged. The vulnerable revision is still new with
operation_absent_from_comparable_base, the fixed twin is analyzed and quiet, the
unchanged revision is not a regression.

The remaining 138 unresolved questions are all context_scope_empty and sit in three
files, none of them access control. They are function-scoped questions whose named
function the query cannot locate in the graph, so the scope it describes is honestly
empty. That is the query-to-operation identity problem M1a recorded at the witness
end, showing up at the scope end, and it is task 12.3 rather than something to patch
by widening a question to its whole file.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 11:36:51 +02:00
Marius CiepluchandClaude Opus 5 a91551e547 feat(contributor-scan): give each family its own change context (M1b 12.2)
Access control and configuration could not complete their change context on any
repository, which left 231 of 436 questions unresolved on the measured Node case
and made two of the three shipped families unable to answer.

The gap was larger than a missing query block. The evidence pass dispatches taint
only, because only taint has an evidence vector to rank by, so these families were
never asked for context at all. A new context pass asks each family with the flag
set.

Each query describes its own scope, which the owner research required and which
matters: copying taint's projection would describe a reachability walk these
queries never perform. Dominance exports the file's methods labelled operation,
guard or sibling, because its claim is that a file contradicts itself. Configuration
exports the file's methods labelled setting or sibling, because it asks what a file
declares. Both report an empty scope explicitly rather than implying one.

A family whose context query answers nothing keeps no context and stays unresolved,
so none can complete on another family's context. A control asserts that directly.

Confirmed on a real NodeGoat graph through the production backend: both families
produce a context summary and seven scoped methods where they previously produced
nothing. All seven are siblings for that file, which is the honest answer since it
declares no access-control operation and no matched setting.

1,405 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 11:19:30 +02:00
Marius CiepluchandClaude Opus 5 4ee31142ab fix(pre-push-safety-net): take the operation symbol from the call (M1b 12.1)
Admission extracted everything before the first parenthesis, so the operation for
`const preTax = eval(req.body.preTax)` was `const preTax = eval`. No real
declaration can enumerate every assignment a codebase might write, so admission
would have rejected genuine findings on operation_semantics_mismatch. The M1b
demonstration only passed because its declaration listed the assignment spellings.

The symbol is now the trailing qualified callee. Receiver spelling stays part of
it, so `$wpdb->prepare` and `prepare` remain different operations and a declaration
cannot widen from one to the other. A spelling that is not a qualified name, such
as a dynamic `handlers[name](x)`, yields the statement text, matches no declaration
and stays ineligible exactly as before.

Controls cover the assigned call, the receiver distinction, the dynamic spelling,
and that a declaration for a different operation still does not cover it. The
recorded-veto report reads the same helper instead of repeating the extraction.

1,403 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 09:56:59 +02:00
Marius CiepluchandClaude Opus 5 6653b6aca8 spec(pre-push-safety-net): record the M2 direction, do not rebuild a graph per push
Both attacks on engine startup are measured and neither reaches the gate. The
session is equivalent for every shipped query kind and helps a long transaction,
326 s to 243 s, but its own 9 to 13 second startup is a third of a 30-second budget
so it usually declines to start. Removing JavaScript's second engine start is real,
11.0 s against 21.0 s on identical graphs, and was reverted for an asymmetric
census gate that made 12 transactions pay two starts.

What the profile names is not startup. A changed-code transaction reaches its
deadline inside the build: head build 25.2 s of 30 s, query time 0.0 s. The cost is
building a fresh whole-repository graph for every pushed revision.

Direction accepted: reuse the graph across revisions rather than optimise the
rebuild. M2 stays unmet, the gates stay unchanged, and the work is not scheduled
here. Group 12 returns to security capability, which is where the remaining release
risk sits.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 09:53:35 +02:00
Marius CiepluchandClaude Opus 5 ed0f12b593 revert: JavaScript one-start build did not deliver in the push path (M2 11.6)
Reverts f04a839 on its own measurement. The seven-class profile rerun recorded
joern_parse_census_incomplete on 12 transactions: the faster build was rejected by
its own census gate and the frontend plus overlay pass ran as well, so those
transactions paid two engine starts instead of one. Every workload kept the same
p50 and the verdict stayed NO-GO.

The cause is an asymmetric gate. The frontend path is not census-gated at build
time, so whatever it produces is accepted, while the joern-parse path was gated and
the same shortfall rejected it. Holding the new path to a standard the old one
never has to meet guarantees the fallback fires. The probe tree and the
materialized push snapshot also differ, so equivalence on the checkout did not
transfer to the real path.

What still stands is the underlying timing fact, measured under the production
retention environment on identical graphs, same 44 files, 579 methods, file names,
overlays, Gruntfile and eighteen test files: 11.0 s against 21.0 s. JavaScript does
pay two engine starts where every other language pays one. A future attempt needs a
gate both paths are held to, or a direct comparison against what the frontend would
produce, and must be measured in the push path rather than on a checkout.

The profile also names the remaining cost plainly: a changed-code transaction
spends its budget before any query runs, head build 25.2 s of a 30 s deadline with
query time 0.0 s. Building one new graph per push is the cost, not querying it.

1,401 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 09:04:44 +02:00
Marius CiepluchandClaude Opus 5 f04a839181 perf(cpg): build JavaScript in one engine start, gated on the census (M2 11.6)
JavaScript was the only language paying two engine starts: the declared frontend
and then a separate overlay pass. joern-parse applies the overlays itself. On the
pinned Node tree that is 11.0 s against 21.0 s.

The declared frontend stays as the fallback, because it is what the versioned
retention adaptation and the explicit exclusion policy run through, and the census
decides per build: a graph that does not hold every readable declared file is
rebuilt through the frontend rather than trusted. Exclusions keep the frontend path
outright.

Measured under the production retention environment, both paths produce the same 44
files, the same 579 methods, the same file names, the same five overlays, the
Gruntfile and all eighteen first-party test files, with no file present in one
graph and missing from the other.

My first measurement did not establish that. It ran without the retention
environment, both paths dropped to 25 files, and the apparent equivalence was two
runs discarding the same things. The existing control named for that decision
failed against the change built on it, which is exactly what it is for; the
evidence was wrong, not the control. That control now asserts the intent directly:
the faster path is tried first, and a short census rebuilds through the frontend.

1,401 tests passed, nine skipped. A build saving is not the latency gate; the
profile rerun decides that. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 08:44:48 +02:00
Marius CiepluchandClaude Opus 5 7acb897d77 fix(cpg): stop a failed session start from consuming the transaction budget
The seven-class profile showed a 25.3 s head "build" of which the trace attributes
only 4.5 s to the frontend. The remaining twenty seconds were the session's startup
poll, which was bounded by `min(startup_timeout, deadline - now)` and therefore ran
until the transaction deadline was exhausted. The fallback then had nothing left,
so the transport's failure mode consumed exactly the budget it existed to save.

A session now refuses to start unless enough budget survives for the work it exists
to serve, and the wait is bounded by its own allowance rather than by the whole
deadline. Under the 30-second hook deadline it declines immediately instead of
burning what is left.

Controls cover both: refusing is immediate and starts no process, and a startup
wait ends at its allowance nowhere near a long deadline.

This does not make the gate reachable; it removes a self-inflicted cost that was
hiding the real one. 1,401 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 08:35:57 +02:00
Marius CiepluchandClaude Opus 5 ffbd864195 test(runtime): measure the seven-class profile with the session; NO-GO (M2 11.5)
The unchanged seven-class profile, three samples each, with the engine session
enabled. Every changed-code workload still times out at the 30-second deadline, and
identical-tip moved the wrong way: p50 rose from 18.5 s to 24.2 s.

The cause is in the artifacts rather than inferred. Thirteen transactions report
session_unavailable:session_startup_timeout, so under the real deadline the session
usually never starts and everything falls back to the disposable path. Startup is 9
to 13 seconds, about a third of the whole budget, and it is spent before any
analysis happens. Where a session did start, on cached-graph transactions whose
remaining work is small, it cost more than it saved.

That is the honest shape of the result: a transaction-owned session amortizes
engine startup across the invocations inside one transaction, and a warm
changed-code push does not have enough of them. It helps a long development-budget
transaction, 326 s to 243 s measured earlier, and cannot help a 30-second one.

Correctness is unchanged throughout: the same three pinned revisions through the
production rules with the session enabled reproduce the M1 verdict exactly.

Group 11 as designed cannot reach the gate, so the dominant remaining cost is named
and the decision is raised rather than worked around. Task 11.6 records the
candidates, since two of them change the product's shape: a session shared across
transactions is a resident local service, which the accepted milestone spec does
not imply, and analysing less is a scope change rather than a speed change. The
30-second and 2-second gates stay unchanged.

Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 08:12:37 +02:00
Marius CiepluchandClaude Opus 5 7fc6713e78 test(cpg): confirm the session stays alive and halves query time (M2 11.4)
The leak was the whole cause. With the session closed by the transaction that owns
it, a session-enabled replay of the same pinned revisions under the same budget
keeps its session for the whole run: no session_query_failed appears on either
side, where the previous run reported one per side and fell back to the disposable
path for everything.

Measured on the same pins, instrument and budget, changing only the transport:

  vulnerable revision   326.4 s -> 243.3 s
  head query time        63.6 s ->  32.0 s
  base query time        85.1 s ->  51.2 s
  dominance and config   ~14 s each -> ~4 s each

Detections are unchanged: witness present, fixed twin analyzed and quiet, unchanged
revision not a regression, kill criterion not triggered. Speed bought by losing a
detection would not be speed.

This is a cold transaction on one teaching repository under a 900-second
development budget. It is not the hook latency gate, which is warm changed-code p95
against the frozen seven-class profile, and that is task 11.5. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-17 07:50:04 +02:00
Marius CiepluchandClaude Opus 5 566b862b0a fix(cpg): end an engine session with the transaction that owns it (M2)
The first session-enabled replay produced correct detections and no speedup,
reporting session_query_failed:census:session_request_failed on both sides. A
targeted probe ruled out the sequence I suspected: an overlay followed by a census
in one session both answer correctly, with the census confirming the overlays were
saved.

What the probe exposed instead is a leak. A build runs on a copy of the backend, so
the session started during the build belonged to that copy and was never closed,
leaving a JVM holding the configured 2 GB heap for the life of the process. Two
builds per transaction meant two of them, which is enough to exhaust a 3 GB
container and make every later request fail. It also matches the host running out
of memory during these runs.

The build now closes any session its copy started, and the driver ends the session
with the transaction, since that is what transaction-owned means. Controls cover
both: a build leaves nothing alive on the copy that started it, and a completed
scan closes the backend's session. Closing never fails a scan that found things.

This is a genuine defect found by measurement rather than by the suite, which
passed throughout. Whether it was the whole cause of the census failure is not yet
established; the next session-enabled run will say.

1,399 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 21:15:00 +02:00
Marius CiepluchandClaude Opus 5 d2362cd18d fix(cpg): carry the engine's own account when a session request fails (M2)
The session-enabled NodeGoat run reported session_query_failed:census:
session_request_failed and fell back, which is correct behaviour but says nothing
about why. A bare transport failure cannot distinguish a timeout from a crash from
an out-of-memory kill, and the only account of which it was is in the server's own
log.

A transport failure now carries the exception and the tail of that log, and the
backend logs it with the process exit code alongside its own label. A control
asserts an OutOfMemoryError in the log reaches the caller.

This is diagnosis, not a repair: the run's timings were unchanged because the
session died early and everything fell back to the disposable path.

1,397 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 21:00:24 +02:00
Marius CiepluchandClaude Opus 5 fc69055a32 feat(cpg): add an explicit environment switch for the engine session (M2 11.3)
A measured run needs to enable the transport without changing any default. The
switch reads as enabled only for the exact value "1"; unset, empty, "0", "true"
and anything else leave the disposable path in place, and an explicit constructor
argument still wins in both directions.

1,396 tests passed, nine skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 20:43:50 +02:00
Marius CiepluchandClaude Opus 5 87f7976a58 perf(cpg): run the overlay pass in-session and verify it saved (M2 11.4)
The overlay was the last fixed cost bypassing the session: a whole JVM start,
15.8 s in the measured Node runs, while the census and query batches already route
through it.

`save` writes under the process working directory, which for a session is its own
private scratch rather than the build scratch, so the saved graph is now looked for
in both. Finding neither keeps the raw graph and records it rather than reporting
success, because a graph without its dataflow layer has that layer recomputed by
every later batch, 54 s each on a 637-file plugin, with nothing otherwise saying so.

The disposable invocation remains the fallback and runs unchanged whenever the
session is off or cannot answer.

Controls cover both new paths: an engine that reports success but saves no graph
keeps the raw one and notes it, and a graph saved under the session's workspace is
found and applied.

1,395 tests passed, nine skipped. The transport stays off by default; task 11.5
measures against the frozen profile. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 20:41:11 +02:00
Marius CiepluchandClaude Opus 5 d93dee9f1a test(cpg): prove transport equivalence for every shipped query kind (M2 11.2)
A session is only usable if it returns exactly what the disposable invocation
returns. On a real NodeGoat graph, taint, dominance and configuration all produce
byte-identical payloads through both transports, with parameters built by each
arbiter's own request function through the driver's family lookup rather than a
hand-written shape.

The failure shapes that would make a session dangerous rather than merely slow are
correct too. An empty answer stays present and distinguishable from an unavailable
one, which is the distinction that once turned a failed scan into a clean bill of
health. An unknown query and an absent graph are both unavailable. Two graphs in
one session each answer about the graph the request names.

Amortized over five taint requests the session took 22.5 s against 72.0 s
disposable, all five answered and identical to each other.

Two instrument defects are recorded rather than hidden. The first run created a
fresh backend per comparison, so every session served one request and paid full
startup, measuring the session's worst case and making it look slower; that is why
the amortized comparison exists. The switched-graph check crashed on a scratch
directory it never created, so it produced no verdict at all.

These timings are orientation, not the latency gate. Task 11.5 measures against the
frozen seven-class profile, and the transport stays off by default until then.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 20:37:47 +02:00
Marius CiepluchandClaude Opus 5 7e3f12bc00 feat(cpg): add a transaction-owned engine session and route it opt-in (M2 11.1, 11.3)
JVM startup, not analysis, is what a push scan spends its time on: each disposable
script invocation costs 13 to 23 seconds while the work takes under a second.

The session owns one Joern server per transaction on loopback with an OS-chosen
port and no published port. Three properties are load-bearing, and the probe that
settled the design demonstrated why each is needed.

An HTTP status is not an answer. The server returns success even for a compile
error, with the diagnostic in its body, so every request redirects the evaluated
code's console output into a fresh private file behind a nonce, and the nonce is
verified before the answer is accepted. The engine's body is retained for exactly
the case where the receipt is missing, without which that failure explains nothing.

A poisoned session answers nothing afterwards, and cancellation kills the process
group rather than the launcher, which this project has already run a machine out of
memory by neglecting.

What the probe established about the engine, after three iterations that each
failed loudly: runScript is absent from the server REPL; a shipped @main script
cannot be nested in a block expression, so definitions go to the top level
unwrapped; and the REPL shadows Console with io.joern.console.Console, which has no
withOut, so capture must qualify scala.Console. With that fixed, both mechanisms
answer identically to the disposable invocation, 0.8 s against 9.8 s.

The backend routes batch and single queries through the session only when
session_transport is enabled, which it is not by default. Any failure falls back to
the disposable invocation rather than reporting a zero, and each script is defined
under its own name because every script declares exec.

Equivalence for taint, dominance and configuration is task 11.2 and gates any
measured run. 1,391 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 20:24:34 +02:00
Marius CiepluchandClaude Opus 5 cec1c4a428 feat(pre-push-safety-net): render the experimental evaluation explanation (M1b 10.10)
The M1b exit run established the comparison but could not show a developer-readable
consequence or repair direction, because both belong to an evaluated capability
declaration and the installed registry is deliberately empty.

The experimental evaluation output now renders, for each change-attributed finding,
its family, exact location, witness, change attribution, provenance, engine method
and operation, together with a consequence and repair direction produced from a
supplied declaration through the same grounded-template path the admitted case
uses. No prose is invented: when no declaration matches, or a template is not
grounded in the operation and context, or the declaration does not cover the
operation symbol, the explanation is absent and the reason is stated.

Declarations reach this rendering only. The runner never passes them to admission,
so nothing in that file can enable a capability or emit an alert however it is
written, and a control asserts that even a declaration claiming PASS admits
nothing. The flag requires explicit base/head replay and the recorded-veto flag.

The evaluation instrument generates its own unreviewed declaration bound to the
run's analysis semantics, asserting that it is neither enabled nor PASS, so the
demonstration cannot drift into a qualified claim.

1,377 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 19:59:12 +02:00
Marius CiepluchandClaude Opus 5 57d6ddcbba test(pre-push-safety-net): record the M1b exit run (task 10.9)
All three pinned NodeGoat revisions replayed through the production rules with
exact pins and reopened source receipts.

The vulnerable revision reports all three sites new with
operation_absent_from_comparable_base, and admission now rejects on
capability_unavailable alone: dependency_unresolved is gone, which verifies 10.7
on real data since those gaps belonged to unrelated questions. The fixed twin is
analyzed and quiet with three base-only findings. The unchanged revision reports
all three sites unchanged with base locations resolved to the same lines.

Every finding carries its question, the engine-reported method and its witness.
The head census is the complete 44 files and all 436 base questions complete.
Zero admitted defects on every revision.

All five vetoes M1a recorded are repaired, plus two the repairs exposed. What
remains is recorded rather than repaired: 231 head questions still report an
incomplete change context from families whose queries export none, which did not
block this case and whose families stay experimental, and the vendor exclusion
and correspondence ambiguity remain reported as the limits they are.

M1b is not yet verified. The milestone also asks for a developer-readable finding
carrying a supported security consequence and a concrete repair direction, and
both are owned by an evaluated capability declaration that is deliberately empty.
Writing that text without a declaration is what admission exists to prevent, so
task 10.10 renders it through an explicitly experimental declaration with
eligibility still disabled.

Runtime, independent qualification and rollout all remain NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 19:11:03 +02:00
Marius CiepluchandClaude Opus 5 541ee5813e feat(pre-push-safety-net): carry query-to-operation provenance (M1b 10.8)
M1a recorded that the NodeGoat target is selected through a file-scoped question
naming no function while the engine reports method <lambda>2, and that this did
not block the comparison once the question completed. It was still only visible by
digging the method out of a serialized detail blob.

Evidence now carries the engine-reported method beside the question that produced
it, and the relationship between them is recorded: exact when the question named
that method, file_scope when the question named no function and the engine
supplied one, unresolved when the two disagree, and method_unreported when the
engine named none.

Recorded, not applied. No region is renamed, no benchmark function name is
injected into a region, and no coincident site is counted. The recorded-veto
report now reads the field instead of reparsing the detail blob.

1,371 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 18:55:31 +02:00
Marius CiepluchandClaude Opus 5 9a67f54ea1 fix(pre-push-safety-net): bind dependency gaps to the candidate (M1b 10.7)
The runner handed every candidate the whole transaction's coverage reason list as
its unresolved dependencies, so admission always answered dependency_unresolved.
That list even included other candidates' comparison outcomes, so one unresolved
comparison anywhere marked every other claim as having an unresolved dependency.

Gaps are now attributed. A boundary carrying a question id counts only when that
question is in this candidate's required comparison scope, the same scope 10.2
established. A gap nothing attributed to any question always counts, because
unknown ownership is not evidence of independence. Integrity degradations count.

A declared exclusion, a correspondence ambiguity and a bounded reachability do
not, because each was already weighed against this specific claim: the first is a
scope choice, the second is decided per operation, and the third is decided per
claim by the comparison.

Found by the M1b production rerun rather than by the suite: the comparison reached
a correct new verdict and admission still rejected it for a dependency belonging
to unrelated questions. It is not an M1b exit blocker, since eligibility stays
disabled, but it would have blocked admission forever once capabilities qualify.

1,371 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 18:52:04 +02:00
Marius CiepluchandClaude Opus 5 b8b46f1c19 test(pre-push-safety-net): record the M1b production comparison on NodeGoat
Rerunning the three pinned revisions after tasks 10.1-10.6 shows the production
comparison classifying all of them with no lifting, and agreeing exactly with the
recorded view. That agreement is the point: the vetoes M1a recorded were repaired
rather than bypassed.

The vulnerable revision is new with operation_absent_from_comparable_base, and the
three eval witnesses at contributions.js lines 32 to 34 are attributed to the
change that introduced them. The fixed twin is analyzed and quiet with three
base-only findings. The unchanged revision is unchanged. In M1a every one of the
436 questions was demoted on both sides and production reported
witness_identity_unresolved on every finding.

All three artifacts still report finding_status none with zero admitted defects.
Admission rejects on capability_unavailable, the deliberate eligibility gate, and
on dependency_unresolved, which the rerun exposed as coarse plumbing: the runner
passes every coverage reason as every candidate's dependency gaps. That is now
task 10.7, ahead of M4, since it would block admission forever once capabilities
qualify. It is not a comparison failure and not an M1b exit blocker.

Three comparisons on one inspected teaching repository under a development budget.
No capability enabled, no alert emitted, no latency claim. Runtime, independent
qualification and rollout all remain NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 18:06:39 +02:00
Marius CiepluchandClaude Opus 5 bc68000e93 fix(pre-push-safety-net): apply the exclusion distinction inside comparison too
The NodeGoat rerun after 10.4-10.6 showed both target questions completing, which
is what those tasks were for, but the comparison still reported
head_context_incomplete. Two checks inside comparison had not been updated with
the same distinction: any degradation blocked, and any scope boundary blocked.

A declared exclusion and a correspondence ambiguity both reached those checks. The
first is a scope choice rather than a failure to read the graph, and the second is
decided per operation since 10.3, so neither should block a claim. Both remain in
aggregate coverage. Integrity degradations, unknown ownership and every other
boundary still block, and bounded reachability is still filtered separately
because it blocks only claims resting on a flow not having been traced.

1,368 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 17:51:49 +02:00
Marius CiepluchandClaude Opus 5 c1cc332bc1 feat(pre-push-safety-net): stop declared exclusion vetoing first-party answers (M1b 10.6)
vendor_semantics_unresolved was grouped with the graph integrity failures, so any
declared exclusion demoted every otherwise complete answer in the repository to
graph_incomplete. Every real repository excludes its dependencies, so this fired
on all of them, including on a complete first-party census with successful query
receipts.

Graph boundaries are now split by what they mean. An integrity failure says the
graph could not be read or is incomplete, so an answer drawn from it is demoted:
unparsed files, an empty or sharded graph, an incomplete or unavailable census,
an unresolved symlink, an unsupported frontend, unreadable source, an ambiguous
frontend path and unvalidated TypeScript property support all keep that behavior.
A declared exclusion is a deliberate scope choice. It stays physical, stays out of
targets and graphs, and stays reported in coverage, but it is not evidence that a
first-party answer is wrong.

Whether a particular answer needed the excluded semantics is carried by that
question's own unresolved call destinations, recorded per question in 10.5, rather
than assumed for every answer in the repository.

Controls exercise the real partition logic rather than an injected reason: a
vendored directory leaves the answer completed with the exclusion still reported,
an unresolved symlink still demotes, and the two together still demote.

1,365 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 17:35:00 +02:00
Marius CiepluchandClaude Opus 5 b202aabd8b feat(pre-push-safety-net): bind the reachability boundary to what it limits (M1b 10.5)
dynamic_external_or_depth_context_unresolved was applied as though it invalidated
everything about a question. It does not. Bounded reachability can hide further
flows; it cannot invalidate a flow that was traced, and it cannot remove a call
from a structurally scoped enumeration.

At scan level the boundary is now recorded against its owning question without
demoting that question's answer. An unavailable context projection, an empty
context scope and an unreadable method extent are unknown ownership and still
demote.

In comparison it no longer blocks the head's own claim, since the finding rests
on a path that was actually traced, and no longer blocks an absence read off the
enumerated inventory. It still blocks every claim that rests on a flow not having
been traced, including the anchored source-connection branch, which now reports
base_reachability_bounded rather than silently passing.

The query declares the basis of its enumeration. A file- or nesting-scoped
question walks calls structurally, so an unresolved destination cannot remove one
from scope. Following the call graph makes the reachable set the scope, so the
same unresolved destination can shrink what was enumerated; that inventory cannot
establish absence while reachability is bounded, and says so.

One prior control asserted a question's own reachability gap blocks its own
claim. It is updated with its reason and split, so an own projection gap is still
proven to block while an own reachability gap no longer does.

1,362 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 17:26:16 +02:00
Marius CiepluchandClaude Opus 5 c73efd2bde feat(pre-push-safety-net): export the enumerated operation inventory (M1b 10.4)
A taint answer reports traced flows, so an absent row means either that the
operation is not present or that no traced path reached it. Change comparison
needs that distinction to say a change introduced an operation, and reachability
is deliberately bounded, so the two can never be separated from flow rows alone.
That is why the veto guarding the confusion is permanently true on any real
Express application, and why it could be neither discharged nor safely removed.

The taint query now exports one row per operation it enumerated in scope, plus a
completion marker. The marker is what makes an empty inventory meaningful:
without it a scope that was never enumerated and a scope holding no such
operation are indistinguishable.

Comparison separates three outcomes on a changed line: the operation is present
in the base without a traced flow, which is a reachability statement and not
novelty; the operation is absent from an enumerated base, which establishes that
the change introduced it; or the base scope was never enumerated, which stays
explicitly unresolved. A malformed inventory row is not an empty inventory.

The arbiter now skips any row carrying a kind, so an inventory row cannot be
normalized into a flow with an empty source that names no attacker input.

Verified against the real engine on an authored two-revision JavaScript control:
the operation is enumerated on both sides, the base carries it with zero traced
flows, the zero-operation marker appears for an empty scope, and answers and
findings are unchanged. No new sink inference, alias resolver or second source
IR. 1,354 tests passed, nine skipped. Rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 16:46:00 +02:00
Marius CiepluchandClaude Opus 5 86980d8aa0 feat(pre-push-safety-net): measure M1b step 1 and record the inventory prerequisite
Rerunning the three NodeGoat revisions after tasks 10.1-10.3 shows the recorded
view now classifies the case: the vulnerable revision reaches new with
operation_absent_from_comparable_base, and the unchanged revision reaches
unchanged with its base location resolved to the same lines. Both previously
stopped at operation_correspondence_unresolved and witness_identity_unresolved.

The rerun also caught a defect in the repair itself: the new novelty reason was
missing from the admission allowlist, so admission rejected the candidate as
change_unsupported. Fixed with a control.

The production path is still blocked by the last two vetoes, and the measurement
shows neither can be scoped or relaxed as the plan described. A taint answer
reports traced flows, so an absent row means either that the operation does not
exist in the base or that no traced flow reached it.
dynamic_external_or_depth_context_unresolved exists to stop the second being read
as the first, and removing it would turn a wrong absence into a wrong new
finding. It is also computed over every in-scope method, so on any real Express
application it is always true and can never be discharged, which is why the tool
reports nothing on real projects.

The resolution is new evidence rather than a relaxed rule: export the operations
the query actually enumerated, separately from the flows it traced, so absence is
established from the inventory while a traced-flow claim still requires complete
reachability. That is now task 10.4, ahead of the two vetoes it unblocks, and it
adds no detector, alias resolver or second source IR.

1,348 tests passed, nine skipped. No capability enabled; rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 16:03:16 +02:00
Marius CiepluchandClaude Opus 5 8ff1d9a638 feat(pre-push-safety-net): scope comparison completeness and ambiguity (M1b 10.2, 10.3)
Two vetoes M1a recorded were global where the evidence is local.

Comparison required every selected base question to complete, so one unsupported
family anywhere prevented an otherwise complete target comparison. Completeness
is now demanded over the required scope: the counterpart question plus the
dependencies the change context actually recorded for it. Every other gap stays
in aggregate coverage and is still reported, never discharged. An incomplete
counterpart, an incomplete recorded dependency, an unknown-ownership boundary
and a truncated base scan all still yield unknown.

Correspondence ambiguity was recorded file-wide and inherited by every question,
so repeated unchanged lines in one file invalidated every question in the
repository and blocked the whole transaction comparison. It is now decided per
operation by the correspondence classifier: an operation needing an ambiguous
anchor stays uncomparable, while a uniquely mapped or newly introduced operation
in the same file is unaffected. The mapping is still withheld and still reported.

On the NodeGoat case the ambiguity is on the target's own file, so file-level
scoping would not have been enough; only per-operation resolution is.

1,347 tests passed, nine skipped. No capability enabled; rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 15:45:57 +02:00
Marius CiepluchandClaude Opus 5 7e0d01d9e8 feat(pre-push-safety-net): classify an operation the change introduced (M1b 10.1)
M1a measured that a newly introduced operation sits on a changed line, has no
base line anchor, and therefore stayed unclassified even with every completeness
veto lifted. Establish absence over the complete counterpart answer instead of
over a line anchor.

Correspondence now reports why a head operation does not map: `changed_line`
inside a recorded head edit, `ambiguous` where several base lines could
correspond, or `unmapped` where no anchor exists. Only the first supports
reasoning about an introduced operation, and only when the base counterpart
answer is complete and contains no operation with that spelling.

Movement is matched on the operation spelling alone, so a benign identifier
rename that changes only the source text reads as movement rather than a new
flow. The existing rename control caught that; the key was too brittle before.

One prior control is superseded and updated with its reason: a new operation on
a changed line in a comparable existing file is now new rather than unknown.
Movement, ambiguity, unmapped operations, discharged or unentailed claims and an
incomplete or failed base all remain unknown.

1,337 tests passed, nine skipped. No capability enabled; rollout stays NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 15:38:13 +02:00
Marius CiepluchandClaude Opus 5 3224884d8f feat(pre-push-safety-net): record admission vetoes and meet M1a on NodeGoat
Add group 9 to the approved task plan as the bounded owner amendment release
milestone M1a requires, implement the task-local recorded-veto evaluation flag,
and run the three pinned NodeGoat revisions under it.

The flag re-runs the production comparison and admission over a lifted copy of
the same evidence and records each veto instead of applying it. Recognized
completeness vetoes are lifted and logged; failures never are, because silence
from a failure is not a quiet fixed twin. It is off by default, accepted only on
explicit base/head replay, refused in Git stdin mode, labels every payload it
produces, and admits no defect even when a lifted candidate otherwise would.

M1a outcome, inside its one-week box and with expectations frozen on disk first:
the three entailed <lambda>2 witnesses are present at contributions.js 32-34; the
fixed twin was genuinely analyzed and quiet, answering its target question with
zero rows and no deadline reason while the same findings appear as base-only; the
unchanged revision was analyzed and recorded unchanged. The kill criterion did
not trigger.

Five vetoes fired and were recorded. M1a also corrected the diagnosis: with all
five lifted the detection still stops at operation_correspondence_unresolved,
because a newly introduced operation sits on a changed line and has no base
anchor. The unchanged revision resolves its base location on the same lines and
classifies as unchanged, which isolates correspondence as the mechanism. That
fifth veto was not in the earlier four-veto diagnosis and is now M1b's central
repair.

Diagnosis only. No defect admitted, no capability enabled, no gate changed, no
latency claim; the 900-second development budget cannot measure the hook gate.
Rollout and task 8.3 remain NO-GO.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-16 15:31:34 +02:00
Marius CiepluchandClaude Fable 5.1 8b66d5cb75 spec(pre-push-safety-net): record M1–M5 release milestones and split M1 into M1a/M1b
Add release-milestones.md as the executable outcome contract: prove one useful
Node detection, make it fast, verify PHP/Python transfer, qualify on untouched
cases, then enable a narrow advisory rollout. Link it from the roadmap, safety-net
steering, requirements, tasks, status and spec.json.

Split M1 after review. The raw NodeGoat <lambda>2 witness already exists in the
2026-09-14 post-census bundle; four admission vetoes discard it. M1a runs the
vulnerable/fixed/unchanged revisions under a task-local experimental flag that
records each veto instead of applying it, with a one-week box and a kill
criterion if the fixed twin is not quiet. M1b repairs only the vetoes that fired
and reruns through production rules. Existing approvals and thresholds unchanged.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-14 20:48:23 +02:00
Marius Ciepluch 1ad472c68d test(release): measure repaired reuse and local Joern lifecycle 2026-09-14 13:18:51 +02:00
527 changed files with 154733 additions and 6196 deletions
+27
View File
@@ -0,0 +1,27 @@
# Build context for the root Dockerfile. The image copies pyproject.toml, README.md, src/, benchmarks/ and
# ops/frontend-retention/install.py; everything below is either never copied or is evidence the image does not
# need. plane/Dockerfile.runner has its own plane/Dockerfile.runner.dockerignore.
# Local state, never part of an image (agent worktrees alone run to hundreds of MB; .env holds keys).
.git
.claude
.venv
.env
.env.*
!.env.example
**/__pycache__
**/*.pyc
.mypy_cache
.ruff_cache
.pytest_cache
.openultrasast
reports
*.sarif
# The measurement archive (about 20 MB) is evidence, not runtime input. These records stay:
# ops/smoke_report.py reads the two 2026-09-13 ones from /app inside the image (ops/README.md).
benchmarks/measurements/*
!benchmarks/measurements/2026-09-13-delta-comparison-smoke.json
!benchmarks/measurements/2026-09-13-admission-policy-smoke.json
# A label source of `ousast learn` (src/openultrasast/learn/sources.toml).
!benchmarks/measurements/2026-09-30-harnessx-removal-baseline/pairs.json
+21 -9
View File
@@ -1,12 +1,24 @@
# Copy to .env and fill in. .env is gitignored.
OPENROUTER_API_KEY=
# Copy to .env and fill in. .env is gitignored and never committed.
# Note: values in .env never override variables already exported in your shell
# (e.g. from ~/.bashrc). Unset a stale export, or the shell value wins.
# Prove-budget and `ousast index` embeddings via OpenRouter /embeddings.
# LLM calls (verify, roles, the decision engine): DeepSeek.
DEEPSEEK_API_KEY=
# DEEPSEEK_BASE_URL=https://api.deepseek.com
# Embeddings only: OpenAI text-embedding-3-small through OpenRouter.
OPENROUTER_API_KEY=
OPENROUTER_EMBEDDING_MODEL=openai/text-embedding-3-small
# Any OpenAI-compatible chat endpoint works for the hunter: set the key above to that
# provider's key and override the base URL. Example for DeepSeek:
# OPENROUTER_API_KEY=sk-deepseek-...
# OPENROUTER_BASE_URL=https://api.deepseek.com/v1
# Then set [models].hunter in openultrasast.toml (or --hunter-model) to that provider's model id.
OPENROUTER_BASE_URL=
# Plane memory store. Default is a local FileStore under ~/ousast-results/plane/memory.
# For an S3-compatible server (RustFS is the tested one): OUSAST_MEMORY=s3://<bucket>[/<prefix>], or
# OUSAST_MEMORY=s3:// to use S3_BUCKET. Needs the s3 extra (boto3). Example for a RustFS server:
# OUSAST_MEMORY=s3://sast-memory
# S3_ENDPOINT=https://files.example.com
# S3_REGION=us-east-1
# S3_BUCKET=sast-memory
# AWS_ACCESS_KEY_ID=
# AWS_SECRET_ACCESS_KEY=
# AWS_SESSION_TOKEN=
# The bucket must have versioning enabled, a lifecycle rule expiring runs/, S3 Select and readable object tags
# (see docs/rustfs.md); the store verifies them at startup and refuses to run otherwise.
+3
View File
@@ -33,3 +33,6 @@ vector-store/
# own scratch dir so it never lands here, but a maintainer querying a CPG by hand from the repo root
# will create one.
workspace/
# MkDocs build output (mkdocs build)
site/
+78
View File
@@ -0,0 +1,78 @@
# Brief: an AI service plane for OpenUltraSAST (Kubernetes-style, at the model level)
Status: brief for maintainer decision (2026-09-29). Not a spec yet; no approvals reset.
## Why now
Two measurements say the current pipeline cannot be developed further at its token cost:
- Development: 90% of the tokens of the last session were tool I/O (inline scripts and their output), because
every stage lives in ad-hoc scripts run from one long conversation (`dev-tokens-are-tool-io` memory).
- Pipeline: verification costs $0.009-0.022 per candidate for two passes because every hunt re-derives the same
per-repository facts and every pass re-sends its whole conversation; resume, budgets and pass agreement are
hand-rolled per script (`results-v2-model-pipeline.json`).
Google's `ax` (open-sourced 2026-09; Apache-2, Go, early) frames agent work as a Kubernetes-style service:
declarative `Task` / `Workspace` / `Model` manifests, a reconciler, sandboxed isolated execution, suspend and
resume, and a runner contract any binary can honour. Its software needs Kubernetes and Agent Substrate, lists
only Google and Anthropic providers, and has no pipelines, artifacts or budgets yet. Its *design* is what we
adopt; its software is a later deployment target.
## The model
| ax primitive | OpenUltraSAST meaning |
|---|---|
| `Workspace` | one pinned checkout: repository, commit, product-file filter; materialised once, bound by many tasks |
| `Model` | provider, model id, credentials, prices, per-task budget (usd, calls); DeepSeek via an adapter |
| `Task` | one concern, isolated: declared inputs (artifacts), declared outputs, resource limits, idempotent per unit, resumable |
| runner contract | our Python entrypoint reads the task and workspace YAML from the environment and writes artifacts |
| reconciler | a local loop over manifests today (one engine container at a time on the 7 GB host); `ax` later, unchanged manifests |
What ax lacks and we add as a thin layer: a `Run` manifest that lists tasks with dependencies (a DAG), artifact
locations, and budgets. This layer is what the reconciler executes; the tasks themselves are ax-shaped.
## The concerns as tasks
1. `classify-sinks` -- workspace -> `candidates.json` (model-read, no tools). Exists: `model_sinks.py`.
2. `triage` -- candidates -> `kept.json` with `data_origin` per function (no tools). Exists inside `verify_batched.py`.
3. `repo-facts` -- workspace -> `facts.json`: per-file callers and entry points, computed once, no model. New;
this is the memory layer, and it is an artifact, not a cache.
4. `verify` -- kept + facts, `pass=n` -> `verdicts.jsonl`, one hunt per file, budgeted. Exists as a script.
5. `judge` -- verdicts -> `agreed.json`: agreement across passes, then a no-tools judgement over each agreed
finding's rationale and code. Agreement exists; the judge does not.
6. `recheck-fixed` -- fixed workspace + agreed -> `fixed_alerts.json`. Exists (`--fixed`).
7. `score` -- all of the above + the population record -> `score.json`. Exists (`score_batched.py`).
Each task: one manifest, one entrypoint, one fixture test, one budget. A task that fails leaves no partial
truth: it is unfinished, never "clean" (the 402 lesson).
## What it buys, measured or estimated
- Tokens per task bounded by declared inputs; `repo-facts` replaces the 1-2 tool turns per hunt that today
re-grep callers (each turn re-sends the whole context); estimated 20-40% per hunt, to be measured on the
46-candidate validation set.
- Two passes and the judge become tasks with their own budgets instead of loops inside a script.
- Development in small contexts: one task type at a time, against a fixture; the transcript measure is the
check.
- Portability: the same manifests run on the laptop reconciler now and on `ax` on a cluster later.
## What it does not buy
Recall and precision come from the model and the questions asked, not from orchestration. The open problems
recorded in the strategy review (judge quality, the config-family question, pass stability) remain the
detection work; this design makes them affordable to attack.
## Risks
- `ax` is early ("heavy development", "major breaking changes"); we depend on its shape, not its releases.
- Its provider list excludes DeepSeek; an adapter task image is ours to keep.
- A local reconciler is new code; it must stay thin (manifests, DAG, artifacts, budgets, resume) or it becomes
the next thing that eats tokens.
## Decisions needed
1. Adopt this as a spec (`/kiro-spec-init` on this brief) and pause further script-level pipeline changes.
2. Manifest format: ax's `ax.io/v1alpha1` kinds verbatim plus our `Run` kind, so the later move is a rename of
the reconciler, not of the manifests.
3. First increment: `repo-facts` + `verify` as tasks under the local reconciler, measured against the
46-candidate set for cost and recall before the other concerns move.
+254
View File
@@ -0,0 +1,254 @@
# Design Document
## Overview
A thin service plane under `src/openultrasast/plane/`: manifest schemas in ax's shape plus a `Run` kind, an
ax-backed reconciler that submits a Run's tasks to ax on this host's kind cluster in dependency order with
persisted status and resume, a runner that honours ax's contract and delivers artifacts, and task entrypoints.
ax (with Agent Substrate) is the only executor; there is no local subprocess path. The pipeline's logic lives only in the task entrypoints; the
reconciler and runner know nothing about prompts, models or scoring (Requirement 3.4). The first increment moves
`repo-facts` and `verify` (plus a model-free `agree` step) and measures them on the 46-candidate set
(Requirement 6); the other concerns stay as scripts until that measurement is recorded.
## Architecture
```
plane/ manifests (YAML), checked in
models/deepseek-flash.yaml Model
workspaces/<case>-<pin>.yaml Workspace, generated from population-v2.toml by `ousast plane workspaces`
tasks/repo-facts.yaml Task
tasks/verify.yaml Task
runs/validation-46.yaml Run: DAG, artifacts, budgets
src/openultrasast/plane/
manifests.py load + validate Task / Workspace / Model / Run (PyYAML, declared dependency)
reconciler.py Run execution, status, resume (<= 500 lines, test-enforced)
runner.py ax runner contract: AX_TASK_YAML / AX_WORKSPACES_YAML -> entrypoint
doctor.py `ousast plane doctor`: the four checks of Requirement 3.5
budget.py spend and call metering from provider usage fields
tasks/repo_facts.py source-only facts artifact
tasks/verify.py one hunt per file with facts in the prompt; per-unit resume
tasks/agree.py pass agreement, no model
~/ousast-results/plane/<run>/ state.json + one directory per task with summary.json and artifacts
```
Locally the reconciler runs a task as a subprocess of the same interpreter with the ax environment set; in a
container the same module is PID 1 (`ax-task-runner`), built as a target of the root `Dockerfile` that extends
`openultrasast:dev`. Nothing in a task depends on which of the two started it.
## Components and Interfaces
### Manifests (`manifests.py`)
- `Task`, `Workspace`, `Model`: `apiVersion: ax.io/v1alpha1`, fields exactly as ax documents; unknown fields
are rejected with the path named. `Model.spec.provider` accepts ax's `google` and `anthropic`, and `deepseek`
only when `metadata.annotations["openultrasast.io/provider-extension"] == "true"` (Requirement 1.4): the
deviation is declared, not hidden in another field.
- Field names are ax's verbatim (`ax.proto`; its API server decodes with `protojson`, so an unknown field fails
`ax apply`): `metadata` is `name` and `atespace` (not `namespace`), and a Workspace `git` entry is `name`,
`repo`, `branch`, `dir`, `depth` (no `commit`; a `commit` there is rejected with a pointer to the annotation).
`metadata.annotations` is this project's only extension, and nothing under it reaches ax: the reconciler strips
it and carries it to the task as env. The Model binding (`openultrasast.io/model`) becomes `OUSAST_MODEL` and
`OUSAST_MODEL_PARAMS`. Commit pins (`openultrasast.io/git-commits: "<git name>=<40-hex sha>,..."` on a
Workspace) become `OUSAST_GIT_PINS` = `{"<workspace>/<git name>": "<sha>"}`, which the runner checks out after
fetching the way ax's `internal/workspace/setup.go` does, with the same `dir` and `depth` rules.
- `Run`: `apiVersion: openultrasast.io/v1alpha1`. `spec.tasks[]`: `name`, `task` (a Task manifest name),
`dependsOn[]`, `inputs{name: "<task>/<artifact>"}`, `outputs[]`, `budget{usd, calls}`, `serialize` (a label
for tasks that must not overlap; engine-bearing tasks carry `engine`). The reconciler passes budgets and
artifact paths to a task as `env` entries (`OUSAST_BUDGET_USD`, `OUSAST_BUDGET_CALLS`, `OUSAST_INPUT_<NAME>`,
`OUSAST_OUTPUT_DIR`), which is ax's own mechanism for task configuration, so a Run never adds a field to a
Task. Inputs another task produced are never rendered into a Workspace (ax puts every bound Workspace into one
env value, `AX_WORKSPACES_YAML`, and Agent Substrate rejects values over 32768 characters): the Task carries
`OUSAST_INPUTS` = `{"<NAME>": "<producer>/<artifact>"}`, and after the start request the runner fetches each
from the receiver's `GET /inputs/<producer>/<artifact>` (same headers, dial and retries as delivery; only a
running task's declared inputs are served) to its `OUSAST_INPUT_<NAME>` path; a failed fetch delivers a
failed summary naming the input and the command never runs.
### Runner (`runner.py`)
Reads `AX_TASK_YAML` and `AX_WORKSPACES_YAML`, materialises each workspace's `git` entries at the pinned commit
into the bound path (locally a `git archive` from the case cache, as `evaluate.export` does today), writes
`spec.files`, then runs `spec.command` as ax's runner contract requires (ax `docs/runner.md`,
`docs/sandbox.md`): a child process `python -m openultrasast.plane.tasks.<name> <args>` in its own process group,
with the first workspace as working directory and `AX_METADATA_URL` pointing at the runner's own server. It serves
`/healthz`, `/readyz` (503 until workspaces are ready; ax probes `/readyz?check=workspace`) and ax's two metadata
paths on port 80 (Requirement 4.2). Workspaces are prepared on the first boot only; `/workspace` survives suspend
and resume. After the child exits the runner reads `summary.json` (a crash without one becomes `failed` with the
exit code and stderr tail), delivers the output directory (Requirement 4.4), writes the completion marker
`/workspace/.ousast-state/<task>.done.json` (`exit`, `status`, `delivered`) and then keeps running as PID 1 until
SIGTERM: an exiting runner stops the container, which the first live run showed as every resume timing out. A
boot that finds the marker does not rerun the command (a resumed actor must not repeat billed work); a marker
with a failed delivery retries the delivery only. SIGTERM (ax's stop and suspend) is forwarded to the command's
process group, followed by SIGKILL after `OUSAST_TERM_GRACE` (10 s), and the runner exits 0.
`AX_RUNNER_EXIT_AFTER_COMMAND=1` is a unit-test escape that exits with the task's code after delivery.
The command never starts by itself (Requirements 4.5, 4.6). Once the workspaces are ready the runner serves
`POST /ousast/v1/start` on port 80 with the body `{"run", "task", "credentials"}`: 503 before the workspaces are
ready (the reconciler retries), 409 when `run`/`task` differ from `OUSAST_RUN` and the Task's name, 409 for a
second request on the same boot, 409 with the reason when the completion marker records a delivered run, 400 for
a malformed body or a credential name outside `^[A-Z][A-Z0-9_]*$` (or under `AX_`/`OUSAST_`), and 202 once.
Only then does the command run -- or, when the marker records a failed delivery, only the delivery is retried; a
runner failure (bad manifests, a workspace that cannot be materialised) is likewise delivered only after a start,
or refused with its reason. The credentials stay in the runner's memory and are merged into the child's
environment only; they are never logged or written, and their values are redacted from the child's echoed
stderr and the stderr tail a crash summary keeps. `AX_RUNNER_AUTOSTART=1` (unit tests only; required with
`AX_RUNNER_HTTP=0`) starts without a request.
**Golden snapshot.** Agent Substrate boots every new ActorTemplate once as a temporary "golden" actor (atespace
`ate-golden`, a UUID name) to capture a snapshot. ax bakes `AX_TASK_YAML`, `AX_WORKSPACES_YAML` and `spec.env`
into the template, and inside the sandbox the hostname is always `actor` with no identity variable, so the golden
boot and the real one are indistinguishable from within (observed on the live cluster, 2026-09-29: the command
ran in both). The distinction exists only outside: Substrate's router (`svc/atenet-router` in `ate-system`)
routes on `ate-target-actor: <atespace>/<name>` to the task's actor, resuming it if suspended, and never to the
golden one. Hence the start request, and hence credentials only in it: a template, a rendered Task or an ax
manifest is visible to the golden boot and to anyone who can read the cluster's objects.
### Reconciler (`reconciler.py`, ax-backed)
`ousast plane run <Run.yaml>`, `ousast plane status <run>` and `ousast plane doctor`. State: `state.json` with
per-task status in `pending | running | suspended | done | failed | unfinished`, timestamps, spend. Execution:
topological order; for each ready task the reconciler renders the Task manifest with the run's env
(`OUSAST_BUDGET_*`, `OUSAST_INPUT_*`, `OUSAST_OUTPUT_DIR`, `OUSAST_ARTIFACT_URL`), applies it and its
Workspaces through the `ax` CLI (`ax apply`, `ax resume`, `ax get`, `ax delete`; the CLI path is injectable so
tests use a fake). ax creates a Task `Suspended`; it runs only after `ax resume task`, and the first resume of a
new image can fail with `DeadlineExceeded` while Agent Substrate builds its golden snapshot (ax then reports
`Failed`), so the reconciler retries a DeadlineExceeded/Unavailable resume with backoff for up to
`OUSAST_RESUME_TIMEOUT` (900 s). ax has no Completed phase and never reports that a command exited: completion is
the artifact delivery. `Failed` after a resume went through, or the task disappearing, before delivery fails the
task with ax's condition message; no delivery within `OUSAST_TASK_TIMEOUT` (7200 s) fails it too; `ax delete task`
follows either way. Artifacts: the reconciler serves a small HTTP receiver on the host (reachable from actors
by name through the egress gateway, see Egress below); the runner posts the output directory there after its command exits. Tasks sharing a
`serialize` label never overlap; everything else may run in parallel up to `--workers`. A task's outcome is
read from its delivered `summary.json`: `done` only when the task says every unit finished; `unfinished` on a
budget stop; `failed` on an account or authentication error, a crash, or a missing delivery. `status` prints
the token attribution table (Requirement 7.3) from the delivered summaries and writes `attribution.json`. On rerun, `done` tasks are skipped and
the others re-executed; per-unit resume is the task's own duty (Requirement 2.2). No prompts, scoring or model
calls anywhere in this module; a test counts its lines.
Start (Requirements 4.5, 4.6; `router.py`, kept out of the reconciler's line budget): for the whole run the
reconciler holds one `kubectl --context kind-ousast -n ate-system port-forward svc/atenet-router :80` (random local
port read from kubectl's `Forwarding from 127.0.0.1:NNNN`; the child is terminated when the run ends;
`OUSAST_ROUTER_URL` replaces it, as in the tests). When ax reports `Running` after an accepted resume -- again
after a re-resume of a suspended task, where an "already started on this boot" answer means the command
survived and still runs -- it posts the start body with `ate-target-actor: <atespace>/<task>`,
retrying 502/503/504 and connection errors with backoff up to `OUSAST_START_TIMEOUT` (300 s); a 409 or the timeout
fails the task with the response text unless it already delivered. Credentials come from the Model binding: the
bound Model's `spec.secretKey.key` names the variable (`DEEPSEEK_API_KEY` for `deepseek-flash`), read from the
reconciler's own environment (`ousast` loads `.env` at startup); a Model-bound task whose variable is unset fails
before `ax apply`, and a task without a Model gets none. The value is never rendered, logged or written.
#### Egress (`egress.py`)
Every actor connection except DNS goes through Agent Substrate's egress gateway (`atenet-egress`), which denies
an actor without an `EgressPolicy`. Three pieces make the plane work under that rule:
- **Receiver by name.** `ops/ax/receiver-service.yaml.tmpl` (applied by `up.sh`) is a selector-less Service
`ousast-receiver` in `ax-system`, port 80, with an EndpointSlice on the kind gateway at `OUSAST_ARTIFACT_PORT`
(fixed, 18090), where the reconciler binds its receiver. Tasks get
`OUSAST_ARTIFACT_URL=http://ousast-receiver.ax-system.svc.cluster.local/` plus `OUSAST_ARTIFACT_DIAL=<the
Service's ClusterIP>:80` (looked up with kubectl unless set): the actor cannot resolve cluster names, so the
runner connects to the dial address and sends the URL's host as `Host`. The gateway decides the policy on
`Host` but connects to the address the actor dialled (measured: dialling the kind gateway on port 80 was allowed
and then refused upstream), so the dial address is the ClusterIP, which kube-proxy maps to the host.
`OUSAST_ARTIFACT_HOST` set keeps the old direct URL (tests).
- **Per-task policy.** `policy_for` grants `http` on port 80 to the receiver name, and `tls_passthrough` on 443
to the Git hosts of the bound Workspaces' `https` repos and to the bound Model's
`openultrasast.io/egress-hosts` (`api.deepseek.com` for `deepseek-flash`, validated as a plain DNS name).
Nothing else: no IP address or wildcard, and passthrough only -- an `https` rule intercepts TLS and the actor
would need the gateway CA, which ax templates do not mount.
- **Order.** `ax apply` creates the actor; the reconciler writes the policy with `kubectl ate create
egress-policy` (update when one exists), reads it back, waits `OUSAST_EGRESS_SETTLE` (11 s: the gateway caches a
missing policy as deny for 10 s), then resumes. kubectl-ate has no delete verb; Substrate's store deletes the
policy with its actor, so after `ax delete task` the reconciler confirms it is gone.
### Budget (`budget.py`)
Wraps a chat client: sums the provider's usage fields per call (`prompt_tokens`, `prompt_cache_hit_tokens`,
`completion_tokens`), prices them from the Model's parameters, raises `BudgetExhausted` at the ceiling and
`AccountError` on 401/402 or an "insufficient balance" body. A model without prices makes the meter report
`unpriced` and refuse to run under a `usd` budget (Requirement 2.4).
### Task contract (all tasks)
Inputs from `OUSAST_INPUT_*` paths and the bound workspace only; outputs under `OUSAST_OUTPUT_DIR`; progress in a
`units.jsonl` appended per finished unit and read back on start; `summary.json` written last with
`status`, `units_done`, `units_total`, `usd`, `calls`, `usage`. A task never writes `summary.json` with
`status: done` unless `units_done == units_total`.
### `repo-facts` (`tasks/repo_facts.py`)
Source only, no model. For every product file (the `enumerate_source_files` + product filter of
`model_sinks.py`): the functions it defines (the per-language declaration patterns of `sink_candidates.py`),
and for each function name in the candidate list (input `candidates.json`) the call sites in other product
files: `path`, `line`, the enclosing function, capped at 40 per name. Output `facts.json`, keyed by file,
sorted, deterministic for a workspace (Requirement 5.2).
### `verify` (`tasks/verify.py`)
The hunt of `verify_batched.py` (system prompt, one hunt per file, up to 6 candidates, 6 steps, candidate-
anchored sites) moved into the package, with two changes: the prompt lists each candidate's known call sites
from `facts.json` ("Known callers: a.py:12 in handler(), ..."), and each hunt records its tool turns and usage
in `units.jsonl`. `pass` is an env value from the Run, so two passes are two task instances with distinct output
directories. Candidates and triage come from the existing `independent-v2-modelsinks` and batched-check
artifacts for the increment; the `classify` and `triage` tasks are later specs.
### `agree` (`tasks/agree.py`)
Reads the pass outputs, emits `agreed.json` and `disputed.json` by the same rule as `verify_batched.agreement`,
plus the per-candidate cost, turns and the declared-site match (`evaluate.matches_v2`) for the measurement.
The rule is reproduced in the package (`agree.matches_declared`, parity-tested against `matches_v2` on the recorded
reference outputs) and the generator writes the fix ranges it matches against into each case's `case.json`, so
the runner image carries no benchmark tree.
**Measurement.** Requirement 6.2's cost is judged on `cost_per_candidate_with_recorded_triage`: the plane's two
verify passes plus the recorded triage cost of the case, over the candidates before triage, summed across the run.
The reference's $0.022 (`verifier-batched-check-2026-09-29.json`) paid for per-file triage and the hunts while the
plane applies the recorded triage without a model call, so `cost_per_candidate` (plane spend alone) is reported but
not judged; the triage share is derived per file from the recorded spend minus the priced hunt usage (about $0.087
of the $1.01) and labelled as derived in `case.json`. Sites are judged as `declared_sites_matched_agreed` against
`declared_sites_in_set`, the reference's 16 of 20.
The tie-break (Requirement 6.4) appends per case `<case>-vc`, `verify` pass c restricted by `OUSAST_INPUT_ONLY`
to the agree task's `disputed.json` (an empty list is `done` with no model call), and `<case>-final`, `agree` over
passes a, b and c, which decides 2-of-3 where pass c asked and keeps the a/b rule elsewhere, so pass c alone never
decides and `metrics.tiebreak` counts the flips; the first 60 entries stay byte-identical, so a rerun keeps their
`done` state, executes only the 30 new tasks, and pass c's cost adds to `usd_total` for the 6.2 judgement.
## Data Models
- `facts.json`: `{ "<path>": { "functions": ["name", ...], "callers": { "<fn>": [ {"path", "line", "in"} ] } } }`
- `units.jsonl` (verify): one row per file hunt: `{"path", "candidates", "flagged": [...], "turns", "usage",
"usd"}`; rows with `"error"` are re-asked on resume.
- `summary.json`: `{"status", "units_done", "units_total", "usd", "calls", "usage", "reason"?}`
- `state.json` (run): `{"run", "started", "tasks": {"<name>": {"status", "started", "finished", "usd"}}}`
## Error Handling
- Manifest errors stop before any task starts, naming the manifest, field and rule.
- `AccountError` inside a task: the task writes `summary.json` with `status: failed` and the provider's
message, exits 2; the reconciler marks the run `failed` and starts nothing further (Requirement 2.3).
- `BudgetExhausted`: `status: unfinished`, exit 3; dependants are not started; a rerun with a larger budget
continues from `units.jsonl`.
- A crash: `failed`, traceback in `summary.json`; `units.jsonl` is still valid for resume.
- Two runs of the same Run name never share a directory: the reconciler refuses to start a second run while one
is `running` (a lock file with the PID), which also keeps engine tasks to one at a time on this host.
## Testing Strategy
- `tests/test_plane_manifests.py`: valid ax manifests load; an unknown field, a bad provider without the
annotation, and a Run with a cycle are rejected with the field named.
- `tests/test_plane_reconciler.py`: a fake `ax` CLI with ax's real lifecycle (apply -> `Suspended`, a first
resume failing DeadlineExceeded, then `Running` forever, and a scripted `summary.json` delivery only after a
resume went through) drives a DAG: order, `serialize` non-overlap, skip-on-rerun, `unfinished` and `failed`
propagation, resume retry and timeout, `Failed` with a condition message, a vanished task, the task timeout
without delivery, the lock, the attribution table; a
test asserts the module is under 500 lines. `tests/test_plane_ax_live.py` (marker `ax`, skipped unless
`ousast plane doctor` passes) applies one trivial Task on the real cluster.
- `tests/test_plane_runner.py`: the env contract on a fixture workspace; `/readyz` 503 then 200; the command as a
child in its own group in the first workspace; the runner serving on after the command; SIGTERM reaching the
command's group; the completion marker preventing a rerun and retrying a failed delivery alone.
- `tests/test_plane_repo_facts.py`: a three-file tree; callers found across files, none from tests or vendor;
identical output on a second run.
- `tests/test_plane_verify.py`: a scripted client; the prompt carries the callers; `units.jsonl` resume asks
only the unfinished file; a scripted 402 yields `failed`.
- Measurement (Requirement 6): `plane/runs/validation-46.yaml` executed twice (passes a and b); the result
compared with `verifier-batched-check-2026-09-29.json` and recorded in `benchmarks/independent/`, with the
transcript measure of the increment's development session next to it.
@@ -0,0 +1,157 @@
# Requirements Document
## Introduction
The model-driven pipeline (classify sinks, triage, verify, judge, recheck, score) is a set of scripts run from
one long conversation, each with its own resume, budget and pass logic. Measured 2026-09-29: 90% of development
tokens are tool I/O around those scripts, and verification re-derives the same per-repository facts in every
hunt and every pass. This feature restructures the pipeline as a Kubernetes-style service plane at the model
level, in the shape of google/ax: declarative `Task`, `Workspace` and `Model` manifests, a reconciler that
executes them in isolation with resume, and a runner contract. The plane runs on ax itself: a `kind` cluster
with Agent Substrate and the ax controller on this host (maintainer decision 2026-09-29: "switch to ax
completely now"). What ax does not yet carry -- task dependencies, artifacts, token budgets (on ax's roadmap),
a DeepSeek provider -- is carried by our thin `Run` layer and dissolves into ax as ax gains it.
## Boundary Context
- **In scope**: the manifest kinds and a `Run` kind for dependencies, artifacts and budgets; bring-up of ax on
this host (kind + Agent Substrate + ax controller) with a doctor command; an ax-backed reconciler that
submits Tasks to ax (one engine-bearing task at a time) and is the only executor; the runner contract and a
runner image; the first two tasks, `repo-facts` and
`verify`, with the existing scripts as reference behaviour; measurement on the 46-candidate validation set.
- **Out of scope**: a remote or multi-node cluster; contributing providers or budgets to ax upstream (a
follow-on once the increment is measured); changing what the model is asked (prompts, families, pass rules)
except as required to consume `repo-facts`; the judge stage, the config-family question and population v3
(they become tasks later, under their own specs); any change to `evaluate.py`'s protocol scoring.
- **Adjacent expectations**: the independent-population protocols and records stay as they are; `finding_dump`
and the engine path are untouched; `vulnerabilities-over-plumbing` applies -- this is plumbing that must pay
for itself in measured tokens and unchanged or better recall.
- **Target state (maintainer, 2026-09-29)**: this plane replaces HarnessX as the agentic plane; HarnessX is
removed by its own spec (`harnessx-removal`) once Requirement 6's measurement shows value; the `Run` layer
shrinks as ax gains dependencies, budgets and artifacts. No new HarnessX feature is added meanwhile.
## Requirements
### Requirement 1: Manifests are ax's kinds plus a Run kind
**User Story:** As the maintainer, I want the pipeline declared in manifests that ax would accept, so that
moving to a cluster later changes the reconciler and not the pipeline.
#### Acceptance Criteria
1. `Task`, `Workspace` and `Model` manifests use `apiVersion: ax.io/v1alpha1` and only fields ax documents
(`spec.image`, `spec.command`, `spec.env`, `spec.resources`, `spec.workspaces`; `spec.git`, `spec.files`;
`spec.provider`, `spec.model`, `spec.secretKey`, `spec.parameters`).
2. A `Run` manifest (our kind) lists tasks with `dependsOn`, the artifacts each task produces and consumes, and a
per-task budget (`usd`, `calls`); it contains nothing a task needs to execute.
3. A manifest that violates its schema is rejected before anything runs, with the field named.
4. A DeepSeek model is expressed as a `Model` with a provider adapter noted in the manifest; no ax field is
repurposed to carry it.
### Requirement 2: A task is isolated, idempotent and resumable
**User Story:** As the maintainer, I want a stopped or failed run to resume without repeating finished work and
without ever recording partial work as a result.
#### Acceptance Criteria
1. A task reads only its declared inputs and the bound workspace, and writes only its declared artifacts.
2. Work is recorded per unit (file, candidate) as it completes; a restarted task skips completed units and
repeats unverified ones.
3. A task whose model calls fail on account or authentication errors stops at once with a distinct status;
its artifacts are marked unfinished, never complete.
4. A task's spend is metered from the provider's usage fields and stops the task at its budget; a task with no
priced model reports "unpriced", not zero.
### Requirement 3: An ax-backed reconciler executes a Run on this host's ax
**User Story:** As the maintainer, I want to run a whole pipeline with one command on the laptop, with every
task executed by ax on the local cluster, so that the same manifests move to a bigger cluster unchanged.
#### Acceptance Criteria
1. The reconciler submits each task to ax as a `Task` (with its bound `Workspace`s and `Model`) in dependency
order, at most one engine-bearing task at a time, reads task outcome from ax's status plus the task's
delivered `summary.json`, and passes artifacts between tasks by placing a producer's outputs where the
consumer's manifest expects them. It never executes a task as a local subprocess; ax is the only executor.
2. Task status (`pending`, `running`, `suspended`, `done`, `failed`, `unfinished`) is persisted and shown by
one status command.
3. A suspended or interrupted run resumes from persisted status (Requirement 2.2) with the same command.
4. The reconciler is under 500 lines and holds no pipeline logic: no prompts, no scoring, no model calls.
5. `ousast plane doctor` verifies the host's ax in one command: kind cluster reachable, Agent Substrate control
API up, ax controller ready, runner image loaded; each check names what is missing and how to bring it up.
6. Bring-up on this host (kind + Agent Substrate + ax) is scripted, documented with its measured memory and
CPU footprint, and smoke-tested with one trivial Task before any pipeline task runs.
### Requirement 4: The runner contract is ax's
**User Story:** As the maintainer, I want each task entrypoint to run unchanged inside an ax task container
later.
#### Acceptance Criteria
1. An entrypoint receives its task and workspace manifests as YAML in `AX_TASK_YAML` and `AX_WORKSPACES_YAML`
and takes nothing else from the command line except what the manifest's `command` states.
2. The runner image serves `/healthz` and `/readyz` on port 80, `/readyz` returning 503 until workspaces are
materialised, as ax's runner contract requires; the image contains this package and is loaded into kind.
3. One fixture test per task type runs the entrypoint against a small workspace and checks its artifacts.
4. Because ax has no artifact channel, the runner delivers the task's output directory (including
`summary.json` and `units.jsonl`) to the reconciler's artifact endpoint named in `OUSAST_ARTIFACT_URL`
before it reports completion; a delivery failure is reported as `failed`, never as done.
5. The command never runs in Agent Substrate's golden-snapshot boot, which executes the same image with the same
task environment (observed on the live cluster, 2026-09-29): the runner prepares workspaces, reports ready and
waits; the reconciler starts the command with one request to the task's actor through Substrate's router
(`ate-target-actor: <atespace>/<task>`), which never addresses the golden actor. A second start request for
the same task is refused.
6. Provider credentials reach a task only in that start request, held in the runner's memory and passed to the
command's environment; they never appear in a manifest, a rendered Task, an ax template or any file.
### Requirement 5: `repo-facts` is the memory layer
**User Story:** As the maintainer, I want per-repository facts computed once and reused by every hunt and pass,
so that verification stops paying tool turns for the same lookups.
#### Acceptance Criteria
1. `repo-facts` produces, without model calls, per product file: the functions defined, and for each candidate
function the call sites in other product files (path and line) -- from source text only, never from git.
2. The artifact is deterministic for a given workspace and is reused across tasks and passes unchanged.
3. `verify` includes each candidate's callers from `repo-facts` in its prompt and reports the number of tool
turns per hunt.
### Requirement 6: The first increment is measured before the rest moves
**User Story:** As the maintainer, I want proof that the restructuring saves tokens without losing recall
before the remaining concerns are converted.
#### Acceptance Criteria
1. `repo-facts` + `verify` run on the 46-candidate validation set through the reconciler on ax, two passes, and
report cost per candidate, tool turns per hunt and declared sites agreed.
2. Acceptance: declared sites agreed at least 16 of 20 (the current value) and cost per candidate below the
current $0.022 for two passes; otherwise the increment is reported as not met and the design revisited.
3. The development transcript measure (`benchmarks/dev/token_report.py`) is run at the end of the increment and
recorded next to the result.
4. After the first measurement (plane-increment-1.json: 14/20 agreed, 19/20 in either pass), agreement is decided
2-of-3 with a third pass run only on candidates passes a and b dispute; the gate of 6.2 is re-evaluated on that
decision, and the result is recorded as plane-increment-2.json next to the first.
### Requirement 7: Routing is a declared concern of the plane
**User Story:** As the maintainer, I want which model serves which concern to be a manifest decision with a
budget, not a prompt trick inside one conversation, so that token spend is structured by component.
#### Acceptance Criteria
1. Every `Task` in a `Run` binds exactly one `Model`; a Run may bind different Models to different tasks
(a cheap model for reads and triage, a stronger one for verification and judgement), and the reconciler
passes the bound Model's identity and prices to the task.
2. A task never chooses or switches its model at runtime; the Model it was bound to is recorded in its
`summary.json`.
3. `ousast plane status <run>` prints a token attribution table: one row per task and Model binding with
calls, input tokens, cache-hit tokens, output tokens and USD, a total row, and a per-Model subtotal; every
number comes from the provider's usage fields recorded in the tasks' `summary.json`, never estimated. A
`--units` flag breaks a task down per unit (file or candidate) from `units.jsonl`. The same table is
written to `attribution.json` in the run directory so runs can be compared.
4. A task that needs a second model for a sub-step (a judge over a verifier's output) is a separate task with
its own Model binding, never a second client inside one task.
+22
View File
@@ -0,0 +1,22 @@
{
"feature_name": "ai-service-plane",
"created_at": "2026-09-29T15:40:00Z",
"updated_at": "2026-09-29T17:55:00Z",
"language": "en",
"phase": "implementation",
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": true
}
},
"ready_for_implementation": true
}
+146
View File
@@ -0,0 +1,146 @@
# Implementation Plan
Order follows the dependency chain: ax bring-up (parallel with 1-2) -> manifests -> budget -> runner ->
ax-backed reconciler -> tasks -> increment run on ax -> measurement. Every task ships with its tests; a task is done when its tests pass in the host suite, ruff and
mypy are clean, and the commit is gated on pytest's own exit code. Script-level pipeline work stays paused.
- [x] 0. ax bring-up on this host
- Install `kind`, `kubectl`, the `ax` CLI and `ko` (user-local). `ops/ax/up.sh`: create the kind cluster and
install Agent Substrate with its `hack/create-kind-cluster.sh` and `hack/install-ate-kind.sh
--deploy-ate-system`, deploy the ax control plane with `ko` to the kind registry, apply one trivial Task
and wait for it to finish; `ops/ax/down.sh` tears it down. Record the measured idle footprint (RSS, CPU)
in `ops/ax/README.md` next to the 7 GB host constraint.
- `ousast plane doctor` (in `reconciler.py` or a sibling `doctor.py`): the four checks of Requirement 3.5.
- Egress: `up.sh` applies Substrate's agentgateway egress gateway (`atenet-egress`, prebuilt image, rendered
with `kubectl kustomize`) when absent, and the receiver Service `ousast-receiver.ax-system`
(`receiver-service.yaml.tmpl`, endpoint the kind gateway at port 18090). The reconciler writes each task's
egress policy (`egress.py`: receiver over HTTP, Git and Model hosts as TLS passthrough) between `ax apply` and
resume; the runner delivers to `OUSAST_ARTIFACT_DIAL` (the Service's ClusterIP, port 80) with the receiver's name as
`Host`.
- Stop rule: if the cluster does not come up within one working session, report the exact failure and stop;
the decision to work around it is the maintainer's.
- _Requirements: 3.5, 3.6_
- [x] 1. Manifest schemas and validation
- Add `pyyaml` as a declared dependency in `pyproject.toml`.
- `src/openultrasast/plane/manifests.py`: dataclasses and loaders for `Task`, `Workspace`, `Model`
(`ax.io/v1alpha1`, ax's fields only) and `Run` (`openultrasast.io/v1alpha1`: tasks, dependsOn, inputs,
outputs, budget, serialize); reject unknown fields, a provider outside ax's two without the
`openultrasast.io/provider-extension` annotation, and a Run with a cycle or an input naming no task.
- `tests/test_plane_manifests.py`: valid manifests load; each rejection names the field.
- _Requirements: 1.1, 1.2, 1.3, 1.4_
- [x] 2. Budget metering
- `src/openultrasast/plane/budget.py`: a client wrapper that sums provider usage per call, prices from the
Model's parameters, raises `BudgetExhausted` at the ceiling and `AccountError` on 401/402 or an
"insufficient balance" body; an unpriced model reports `unpriced` and refuses a `usd` budget.
- Tests with a scripted client: ceiling stop, account error, unpriced refusal, cache-hit pricing.
- _Requirements: 2.3, 2.4_
- [x] 3. Runner contract
- `src/openultrasast/plane/runner.py`: read `AX_TASK_YAML` / `AX_WORKSPACES_YAML`; materialise `git`
entries at the pinned commit from the local case cache and write `spec.files` (first boot only); run
`spec.command` as a child `python -m openultrasast.plane.tasks.<name>` in its own process group, cwd the
first workspace, `AX_METADATA_URL` set; `/healthz` `/readyz` and ax's metadata paths on port 80 (503 until
workspaces are ready; disabled only under `AX_RUNNER_HTTP=0` for unit tests); after the command exits,
POST the output directory as a tar stream to `OUSAST_ARTIFACT_URL` (report `failed` if delivery fails),
write `/workspace/.ousast-state/<task>.done.json`, and keep serving until SIGTERM (ax's contract for PID 1);
a boot with the marker never reruns the command and retries only a failed delivery; SIGTERM goes to the
command's group, SIGKILL after `OUSAST_TERM_GRACE`, exit 0. `AX_RUNNER_EXIT_AFTER_COMMAND=1` (tests only)
exits after delivery with 0 done, 2 failed, 3 unfinished. The command never starts by itself: after the
workspaces are ready the runner waits for `POST /ousast/v1/start` (`{run, task, credentials}`; 503 before
ready, 409 on a run/task mismatch, a second request, or a marker recording a delivered run; 202 once),
merges the credentials into the child's environment only and redacts them from its echoed stderr, so
Agent Substrate's golden-snapshot boot runs nothing (Req 4.5, 4.6). `AX_RUNNER_AUTOSTART=1` (tests only)
starts without a request. `plane/Dockerfile.runner`: this package on Python 3.12, installed to
`/usr/local/bin/ax-task-runner`, loaded into kind by `ops/ax/up.sh`.
- `tests/test_plane_runner.py`: the env contract on a fixture workspace; readyz 503 then 200; delivery to a
local receiver; delivery failure yields `failed`; the child's cwd and `AX_METADATA_URL`; the runner stays
up after the command; SIGTERM kills a long-running command's group; the marker prevents a rerun; no command
before a start request, 409 on a wrong run/task and on a second start, the credential reaches the child
(its hash in the artifact) and no file or log record, a crash printing it is redacted, and of two runners
with identical env only the addressed one runs (the golden scenario).
- _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6_
- [x] 4. ax-backed reconciler and CLI
- `src/openultrasast/plane/reconciler.py` (under 500 lines, no pipeline logic, no local subprocess
executor): `state.json`, statuses, topological submission through the `ax` CLI (path injectable),
`ax resume task` after `ax apply` (a created Task is `Suspended`), retrying DeadlineExceeded/Unavailable
with backoff up to `OUSAST_RESUME_TIMEOUT`; completion is the artifact delivery, never an ax phase (ax has
none for a finished command); `Failed` or a vanished task before delivery, or no delivery within
`OUSAST_TASK_TIMEOUT`, fails the task with ax's condition message; the artifact HTTP receiver, `serialize` labels never overlapping, `--workers`, budgets and
artifact env passed in the rendered Task manifest, outcome read from the delivered `summary.json`, skip
`done` on rerun, PID lock per run, `ax delete` of finished Tasks. Once ax reports `Running` after a
resume, one start request per resume through Agent Substrate's router (`src/openultrasast/plane/router.py`:
`kubectl port-forward svc/atenet-router` for the run, or `OUSAST_ROUTER_URL`; header
`ate-target-actor: <atespace>/<task>`; 502/503/504 and connection errors retried up to
`OUSAST_START_TIMEOUT`, a 409 final), carrying the bound Model's `secretKey.key` variable from the
reconciler's environment; a Model-bound task whose variable is unset fails before `ax apply`.
Task-produced inputs travel as `OUSAST_INPUTS`, not as a rendered Workspace: the runner fetches them after
the start from the receiver's authorised `GET /inputs/<producer>/<artifact>` (a 182 KB `facts.json` in
`AX_WORKSPACES_YAML` exceeded Substrate's 32768-character env limit on the live cluster, 2026-09-29).
- `ousast plane run <Run.yaml>`, `ousast plane status <run>` (token attribution table per task and Model
from the tasks' `summary.json` usage; `--units` from `units.jsonl`; also written to `attribution.json`),
`ousast plane workspaces <population.toml>` (generates Workspace manifests per case pin).
- `tests/test_plane_reconciler.py`: a fake `ax` CLI with ax's real lifecycle (Suspended, a first resume
failing DeadlineExceeded, Running without a Completed phase) drives a DAG: order, non-overlap,
skip-on-rerun, `unfinished` and `failed` propagation, resume retry and timeout, `Failed` with a condition
message, the task timeout without delivery, the lock, the attribution table, and the
line-count assertion; with a fake router: one start per task after the accepted resume and before the
delivery, the credential in the verify start only and in no file under the run or tmp dirs, a missing
secret failing the task, a refused start failing it. `tests/test_plane_router.py`: port parsing from a
fake kubectl and its termination on exit, retry on 503/502 then 202, 409 raised with redacted text, the
start timeout. `tests/test_plane_ax_live.py` (marker `ax`, skipped without a live cluster): one
trivial Task end to end.
- _Requirements: 3.1, 3.2, 3.3, 3.4, 2.2, 4.5, 4.6, 7.1, 7.3_
- [x] 5. `repo-facts` task
- `src/openultrasast/plane/tasks/repo_facts.py`: product files, functions per file (per-language
declaration patterns), call sites per candidate function across product files (path, line, enclosing
function, cap 40), sorted and deterministic; `summary.json` per the task contract.
- `tests/test_plane_repo_facts.py`: three-file tree; cross-file callers found, tests and vendor excluded,
identical output on a second run.
- _Requirements: 5.1, 5.2, 2.1_
- [x] 6. `verify` and `agree` tasks
- `src/openultrasast/plane/tasks/verify.py`: the batched hunt moved from `verify_batched.py` (one hunt per
file, up to 6 candidates, 6 steps, candidate-anchored sites) with known callers from `facts.json` in the
prompt, tool turns and usage per hunt in `units.jsonl`, per-unit resume, `pass` from env, budget wrapper
from task 2; `summary.json` never `done` unless every unit finished, and records the bound Model and
the summed usage fields (prompt, cache-hit, output tokens, calls, usd).
- `src/openultrasast/plane/tasks/agree.py`: agreement across pass outputs, per-candidate cost and turns,
declared-site matching via `evaluate.matches_v2`.
- `tests/test_plane_verify.py`: scripted client; prompt carries the callers; resume asks only the
unfinished file; a scripted 402 yields `failed`; agree on two scripted passes.
- _Requirements: 5.3, 2.1, 2.2, 2.3, 7.2_
- [x] 7. Manifests for the increment
- `plane/models/deepseek-flash.yaml` (annotated extension, prices in parameters);
`plane/tasks/repo-facts.yaml`, `plane/tasks/verify.yaml`, `plane/tasks/agree.yaml`;
`plane/runs/validation-46.yaml`: repo-facts -> verify pass a, verify pass b -> agree, budgets per task,
inputs from the recorded candidate and triage artifacts.
- Workspaces generated for the 15 validation cases' vulnerable pins; the runner image rebuilt and loaded.
- _Requirements: 1.1, 1.2, 6.1_
- [x] 8. Measurement of the first increment
- Run `validation-46` through the reconciler on ax (needs DeepSeek credit); record cost per candidate, tool turns
per hunt, declared sites agreed, against `verifier-batched-check-2026-09-29.json`.
- Run `benchmarks/dev/token_report.py` on the increment's development session.
- Record `benchmarks/independent/plane-increment-1.json` with the gate result (met or not met) and commit;
if not met, revise the design before task 9.
- _Requirements: 6.1, 6.2, 6.3_
- [x] 8.1 Tie-break pass and re-measurement
- `verify` takes `OUSAST_INPUT_ONLY` (pass c on the disputed candidates only); `agree` takes
`OUSAST_INPUT_PASS_C` and decides 2-of-3 where pass c asked, the a/b rule elsewhere (`metrics.tiebreak`).
- `validation-46` gains `<case>-vc` and `<case>-final` per case, appended so the first 60 entries keep their
state; an offline replay with an empty pass c reproduces the recorded 14/20.
- Rerun `validation-46` on ax (only the new tasks run), re-evaluate the 6.2 gate on the final decision and record
`benchmarks/independent/plane-increment-2.json` next to the first.
- _Requirements: 6.2, 6.4_
- [x] 9. Handover
- `ops/README.md`: how to run a Run locally and what an ax deployment would need (Kubernetes, Agent
Substrate, the runner image, the provider extension); memory note updated; scripts that the tasks replaced
marked as reference-only in their docstrings.
- _Requirements: 4.2_
@@ -1,5 +1,7 @@
# Requirements Document
> **Status 2026-10-02:** dormant. Tasks were approved but none of the 24 was started; the absence-bug case it targets is handled in practice by the dominance arbiter (`model/dominance.py`) and `semantic/obligations/`. Reopen it only with a measured gap the arbiter misses.
## Introduction
AppSec engineers and maintainers of OpenUltraSAST scan web code, increasingly vibe-coded or agent-written, in which the dominant bugs are absences rather than flows: a handler queries a user-owned record without constraining it to the caller, a privileged route has no guard on its path, an identity is read from the request body instead of the authenticated context, a security control stays at its permissive default. Today the detector, the overlay, and the variant search model only source-to-sink flows, so these pairs score zero and cannot teach a mechanism (6 of 19 non-teaching vibe-py pairs; 15 of 29 agent-vfc rows carry absence labels). This spec adds obligations: an operation that must be discharged by a guard, an identity constraint, or a safe value before it is reached. The application's own policy, recovered from the consistency of its sibling handlers and optionally declared in a versioned file, says which operations carry which obligations. Violations are reported with the missing discharger and its evidence, measured leave-one-out on the absence pairs, and admitted into the detector through the existing improve-loop gate.
@@ -1,5 +1,7 @@
# Requirements Document
> **Status 2026-10-02:** superseded by `model-grounded-detection` (spec.json `superseded_by`). 4 of 20 tasks were done and the tasks were never approved; none will be. Kept as decision history; nothing in src, tests or CI points here.
## Introduction
The detector's instability is structural, not statistical. Measured on 2026-09-06 over twelve injection pairs at
+32
View File
@@ -51,6 +51,38 @@ for someone who does not have Joern.
- Building a sandbox. Req 10 of the predecessor stands: adopt, never build.
- Changing the gates, the taxonomy, or the fact-table schema.
## The fact tables are an ontology, and it is general-purpose — 2026-09-21
The `[[source]]`, `[[sink]]`, `[[sanitizer]]`, `[[operation]]`, `[[discharger]]` and `[[layout]]` rows
are not a pattern list. They are an ontology of program elements and their security relations: what
carries untrusted input, what is dangerous under which weakness class, what discharges which
obligation, what a value's shape has to be for a call to mean what the table says. The engine reasons
over that ontology; it is where nearly all of this tool's accuracy lives, and the 2026-09 family
census measured that directly. Five of seven families established nothing, and every one of them
failed for an ontology reason with the flow engine ready and nothing to aim at: no HTTP client for a
server-side request forgery, no collection read for an object-reference defect, no WordPress spelling
for a deserialization, and only browser sinks for a server-rendered injection.
**This is a general-purpose pre-commit hook, not an application-specific one.** The ontology describes
LIBRARIES, FRAMEWORKS and LANGUAGE CONSTRUCTS. It does not describe repositories. That distinction is
the difference between a tool that transfers and a set of rules that happen to fit whatever was last
measured, and it constrains how the ontology may grow:
- An entry names an API and is justified by what that API DOES. `needle.get` is admissible because
needle is an HTTP client and the argument is a destination, not because NodeGoat calls it.
- An entry carries provenance and a weakness class, so a reader can check the claim without trusting
the person who added it.
- An entry must not encode a path, directory name or identifier peculiar to one project. A plugin's
`includes/lib/` was measured and declined on exactly this ground: it would have fixed one subject and
claimed something false about every other.
- A measured gap is a CANDIDATE, never an answer. The coverage instrument below reports what a
repository calls that nothing models; each candidate must then be resolved to a library API with
provenance before it enters the ontology. Accretion by subject is how a general tool becomes a
fixture of its benchmark.
The corollary is that ontology growth should be driven by measurement rather than by whatever
repository is in front of the maintainer, which is what task 7.1 exists to make possible.
## Boundary Commitments
### This Spec Owns
@@ -89,3 +89,19 @@ The partial-census correction now preserves actual counts and missing filenames,
with missing/empty/malformed census still preventing completed questions. Runtime amortization and VAmPI context/witness evidence remain open.
Source: [Joern 4.0.625 AstGenRunner](https://raw.githubusercontent.com/joernio/joern/v4.0.625/joern-cli/frontends/jssrc2cpg/src/main/scala/io/joern/jssrc2cpg/utils/AstGenRunner.scala).
## 2026-09-14 post-census measurement and lifecycle experiment
The full new profile establishes compatible graph/query reuse on identical tips
(18.521–18.874s), but all 18 other transactions time out and 0/24 target checks complete.
The isolated local-session census probe is promising: four warm requests take 0.221–0.467s
versus 14.292s for disposable execution, with confirmed hanging-operation cancellation
and process-group reaping in 0.043s. Startup and load still cost 12.805s and 3.636s.
The first transport attempt correctly failed on HTTP success without a response payload;
all attempts are retained. No production session integration or query equivalence is claimed.
[Remediation research](release-remediation-research.md) records the exact results and
the shared context-contract gaps: missing non-taint family producers, global vendor and
line-correspondence vetoes, and unresolved query-to-operation identity. Preserve the gaps
until actual question-owned dependency/correspondence evidence can resolve them. Task 8.3
and independent qualification remain blocked; the alpha's eligibility remains disabled.
@@ -0,0 +1,174 @@
# Release remediation research — 2026-09-14
Status: investigation under the maintainer's instruction to continue towards release.
This record does not approve rollout, replace the existing design, or mark a production
remediation complete. The published candidate is `v1.2.0-alpha.1`, commit `04f8880`.
## Summary
Two independent constraints remain after task 2.19 restored Node's complete named census:
engine lifecycle cost and the scope of completeness evidence. A faster engine cannot
repair missing context; suppressing completeness gaps cannot prove useful detection.
Both belong in the existing contributor-scan boundary, with comparison/admission changes
owned by pre-push-safety-net. The ranker continues to select all analysis work.
## Engine lifecycle investigation
The earlier task 2.3 decision rejected server mode on the measured large C workload:
evaluation dominated its elapsed time. That decision remains valid for that measurement.
It does not establish the best lifecycle for small, frequently changed API repositories.
The fresh Node priming transaction reopened the declared inputs and completed in 338.157s.
Its two frontends took 5.536s and 5.451s; overlays 15.779s and 15.239s; standalone census
13.716s and 13.708s. Each of eight query invocations took another 13.264–22.783s. These
are individual process receipts, not estimates of exclusive JVM startup time. The remaining
transaction time includes Python orchestration, cache publication and other work and must
not be attributed to the JVM without a separate measurement.
The first identical-tip replay took 18.874s with two graph hits, 1,512 query hits, and no
engine subprocesses. Thus compatible persistence now works on this workload. Every head
question still has incomplete change context; base questions have `graph_incomplete` from
the vendor boundary described below. Identical-tip timing is not changed-code qualification.
Joern documents a local interpreter server with synchronous and asynchronous query APIs:
[official server contract](https://docs.joern.io/server/). This supplies a candidate transport,
not a sandbox. Before adopting it, probe the actual pinned 4.0.625 image in a disposable,
network-disabled container with no published ports, a private graph copy and process-group
cleanup. Compare an actual named census against disposable script execution, record startup,
load and repeated request times separately, and test cancellation of an in-flight request.
A census microbenchmark cannot establish security-query equivalence or hook p95.
### Completed lifecycle probe
[Full receipts and all three instruments](../../../benchmarks/measurements/2026-09-14-joern-session-lifecycle.json)
retain the failed first attempt as well as both successful file-receipt attempts.
The first attempt received HTTP `success=true` with an empty stdout for the census;
the `println` payload went to the process log. It failed rather than interpreting that
response as a zero. The repaired instrument writes a fresh private result file per
request and checks a unique receipt before accepting its census.
The final attempt reopened all 44 source files (95,411 bytes), verified the cached graph
digest, loaded a private copy, and matched names, counts and overlays against disposable
execution. It confirmed entry into a deliberately hanging operation before cancellation,
then verified the entire process group disappeared. The cached graph remained unchanged.
| Final attempt phase | Observed seconds |
|---|---:|
| Disposable census invocation | 14.292 |
| Server startup through usable request | 12.805 |
| Graph load | 3.636 |
| First census request | 1.402 |
| Four warm census requests | 0.339, 0.467, 0.353, 0.221 |
| Kill and process-group reaping | 0.043 |
Decision: a transaction-owned local session is a promising candidate for the owner
implementation. The adapter needs explicit request receipts; HTTP status alone is not
an answer contract. This probe does not cover full taint/dominance/config equivalence,
multiple graph epochs, concurrent transactions, overlays in-session or production
failure recovery. It neither overturns the large-C evaluation result nor proves 30s p95.
### Completed full runtime profile
The [post-census scorecard](../../../benchmarks/measurements/2026-09-14-nodegoat-post-census-feasibility.json)
and its compressed 115-file evidence bundle retain all 21 planned transactions plus
priming, raw scans, process traces, CLI output and exact instruments. All 22 scan artifacts
match the frozen runtime/core provenance; recomputing the scorecard reproduced its result.
There were no missing cases or instrument failures. Eighteen transactions timed out;
the three identical-tip cases took 18.521–18.874s. Completion remains 0/24 target checks;
maximum observed cancellation/report overrun was 0.8732s. Runtime and rollout remain NO-GO.
Verification: seven runtime-scorecard tests pass; JSON/bundle round-trip, hashes, population,
provenance and scorecard recomputation pass. Production source is unchanged from the
released alpha; this is evidence validation, not a new full implementation verification.
## Completeness diagnosis
### Missing family context producer
`model/scan.py:_evidence_pass` dispatches only taint queries and records their context by
the exact family-specific question identity. `affected_context` requires `context_summary`
and readable method extents for every question. Neither `dominance.sc` nor `config.sc`
exports that contract. Consequently access-control and configuration questions cannot
complete change context through the production path. This is deterministic missing
integration, not evidence that the ranker selected the wrong region.
Do not copy taint context into other families: dominance deliberately evaluates sibling
operations across a file; configuration has different scope; source, sink and guard
summaries have different obligations. A producer must describe the actual scope of the
query it accompanies. Missing extents, unresolved external calls and unsupported reasoning
remain explicit. Context relationships alone never prove an exploitable flow or guard loss.
### Exclusion is currently a global completion veto
`model/partitions.py` records `vendor_semantics_unresolved` when it excludes any vendor
path. `model/scan.py` includes that reason in `graph_gaps`, changing every otherwise
completed outcome to `graph_incomplete`. In the fresh Node prime, this is the only base
degradation, despite a complete 44-file first-party census and successful query receipts.
Vendor exclusion must remain physical. The missing contract is which question actually
requires excluded semantics. Distinguish input-integrity failures, which invalidate the
affected graph, from an excluded dependency whose relevance needs query evidence. Only
an explicit, complete dependency proof or versioned summary can discharge that question's
boundary; lack of a recorded dependency is not such proof.
### Line ambiguity propagates beyond the affected correspondence
`push/snapshot.py` conservatively omits non-unique unchanged lines and records a file-level
`ambiguous_line_correspondence`. `model/regions.py:affected_context` inherits any existing
boundary into every question. `push/policy.py:compare_evidence` also rejects a candidate
when any transaction context boundary exists. Repeated unchanged lines can therefore
invalidate an otherwise distinct, uniquely mapped operation elsewhere.
Retain ambiguous mappings as unavailable. A future repair must bind the required base/head
correspondence to the specific operation and dependency context, with absent, renamed,
removed-guard and ambiguous-operation controls. Aggregate coverage must retain every gap.
Do not delete the ambiguity check or treat lexical alignment as a semantic proof.
Comparison currently also requires all selected base questions to complete, no base
degradations, and no non-tier-zero deferred question. That prevents a complete target
comparison from surviving an unrelated unsupported family. Any amendment must establish
the complete base counterpart and its required scope explicitly; a missing raw finding
in an incomplete base scan cannot establish novelty. This change belongs to the push
comparison policy, consuming evidence ownership supplied by the shared scan contract.
### Query identity and witness identity differ
The Node contribution target is selected through a file-fallback injection question
(`function=None`), while raw rows report `<lambda>2` at lines 32–34. The declared benchmark
target is `handleContributionsUpdate`. VAmPI similarly has a raw SQLI site without an exact
function target question. The benchmark correctly refuses exact-target completion today.
Production normalization must carry verified query-to-operation provenance; do not inject
benchmark function names into regions or change the denominator to count coincident sites.
## Bounded remediation sequence
1. Finish and preserve the unchanged seven-class runtime profile, including all failures.
Run the isolated lifecycle probe afterwards so the experiments do not contend for CPU.
2. Use those results to choose one engine lifecycle amendment. If a session is promising,
keep it transaction-owned inside the existing backend: one deadline, serialized requests,
explicit graph identity/epoch, private leases, failure poisoning, kill-and-reap cancellation,
and fresh census on each answer. No resident laptop service is implied. Compare every
shipped query kind with the disposable path before any production switch.
3. Specify question-owned completeness across scan and comparison boundaries. First add
authentic family context producers, then dependency/correspondence ownership, with paired
PHP/JavaScript normalized controls and VAmPI regressions. Unknown ownership remains a
blocker. A family that cannot provide this evidence stays experimental.
4. Refreeze runtime and semantic identities. Rerun changed-code latency/completion and the
independent reviewed vulnerable/fixed/benign populations together. Regenerate eligibility
only after the existing joint gates pass; then repeat packaged acceptance validation.
This sequence separates runtime transport from evidence semantics. It adds no detector,
second IR, language-specific rank boost, vendor graph expansion or differential CPG mutation.
## Verification needed for implementation
| Boundary | Required evidence |
|---|---|
| `cpg/backend.py`, candidate local session transport | Actual pinned launch/API; full query equivalence; malformed/absent response; switched graph isolation; concurrent transactions; deadline, crash and orphan cleanup |
| `cpg/queries/{taint,dominance,config}.sc`, `model/scan.py` | Context from actual query scope; authentic per-family methods; answered-empty distinct from unavailable; unchanged rank decisions |
| `model/contracts.py`, `model/regions.py`, `model/partitions.py` | Explicit gap ownership; first-party input failures remain blocking; dependent vendor/unknown paths unresolved; unrelated exclusion never silently proves completeness |
| `push/snapshot.py`, `push/policy.py` | Exact operation correspondence; guard deletion; same-named and ambiguous operations; incomplete base cannot establish novelty; aggregate partial coverage retained |
| Evaluation and eligibility | Frozen identities and populations; unchanged 30s/2s and precision/recall/completion gates; no capability enabled by a microbenchmark |
Production changes require task-local review and fresh ordinary scan gates, runtime smoke,
and downstream acceptance evidence. This research record is not that verification.
+89
View File
@@ -1210,3 +1210,92 @@ checkout that can actually demonstrate detection.
pre-push-safety-net task 3.4: pinned JavaScript frontend filters remove declared
first-party tests. The maintainer approved its dependency adaptation and acceptance checks;
task 2.18 now implements it before downstream partition completion.
## Group 7 — Grow the ontology by measurement, not by subject (2026-09-21)
Named by the family detection census and the week of repairs after it. Five of seven families
established nothing and every one failed for an ontology reason, so this group is where detection
efficiency actually lives. The design amendment above states the constraint these tasks work under:
the ontology describes libraries, frameworks and language constructs, never repositories.
- [ ] 7.1 Report what a repository calls that the ontology does not model
- Census the call names, and the receiver-qualified spellings, that appear in a repository's first-party graph; subtract everything any fact table names; rank the remainder by how often it is called and by how close it sits to a modelled source. The output is a measured worklist of what this tool is blind to here.
- It needs no dataflow, which is why it is cheap: the same structural pass that answers 17,700 requests in 44 s already walks every call.
- **It reports candidates, not entries.** A candidate becomes ontology only when someone resolves it to a library API, states the weakness class and records provenance. Nothing may be added because a subject called it.
- Three defects this would have named in one run rather than three investigations: `needle.get` absent while the family found the redirect two files away, `find` absent so the documented object-reference defect raised no obligation, and `maybe_unserialize` absent so the only deserialization site in 1,164 PHP files was invisible.
- Done when the instrument reports a ranked gap list for each measured subject, every entry carries how often it is called, and adding nothing to the ontology changes no result.
- _First run 2026-09-21 on NodeGoat:_ 18 of 175 distinct call names modelled, covering 12% of 1,185 calls. The top product candidate is `res.render`, 23 calls with five in a method reading untrusted input — the same gap task 14.3 reasoned its way to by hand, found here in one run, which is the validation that matters. It also surfaced `Math.random` in the user data-access module: weak randomness is a weakness class this ontology has no family for, so no census could have found it, because the family list is itself the blind spot.
- _It had to be taught what is not an API._ The first run ranked `_tmp_3.push` and `__ecma.Array.factory` first, which is how the frontend lowers a destructuring and an array literal, and a Cypress suite and a Gruntfile held four of the next five. Both are now separated, because noise at the top of a worklist is how a worklist stops being read. Evidence: `benchmarks/measurements/2026-09-21-ontology-coverage-nodegoat.json`.
- _Boundary: shipped semantic facts, evaluation harness_
- _Requirements: 6.2, 8.1_
- [ ] 7.2 Key sanitizers to the weakness they actually prevent
- The sanitizer list is flat across families, and that forces a false choice which is now measured: adding WordPress's `esc_attr` removes eighteen output-encoding false positives on one plugin and simultaneously hides a real SQL injection, because no HTML escaper stops SQL injection. `php.toml` has carried a comment predicting exactly this since before the measurement existed.
- Key each sanitizer to the weakness class it discharges, so a family consults the cleansers that apply to it. Do not solve it by choosing which defect to keep.
- Done when an HTML escaper clears an output-encoding flow and leaves an injection flow standing, on the measured plugin, with both outcomes recorded.
- _Boundary: shipped semantic facts_
- _Requirements: 6.2, 8.1_
- [ ] 7.3 Do not arbitrate a question whose evidence says there is nothing to find
- The structural pass answers 17,700 requests in 44 s; arbitration costs one to two seconds per question, and a 673-file plugin asks 2,526 of them. Gate arbitration on the evidence vector already computed: a question whose scope holds no modelled sink, or no reachable source, cannot produce a flow and does not need the expensive pass.
- This attacks the scale limit from the opposite side to graph reuse and can be sized today from data already on disk, by counting how many of those 2,526 questions had any sink in scope at all.
- Done when the arbitrated count falls without any finding being lost on the measured subjects, and the questions skipped are recorded as skipped rather than as answered.
- _Implemented and measured 2026-09-22, and the mechanism already existed._ Tier-zero pruning defers a question whose evidence vector says no sink of its family is in reach. It was switched off for the WHOLE SCAN whenever any of twelve degradation reasons appeared, and a multi-language repository always carries two of them, so the measured plugin arbitrated every question it could ask. Third instance this week of the same blast radius. A prune blocker that names a partition now withholds pruning from that partition's questions and nobody else's, and `cross_partition_semantics_unresolved` leaves the blocker set entirely for the reason it left the integrity gaps.
- _The arbitrated count falls and no finding is lost._ At 25 regions: 127 questions to 34, with 93 recorded deferred as `tier_zero`, and 19 findings before and after, identical family by family. At 500: 2,526 to 406, with 2,120 deferred. A family whose every question was pruned now reports "no sink of this family in reach" rather than "region offered, no question asked", because nothing to ask is a different statement from never offered.
- **It is not sufficient, and that is the useful half of the result.** The 500-region run still fails in the same way at the same cost, because 406 questions at roughly one and a half seconds each still exceed the 300-second per-batch ceiling. This lever moved the volume; the ceiling is a separate limit and pre-push 13.3 still cannot reach the population that holds its target without it. Evidence: `benchmarks/measurements/2026-09-22-tier-zero-pruning.json`.
- _Boundary: contributor-scan scan budget_
- _Requirements: 4.1, 6.3, 8.1_
- [ ] 7.4 One matcher, used by every query
- Four query files carry near-copies of the same token matcher and each acquired the same defect separately: the configuration matcher needed a receiver check the dominance matcher already had, and the sink matcher searched a whole node's text including its comments. Six instances of one bug class are listed in the pre-push design's invariants.
- Extract one matcher with callee confinement and word boundaries built in, and have taint, dominance and config use it. A rule in a document gets rediscovered; a rule in a constructor does not.
- Done when one implementation is shared, each caller's behaviour is unchanged on the measured subjects, and a new query cannot match text that merely contains a name.
- _Boundary: contributor-scan queries_
- _Requirements: 6.2, 8.1_
- [ ] 7.5 A degradation must declare the scope it reaches
- Five instances of one bug class are listed in the pre-push design's invariants, two of them found this week, and two of those cost the declared target its entire admissibility. A degradation is currently a bare dict, so nothing forces a new reason to say whether it reaches one question, one file, one partition or the whole read.
- Make it a record with a required scope, and let the demotion and the admission policy read that field rather than a hand-maintained membership set.
- Done when a degradation cannot be constructed without a scope, the existing reasons carry theirs, and the measured subjects' outcomes are unchanged.
- _Boundary: contributor-scan degradation vocabulary, pre-push admission_
- _Requirements: 6.3, 8.1_
- [ ] 7.6 Golden findings per subject, so a silent change cannot pass
- Nothing fails today when a change alters what NodeGoat or VAmPI report. Four changes in one week altered detection and each time the census had to be re-run from memory; one false positive was caught by reading a witness rather than by a test.
- Commit an expectation per subject at the finding level, site and family and rung, and diff against it. Keep it honest about what it is: a regression guard on development subjects, never evidence of qualification.
- Done when a deliberate change to a matcher or a fact table fails the diff, and the expectation records the ruleset and engine identities it was taken under.
- _Boundary: evaluation harness_
- _Requirements: 8.1, 8.2_
- [ ] 7.7 Budget a batch by the sinks it implies, not by the requests it holds
- **Named by the 2026-09-22 identification.** A taint batch costs what its matched sink NODES cost, because `reachableByFlows` is asked per sink. Measured at a fixed 100 requests on one graph: SQL injection 147 s, path 194 s, output encoding 704 s — and output encoding declares seven sink names against path's sixteen. What separates them is that `echo` and `print` match thousands of calls in a WordPress plugin.
- The driver currently splits a failed batch into eight parts by request count, which cannot help: each part carries the same everywhere-sinks.
- The count needed is already computed. `taint.sc` memoises its candidate sink list per sink set, and the evidence pass reports the in-scope sink list per request before any dataflow runs. A driver that reads it can size a batch by expected work, and can report a family whose sinks are everywhere as EXPENSIVE rather than as failed.
- Done when a batch is sized by the sink nodes it implies, the measured plugin's arbitration completes within its ceiling, and a family too expensive to arbitrate is reported as such rather than as a timeout.
- _Implemented and measured 2026-09-22._ On the 500-region population where every earlier run produced nothing: **153 of 406 questions completed and 125 findings, inside the deadline**, against 0 and 0. 380 requests carrying 5,807 sink visits split into 11 batches, each answering in 80 to 197 s, none near the 300 s ceiling. Every family that can speak on this subject now does: injection 49 completed and 24 findings, path 64 and 15, output encoding 32 and 70, untrusted destination 8 and 16. Eight batches were left unasked to keep time for arbitration, and that is recorded rather than silent.
- _Four defects, each hidden by the one before it, and three of them the same rule._ Batches sized by request count when cost is per matched sink. Packing heaviest-first, which starved injection — the one family with a true positive here — of all 114 of its questions. Any per-query timeout cancelling the whole scan, abandoning batches with 33 minutes of budget left. And no budget kept for arbitration, which removing that cancellation exposed: rows are not findings, so the query phase ran the full deadline and completed nothing, where the cascade had been an accidental guard.
- _The reserve had to be checked per BATCH._ Checked between query kinds it changed nothing at all, because one kind is a single loop iteration that runs every one of its batches inside it. The granularity of a budget check has to match the granularity of the work it guards. Evidence: `benchmarks/measurements/2026-09-22-sink-weighted-batching.json`.
- _What it does not claim:_ 253 of 406 questions still went unanswered and one batch still records a timeout; the raw finding count is dominated by output encoding, the family already measured to be imprecise on PHP; and the reserve share of a fifth is a first value, not a tuned one.
- _Boundary: contributor-scan scan budget, cpg backend_
- _Depends: 7.3_
- _Requirements: 4.1, 6.3, 8.1_
- [ ] 7.8 Portion the work instead of capping it: a kill keeps what finished
- **Asked for by the maintainer on 2026-09-22, after 7.7.** A hard per-invocation ceiling is all-or-nothing: a batch killed at 301 s loses every answer it had already computed, and the retry then pays a fresh JVM start per part to ask them all again. Measured on the plugin: 116 requests killed at 300 s, eight restarts, and the same everywhere-sinks in every part.
- Three coordinated changes replace the cap as the thing that governs cost. The taint query STREAMS: the census first, then one line per request as it is answered, flushed as it goes, inside the unchanged fence. The driver harvests a killed process's stdout and assembles every complete line, marking the result partial; the retry then asks only the remainder. And portion sizing ADAPTS: after each portion the observed seconds per sink visit, less the fixed start, resizes the rest toward a target duration, and a portion that did not answer halves the budget rather than being fitted. The ceiling stays as a safety net that the sizing is meant to make rare.
- Legacy single-document payloads must keep parsing unchanged; the other queries do not stream and need not. A census line is one whose only key is the census — a first cut misread a legacy document that carried the census beside its answers as a census line and dropped every answer it sat beside, which a control caught.
- Done when a killed taint batch hands back the answers it finished and the retry asks only the rest; portion sizes follow observed cost; the streamed and legacy forms both parse; and the plugin population completes more questions per unit of budget than under 7.7 with no finding lost.
- _Status 2026-09-22:_ implemented; host controls pass for the assembler in both forms and under a kill, for the remainder-only retry, and for the sizer; the joern-gated check passes on the real engine — the taint query streams one line per request, and the comment-sink and literal-destination controls hold under the streamed form. The plugin measurement is running on a frozen export of the committed source.
- _Two things the measurement exposed on the way, both fixed:_ the evidence pass was one unportioned batch protected only by the ceiling — 22,125 requests on a plugin push transaction — and now runs in portions of 4,000; and a first cut of the harvest read a timed-out process's pipe BEFORE killing it, which blocks forever because a pipe whose writer is alive never reaches end-of-file. The suite's own hanging-engine control caught that after twenty minutes of what I mistook for CPU starvation.
- _A stated limit:_ the reuse layer's re-query of `missing` ids calls the backend batch directly and so bypasses sink-weighted portioning. Those re-queries are the small remainder of a cached scan, and streaming still keeps what a killed one finished, so it is recorded here rather than fixed. _Corrected 2026-09-23:_ the limit was stated backwards and was worse. The reuse layer wrapped only `run_batch`, and the portioner asks `run_batch_once`, so every portioned query bypassed the reuse cache — and its source-root normalization — entirely. Fixed under 7.9: both paths go through the cache.
- _End-to-end tuning, 2026-09-23, the honest version._ The first end-to-end run REGRESSED completion (64 of 406 against 7.7's 153) while finding as many defects, and each fix exposed the next: the adaptive sizer read one portion at 953 s because it timed `run_batch`, whose own split-retry folded in, and collapsed; giving the portioner a single-attempt path (`run_batch_once`) fixed the timing but only after it was wired on the PARTITIONED graph too, which every plugin uses and which the first wiring missed; and even then completion sat at 56 of 406 because the portioner iterated a fixed list and dropped the remainders of killed portions, stopping at 1,263 s of a 3,600 s budget. The queue re-asks those remainders. The sizer now adapts both ways in the trace — 268 up to 535 after a cheap portion, halved to 267 to 133 on kills — and re-asks what a kill left. Each step is unit-tested: the assembler in both forms and under a kill, the single-attempt path, coverage-halving, and queue termination. _Final end-to-end 2026-09-23, mixed and told straight._ Against the hard-cap-plus-reserve baseline on the same 500-region PHP run: findings IDENTICAL at 125 either way, completion slightly LOWER under portioning (131 of 406 against 153), wall time higher (2,982 s against 2,338). Portioning does not beat the cap on throughput here; it matches it on what it finds. What it buys is kill-safety -- the maintainer's actual ask -- and reaching the same 125 findings is the evidence that streaming loses nothing the cap kept while no longer discarding a near-complete batch when the ceiling falls mid-answer. Completion did not rise because the requeue re-asks a killed portion's remainder, and on this subject those are the expensive output-encoding questions that keep hitting the ceiling in smaller pieces; telling a transient kill from a genuinely-too-expensive family is the next lever, not more portioning. Neither approach is release-ready regardless: 32-38% completion against a 95% gate, on development data that qualifies nothing. Evidence: `benchmarks/measurements/2026-09-23-portioning-vs-hard-cap.json`.
- _Boundary: cpg backend, contributor-scan scan budget, taint query_
- _Depends: 7.7_
- _Requirements: 4.1, 6.3, 8.1_
- [x] 7.9 Size portions from what each answer cost, and tell a transient kill from an expensive question
- **The next lever named by 7.8, and the trace said it was a different lever.** Re-read on 2026-09-23, the 7.8 run's sizer did not lose completion to expensive families so much as to a death spiral: after two kills the budget fell to 3 sink visits per portion and never recovered. Portions of 3–6 visits finished in 25–30 s, which under the assumed 70 s fixed start read as "no information", so the budget could shrink and never grow — and the rest of the run paid a JVM start per handful of requests.
- The instrument first: the taint query streams each answer's own cost (`ms`), carried as a reserved `__timing__` list through shards, partitions and reuse. The sizer then MEASURES what it used to assume — the fixed start as the wall time the answers do not explain (median over whole portions), and a seconds-per-sink-visit rate per family from that family's own answers. Portions are cut in predicted seconds and grow request by request from the head of the queue.
- A kill names its culprit. Answers stream in request order, so the first unanswered id was running at the kill and every id after it never started: those go straight back to the queue, the culprit is asked alone after everything else, and its family is costed at the lower bound it proved. A request that fails alone after more than a portion's target is `query_too_expensive`, reported apart from `query_failed`.
- Two defects fixed on the way: a scan small enough for one portion took a single attempt and lost whatever a kill cut off (it now goes through the queue); and requests the arbitration reserve held back were reported as `query_failed` although they were never asked (now `query_budget_reserved`).
- _Measured 2026-09-23, same 500-region PHP run, 3,600 s, frozen at `1b28465`:_ **281 of 406 questions completed, against 153 (hard cap) and 131 (7.8)**, in 2,987 s. Findings per family never fell: injection 24→33, output encoding 70→125, path 15→35, untrusted destination 16→31; 224 in total against 125. The declared target `classes/class.memberorder.php:936:getMemberOrderByCode` is found at `model_entailed`. The measured fixed start was 22–24 s throughout. **No portion was killed in the whole run** — every one answered whole — so the gain is the sizing, and the culprit path is covered by host controls but was NOT exercised end to end here.
- _Limits, stated:_ one subject, one budget. The census keeps ten examples per family, so it shows per-family counts did not fall, not that the earlier findings are a strict subset. The extra findings are unadjudicated: output encoding on PHP `echo` is the family most likely to carry false positives, and 224 is a count, not a precision. 99 taint questions were still held for arbitration, and access control, config and prototype were never reached (0 of 39) — the ranker puts them last, and the reserve cuts the tail. Completion is 69%, still short of the 95% M4 gate; this is development data and qualifies nothing. Evidence: `benchmarks/measurements/2026-09-23-measured-portioning.json`.
- _Boundary: cpg backend, taint query, contributor-scan scan budget, push reuse_
- _Depends: 7.8_
- _Requirements: 4.1, 6.3, 8.1_
@@ -1,5 +1,7 @@
# Requirements Document
> **Status 2026-10-02:** complete, 17 of 17 tasks (spec.json phase corrected to `implementation-complete`). Its `[variants]` config keys were retired on 2026-10-02: the variant search they configured (`semantic/variants.py`) had already been deleted.
## Introduction
OpenUltraSAST's labeled pair corpus measures the detector but does not improve it, and its mechanism memory is empty because only sandbox-proven findings may write to it. This spec turns every trusted pair into a mechanism record, adds a structural variant search that proposes matches on any scanned tree, measures the corpus's own detection rate leave-one-out, and gives the improve loop a lever that can raise recall under the existing false-positive ceiling and per-profile holdout gate.
@@ -1,9 +1,9 @@
{
"feature_name": "corpus-seeded-mechanisms",
"created_at": "2026-09-05T06:18:31Z",
"updated_at": "2026-09-05T16:30:00Z",
"updated_at": "2026-10-02T00:00:00Z",
"language": "en",
"phase": "implementation",
"phase": "implementation-complete",
"approvals": {
"requirements": {
"generated": true,
+2
View File
@@ -1,5 +1,7 @@
# Brief: flow-aware-ranking — spend the expensive query where the cheap evidence is strongest
> **Status 2026-10-02:** dormant. Brief only and unapproved; its 2026-09-12 amendment placed ranking inside the pre-push safety net, where the ranker now lives.
**Status:** brief only, unapproved.
## Current amendment — 2026-09-12
+30
View File
@@ -0,0 +1,30 @@
# Brief: remove HarnessX after the plane proves value
Status: brief only, gated. Not started until `ai-service-plane` Requirement 6 records a met gate.
## Inventory (read-only, 2026-09-29)
- Code: `src/openultrasast/harness_ext.py` (97 lines) and `hunter_harness.py` (197) exist only for HarnessX;
guarded branches in `cli.py` (372-512), `fusion.py` (LLM panels, ~100 lines), `verify_judge.py` (judge path,
~90), `stage_processors.py` (`host_under_harnessx`, ~40), `config.py` (`HarnessxConfig`). About 575 of
30,243 lines (1.9%). Every path falls back deterministically when the extra is absent and records a
degradation (`harnessx_extra_unavailable`; fallbacks `run_hunter_pool`, `structural_verifier`,
`deterministic_panels`). `evolve`, `regress`, `benchmark`, `gate` and quick mode do not depend on it.
- Comment-only mentions (re-verified 2026-09-30): `gate.py:174`, `improve/evolve.py:9`, `slot_contract.py:7-13`.
- Packaging: `pyproject.toml` extra `harnessx` pinned to a git commit; mypy override; `uv.lock` entry.
- Tests: 14 files touch it; dedicated `test_harness_ext.py`, `test_hunter_harness.py`, `test_cli_hx_dispatch.py`,
`test_verify_judge.py`; `tests/conftest.py` fixture `assert_cold_of_harnessx`.
- Docs: README sections "Configuring the HarnessX agentic plane" and "The HarnessX self-improving cycle";
`docs/examples.md` section 6; `docs/threat-model.md` egress and spend caps.
- Specs: `harnessx-self-improving-rulesets` (approved, 43/43 done, Requirement 1 is direct adoption);
`learning-harness` (30/33); `three-stage-scan` framed as its follow-up; `openrouter-sast-harness` (no
spec.json); mentions in four more; steering `overview.md:175`.
## Shape of the removal
1. Replace the three guarded paths with plane tasks (hunter pool -> `verify`; judge -> `judge`; panels ->
`judge`/`agree`) so their fallbacks become the only path, then delete the two modules, the config block,
the extra, the override and the fixture.
2. Retire the four specs' HarnessX requirements with a dated note, not a rewrite; update README and docs.
3. Gate: the plane increment met (recall and cost), and the replaced features measured no worse on the
development pins before deletion.
+507
View File
@@ -0,0 +1,507 @@
# Design Document
## Overview
HarnessX leaves the product in one reviewable sequence, and the plane's memory layer takes over what HarnessX was
meant to drive. Three facts from the code shape the design:
- **No HarnessX capability is measured, and one of them never existed.** The LLM paths are reached only through
config keys, not CLI flags (`scan` has `path`, `--mode`, `--config`, `--fail-on`; `cli.py:163-167`), and
`benchmarks/` holds no run of any of them (no `hx-hunter:` finding, no `llm-panels` decision). The
`MetaAgent.evolve` proposer is only a docstring (`improve/evolve.py:8-9`, `harness_ext.py:7-8`). No code calls it.
- **The plane already does the LLM work better, and outside `ousast scan`.** `verify` (two passes, known callers
from `repo-facts`) plus `agree` (2-of-3 with the tie-break) met the ai-service-plane gate
(`benchmarks/independent/plane-increment-2.json`: 16/20 declared sites agreed, $0.0205 per candidate). The
standard-mode LLM hunter that users actually reach, the tool hunter at the MAP stage (`cli.py:631-640`,
`tool_hunter.resolve_hunter_client`), is not HarnessX and stays.
- **Everything deterministic already runs without the extra.** Every guarded branch falls back to a deterministic
function (`run_hunter_pool`, `verify_findings`, `fuse_finding`). Removal therefore makes the fallback the only
path. Quick mode, `benchmark`, `pairs`, `improve` and the gates never enter a guarded branch.
So every HarnessX capability in `ousast scan` is **retired** with its deterministic path kept. The plane's `verify`
and `agree` tasks are the documented agentic path (`ousast plane run <Run>`). The proposer is **carried by the
memory layer**: a persistent store, keyed by repository and pin, that plane Runs write through a model-free
`remember` task. From that store a deterministic generator emits `RuleStatusEdit` proposals into the unchanged
`EvolveValidator` and gate. The improvement loop becomes a plane Run whose only Model-bound tasks are the verify
passes.
## Architecture
```
Before After
ousast scan --mode standard ousast scan --mode standard
hunter_pool: HxScanOrchestrator | run_hunter_pool hunter_pool: run_hunter_pool (only path)
verify: verify_judge (llm-judge) | verify_findings verify: verification.verify_findings
fusion: llm panels | deterministic panels fusion: deterministic panels (fuse_findings)
MAP: tool hunter ([models] hunter, DeepSeek) MAP: tool hunter (unchanged)
harness_ext.py, hunter_harness.py, verify_judge.py deleted
stage_processors.host_under_harnessx deleted; deterministic pipeline kept
[harnessx] config, extra, mypy override, lock one load warning; extra/override/lock gone
improve: benchmark per-rule proposer (+ MetaAgent improve: benchmark proposer + memory proposer (--memory)
"plugs in" -- never implemented)
plane Run (per case)
facts -> va, vb -> agree -> vc -> final (existing)
alerts (new, no model) ------------------+
remember (new, no model) <- facts, final, alerts, va/vb/vc
plane Run (loop tail, once)
measure -> propose -> improve(validate+gate) (no model)
~/ousast-results/plane/memory/ the store: facts by content, rows by repo + pin
```
New code: `src/openultrasast/plane/memory.py` (store, ingest, seed; host side),
`src/openultrasast/plane/tasks/{alerts,remember,measure,propose,improve}.py`,
`src/openultrasast/improve/memory.py` (deterministic proposal rules). None of it is in the reconciler. The
reconciler gains only `mark_done` (below). It is 465 lines today, and `tests/test_plane_reconciler.py:759-761`
keeps it under 500.
## Components and Interfaces
### 1. Fate of each HarnessX capability (Requirement 1)
| Capability | Where today | Fate | What the user loses |
| --- | --- | --- | --- |
| LLM hunter pool (`HxScanOrchestrator`) | `hunter_harness.py:49-185`; gated in `cli.py:400-456` (`hx_hunter` at :405, orchestrator at :438-448, degradation at :452-455) | **Retired.** `run_hunter_pool` becomes the only standard-mode hunter pool. LLM hunting of a repository stays available in two places: the MAP-stage tool hunter in `ousast scan --mode standard/deep` with `[models] hunter` (`cli.py:631-640`, not HarnessX), and plane `verify` via `ousast plane run plane/runs/validation-46.yaml` | `hx-hunter:<path>:<line>` findings from an anthropic/openai/litellm agent (`hunter_harness.py:155`). No run of this path is recorded under `benchmarks/`, so no measured TP is lost. |
| LLM judge | `verify_judge.py:22-63`; gated in `cli.py:406, 499-511` | **Retired.** `verification.verify_findings` is called directly and `verify_judge.py` is deleted (it is only the dispatcher and the judge). The plane's LLM verification is `verify` + `agree` (+ `vc`/`final` tie-break) | Per-finding `plausible/unsupported` verdicts that set `ACCEPTED/REJECTED` (`verify_judge.py:83-88`). Unmeasured. `[models] judge` (model-grounded-detection, `model/endpoint.py:195`) is a different capability and stays. |
| LLM fusion panels | `fusion.py:194-326` (`_fuse_with_llm_panels`, `_run_llm_panels`); gated in `cli.py:519-541` | **Retired.** The deterministic panels (`_panel_a`, `_panel_b`, `_reconcile`, `fusion.py:101-167`) become the only path. `fuse_findings_dispatch` is renamed `fuse_findings(findings, verifications, *, high_assurance, mitigated_ids)`. The plane counterpart of independent cross-examination is `agree` across independent passes with the 2-of-3 tie-break | Steel-man LLM panels (`decision_source: "llm-panels"`). Unmeasured. `fusion.json` for deterministic decisions is unchanged. |
| Deterministic stages hosted under HarnessX | `stage_processors.py:129-156` (`host_under_harnessx`); only caller is the skipped test `test_slot_contract.py:119-133` | **Retired, no replacement.** ax runs whole tasks, so a model-disabled in-loop host has nothing to host. `stage_processors`/`slot_contract` stay as the standalone deterministic pipeline that `test_gate.py:147-148` pins | Nothing. The function has no production caller. |
| LLM evolution proposer (`MetaAgent.evolve`) | docstring only: `improve/evolve.py:8-9`, `harness_ext.py:7-8` | **Carried by the memory layer** (Requirement 6): `improve/memory.py` rules over the store, fed into `run_round` as extra proposals | Nothing implemented is lost. |
No CLI flag is HarnessX-specific, so Requirement 1.3 applies to config keys. A key that asked for a retired
capability makes `load_config` raise `RetiredConfigError(ValueError)`, and the CLI prints the message and exits 2.
A silent downgrade is not allowed:
| Key | Message (abridged) |
| --- | --- |
| `[models] verifier` | "`[models] verifier` was retired with HarnessX on <date>: LLM verification runs on the plane (`verify` + `agree`, `ousast plane run`, see ops/ax/README.md). Remove the key." |
| `[fusion] panel_model`, `[fusion] decider_model` | "... LLM fusion panels were retired; fusion is deterministic; independent LLM agreement is the plane's `agree` task ..." |
`[harnessx]` is different (Requirement 3.3). It only tuned budgets and never requested a capability, so it loads
with **one** `logging.warning` on logger `openultrasast.config` that names the plane ("`[harnessx]` is ignored:
HarnessX was removed; agentic work runs on the ax plane with per-task budgets, see ops/ax/README.md"). The section
is otherwise ignored. `[models] hunter` keeps its meaning (tool hunter, obligations intent adjudication at
`cli.py:1237`), so it gets no message. With the extra installed it no longer starts a second, HarnessX hunter. The
release notes say so.
Requirement 1.2 (moved capabilities have a CLI path, a Model per task and a fixture test) is met by the existing
`verify`/`agree` tasks (`ousast plane run`; Model `deepseek-flash` bound per Task; `tests/test_plane_verify.py`,
`tests/test_plane_scoring_parity.py`, `tests/test_plane_tiebreak.py`), and by the new memory and loop tasks below, each with its own fixture test. No
capability is moved into a new Model-bound task.
### 2. Removal mechanics (Requirement 2)
**Source.**
- Delete `harness_ext.py` (97 lines), `hunter_harness.py` (197) and `verify_judge.py` (130).
- `cli.py`: drop the imports at :44, :46 and :94. Drop `harnessx_present`, `hx_hunter` and `hx_verify` (:400-406),
the orchestrator branch and its degradation (:438-456), and the verify degradation (:499-502). Call
`verify_findings(findings)` at :503-511. Drop `hx_fusion` and its degradation (:521-527). Call
`fuse_findings(findings, verifications, high_assurance=config.fusion.high_assurance)`. The `verifier_model`
local goes away.
- `fusion.py`: drop the import at :24, the `panel_model`/`decider_model`/`provider`/`use_harnessx` parameters, and
`_with_degradation`, `_fuse_with_llm_panels`, `_run_llm_panels` and `_text` (:229-338). Rewrite the module
docstring (:10-14) and update `__all__`.
- `stage_processors.py`: drop `host_under_harnessx` (:129-156) and its `__all__` entry, and rewrite the module
docstring (:10-11). `slot_contract.py:7-14` becomes "the zero-dependency substrate for the deterministic
pipeline".
- `config.py`: drop `HarnessxConfig` (:99-107), `ResolvedConfig.harnessx` (:201), `_load_harnessx` (:345-351) and
its call (:230). Drop `ModelConfig.verifier` (:37), `FusionConfig.panel_model/decider_model` (:123-124) and
their loaders (:266, :366-367). Add `RETIRED_KEYS` and the one-warning rule above. `harness.py:183` keeps the
`"verifier": None` literal in `model_roles`, so `harness.json` keeps its schema byte for byte (the key has always
been `null` in the baseline config).
- Comment-only mentions: `gate.py:174` ("a model-backed stage"), and `improve/evolve.py:8-9`, which now names the
memory proposer.
- `plane/Dockerfile.runner:23` loses "no HarnessX".
**Packaging.**
- Delete the `harnessx` extra and its comment (`pyproject.toml:14-22`) and the `harnessx.*` mypy override and its
comment (:67-71).
- `uv lock` regenerates `uv.lock`. The `harnessx` package (:1062), the extra (:2145-2146), the requirement
(:2169) and `provides-extras` (:2178) go.
- Extras that remain: `semantic`. There is no `all` extra. `pip install .` and `pip install '.[semantic]'` in a
fresh venv are the Requirement 2.2 check.
- No source or test imports a HarnessX transitive dependency. The only third-party imports are `yaml`,
`tree_sitter` and `pytest` (checked with an import scan), so dropping the lock entry cannot break an import.
**Tests** (14 files plus `conftest.py`):
| File | Action |
| --- | --- |
| `test_harness_ext.py`, `test_hunter_harness.py`, `test_verify_judge.py`, `test_cli_hx_dispatch.py` | **Delete.** Each exercises only HarnessX or its dispatcher. |
| `conftest.py` | Rename fixture `assert_cold_of_harnessx` to `assert_cold_import(code)`: a fresh interpreter runs `code`, with no module-name guard. `test_semantic_extra.py:28-35` is its remaining user (the tree-sitter check brings its own assertion). |
| `test_gate.py` | Delete `test_harnessx_stage_stays_within_tolerance_of_baseline` (:124-150) and the `has_harnessx` import (:23). Replace the two "never import" tests (:155-175) with `test_governance_planes_import_only_stdlib_and_declared_dependencies`, an allow-list (stdlib, `openultrasast`, `yaml`) over module-level imports of `GOVERNANCE_SCORING_BENCHMARK_PLANES`. It covers any extra without naming one. Update the docstring at :5-6. |
| `test_scan_pipeline_integrity.py` | Rename `test_standard_scan_extra_absent_matches_deterministic_baseline` to `test_standard_scan_with_a_hunter_model_keeps_deterministic_findings` (config `hunter` only). Rename `test_quick_scan_is_unaffected_by_the_extra` to `test_quick_scan_records_no_degradation`. Drop the `has_harnessx` monkeypatch and the `harnessx` parameter. Recompute the expected degradation set from the post-removal run, since `hunter_pool`/`verify` entries no longer exist. |
| `test_slot_contract.py` | Delete `test_host_under_harnessx_disables_the_model` and the imports at :16 and :22. Keep the rest. |
| `test_config.py` | Replace `test_harnessx_defaults_when_section_absent` and `test_load_config_reads_harnessx_section` with `test_retired_agentic_section_loads_with_one_warning` (caplog: exactly one record naming the plane) and `test_retired_llm_keys_fail_naming_the_replacement` (verifier, panel_model, decider_model). |
| `test_fusion.py` | Delete `test_dispatch_records_degradation_when_llm_requested_but_extra_absent` (:142-146) and the import at :16. Rename the dispatch calls to `fuse_findings`. |
| `test_tool_hunter.py` (:229, :266), `test_scan_deep.py` (:144) | Drop the `has_harnessx` monkeypatch line. |
| `test_semantic_extra.py` | Use `assert_cold_import`. |
| `test_module_audit.py` | The subsystem tuple at :31 becomes `("fusion", "mcp", "skills")`. A new assertion checks that the three deleted modules are absent from the audit. |
**Module audit.** `model/audit.py:76-77` (`harness_ext`, `hunter_harness`) are deleted, and `:78`
(`stage_processors`) is reworded to "deterministic slot-contracted scan stages; pinned by the zero-dependency
guard in test_gate". Regenerate `benchmarks/measurements/2026-09-08-module-audit.json` in place with `to_dict(audit(SRC))`.
`test_the_manifest_matches_the_tree` pins that path. The diff shows three rows removed and no orphan.
**Search (Requirement 2.4).** `grep -rni harnessx src tests pyproject.toml README.md` may match only lines tagged
`retired 2026-..`:
- the `RETIRED_KEYS` entry and warning text in `config.py`
- the two retirement tests in `test_config.py`
- the one README line in the plane section
Requirement 3.3 cannot be implemented or tested without naming the section, so these lines count as retirement
notes. The check is a test (`test_no_harnessx_outside_retirement_notes`) that fails on any untagged match. It
builds its pattern at runtime, so it does not match itself.
### 3. Equality proof (Requirement 3)
The spec's "`ousast regress`" is not a subcommand (`cli.py:133-251`). Regress exists only as the deep-mode
sandbox stage. On this project the regression command over the development pins is `ousast pairs`
(README.md:38): the vuln-vs-fixed replay over local fixtures and vendored pinned VFC pairs, offline by default.
The benchmark replay is README.md:35's manifest. The baseline also covers the two paths the removal touches
directly: one standard-mode scan (verify and fusion) and one `improve --dry-run` (evolve). Everything is
quick or standard mode with no Joern on the host, so each command takes minutes, not hours, on this 7 GB host.
**Environment (instrument first).**
- Run from a `git archive HEAD` export in the scratchpad (the source is frozen, per the long-runs rule).
- `uv sync --frozen --extra semantic`, which drops the installed `harnessx`. It is installed in `.venv` today.
- `OPENULTRASAST_SKIP_DOTENV=1` and `env -u DEEPSEEK_API_KEY -u OPENROUTER_API_KEY -u ANTHROPIC_API_KEY -u OPENAI_API_KEY`,
plus an empty `--config`, so no command can reach a model.
The harness asserts `find_spec("harnessx") is None`, `find_spec("tree_sitter") is not None`, and a
`uv pip freeze` hash. It also prints the byte size of each input manifest, and fails on any non-zero exit code.
**Commands** (one script, `benchmarks/harnessx_removal_equality.py record|compare <dir>`):
1. `pytest -q -p no:cacheprovider --junitxml=suite.xml`, which gives outcome per test id.
2. `ousast pairs --json > pairs.json`. Instrument check: pair count > 0 and at least one TP.
3. `ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick`. Keep
`benchmark_result.json`, `calibration_records.json` and `external_baseline_deltas.json`. Instrument check:
`expected > 0`.
4. `ousast scan benchmarks/fixtures/python-vulnerable --mode standard --config empty.toml`. Keep
`findings.json`, `verification.json`, `fusion.json`, `report.sarif`, `harness.json` and `manifest.json`.
5. `ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --no-pair-gate`, stdout plus
the dry-run journal.
**Stored.** `benchmarks/measurements/<date>-harnessx-removal-baseline/`: the normalized outputs, and `env.json`
(commit, freeze hash, instrument checks). The script's exclusion list is written into that directory's
`README` line.
**Comparison.** JSON is parsed, normalized and re-serialized with sorted keys. Everything else is compared as
raw bytes. Excluded:
- keys `runtime_seconds`, `scan_id`, `timestamp`, `started`, `finished`, `created_at`, `generated_at`, and any
key ending `_seconds` or `_ms`
- run directory names (timestamped, `benchmark.py:186`)
- `trace/events.jsonl` as a whole (timings)
- the export's absolute root path, replaced by `<ROOT>`
For the suite, every test id that survives must keep its outcome. Deleted and renamed ids are compared through
the map in the Tests table above, and any other difference is a failure. Any difference in 2-5 blocks the merge.
### 4. Memory layer (Requirement 6.1, 6.4)
**Store** (`plane/memory.py`). The store lives at `results_root()/memory/` (`~/ousast-results/plane/memory/`,
honouring `OUSAST_RESULTS`, `reconciler.py:58-59`). It is JSON files, not SQLite: the rest of the plane is JSON,
diffable and greppable, and one run at a time holds the run lock, so SQLite's concurrency buys nothing.
```
memory/
index.jsonl one row per ingested (run, task): sha256 of its memory.jsonl, row count
facts/<sha256>.json facts.json by content (identical facts stored once)
repos/<host>__<owner>__<name>/<pin>.jsonl the rows of one repository + 40-hex pin
```
Ingest is idempotent: an `index.jsonl` hit skips the delivery, and rows carry a deterministic `id`. Ingest also
refuses to write when `shutil.disk_usage` reports less than 1 GiB free. The disk is at 98% today, so it names
the store path in the error.
**Storage backends (maintainer, 2026-09-30: asked to use the object-store features of the S3 server he runs,
including its JSON support).** The store is written against one interface, `MemoryStore` (`put_row`, `put_facts`, `get_facts`,
`rows(repo=..., pin=..., kind=..., where=...)`, `index`), with two backends of identical behaviour:
- `FileStore`: the layout above, for this laptop and the unit tests.
- `S3Store`: an S3-compatible bucket (boto3; RustFS is the tested server), and the store of record once the plane runs on a separate
Kubernetes cluster. The layout maps onto one bucket with the same key paths:
- **Object metadata and tags** on every row object: `repo`, `pin`, `kind`, `family`, `run`, `population`,
`split`. Rules list by prefix (`repos/<repo>/`) and filter by tag instead of scanning the bucket.
- **Server-side JSON queries**: `rows(where=...)` pushes the filter down with S3 Select
(`SelectObjectContent`, JSON Lines input, JSON output), so a proposal rule reads matching rows, not whole
files. Support is probed once per process against the server (a small Select on a known object); when the
server lacks it, the backend fetches and filters locally, logs the
fallback once, and returns the same rows. A query never silently returns nothing because a feature is
missing.
- **Versioning** on the bucket: each proposal's provenance records the object key and version id of every
memory row it cites, so its evidence cannot change underneath it (Req 6.4). Object lock (governance mode) is
optional for the `proposals/` prefix.
- **Lifecycle rules**: raw run outputs under `runs/` expire after a configurable number of days; `repos/`,
`facts/` and `proposals/` are kept. The near-full laptop disk stops being the store's limit.
- **Presigned URLs** (the step that removes the host receiver for the Kubernetes move, not built in this
spec): the reconciler can hand a task presigned PUT URLs for its outputs and GET URLs for its inputs in the
start request, so tasks write to the bucket directly and never hold credentials. The interface keeps
`presign_put`/`presign_get` so that follow-on changes the reconciler, not the store.
- **Bucket notifications** (follow-on): an event on new `repos/` rows can trigger the loop Run
(event-driven loop engineering).
- **Configuration**: `OUSAST_MEMORY` selects the backend (`file:///path` or `s3://<bucket>[/<prefix>]`); the S3
endpoint and credentials come from `.env` (`S3_ENDPOINT`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`,
optional `S3_REGION`), never from a manifest, never printed. Client: boto3 as an optional extra
(`openultrasast[s3]`); the core install stays PyYAML-only.
- **Tests**: the same contract tests run against `FileStore` and against `S3Store` pointed at an S3-compatible
endpoint (skipped unless `OUSAST_MEMORY_TEST_S3` is set), including the Select probe with its fallback
forced, versioned provenance, and metadata filters. The disk-space refusal applies to `FileStore` only.
**`remember` task** (`plane/tasks/remember.py`, no Model, budget `{usd: 0, calls: 0}`, one per case). It reads the
case's delivered artifacts as inputs: `facts.json`, the `units.jsonl` of `va`/`vb`/`vc`, `final/agreed.json` and
`final/disputed.json` with the per-candidate rows, and `alerts.jsonl`. It takes the repository URL and pin from
the bound Workspace (`AX_WORKSPACES_YAML`, `OUSAST_GIT_PINS`) and the population from the Run annotation
`openultrasast.io/population`. It writes `memory.jsonl` and passes `facts.json` through. Task code interprets
the artifacts, so the reconciler and runner stay content-blind (ai-service-plane Req 3.4). After `reconciler.run`
returns, `cli._plane` calls `memory.ingest(run_dir)`. `ousast plane remember <run>` does the same by hand.
**Row kinds** (every row has `id`, `kind`, `repo`, `pin`, `run`, `task`, `population`, `split`, `image`):
- `facts`: `sha256`, `candidates_digest` (`repo_facts._digest`, `repo_facts.py:188`), `files`.
- `verdict`: `candidate` (`path::function`), `family`, `site`, `passes {a, b, c}` (true/false/null), `final`
(`agreed | rejected | disputed`), `tiebreak` (bool), `site_match` (bool/null), `usd`, `turns`.
- `unit_cost`: `pass`, `path`, `usd`, `calls`, `usage`, `turns`, `model`.
- `alert`: `rule_id`, `rule_status` (`enabled | shadow`), `path`, `line`, `function`, `pin_role`
(`vulnerable | fixed`).
- `proposal_outcome`: `edit_key`, `evidence_digest`, `outcome` (`accepted | reverted | rejected`), `gate_run`.
**Fact reuse.** A facts entry is valid for the key (repo, pin, candidates digest, runner image digest). The image
is in the key because the host's source and the image's source can differ, and facts computed by other code are
not the same facts. `ousast plane workspaces --validation-set` writes the key as the Task annotation
`openultrasast.io/memory-key`. Annotations are already stripped before ax (`manifests.py:4-6`). Before
`reconciler.run`, `cli._plane` calls `memory.seed(run_manifest)`. For each Run entry whose Task carries a key with
stored facts and no state yet, seed writes `<run>/<task>/facts.json` and a `summary.json`
(`status: done, units_done == units_total == 1, reused: {"sha256", "from_run"}`). It then calls
`reconciler.mark_done(run, task, reused=...)`, a public helper of about 10 lines that takes the run lock, sets
the state and releases it. The reconciler already skips `done` tasks (`reconciler.py:342`). The receiver already
serves any regular file under the run directory as a declared input (`egress.py:242-246`). A changed pin, a
changed candidate set or a rebuilt image recomputes.
### 5. Proposals from memory (Requirement 6.2, 6.4)
**Interface.** The proposal surface HarnessX would have used is `run_round` (`improve/evolve.py:149-247`). It
builds `RuleStatusEdit`s (`improve/validator.py:36-47`), filters them through the journal's novelty gate
(`journal.reverted_edit_keys`, `journal.py:26-38`), validates each with `EvolveValidator.validate`
(`validator.py:75-81`, which accepts only `RuleStatusEdit` and `PolicyConstantEdit`), smoke-replays and gates.
`improve/memory.py` exposes:
```
propose_from_memory(rows, ruleset_by_id, current_ledger, journal_rounds, *, excluded) -> list[MemoryProposal]
MemoryProposal = (edit: RuleStatusEdit, provenance: dict)
```
`run_round` gains `proposals: Sequence[MemoryProposal] = ()`. It appends their edits after the benchmark's edits
and keeps the first edit per `key()`. The validator and the gate are untouched. `ousast improve` gains
`--memory [DIR]` (off by default, so the deterministic `improve` output is byte-identical) and
`--qualify-population NAME` (repeatable).
**Rules** (the thresholds are module constants and part of the provenance record). Join key: an `alert` and a
`verdict` match when repo, pin and `path::function` are equal. A `disputed` final counts as no evidence in
either direction.
- **M1, repeated false alerts** (`enabled -> shadow`). Rule R qualifies when both hold:
- at least 3 R alerts sit at candidates whose final is `rejected`, or on a `fixed` pin inside the case's declared
fix range, across at least 2 distinct repositories;
- no R alert sits at an `agreed` candidate or a `site_match` site.
- **M2, missed declared sites** (`shadow -> enabled`). Shadow rule R qualifies when all three hold:
- its shadow alerts hit at least 2 `agreed` candidates with `site_match == true`, across at least 2
repositories;
- no enabled rule alerts at those candidates;
- R has no alert on a `fixed` pin at the same function and at most 1 alert at a `rejected` candidate.
- **M3, frequently disputed families.** Families with at least 5 candidates and a dispute rate of at least 0.3 go
to `signals.json` as advisory input for plane routing (for example, scheduling pass `c` up front). They are not
an edit and never reach the validator.
**Provenance.** Each proposal's `rationale` is `memory:<proposal_id>`, where `proposal_id` = sha256 of
(rule, edit key, sorted evidence row ids, thresholds). The sidecar `memory_proposals.jsonl`, next to the journal,
holds:
- `proposal_id`, `edit_key`, `rule_id`
- `evidence` (the row ids plus their repo, pin, run and task)
- the counts that met each threshold
- `excluded` (populations and pins removed by the guard, with counts)
- the store's `index.jsonl` digest at proposal time
`evolve._record` (`evolve.py:366-396`) adds `"evidence": proposal_id` to an edit's journal entry only when the
edit came from memory. Otherwise the journal is unchanged byte for byte. A reader follows journal, then sidecar,
then store rows, then the run directory.
**Novelty.** `journal.reverted_edit_keys` does not block a rule edit that carries a rationale (`journal.py:36`), so
a memory proposal would otherwise come back every round. The proposer itself skips any key whose reverted journal
entry has the same evidence digest. New evidence is the "new rationale" that allows a retry.
**Train-on-test guard.** A row is excluded from proposals when any of these holds:
- (a) its `population` is one of `--qualify-population`, which is also where the proposal will be gated or
qualified;
- (b) its (repo, pin) is a case of the benchmark manifest being gated;
- (c) its (repo, pin) is a `split = "holdout"` pair of the pair catalog, which the per-profile holdout clause
(`evolve.py:204-214`) measures.
This is the rule the closed-loop leak broke, where a holdout pair taught the shape that "recovered" it. Here the
filter runs before any rule sees a row. Concretely, rows from `validation-46` (population
`independent-v2/validation`) never propose edits that are then qualified on `independent-v2/validation`. The
tie-break was designed on that set, so it is development evidence only.
### 6. The loop as a plane Run (Requirement 6.3)
`ousast plane workspaces --validation-set ... --loop` extends the generated Run. It also emits a Workspace for each
case's fixed pin (the population's `fixed` sha, already read by `generate.fix_ranges`, `generate.py:89`):
| Task (per case unless noted) | Model | Budget | Inputs -> outputs |
| --- | --- | --- | --- |
| `<case>-facts`, `-va`, `-vb`, `-agree`, `-vc`, `-final` | verify passes: `deepseek-flash`; others none | as generated today (`generate.verify_budget`) | unchanged |
| `<case>-alerts` (new) | none | `{0, 0}`, `serialize: engine` | vulnerable + fixed Workspaces -> `alerts.jsonl` (quick-mode `quick_scan_findings` with enabled and shadow rules, function-attributed) |
| `<case>-remember` (new) | none | `{0, 0}` | facts, va, vb, vc, final, alerts -> `memory.jsonl`, `facts.json` |
| `loop-measure` (once) | none | `{0, 0}` | every `remember` + `memory-snapshot/rows.jsonl` -> `measure.json` (the increment metrics: declared sites agreed, cost per candidate, dispute rates) |
| `loop-propose` (once) | none | `{0, 0}` | measure, snapshot, journal -> `proposals.jsonl`, `memory_proposals.jsonl`, `signals.json` |
| `loop-improve` (once) | none | `{0, 0}`, `serialize: engine` | proposals; Workspace = this repository at the commit under test -> `gate.json` (the `RoundOutcome`), `journal.json`, `rule_policy.json` (only when accepted) |
Before the Run starts, `memory-snapshot` is seeded into the run directory like reused facts. It holds the store
rows after the guard, since a sandboxed task cannot read the host store. A `{usd: 0, calls: 0}` budget makes
`budget.py` refuse any call, so a model-free task that tries one fails loudly rather than spending. Attribution
per task is the existing `ousast plane status` table.
**The gate's verdict** lands in `~/ousast-results/plane/<run>/loop-improve/gate.json` and is ingested as
`proposal_outcome` rows. An accepted `rule_policy.json` is adopted into the repository only by a maintainer
commit, as any ledger change is today (README "Rule-level loop").
### 7. Documentation and specs (Requirement 4)
- **README.** Replace "Configuring the HarnessX agentic plane" (README.md:293-329) with "The agentic plane (ax)":
what runs there (repo-facts, verify, agree, memory), `ousast plane run|status|remember`, a pointer to
`ops/ax/README.md`, and one tagged line saying HarnessX was retired. Rename "The HarnessX self-improving cycle"
(:330) to "The self-improving cycle". Its calibration ledger and `ousast improve` content is not HarnessX and
stays. The MetaAgent paragraph (:393-400) is replaced by the memory proposer (`--memory`, provenance, guard).
Drop the `panel_model`/`decider_model` lines at :227-228.
- **`docs/examples.md` section 6** becomes "Running the agentic plane", with the plane doctor, run and status
commands.
- **`docs/threat-model.md:17-20, 32-34`**: egress is the per-task EgressPolicy (receiver, Git hosts, Model
`egress-hosts`, passthrough only). Credentials travel only in the start request. Spend is the per-task
`usd`/`calls` budget in the Run.
- **Retirement notes.** A dated note ("Retired 2026-..: HarnessX was removed by `harnessx-removal`; the agentic
plane is ax (`ai-service-plane`), proposals come from the plane memory. History below is unchanged.") goes at
the top of the requirements (and design, where present) of `harnessx-self-improving-rulesets`,
`learning-harness`, `three-stage-scan` and `openrouter-sast-harness`, and in `.kiro/steering/overview.md` near
:175. The other specs that mention HarnessX in passing keep their history untouched.
- `RELEASE_NOTES.md` gets one entry listing the retired keys. Kiro tooling stays out of user docs.
## Data Models
- `memory.jsonl` (remember output) and `repos/<repo>/<pin>.jsonl`: the rows above, one JSON object per line,
sorted keys, stable `id` = sha256(kind, run, task, candidate|path|rule_id+line).
- `index.jsonl`: `{"run", "task", "sha256", "rows", "ingested"}`.
- `memory_proposals.jsonl`: `{"proposal_id", "edit_key", "rule_id", "rule": "M1|M2", "evidence": [...],
"counts": {...}, "thresholds": {...}, "excluded": {...}, "store_digest"}`.
- Journal edit entry: the existing fields (`evolve.py:381-384`) plus `"evidence"` only for memory edits.
- Equality record: `env.json` `{"commit", "freeze_sha256", "harnessx_importable": false, "tree_sitter": true,
"inputs": {path: bytes}, "exclusions": [...]}`.
## Error Handling
- `RetiredConfigError`: exit 2 with the key and its replacement, before any stage runs.
- A `[harnessx]` section produces exactly one warning, and loading continues.
- `memory.ingest` on a run without `remember` outputs is a no-op that says so. A malformed row fails the ingest
and names the file and line, and nothing partial is written: rows are staged in a temporary file and renamed.
- A low-disk refusal names the store and the free space. The plane run's result is unaffected, and ingest can be
repeated with `ousast plane remember`.
- `seed` never overwrites an existing task directory or a non-pending state. A facts file whose sha256 no longer
matches is dropped from the store with a warning, and that task recomputes.
- `propose_from_memory` with an empty store after the guard returns no proposals, and `run_round` reports
`no_proposals` as today.
- The equality script exits non-zero on the first failed instrument check. It also exits non-zero on any
difference outside the exclusion list and prints the first differing path and key.
## Testing Strategy
- `tests/test_config.py`: the one-warning section and the retired keys (above).
- `tests/test_no_harnessx_references.py`: the tagged-only search of Requirement 2.4.
- `tests/test_plane_memory.py`: ingest is idempotent and content-addresses identical facts. `seed` marks a facts
task done and serves its file, and does not for a changed pin, candidate digest or image. Low-disk refusal
(monkeypatched `disk_usage`).
- `tests/test_plane_remember.py`, `test_plane_alerts.py`, `test_plane_loop.py`: fixture run directories built from
the recorded validation-46 shapes. `remember` rows match `agree`'s per-candidate rows. `alerts` attributes
functions. `measure` reproduces the increment's recorded metrics on the recorded artifacts. `propose` and
`improve` work end to end on a fixture manifest.
- `tests/test_improve_memory.py`:
- M1 and M2 fire at their thresholds and not below.
- A disputed candidate counts neither way.
- The guard removes rows by population, manifest case and holdout pair, so a proposal that exists only because
of a holdout row disappears.
- Provenance ids are stable across runs.
- A reverted memory edit is not re-proposed on the same evidence, and is re-proposed on new evidence.
- `run_round` without proposals writes a journal byte-identical to today's.
- `tests/test_plane_reconciler.py`: `mark_done` under the lock. The existing under-500-lines test stays.
- The existing gate and pair tests, `test_benchmark_output_is_byte_identical_to_golden_baseline`
(`test_gate.py:104`) and the equality script (section 3) are the regression net.
## Requirement Traceability
| Requirement | Design section | Verified by |
| --- | --- | --- |
| 1.1 fate of every capability | 1 (table) | this document; commit 5 |
| 1.2 moved capabilities reachable, Model per task, fixture test | 1 (existing `verify`/`agree`), 4-6 (new tasks are Model-free) | `test_plane_verify.py`, `test_plane_tiebreak.py`, new plane task tests |
| 1.3 retired keys fail naming the replacement | 1 (`RetiredConfigError`) | `test_config.py` |
| 2.1 modules and guards gone | 2 Source | commit 6; audit test |
| 2.2 extra, override, lock gone; installs succeed | 2 Packaging | fresh-venv `pip install .` / `'.[semantic]'` |
| 2.3 tests deleted or renamed; audit regenerated | 2 Tests, Module audit | `test_module_audit.py` |
| 2.4 search returns only retirement notes | 2 Search | `test_no_harnessx_references.py` |
| 3.1 baseline without the extra | 3 | commit 1 record |
| 3.2 byte-identical after, exclusions written | 3 Comparison | commit 7 record |
| 3.3 `[harnessx]` loads with one warning | 1 | `test_config.py` |
| 4.1-4.3 docs, retirement notes, Kiro out of user docs | 7 | commit 8 review |
| 5.1-5.2 order, green steps, no deletion before replacement | Commit sequence | per-commit gates |
| 6.1 store keyed by repo + pin, fact reuse | 4 | `test_plane_memory.py`, `test_plane_remember.py` |
| 6.2 deterministic proposals through validator and gate | 5 | `test_improve_memory.py` |
| 6.3 loop as a plane Run with per-task Models and budgets | 6 | `test_plane_loop.py`; `ousast plane status` attribution |
| 6.4 provenance and train-on-test guard | 5 | `test_improve_memory.py` |
## Commit sequence (Requirement 5), mapped to future tasks
Each commit is gated on the full suite's own exit code, `ruff` and `mypy`.
1. **Baseline** (3.1): the equality script and `benchmarks/measurements/<date>-harnessx-removal-baseline/`,
recorded with the extra uninstalled.
2. **Memory store and `remember`** (6.1): `plane/memory.py`, `tasks/remember.py`, `reconciler.mark_done`, seed and
ingest wiring in `cli._plane`, generator annotations, fixture tests.
3. **Memory proposer** (6.2, 6.4): `improve/memory.py`, `run_round(proposals=...)`, `ousast improve --memory`,
sidecar and journal evidence, guard tests.
4. **Loop Run** (6.3): `alerts`, `measure`, `propose` and `improve` tasks, `--loop` generation, fixed-pin
Workspaces, fixture tests.
5. **Retire the scan paths** (1, 3.3): the `cli.py`, `fusion.py`, `verify_judge.py` and `stage_processors.py`
changes, `RetiredConfigError`, the one warning, and the test renames that follow. At this point nothing imports
`harness_ext`/`hunter_harness`. Their replacements (plane tasks, and the memory proposer from 2-4) have landed.
6. **Delete code and packaging** (2): the two modules, the extra, the mypy override, `uv.lock`, the deleted tests,
the fixture rename, audit rows and the manifest regeneration, and the reference search test.
7. **Equality proof** (3.2): rerun the script, commit the comparison record next to the baseline, and link it in
the commit message.
8. **Docs and specs** (4): the README, `docs/examples.md`, `docs/threat-model.md`, retirement notes, steering and
release notes.
## Risks
- **Existing users of the LLM scan paths.** There is no `--llm` flag. Users who set `[models] verifier` or
`[fusion] panel_model` get a hard error, which is intended (1.3). Users who installed the extra and set
`[models] hunter` lose the second (HarnessX) hunter without an error. Only the release note covers that,
because the key is still valid. No measurement suggests either path found anything the remaining paths miss.
The absence of a measurement is itself the risk.
- **Plane precision is unsolved.** `plane-increment-2.json` agrees 28 candidates, of which 16 are declared
sites. The other 12 are unadjudicated and several may be false. M1 treats a `rejected` final as negative
evidence, while plane recall is 16/20, so "rejected" is not "safe". The thresholds (2 or more repositories, zero
agreed hits) and the unchanged recall/FP gate are the guard. Precision work (judge design) remains out of scope.
- **Evidence is thin.** Today's store would hold 15 cases from one validation population, which is exactly the
population the guard excludes for its own qualification. Expect `no_proposals` until a second population has
run through the loop.
- **Disk.** The home volume is 98% full (2.4 GB free). One validation-46-sized Run adds about 3 MB of facts
(content-addressed; about 0.1 MB when pins are unchanged) plus about 0.5 MB of rows. Ingest refuses below 1 GiB
free. The store has no eviction in this spec.
- **The baseline measures the environment as well as the code.** `.venv` has `harnessx` installed today. A
baseline taken without re-syncing would include it. The script's `find_spec` and freeze checks exist for that.
- **Seeding writes reconciler state from outside the reconciler.** It is confined to `mark_done` under the run
lock. The reconciler's line budget (465 of 500) leaves no room for fact reuse inside the reconciler.
@@ -0,0 +1,125 @@
# Requirements Document
## Introduction
HarnessX is the optional agentic plane (`openultrasast[harnessx]`): an LLM hunter pool, an LLM judge and LLM
fusion panels, each behind a capability guard with a deterministic fallback, plus a lazy bridge that hosts
deterministic stages under a HarnessX processor. The ai-service-plane increment met its gate on google/ax
(`benchmarks/independent/plane-increment-2.json`), which is the condition the maintainer set on 2026-09-29:
"goal is to remove harnessx and to shift to ax fully once we establish we have value." This feature removes
HarnessX from the product and moves every LLM capability it carried either onto the plane or into a recorded
retirement. Inventory: `brief.md` (verified 2026-09-30).
Maintainer, 2026-09-30: "Please remove harnessx completely. the memory layer that saves tokens takes over the
evolutionary improvements as well as loop engineering." So nothing of HarnessX stays, not even as an optional
path, and the self-improving cycle HarnessX was meant to drive (the LLM `MetaAgent.evolve` proposer plugged into
`improve/evolve.py`) is carried by the plane's memory layer instead: what runs record (repository facts,
verdicts, disputes, costs) becomes the input to proposals and to the run-measure-propose-gate loop.
## Boundary Context
- **In scope**: the HarnessX-only modules (`harness_ext.py`, `hunter_harness.py`); the guarded branches in
`cli.py` (scan `--llm` path, `HxScanOrchestrator`), `verify_judge.py` (judge path), `fusion.py` (LLM panels),
`stage_processors.py` (`host_under_harnessx`), `config.py` (`HarnessxConfig`); the `harnessx` extra, its mypy
override and lock entry; the tests that exercise HarnessX; README and docs sections; dated retirement notes in
the specs and steering that name HarnessX.
- **Also in scope**: the plane memory layer as the source of evolutionary proposals and the loop (Requirement 6),
replacing the HarnessX proposer.
- **Out of scope**: changing the deterministic engine, rulesets or `evolve`'s validator and gate (proposals change
source, not how they are validated and gated); new detection logic; qualification on an independent population; the plane's own precision work (judge design) --
where a capability moves to the plane, this spec wires it, it does not improve it.
- **Adjacent expectations**: every deterministic path (quick mode, `regress`, `benchmark`, `gate`, `evolve`)
produces byte-identical output before and after; `vulnerabilities-over-plumbing` applies, so capabilities are
kept or retired by what they detect, reported in TP/FP terms where a measurement exists.
## Requirements
### Requirement 1: HarnessX is removed completely; the plane is the only agentic path
**User Story:** As the maintainer, I want no HarnessX path left, optional or not, and every LLM capability either
on the plane or explicitly retired, so that nothing is lost silently.
#### Acceptance Criteria
1. No HarnessX code path remains, optional or guarded. The design lists each capability -- LLM hunter pool
(`ousast scan --llm`), LLM judge (`verify_judge`), LLM fusion panels (`fusion`), deterministic stages hosted
under HarnessX (`host_under_harnessx`), the LLM evolution proposer -- with its fate: carried by a plane task
(named), carried by the memory layer (Requirement 6), or retired with the deterministic path kept.
2. A capability moved to the plane is reachable from the CLI through the plane (`ousast plane run` with a
documented Run, or a CLI flag that submits one), binds its own Model per task (ai-service-plane Req 7), and
has a fixture test like the other plane tasks.
3. A retired capability's CLI flags and config keys fail with a message that names the replacement or says it
was retired, never a silent no-op.
### Requirement 2: HarnessX code, packaging and tests are gone
**User Story:** As the maintainer, I want no HarnessX code path left to maintain.
#### Acceptance Criteria
1. `harness_ext.py` and `hunter_harness.py` are deleted; no module imports `harnessx`; `use_harnessx`
parameters, `has_harnessx` checks and `host_under_harnessx` are removed; the deterministic fallbacks become the
only code path where a capability is retired.
2. The `harnessx` extra, the mypy override for `harnessx.*` and the lock entry are removed; `pip install .` and
`pip install '.[all]'` (or whatever extras remain) succeed.
3. Tests that only exercise HarnessX are deleted; tests that assert fallback behaviour are kept and renamed to
describe the behaviour, not the absence of HarnessX; the module audit is regenerated with no orphan.
4. A search for `harnessx` (case-insensitive) in `src/`, `tests/`, `pyproject.toml` and the README returns only
the retirement notes of Requirement 4.
### Requirement 3: Deterministic behaviour is unchanged, and proven so
**User Story:** As the maintainer, I want evidence, not assurance, that removing HarnessX changed nothing a
deterministic run produces.
#### Acceptance Criteria
1. Before any deletion, a baseline is recorded without the extra installed: the full host suite, and the outputs
of `ousast regress` and one `ousast benchmark` replay on the development pins, stored under
`benchmarks/measurements/`.
2. After the removal the same commands produce byte-identical outputs (timestamps and durations excluded, and
the exclusion list is written down); any difference blocks the merge.
3. A config file that still carries a `[harnessx]` section loads with one warning that names the plane, and the
section is otherwise ignored.
### Requirement 4: Documentation and specs tell the new story
**User Story:** As a reader of the README or a spec, I want to learn that the agentic plane is ax, not HarnessX.
#### Acceptance Criteria
1. The README sections "Configuring the HarnessX agentic plane" and "The HarnessX self-improving cycle" are
replaced by a short section on the ax plane that points at `ops/ax/README.md`; `docs/examples.md` section 6 and
`docs/threat-model.md` (egress and spend caps) describe the plane's egress policy and budgets instead.
2. Specs that name HarnessX (`harnessx-self-improving-rulesets`, `learning-harness`, `three-stage-scan`,
`openrouter-sast-harness`) and `.kiro/steering/overview.md` get a dated retirement note at the top; their
history is not rewritten.
3. Maintainer-only Kiro tooling stays out of user docs (standing rule).
### Requirement 5: The removal lands in one reviewable sequence
**User Story:** As the maintainer, I want to review the removal in steps that each leave main green.
#### Acceptance Criteria
1. Order: baseline (3.1) -> capabilities moved or retired (1) -> code and packaging removed (2) -> equality proof
(3.2) -> docs and specs (4). Each step is a commit gated on the full suite's own exit code, ruff and mypy.
2. No step deletes code whose replacement (Requirement 1) has not landed and passed its fixture test.
### Requirement 6: The memory layer carries evolutionary improvement and loop engineering
**User Story:** As the maintainer, I want the token-saving memory layer to be what the system learns from, so
that improvement proposals and the improvement loop come from recorded evidence instead of a HarnessX agent.
#### Acceptance Criteria
1. Plane runs persist their memory beyond the run directory in one store keyed by repository and pin: repository
facts (`repo-facts`), per-candidate verdicts per pass with the final decision, disputes and tie-breaks, and cost
and tool turns per unit; later runs reuse facts for an unchanged pin instead of recomputing them.
2. `evolve`'s proposals are generated from that store (for example: families or rules whose candidates are often
disputed, missed declared sites, repeated false alerts), deterministically and without a HarnessX agent; they
pass through the existing validator and gate unchanged.
3. The improvement loop -- run, measure, propose, validate, gate -- is expressible as a plane Run whose tasks bind
their own Models and budgets (ai-service-plane Req 7), so each iteration's token spend is attributed per task.
4. A proposal records which memory entries it came from, so a later reader can trace every accepted change to the
evidence behind it; proposals never train on the population used to qualify them (the train-on-test rule).
+22
View File
@@ -0,0 +1,22 @@
{
"feature_name": "harnessx-removal",
"created_at": "2026-09-30T08:00:00Z",
"updated_at": "2026-09-30T16:00:00Z",
"language": "en",
"phase": "implementation-complete",
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": true
}
},
"ready_for_implementation": true
}
+62
View File
@@ -0,0 +1,62 @@
# Implementation Plan
Order follows the design's commit sequence (Requirement 5): nothing is deleted before its replacement has landed
and passed its fixture test. Every task is a commit gated on the full suite's own exit code, `ruff check`,
`ruff format --check` and `mypy`; the reconciler stays under 500 lines.
- [x] 1. Baseline without the extra
- Equality script (design §3) with its `find_spec("harnessx")` and freeze checks; re-sync `.venv` without the
`harnessx` extra first.
- Record `benchmarks/measurements/<date>-harnessx-removal-baseline/`: full suite, `ousast pairs --json`, the
`python-vulnerable` quick benchmark, one standard-mode scan, `ousast improve --dry-run`; the exclusion list
written next to it.
- _Requirements: 3.1, 5.1_
- [x] 2. Memory store and the `remember` task
- `plane/memory.py`: the `MemoryStore` interface with `FileStore` and `S3Store` (object metadata/tags,
S3 Select JSON queries probed with a local fallback, versioned provenance, lifecycle, presign stubs;
`OUSAST_MEMORY` selects; S3 settings from `.env`; boto3 as an optional extra); JSON rows per
repository + pin, facts by content hash, `FileStore` ingest refuses below 1 GiB free,
`plane/tasks/remember.py` (model-free), `reconciler.mark_done` under the run lock, seed and ingest wiring in
`cli._plane`, generator annotations; fact reuse when repository, pin, candidate set and runner image match.
- Fixture tests: rows from a recorded Run, reuse on an unchanged pin, recompute on a changed one, disk refusal.
- _Requirements: 6.1, 6.4_
- [x] 3. Proposals from memory
- `improve/memory.py` with rules M1 (demote) and M2 (promote) and the advisory disputed-family signal;
`run_round(proposals=...)`; `ousast improve --memory` (opt-in; default output byte-identical); provenance per
proposal in the sidecar and journal; the train-on-test guard (qualification population, gated benchmark cases,
holdout pairs).
- Tests: each rule on crafted rows, the guard drops excluded rows, provenance round-trips, default `improve`
unchanged.
- _Requirements: 6.2, 6.4_
- [x] 4. The loop as a plane Run
- Tasks `alerts`, `measure`, `propose`, `improve` (no Model, zero budget); `--loop` Run generation; fixed-pin
Workspaces; the gate's verdict in `<run>/loop-improve/gate.json`; adoption stays a maintainer commit.
- Fixture tests with the fake ax; one live loop Run on the local ax recorded (no model spend beyond the verify
passes it reuses from memory).
- _Requirements: 6.3_
- [x] 5. Retire the HarnessX scan paths
- `cli.py`, `fusion.py`, `verify_judge.py`, `stage_processors.py` changes; `RetiredConfigError` for
`[models] verifier`, `[fusion] panel_model`/`decider_model`; one warning for a `[harnessx]` section (the only
tagged retirement text allowed in `src`, enforced by a test); tests renamed to the behaviour they assert.
- _Requirements: 1.1, 1.2, 1.3, 3.3_
- [x] 6. Delete HarnessX code and packaging
- Delete `harness_ext.py`, `hunter_harness.py`, `verify_judge.py`'s judge path, the HarnessX-only tests; remove
the extra, the mypy override and the `uv.lock` entry; rename the conftest fixture; audit rows and manifest; the
reference search test (Req 2.4).
- _Requirements: 2.1, 2.2, 2.3, 2.4_
- [x] 7. Equality proof
- Rerun the baseline script; commit the comparison record next to the baseline; any difference outside the
exclusion list blocks the merge.
- _Requirements: 3.2_
- [x] 8. Documentation and specs
- README ax-plane section pointing at `ops/ax/README.md`; `docs/examples.md` §6 and `docs/threat-model.md`
(egress policy, budgets); dated retirement notes in the four specs and `.kiro/steering/overview.md`; release
note naming the silent loss of the second hunter for users who set `[models] hunter`.
- _Requirements: 4.1, 4.2, 4.3_
@@ -1,5 +1,8 @@
# Design Document
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
## Overview
**Purpose.** This feature splits OpenUltraSAST's single overloaded regex layer — which today is detector, severity authority, and volume knob at once — into three governed owners: **rules detect**, a **central CWE policy decides severity and scope**, and a **single project score** becomes the optimization target. On top of that governance plane it adopts the HarnessX runtime exactly where it pays off — the LLM hunter loop, the LLM-judge verifier, and a meta-harness self-improvement loop that adapts rules and policy gates under hard bounds.
@@ -1,5 +1,10 @@
# Requirements Document
> **Status 2026-10-02:** retired with HarnessX (see the note below). Its task 6.3 modules `stage_processors.py` and `slot_contract.py` were deleted on 2026-10-02: nothing imported them. Kept as decision history; nothing in src, tests or CI points here.
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
## Introduction
OpenUltraSAST today fuses three responsibilities into one regex layer: detection patterns also carry their own severity strings and act as the sole volume control, so a noisy rule can dominate scan output, set its own criticality, and fire forever with no central governance. This feature splits those responsibilities into three governed owners — rules detect, a central CWE policy decides severity and scope, and a single project score becomes the optimization target — and adopts the HarnessX runtime exactly where it pays off: the LLM hunter loop, the LLM-judge verifier, and a meta-harness self-improvement loop that adapts rules and policy gates under hard bounds.
+2
View File
@@ -1,5 +1,7 @@
# Brief: ir-consolidation — stop maintaining an IR we did not need, and adopt one we do
> **Status 2026-10-02:** dormant. Brief only and unapproved. The subtraction it asks for still needs a measured recall comparison before the flat IR goes.
**Status:** brief only, unapproved.
## Why now
@@ -0,0 +1,47 @@
# Brief: a trained decision engine and A/B experiments instead of hand-coded detection
Maintainer, 2026-09-30: "you need to go to a different approach with a b testing to extrapolate detection rather
than hardcoding it and train a decision engine".
## Why
Every detection gain so far was a hand edit made after a miss: regex rules, taint vocabulary, guards, prompt
lines, a tie-break rule. Each edit was tuned on the cases that exposed it and had to be re-qualified on a fresh
population; v1 -> v2 showed engine tuning does not transfer (0/17). The plane changed what is possible: every task
records per-candidate outcomes, costs and repository facts into one memory store, and runs are cheap to repeat.
## Direction
- Instruments become signal producers: quick rules, the Joern engine, model sink classification, verify passes,
tie-break, repo facts (callers, reachability), triage. None decides alone.
- A decision engine, trained on labelled outcomes, turns the signals for a candidate into a calibrated probability
that it is a real vulnerability of its family; operating points (block, report, drop) are thresholds on it.
- Changes to the pipeline (prompts, models, pass counts, rule sets, features) are variants compared by A/B
experiments run as plane Runs with pre-registered metrics and confidence intervals, never adopted on a single
run's point estimate.
- Detection on unseen code is extrapolated from measured learning curves and leave-one-population-out results,
reported with uncertainty, and checked once on the frozen v3.
## Data on hand (2026-09-30)
- Pair corpus: 317 vulnerable/fixed pairs, 342 labelled findings (`ousast pairs`), with train/holdout splits.
- Populations: v1 (spent), v2 validation set (46 candidates, 20 declared sites; plane runs recorded), PHP
development corpus (pmpro, v1 PHP, MediaWiki, WP Statistics). v3 (PHP, 16 cases) is being frozen now and stays
untouched until the final check.
- Memory store rows: facts, per-pass verdicts, disputes, tie-breaks, costs, alerts, coverage.
## Risks
- Small labelled data: the engine must be simple and calibrated (regularised linear or shallow trees), with
population-level folds; a deep model would memorise.
- Leakage: labels and features must never come from the population a number is reported on (the train-on-test
rule; see the closed-loop leak).
- Label quality: "rejected by the plane" is not "safe"; negatives come from fixed and benign pins and adjudicated
findings, not from absence of an alert.
## Relation to earlier specs
Replaces the hand-edit loop of `improve` for detection decisions (the memory proposer's M1/M2 status edits become
one variant family among others). `learning-harness` and `harnessx-self-improving-rulesets` are retired with
HarnessX. `finding-feedback-loop`, `constrained-detector` and `model-grounded-detection` are inputs to read in
design, not dependencies.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,162 @@
# Requirements Document
## Introduction
Detection decisions stop being hand-coded. Every instrument becomes a producer of per-candidate signals, a trained
decision engine turns signals into a calibrated probability per candidate and family, pipeline changes are chosen
by A/B experiments run on the plane, and expected detection on unseen code is extrapolated with stated uncertainty.
Maintainer direction and context: `brief.md`.
## Boundary Context
- **In scope**: a feature record per candidate built from every instrument and the memory store; a labelled
dataset with provenance; training, calibration and versioning of the decision engine; A/B experiments as plane
Runs; learning curves and leave-one-population-out estimates; the engine's use in `ousast scan`, `ousast
pre-push` and plane Runs; a final one-time check on frozen population v3.
- **Out of scope**: new instruments (they plug in as signal producers later); deep or large models; any change to
population v3 or its protocol; publishing results (licences still gate that).
- **Adjacent expectations**: instruments keep working standalone; the memory store (harnessx-removal Req 6) is
the system of record; the plane attributes every experiment's token spend per task.
## Requirements
### Requirement 1: Instruments emit signals, not verdicts
**User Story:** As the maintainer, I want every detector's output to become evidence for one decision, so that no
single hand-written rule or prompt decides whether code is vulnerable.
#### Acceptance Criteria
1. For each candidate (repository, pin, file, function, family) the system assembles one feature record from every
instrument that ran: quick-rule hits (rule ids, counts), engine findings and questions completed, model sink
classification, triage, verify pass verdicts and turns, tie-break, repository facts (callers, entry-point
distance), coverage, language and family.
2. A missing instrument is recorded as missing, never as a negative signal (coverage `none` stays distinct from
"ran and found nothing").
3. Feature records are stored in the memory store with the instrument versions and image digests that produced
them, so any record can be rebuilt.
### Requirement 2: Labels come from ground truth with provenance
**User Story:** As the maintainer, I want the engine to learn only from outcomes we can defend.
#### Acceptance Criteria
1. Positive labels: candidates matching a declared site or fix range on a vulnerable pin (populations, pair
corpus). Negative labels: the same functions on fixed pins, benign-control pins, and findings adjudicated false
by the maintainer or a judge task with recorded evidence.
2. A plane verdict alone (agreed, rejected, disputed) is a feature, never a label.
3. Every label records its source (population, pair, adjudication), split, and the date it was created; labels from
a reserved or frozen population are unusable for training until that population's evaluation is recorded.
### Requirement 3: An AI classifier with local memory, compiled DSPy-style
**User Story:** As the maintainer, I want the decision made by an AI classifier that learns from a small, growing
memory of labelled cases, because "We will not be able to scale up data" and "we need to have the decision
engine on AI classifier not ML. with local memory, dspy style" (maintainer, 2026-10-01).
#### Acceptance Criteria
1. The decision engine is a declared language-model program (DSPy-style signature: candidate code, instrument
signals, inferred repository roles in; verdict, family, confidence and rationale out), built in-house on the
plane's metered client so every call stays within per-task budgets, attribution and Model routing; no
statistical model is trained.
2. For each candidate the program retrieves similar labelled cases from the local memory store as in-context
examples: nearest neighbours by the repository-agnostic signal profile, re-ranked by code embeddings
(OpenRouter, embeddings only); identities are never used for retrieval.
3. The program is compiled, not fitted: its instructions and example-selection policy are optimised against a
pre-registered metric on training folds (bootstrap few-shot and instruction search); each compiled program is
versioned in the memory store with its data snapshot, model, prompts, retrieval settings and metrics, and the
same inputs reproduce it.
4. Retrieval never crosses the evaluation boundary: when a number is reported for a repository, examples come
only from other repositories (and other frameworks in leave-one-framework-out folds); a test proves it.
5. The confidence the program reports is calibrated on held-out repositories before operating points are set;
where it does not calibrate, only ADVISORY is offered (Req 7.4).
6. New labels (adjudications, dismissals in the opt-in adaptation layer) improve decisions by entering memory,
without retraining; their effect is measured by A/B experiments like any other change.
### Requirement 4: Pipeline changes are chosen by A/B experiments
**User Story:** As the maintainer, I want changes to prompts, models, pass counts, rule sets and features compared
by controlled experiments, so that nothing is adopted on one run's number.
#### Acceptance Criteria
1. An experiment is declared before it runs: arms (variants as plane Run configurations), unit of randomisation
(candidate, file or repository), metric (detection, precision, cost per candidate), minimum detectable effect,
sample size or sequential stopping rule, and budget.
2. Arms run on the same units where the design allows (paired), with assignment recorded; results and spend go to
the memory store per arm.
3. A variant is adopted only when the pre-registered test shows a difference with its confidence interval
excluding zero, or declared equivalent within a margin for cost-only changes; otherwise the result is recorded
as inconclusive.
4. The first experiment is the known open question: the known-callers prompt line (17 vs 19 sites flagged in any
pass), on the development corpus.
### Requirement 5: Detection is extrapolated, with uncertainty
**User Story:** As the maintainer, I want to know how well the system will detect on code nobody has seen, before
and without tuning on it.
#### Acceptance Criteria
1. Learning curves: engine metrics as a function of memory size (labelled cases available for retrieval) and number of populations, per family,
with confidence bands.
2. Expected recall and precision on unseen code are reported per family and language as intervals from the
population-level folds; a family with too few labels is reported as "insufficient data", never as a number.
3. The extrapolation is checked exactly once against frozen population v3 under protocol v3, after the engine and
its operating points are fixed; the comparison of predicted and measured intervals is recorded.
### Requirement 6: The engine is what users get
**User Story:** As a user scanning a fresh codebase, I want the decision engine's judgement, not a raw list of
rule hits.
#### Acceptance Criteria
1. `ousast scan`, `ousast pre-push` and plane Runs report findings with the engine's probability, the operating
point they passed and the top contributing features.
2. Without a trained model available the tools fall back to today's behaviour and say so in the report.
3. Hand-written rules remain as signal producers; changing their enabled/shadow status no longer changes what is
reported unless the engine's decision changes.
### Requirement 7: Built for unseen repositories -- a repo safety net
**User Story:** As the maintainer, I want the engine judged by how it behaves on repositories it never saw, because
"the tool will never run against the test cases it's being trained with. it's purpose is a repo safety net."
(maintainer, 2026-09-30).
#### Acceptance Criteria
1. Every reported metric comes from folds grouped by repository across all corpora: a repository in an evaluation
fold contributes no row to training.
2. Features are repository-agnostic by an allow-listed schema (no repository, path, file, function or commit
identities), enforced by a test.
3. Two operating points serve the safety net: BLOCK, chosen for >= 95% precision on unseen repositories, and
ADVISORY, trading precision for recall; both reported with out-of-repository intervals. The decision unit is a
candidate in changed code (`pre-push` delta), with whole-repository scans as a secondary mode.
4. Calibration is checked on held-out repositories; where it does not hold, only ADVISORY is offered for that
family.
5. Any adaptation to a user's own repository (facts reuse, their dismissals) is opt-in, separate from the
generalisation model, and never contributes to its reported numbers.
### Requirement 8: No framework or application knowledge as a prerequisite
**User Story:** As the maintainer, I want the safety net to work on any application in a language, because "if
there is hardcoded application specification needed to build trees this is problematic. wordpress isn't the only
php application. same for python, js etc" (maintainer, 2026-09-30).
#### Acceptance Criteria
1. Hand-written knowledge is limited to the language level (built-ins, superglobals, process execution, raw
database drivers, deserialisers). Roles specific to a framework or application (its request objects, query
builders, hooks, sanitisers, guards) are inferred per repository at scan time from its own source and its
dependencies, not required from a hand-written table.
2. Existing framework-specific entries (WordPress, Django, Flask, Express, Spring and others in `ruleset/semantic/`
and `ruleset/<lang>/`) are tagged with their framework and enter the engine only as optional prior features; a
prior is kept only if an A/B experiment on held-out repositories shows it helps, and the engine must still work
with every prior switched off.
3. Evaluation is stratified by framework and includes leave-one-framework-out folds (for example: train without any
WordPress repository, evaluate on WordPress), so a number never depends on having seen the target's framework;
results are reported per framework and for "framework unseen in training".
4. The share of each reported result that depends on framework priors is stated (with priors vs without).
@@ -0,0 +1,23 @@
{
"feature_name": "learned-decision-engine",
"created_at": "2026-09-30T14:00:00Z",
"updated_at": "2026-10-01T10:20:00Z",
"language": "en",
"phase": "implementation",
"approvals": {
"requirements": {
"generated": true,
"approved": true
},
"design": {
"generated": true,
"approved": true
},
"tasks": {
"generated": true,
"approved": true
}
},
"ready_for_implementation": true,
"note": "2026-10-01: AI-classifier design approved; 6.1-6.5 (no model spend) authorised; paid steps need go-ahead."
}
@@ -0,0 +1,149 @@
# Implementation Plan
Order follows the design's commit sequence. Every task is a commit gated on the full suite's own exit code,
`ruff check`, `ruff format --check` and `mypy`. No reported number changes until task 9 adopts a compiled program;
without an adopted program every path is today's. Requirement 3 changed on 2026-10-01 (an AI classifier with local
memory, compiled DSPy-style in-house); tasks 1-5 are unaffected, tasks 6-10 are re-planned for it. Sub-tasks marked
**(budget go-ahead)** spend model money or host hours: each starts only after the maintainer confirms its budget
against the provider balance read just before it, and none starts with a ceiling above that balance. No test makes
a model or embedding call.
- [x] 1. Schema and features
- `learn/schema.py` (closed, typed allow-list; no identities; missing != zero), `learn/features.py` (host
builder), new memory `KINDS`, `plane/tasks/features.py` and the `remember` input; tests enforcing the
allow-list and the exclusion of label-carrying fields.
- _Requirements: 1.1, 1.2, 1.3, 7.2_
- [x] 2. Framework tags
- `ruleset/frameworks.toml`; tag and split every framework/library entry listed in design section 2; `priors=`
in the semantic and quick loaders (default `all`, so today's scan is unchanged -- the golden benchmark test
stays byte-identical); a lint test that no untagged framework entry remains.
- _Requirements: 8.2, 8.4_
- [x] 3. Role inference
- `learn/roles.py` (deterministic wrapper inference), `plane/tasks/roles.py` (model roles, the `model_sinks.py`
approach), the vocabulary overlay for the engine; fixture tests with no model calls.
- _Requirements: 8.1_
- [x] 4. Labels
- `learn/sources.toml` (fail-closed allowed sources, v3 excluded unread), `learn/labels.py`, `ousast learn
labels`, repository-group normalisation and CVE merge, the `assumed_benign` source mined from non-security
commits (separate, never merged); a first label snapshot committed as counts only.
- _Requirements: 2.1, 2.2, 2.3, 7.1_
- [x] 5. Harvest (needs budget go-ahead)
- A plane Run of verify a/b on the verify-family pairs and development cases (~$6, ceiling $10), model roles on
the candidate files (ceiling $10), engine features for the pair corpus on the host (frozen source, one
container at a time, background).
- Done 2026-10-01 (`benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json`): verify $2.54 and roles
$1.40 by attribution. The engine job finished the same day (682 of 686 pair sides; the 2 family-unknown pairs
not run) and the feature records were rebuilt; dev-php pins have no engine run (`missing`).
- Advisory-fix pairs harvested 2026-10-02 (`benchmarks/measurements/2026-10-02-decision-engine-harvest-advisory/record.json`):
194 sides, verify a/b on the 34 untrusted_destination sides and model roles on all ($0.38 metered, ceiling $5),
engine on the host (54 ran, 58 asked no question, 82 no language model); static memory 724 -> 904 examples. The
leak audit flags nothing new in the static profile; the plane profile newly flags `verify.flag_b` and `ms.operation`.
- _Requirements: 1.1_
- [ ] 6. Program, local memory and compilation (design sections 4-5)
- [x] 6.1 Excerpts and the example store (no model)
- `learn/excerpt.py` (function span at the label's pin, bounds 80/4,000 for candidates and 40/2,000 for
examples, numbered lines, no path/repository/commit, advisory ids and security wording in comments
redacted, delta diff); `learn/examples.py` and `ousast learn memory build` (label + feature record +
excerpt -> `example` row; refuses `origin: user` and sources outside `sources.toml`; exits 2 when more than
10% of a source has no excerpt); `example` added to `KINDS`; `put_blob`/`get_blob` on both backends.
- Tests: `test_learn_excerpt.py`, example-builder refusals, blob round trip on FileStore (S3 gated).
- _Requirements: 3.2, 3.6, 7.2_
- [x] 6.2 Metered embeddings and the cache
- `plane/models/openrouter-embedding.yaml` (`input_per_m: 0.02`), `MeteredEmbeddingClient` in
`plane/budget.py`, usage kept by `parse_embedding_response`; cache `embeddings/<model>/<excerpt_sha>.json`;
`ousast learn memory embed`; scripted-client tests.
- **(budget go-ahead)** embedding the store: ~5.0 M tokens, ~$0.10, ceiling $1 (OpenRouter account).
Done 2026-10-01: 616 distinct excerpts of the 724 static examples, 198,819 tokens, 20 calls, $0.0040 metered
(`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`).
- _Requirements: 3.2_
- [x] 6.3 Retrieval and the evaluation boundary (no model)
- `learn/retrieve.py`: per-instrument Gower distance with explicit missing, `n1 = 40`, embedding re-rank
`r = lam(1 - D) + (1 - lam)cos`, `signals_only` mode, balance rule (`k_ret = 6`), `eligible(example, target,
fold)`, near-duplicate drop in evaluation, deployment self-exclusion.
- `learn/folds.py`: compile split (25% of groups), outer grouped 5-fold, leave-one-source-out,
leave-one-framework-out.
- Tests: `test_learn_retrieve.py` including `test_boundary` (every rendered example and demonstration eligible,
for all fold kinds; `memory.seed` excludes the case's group), `test_learn_folds.py`.
- _Requirements: 3.2, 3.4, 7.1, 8.3_
- [x] 6.4 The program (no model in tests)
- `learn/program.py`: the signature, prompt rendering (instruction + demonstrations as a byte-identical prefix,
then retrieved examples, then the candidate), JSON parsing with one retry -> `unsure, parse_failed`,
`Classify(k)` with temperature 0 then 0.7, the score `s` and majority verdict; `responses/<sha>.json` cache
and replay; `temperature`/`logprobs` pass-through in `MeteredClient.complete` and `DeepSeekChatClient._call`.
- Tests: `test_learn_program.py` (no `EXCLUDED_FIELDS` in prompts, prefix identity, parsing, `unsure` cannot
BLOCK, budget ceiling, replay).
- _Requirements: 3.1, 3.3_
- [x] 6.5 Compile, calibrate, evaluate (no model in tests)
- `learn/compile.toml` (pre-registered metric: balanced Brier score; seeds; budget), `learn/compile.py`
(bootstrap few-shot on `C_boot`, 6 proposed + baseline instructions scored on `C_val`, 4 demonstration sets,
alternates for leave-one-framework-out, artifact `programs/<sha>.json`), `learn/calibrate.py` (cross-fitted
Platt, reliability/slope/intercept/ECE, ADVISORY-only downgrade), `learn/evaluate.py` (operating points from
the other folds, repository-cluster bootstrap, strata, "insufficient data"), `ousast learn
compile|evaluate|curve`.
- Tests: `test_learn_compile.py` (`test_reproducible_from_cache` on both backends), `test_learn_calibrate.py`,
`test_learn_decide.py`.
- _Requirements: 3.3, 3.5, 5.2, 7.3, 7.4_
- [x] 6.6 First paid slice **(budget go-ahead: ~$5 expected, ceiling $7)**
- Smoke run on 20 candidates (~$0.10): real token counts, prefix-cache hit share, parse rate; it fixes `k`
(5 if samples 2..k hit the cache for >= 80% of their prompt tokens, else 1) before anything else is spent.
- One compile per profile with data (`static` first; ~$2.2, ceiling $3).
- Outer-fold evaluation, priors off, evaluable families only (675 candidates: ~$2.0 at k = 1, ~$3.0 at k = 5),
with the 30-candidate canary set; report under `benchmarks/measurements/<date>-decision-engine-v1/` (counts
and intervals only, no identities).
- Done 2026-10-01 for family **injection only** (the maintainer's scope for this slice; memory holds every family,
at most 2 contrast examples per prompt): smoke fixed `k = 5` (repeat hit share 0.96); program `c1237a49...`
(proposed instruction 5, balanced Brier 0.128 on 13 `C_val` candidates vs 0.199 baseline); 110 out-of-repository
candidates in 29 groups: ADVISORY recall 0.90 [0.82, 0.97], precision 0.54 [0.44, 0.65]; AUC of `s` 0.74 [0.65,
0.85]; BLOCK not offered (ECE 0.064 > 0.05, and no fold reached a Wilson lower bound of 0.95); canary agreement
0.83; $0.91 metered at list price (provider balance $18.20 -> $17.90). Record:
`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`. The other evaluable families
are not evaluated yet.
- _Requirements: 3.1-3.5, 5.2, 7.3, 7.4_
- [ ] 6.7 Second paid slice **(budget go-ahead: ~$7 expected, ceiling $10; needs a top-up)**
- Leave-one-source-out (~$1.4-2.1), leave-one-framework-out where a framework reaches 10 groups ($0 today),
learning curves over memory size (0/25/50/100%) and number of sources (~$5.2), recorded beside 6.6.
- Partial 2026-10-02 (`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`): after the
advisory-fix harvest (memory 724 -> 904), one compile per evaluable family and its outer-fold evaluation for all
six (injection, untrusted_destination, path, output_encoding, deserialization, access_control); out-of-repository
AUC 0.77-0.96, BLOCK offered for none (calibration holds only for output_encoding, where no fold reaches the
Wilson bound); one memory-size point for injection (724 vs 904 examples, paired: AUC 0.76 vs 0.77). $2.88
metered (ceiling $8). Not done: leave-one-source-out, the 0/25/50% curve points, the source-count curve.
- _Requirements: 5.1, 5.2, 8.3_
- [ ] 7. Experiments (exp-001 and program variants need budget go-ahead)
- `learn/experiments.py`, `register`, `experiment-outcome`, Run annotation checks, `OUSAST_VERIFY_CALLERS`;
exp-001 (known-callers line on/off, ~$7.7 expected, ceiling $15) registered, run and recorded -- an
inconclusive result is pre-registered as likely.
- Program-variant experiments (paired on candidates, repository cluster, arm A replayed from cache at $0): the
machinery and its test; the first variant (`k = 5` vs `k = 1`, or the confidence source) **(budget go-ahead:
~$0.9-1.35 per arm-B pass of 300 candidates, ceiling $2 each)**.
- _Requirements: 3.6, 4.1-4.4_
- [ ] 8. Integration
- `plane/tasks/embed.py` (Model `openrouter-embedding`) and `plane/tasks/decide.py` (Model `deepseek-flash`,
metered, exits 0/2/3), the host seed of the adopted program and the case's boundary-filtered memory; the `scan`
decision stage and `dropped_by_engine.json`; `pre-push` admission with the engine (structural admission reasons
unchanged; concurrency, `undecided: deadline`, a per-push budget); report fields (p, operating point,
program id, verdict, rationale, cited lines); the fallback line (no program, no key, budget exhausted,
insufficient data, calibration failed); the opt-in adaptation layer with user-local memory.
- Tests use scripted clients; a plane fixture Run proves `decide` never receives an example of its own case.
- _Requirements: 6.1-6.3, 7.3, 7.5_
- [ ] 9. Adopt
- `ousast learn adopt`; the packaged `ruleset/decision/program-<profile>.json` with only licence-cleared
demonstrations (the maintainer decides which licences qualify), re-evaluated as its own variant; the canary
drift check before adoption; the priors-on/off experiment per framework with a recorded decision
**(budget go-ahead per experiment, ~$1-2 each)**.
- _Requirements: 3.3, 8.2_
- [ ] 10. The one-time v3 check **(budget go-ahead for the engine's v3 spend, ~$0.0045 per candidate)**
- Commit `prediction-v3.json` first (program id, memory snapshot digest, model ids, operating points, predicted
intervals, spend ceiling from the owner's candidate count); then v3 is run under protocol v3 and
`result-v3-engine.json` is committed. Nothing between the two commits may touch the program, memory snapshot,
features, priors or thresholds. This spec's code never opens v3 files.
- _Requirements: 5.3_
+3
View File
@@ -1,5 +1,8 @@
# Brief: learning-harness
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
## Problem
The tool is a hand-authored static analyzer with a learning loop bolted on the side, and the loop cannot learn. Three deliberate choices cap it: closed vocabularies (sinks, sources, obligation kinds) that grow only by writing a new spec; a corpus that is the only teacher and a shape that is the only lesson; and a ban on retaining anything the model discovers. Results have ceilings by construction, the loop trains on its own test set, and there is no failure analysis feeding the next round. Detection logic is ~16% of the source and the other 84% cannot say whether the 16% works.
+3
View File
@@ -1,5 +1,8 @@
# Design Document: learning-harness
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
## Overview
**Purpose**: Turn OpenUltraSAST from a hand-authored analyzer with a leaking improvement loop into a learning harness: an auto-classifier routes every pair, finding and code region to one of ten verifier-aligned families; one detector configuration per family is evolved one change per round from structured failure facts; a class-aware pair-wise scorer with unscorable denominators and a measured per-family noise floor decides acceptance; only verifier-confirmed claims are reported above suspicion; the split is enforced by construction.
@@ -1,5 +1,10 @@
# Requirements Document
> **Status 2026-10-02:** retired, superseded by `model-grounded-detection` (spec.json `superseded_by`). 30 of 33 tasks were done; the open three will not be. Kept as decision history; nothing in src, tests or CI points here.
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
## Introduction
OpenUltraSAST's detection numbers cannot be trusted and cannot be improved. The static machinery is treated as the detector while the model with tools already outperforms it (12/14 versus 5/14 pair-correct on the vibe-py holdout once scoring is keyed on class and function), the pair scorer keys detection on text tokens rather than on a vulnerability class, the improve loop learns from its own holdout pairs, and no change is attributed to the class it targets, so an improvement in one class silently degrades another. This feature turns the tool into a learning harness: an auto-classifier assigns every labeled pair, finding and code region to one of ten verifier-aligned families; one detector configuration per family is evolved round by round from structured failure facts; a class-aware, pair-wise scorer with unscorable denominators and a measured per-family noise floor decides acceptance; only verifier-confirmed claims are reported above suspicion; and the train/holdout split is enforced by construction. Context, measurements and sources: `brief.md`, `research.md`, `benchmarks/measurements/2026-09-06-*`.
@@ -1,5 +1,7 @@
# Requirements Document: model-grounded-detection
> **Status 2026-10-02:** complete, 16 of 16 tasks (spec.json phase corrected to `implementation-complete`). On 2026-10-02 three of its modules were deleted because nothing imported them: `model/judge.py` (Req 8; `model/pipeline.py` does the judging), `model/calibrate.py` (Req 9; the decision engine's `learn/calibrate.py` is the live calibration) and `model/execution.py`. Deleting `execution.py` closes the Req 10 execution-tier option: a future execution tier is a new design, not this seam.
## Introduction
This feature supersedes `learning-harness` and `constrained-detector`. It replaces the assumption that the LLM
@@ -1,9 +1,9 @@
{
"feature_name": "model-grounded-detection",
"created_at": "2026-09-08T00:00:00Z",
"updated_at": "2026-09-08T10:00:00Z",
"updated_at": "2026-10-02T00:00:00Z",
"language": "en",
"phase": "implementation",
"phase": "implementation-complete",
"approvals": {
"requirements": {
"generated": true,
@@ -1,5 +1,8 @@
# Design: OpenRouter SAST Harness
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
## Architecture Summary
OpenUltraSAST is a harness-oriented SAST system, not a monolithic scanner. The core runtime composes processors, model roles, retrieval, tools, sandbox policy, traces, verification gates, and reporting into a reproducible scan harness.
@@ -1,5 +1,10 @@
# Requirements: OpenRouter SAST Harness
> **Status 2026-10-02:** retired (see the note below). Its OpenCode skill `openultrasast-kiro-impl`, which still sent agents to this tasks.md, was deleted. Kept as decision history; nothing in src, tests or CI points here.
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
## Overview
OpenUltraSAST shall be an independent OpenCode-driven security harness for finding, validating, reporting, and optionally fixing source-code vulnerabilities at scale. It shall use OpenRouter-selected chat models for reasoning, OpenRouter embeddings for semantic retrieval, Docker for isolated analysis, and an explicit evidence ladder to prevent AI suspicion from being reported as verified security impact.
@@ -0,0 +1,7 @@
{
"feature_name": "openrouter-sast-harness",
"language": "en",
"status": "retired",
"updated_at": "2026-10-02T00:00:00Z",
"retired_note": "2026-09-30: HarnessX was removed by harnessx-removal; this spec had no spec.json before 2026-10-02 and is kept as decision history."
}
@@ -1,5 +1,7 @@
# Requirements Document
> **Status 2026-10-02:** complete, 32 of 32 tasks (spec.json phase corrected to `implementation-complete`).
## Introduction
Engineers evolving OpenUltraSAST's overlay cannot trust the pair scoreboard: coverage detections are dropped, vfc labels mostly say `sink = "unknown"`, a quarter of the vfc corpus fails to parse because excerpts start mid-comment, two sast pairs are unlabelable as vendored, and there is no Node/TypeScript/Python web corpus and no agent-authored corpus. This spec makes the scoreboard honest per slice, per mechanism, and per provenance, adds three web and agent-authored slices, and lets the improve loop accept a change only when no provenance profile regresses. It supersedes `real-world-vfc-slice` for the corpus and scorer plane.
+2 -2
View File
@@ -1,9 +1,9 @@
{
"feature_name": "pair-corpus-honesty",
"created_at": "2026-09-04T16:12:17Z",
"updated_at": "2026-09-05T12:30:00Z",
"updated_at": "2026-10-02T00:00:00Z",
"language": "en",
"phase": "implementation",
"phase": "implementation-complete",
"supersedes": [
"real-world-vfc-slice"
],
+62
View File
@@ -24,6 +24,68 @@ an irrelevant old defect, or an unactionable witness can still produce noise.
- Do not build a second ranker/scope heuristic, an incremental CPG engine, a daemon, an automatic fixer,
or a mandatory model service. Do not scan intermediate pushed history in this feature.
## What this hook is, and is not — 2026-09-21
A general-purpose pre-commit hook, not an application-specific one. It may be measured on WordPress
plugins and Node applications, and it may not become a tool that only works on them. The practical
test at review time: would this change still be correct on a repository nobody here has seen? A path,
a directory name or an identifier peculiar to one project fails that test, and one was declined on
exactly this ground when a plugin's bundled library tree would have fixed a single subject while
claiming something false about every other.
The fact tables are an ontology of libraries, frameworks and language constructs, and
`contributor-scan` owns it. Its design records how it may grow and its group 7 makes growth
measurable rather than anecdotal. Nearly all of this tool's accuracy lives there: the 2026-09 family
census found five of seven families establishing nothing, every one of them for an ontology reason
with the flow engine ready and nothing to aim at.
## Two invariants this project keeps rediscovering
Added 2026-09-21 after the same two defects appeared eleven times between them. Each instance was
found by measurement, fixed in one place, and then found again somewhere else, because the RULE was
never written down. Both are cheap to check in review and expensive to find in a benchmark.
### A degradation is owned by the narrowest scope that caused it
A gap recorded at repository scope silences questions it has no evidence about. The test for a
repository-wide gap is whether it could make a reported answer WRONG everywhere, not whether it cost
coverage somewhere.
| instance | the blast radius | where it was fixed |
|---|---|---|
| vendor exclusion acting as a global completion veto | every question in the repository | M1b, task 10.6 |
| line ambiguity inherited by every question | every question in a file | M1b |
| an empty scope blocking another question's comparison | a sibling's answer | task 12.3 |
| a language partition boundary demoting every answer | 4,425 PHP regions, by 36 JavaScript ones | task 13.4 |
| a partition census gap demoting every answer | 4,425 PHP regions, by four build artefacts | task 13.4 |
The fourth and fifth cost the declared v0.1 target its entire admissibility: a WordPress plugin is
PHP plus admin JavaScript, so the product could detect defects on its own target and never admit
one. Neither passing subject could reveal it, because both hold one language.
**In review:** every new degradation reason must state the scope it reaches — one question, one file,
one partition, or the whole read — and a reason that demotes an outcome must justify reaching that
far. A gap that costs coverage without threatening correctness is a declared exclusion: it stays
published and it does not demote.
### A token matches the thing it names, not text that contains it
Every matcher in this project searches text for a declared name, and each time the search was allowed
to look somewhere the name does not appear AS the thing, it claimed something else.
| instance | what it matched instead | where it was fixed |
|---|---|---|
| `gets` inside `fgets` | the safe API reported as the dangerous one, 26 times | task 5.13 |
| `resolve` inside `resolveUrl` | a sanitizer that sanitized nothing | contributor-scan |
| `user` inside `users` in a SQL string | a textbook IDOR read as guarded | authorization-obligations |
| a sink word inside a comment | a prototype finding on a handler with no such call | task 14.1 |
| a dotted setting matched on its last segment | an editor's `setOptions` read as a template engine's | task 14.3 |
| a flow query reading the call's receiver | `res.end()`, which takes no data, reported like a write | task 14.3 |
**In review:** a text match must be bounded AND confined to the part of the node that names the
callee. Whole-node text, an argument subtree, a receiver and a trailing path segment are all places
the name can appear without being what is called.
## Boundary Commitments
### This Spec Owns
@@ -0,0 +1,330 @@
# M1a evidence: the NodeGoat detection with every veto recorded
Run date 2026-09-16. Milestone **M1a** of [release-milestones.md](release-milestones.md).
Instrument `benchmarks/push/m1a_record_vetoes.py`; flag `--experimental-record-vetoes`.
Evidence: [summary](../../../benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json)
and its 11-file [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json.gz).
This record diagnoses. It admits no defect, enables no capability, qualifies no language and
makes no latency claim. All three artifacts report `finding_status: none` with zero admitted
defects; every recorded section carries its experimental label.
## What was declared before running
Expected base/head pins, expected operation and expected outcome for all three comparisons
were written to `m1a-record.json` before the first scan. The declared target is
`app/routes/contributions.js`, `handleContributionsUpdate`, family `injection`, lines 32–34.
| Case | Base | Head | Declared expectation |
|---|---|---|---|
| nodegoat-introduce | fixed `0b453d5a` | vulnerable `c5cb68a7` | raw finding at the three eval lines, recorded novelty new |
| nodegoat-repair | vulnerable `c5cb68a7` | fixed `0b453d5a` | fixed twin analyzed and quiet; three base-only findings |
| nodegoat-benign | vulnerable `c5cb68a7` | benign `f772c59e` | backlog unchanged; never new or worsened |
Each revision's target file was reopened before the run: 2,485, 2,491 and 2,498 bytes with
recorded SHA-256 digests. Each head scan reported a complete 44-file JavaScript census.
Installed engine, facts, query, policy, core and semantics provenance matched the frozen
values in all three artifacts.
## Observed outcome
| Case | Elapsed | Target findings | Target question answered | Deadline reached |
|---|---:|---:|---|---|
| nodegoat-introduce | 349.8 s | 3 | yes, 9 rows | no |
| nodegoat-repair | 153.8 s | 0 | yes, 0 rows | no |
| nodegoat-benign | 261.4 s | 3 | yes | no |
All three ran under an explicitly named 900-second development budget. These durations are
development measurements and cannot satisfy or measure the 30-second hook latency gate.
Verdict recorded by the instrument: raw witness present, fixed twin quiet, unchanged revision
not reported as a new regression, recorded evaluation produced for all three. The milestone's
**kill criterion did not trigger**: the fixed twin was genuinely analyzed, answering its target
question with zero rows and no deadline reason, while the same three findings appear on its
base side as base-only findings. Silence from failure was not accepted as a pass.
## The vetoes that actually fired
Identical for each of the three `eval` witnesses on the vulnerable revision. Five recognized
vetoes fired and were recorded rather than applied:
| Stage | Veto | Evidence |
|---|---|---|
| head question | `change_context_incomplete` | the answered target question carried 9 rows and was then demoted |
| head scan | `dynamic_external_or_depth_context_unresolved` | on the target's own question identity, not an unrelated one |
| head scan | `vendor_semantics_unresolved` | recorded both as a degradation and as a scope boundary |
| base scan | `graph_incomplete` | the base question answered with 0 rows and was then demoted |
| change context | `ambiguous_line_correspondence` | file-level, on the target file |
The earlier owner diagnosis named four independent vetoes and expected query-to-operation
provenance and question-owned vendor relevance to be the two that mattered. Both fired. The
run also confirms the third and fourth, so all four named vetoes are real on this case.
## What M1a corrected in the diagnosis
**A fifth veto is the actual remaining blocker.** With all five recognized vetoes lifted, the
detection does not become new. Its reason advances from `witness_identity_unresolved` to
`operation_correspondence_unresolved`, because the head operation sits on a changed line and
therefore has no base line anchor. The comparison can classify an operation only when it maps
to a base location.
The unchanged revision proves this is the mechanism rather than a coincidence. Its comment-only
edit leaves lines 32–34 unchanged, the base location resolves to the same lines, and the
recorded novelty becomes `unchanged` / `same_supported_mechanism`. Same evidence, same policy,
opposite correspondence outcome:
| Revision | Operation line | Base location | Recorded novelty |
|---|---|---|---|
| vulnerable | changed | unresolved | unknown, `operation_correspondence_unresolved` |
| unchanged | unchanged | resolved to the same line | unchanged, `same_supported_mechanism` |
**Query identity differs from witness identity, as predicted.** The target is selected through a
file-fallback question with `function=None`, while the rows report method `<lambda>2` at lines
32–34. The declared benchmark function is `handleContributionsUpdate`. This is recorded, not
applied: it is why production reported `witness_identity_unresolved` before any lifting, since
an unresolved question yields no comparable operation at all.
**Admission reasons that no flag can remove.** `capability_unavailable` remains on every
candidate because the registry is empty and this flag cannot enable a capability.
`dependency_unresolved` remains because unlifted coverage reasons still reach admission.
## What M1b must repair, ordered by this evidence
1. **Authentic per-family change context.** `change_context_incomplete` demoted every answered
question in all three cases. Dominance and configuration questions need their own context
producers describing their actual query scope; taint context must not be copied into them.
2. **Question-owned vendor relevance.** `vendor_semantics_unresolved` turned an otherwise
complete 44-file first-party census into `graph_incomplete` on every base question. Keep the
exclusion physical; establish which question actually requires excluded semantics.
3. **Operation correspondence for changed lines.** The blocker M1a surfaced. Bind the required
base/head correspondence to the specific operation rather than to a line anchor, with absent,
renamed, removed-guard and ambiguous-operation controls. Do not delete the ambiguity check
and do not treat lexical alignment as semantic proof.
4. **Correspondence ambiguity scoped to its file region.** `ambiguous_line_correspondence` is
recorded file-wide and inherited by every question, including uniquely mapped operations.
5. **Query-to-operation provenance.** Carry verified provenance from the selected question to the
reported operation. Do not inject benchmark function names into regions.
Vetoes that did not fire on this case stay as they are and their families stay experimental.
## Limits
Three comparisons on one inspected teaching repository. NodeGoat is development and transfer
data; it cannot qualify production Node support, and Node evidence cannot qualify PHP or Python.
The lifted view is a diagnostic reconstruction, not a comparison result: it shows what the
existing evidence would support if those vetoes were repaired, and no repair has been made.
Runtime, independent qualification and rollout all remain NO-GO.
## M1b step 1: what the first three repairs changed
Rerun 2026-09-16 after tasks 10.1–10.3, same instrument, same pins, same budget.
Evidence: [summary](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-step1.json)
and its [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-step1.json.gz).
The recorded view now classifies the case correctly. The vulnerable revision reaches
`new` / `operation_absent_from_comparable_base`, and the unchanged revision reaches
`unchanged` / `same_supported_mechanism` with its base location resolved to the same lines.
Before the repairs both stopped at `operation_correspondence_unresolved` and
`witness_identity_unresolved`.
| Revision | Before 10.1–10.3 | After 10.1–10.3 |
|---|---|---|
| vulnerable | unknown, `operation_correspondence_unresolved` | new, `operation_absent_from_comparable_base` |
| unchanged | unknown, `witness_identity_unresolved` | unchanged, `same_supported_mechanism` |
The rerun also caught a defect in the repair itself: the new novelty reason was missing from
the admission allowlist, so admission rejected the candidate as `change_unsupported`. Fixed
with a control.
The production path is unchanged so far, because two vetoes still demote the target
questions: `change_context_incomplete` on the head from the question's own
`dynamic_external_or_depth_context_unresolved`, and `graph_incomplete` on the base from the
vendor exclusion.
## Why the last two vetoes need new query evidence
Neither remaining veto can be scoped or relaxed soundly as the plan originally described,
and the rerun is what showed it.
A taint answer reports traced flows. An absent row therefore means either that the operation
does not exist in the base, or that it exists and no traced flow reached it.
`dynamic_external_or_depth_context_unresolved` exists precisely to stop the second being read
as the first. Removing or narrowing it would let an unreached operation look absent, which
turns a wrong absence into a wrong new finding: the false-positive direction this project
cannot afford.
At the same time the flag carries no discrimination. It is computed over every in-scope
method, and on any real Express application ordinary framework and dependency calls have
unresolved destinations, so it is always true and can never be discharged. A veto that is
always on and can never be lifted is why the tool reports nothing on real projects.
The resolution is neither to keep it nor to drop it, but to supply the evidence that makes
absence provable independently of reachability: export the operations the query actually
enumerated in the question's scope, separately from the flows it traced. Absence is then
established from the inventory, while a traced-flow claim still requires complete
reachability. Vendor exclusion can then stop acting as a global completion veto for the same
reason, since dependency on excluded code becomes visible in the evidence rather than assumed.
That prerequisite is now task 10.4, ahead of the two vetoes it unblocks. It adds no detector,
alias resolver or second source IR.
## M1b outcome: the same case through the production rules
Rerun 2026-09-16 after tasks 10.1 to 10.6, same instrument, same pins, same budget.
Evidence: [summary](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-production.json)
and its [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-production.json.gz).
The production comparison now classifies all three revisions without any lifting, and its
verdicts match the recorded view exactly. That match is the point: the vetoes M1a recorded
were repaired rather than bypassed.
| Revision | Production verdict | Recorded verdict |
|---|---|---|
| vulnerable | new, `operation_absent_from_comparable_base` | identical |
| fixed twin | quiet, analyzed, three base-only findings | identical |
| unchanged | unchanged, `same_supported_mechanism` | identical |
Compared with M1a, where production reported `witness_identity_unresolved` on every finding
and every one of the 436 questions was demoted on both sides.
The head target questions now complete, the base scan completes all 436 questions, and the
three `eval` witnesses at contributions.js lines 32 to 34 are attributed to the change that
introduced them. Timings were 325.7 s, 158.9 s and 260.3 s under the named 900-second
development budget, which measures nothing about the hook gate.
All three artifacts still report `finding_status: none` with zero admitted defects. Admission
rejects on `capability_unavailable`, which is the deliberate eligibility gate, and on
`dependency_unresolved`, which is a coarse plumbing defect recorded as its own task: the runner
passes every coverage reason as every candidate's dependency gaps rather than the gaps that
pertain to that candidate. Neither is a comparison failure.
### What each repair contributed, measured
1. **10.1 operation correspondence.** Moved the vulnerable revision from
`operation_correspondence_unresolved` to a classifiable operation.
2. **10.2 required comparison scope.** Stopped unrelated unsupported families from blocking a
complete target comparison.
3. **10.3 ambiguity scoping.** Stopped one file's repeated unchanged lines invalidating every
question in the repository.
4. **10.4 operation inventory.** Made absence provable from enumeration rather than from the
absence of a traced flow, which is what let 10.5 and 10.6 be done soundly.
5. **10.5 reachability binding.** Let the head questions complete, since bounded reachability
cannot invalidate a flow that was traced.
6. **10.6 declared exclusion.** Let the base scan's 436 answers stand, since excluding
dependencies is a scope choice rather than a failed read.
A seventh change was needed after the first rerun: the same exclusion and ambiguity distinction
was still missing inside the comparison's own degradation and boundary checks, so the verdict
stayed at `head_context_incomplete` even once both scans completed. The rerun is what surfaced
it, not the unit suite.
### Limits
Three comparisons on one inspected teaching repository, under a development budget. NodeGoat is
development and transfer data: it cannot qualify production Node support, and Node evidence
cannot qualify PHP or Python. No capability is enabled, no defect is admitted, no alert is
emitted, and no latency claim is made. Runtime, independent qualification and rollout all
remain NO-GO.
## M1b exit run (tasks 10.1 to 10.8)
Rerun 2026-09-16 after every repair, same instrument, same pins, same 900-second development
budget. Evidence: [summary](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-exit.json)
and its [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-exit.json.gz).
| Revision | Verdict | Admission | Elapsed |
|---|---|---|---:|
| vulnerable | all three sites new, `operation_absent_from_comparable_base` | `capability_unavailable` only | 326.4 s |
| fixed twin | analyzed and quiet, three base-only findings | none to admit | 157.8 s |
| unchanged | all three sites `unchanged`, `same_supported_mechanism` | `capability_unavailable`, `unchanged` | 256.8 s |
The vulnerable revision's only remaining admission reason is the deliberate eligibility gate.
`dependency_unresolved` is gone, which verifies task 10.7 on real data: those gaps belonged to
unrelated questions.
Each finding now carries its provenance explicitly: question function `None`, engine method
`<lambda>2`, witness `eval(req.body.preTax) -> const preTax = eval(req.body.preTax)` at
contributions.js line 32, and likewise for lines 33 and 34. The unchanged revision resolves each
base location to the same line, which is why it reads `unchanged` rather than new.
Source receipts were reopened before the run, and the head census is the complete 44 files.
### Which vetoes were repaired and which remain
All five vetoes M1a recorded are repaired, plus two the repairs exposed:
| Veto | State |
|---|---|
| `change_context_incomplete` on the target question | repaired, question completes |
| `graph_incomplete` on the base from vendor exclusion | repaired, all 436 base questions complete |
| `ambiguous_line_correspondence` inherited repository-wide | repaired, decided per operation |
| `dynamic_external_or_depth_context_unresolved` as a blanket veto | repaired, bound to reachability claims |
| `operation_correspondence_unresolved` on a changed line | repaired via the enumerated inventory |
| comparison's own degradation and boundary checks | repaired after the first rerun exposed them |
| `dependency_unresolved` on every candidate | repaired, gaps attributed per candidate |
What remains, recorded rather than repaired, because each is outside this case or a later
milestone:
- 231 of 436 head questions still report `change_context_incomplete`, from
`context_projection_unavailable` and `context_scope_empty` on families whose queries export no
context. Those are dominance and configuration questions. They did not block this case and
their families stay experimental, so per the milestone they stay as they are.
- The vendor exclusion and the file's correspondence ambiguity remain reported in coverage. That
is intended: they are recorded limits, not repaired absences.
### What M1b has not yet demonstrated
The milestone also asks for a developer-readable evaluation finding carrying a supported
security consequence and a concrete repair direction. Those two come from an evaluated
capability declaration, and the registry is deliberately empty, so nothing renders them today.
Writing that text without a declaration is precisely what admission exists to prevent, so it is
not something to work around here. The comparison half of M1b is verified; the explanation half
waits on a declaration, which is M4's subject.
### Limits
Three comparisons on one inspected teaching repository under a development budget. NodeGoat is
development and transfer data: it cannot qualify production Node support, and Node evidence
cannot qualify PHP or Python. No capability is enabled, no defect is admitted, no alert is
emitted and no latency claim is made. Runtime, independent qualification and rollout all remain
NO-GO.
## M1b explanation: the finding as a developer would read it
Run 2026-09-16 after task 10.10, same pins and budget, with an unreviewed experimental
declaration generated by the instrument and bound to the run's own analysis semantics.
Evidence: [summary](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-explained.json)
and its [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-explained.json.gz).
Three evaluation findings were rendered on the vulnerable revision, one per eval site. For
line 32:
- **Location** app/routes/contributions.js:32, family injection, novelty new with
`operation_absent_from_comparable_base`.
- **Witness** `eval(req.body.preTax) -> const preTax = eval(req.body.preTax)`.
- **Change attribution** `base:app/routes/contributions.js:32-34`.
- **Provenance** `file_scope`: the question named no function, the engine reported `<lambda>2`.
- **Consequence** "A request field reaches const preTax = eval(req.body.preTax), where it is
evaluated as JavaScript, so an attacker controlling that field can execute arbitrary code in
the server process."
- **Repair** "Replace the evaluation at const preTax = eval(req.body.preTax) with a numeric
conversion such as Number(...) or parseInt(...), then validate the result before use."
Both sentences come from the declaration's templates through the grounded-template path, not from
generated prose. The declaration reports verdict `experimental` and `enabled: false`, the
production result still reports `finding_status: none`, and all three artifacts carry zero
admitted defects. The rendering path never reaches admission, so no supplied declaration can
change that.
### A defect this surfaced, recorded not repaired
The operation symbol admission extracts is the text before the first parenthesis, which for
`const preTax = eval(req.body.preTax)` is `const preTax = eval`, not `eval`. The demonstration
declaration had to enumerate each assignment spelling for its coverage check to pass. A real
qualified declaration cannot enumerate every assignment a codebase might write, so admission
would reject genuine findings on `operation_semantics_mismatch`. That is recorded as its own task
ahead of M4; it is not worked around here.
M1b is met: the comparison classifies all three revisions through the production rules, and the
finding reads as a developer would need it to, with every claim traceable to its evidence.
@@ -0,0 +1,288 @@
# Release milestones: prove usefulness, then speed and transfer
Accepted direction: 2026-09-14. The maintainer agreed to the five-step path and asked
for it to be recorded in the roadmap as a spec. This amendment defines the order and
observable exit criteria for the remaining release work within pre-push-safety-net.
It preserves the approved requirements, completed tasks and existing owner boundaries.
It does not approve an unreviewed implementation design or enable any capability.
## Current position
**Next milestone: M3, with M4 population preparation running in parallel. M1a and M1b are met;
M2 is measured NO-GO; none of M1–M5 is verified.** The published `v1.2.0-alpha.1` is an
experimental implementation release, not a qualified advisory rollout. See
[current status](status.md) for measured evidence and limitations.
The immediate goal was one real changed-code defect with a defensible explanation and a quiet
fixed twin. That is met. Task 8.3 remains blocked.
## Strategy amendment, 2026-09-21
Accepted after the family detection census and the PHP correction. Four decisions, in the order
they should be executed. They change sequence and declared scope; they change no acceptance
threshold and no owner boundary.
1. **Close M3 at a scale that completes, before anything else.** PHP transfer is recorded as not
established, and that record came from two instrument defects rather than from the analyser:
a census verdict that checked question completion before findings, and a region budget whose
question count cannot fit any ceiling. Bounded to 25 and 100 regions, Paid Memberships Pro
completes with no timeout and establishes a true-positive SQL injection of the class its own
fact table predicted. Re-run the declared transfer replay at a completing scale and let the
result stand either way.
2. **Reserve and review the untouched M4 population now, in parallel.** M4 already permits
population preparation to proceed before the frozen candidate exists, and it is the longest
lead-time item on this path: nothing can be qualified without it however good the analyser
gets. Starting it after M2 serialises the release for no reason.
3. **M2 is the only engineering item that remains, and it cannot be scoped away.** A changed-code
transaction spends its whole budget in the build with zero query time. Graph reuse across
revisions is the accepted direction. Narrowing what a push analyses is explicitly not a way to
meet the latency gate, so the work is architectural and should have the effort that family
vocabulary work has been absorbing.
4. **Declare the v1 advisory scope as the families whose claims can be ADMITTED.** Admission
requires the entailed rung. Injection, untrusted destination and configuration entail with
witnesses; access control does not, because on any application that splits its route from its
data access the obligation is carried and a carried obligation corroborates by design. Shipping
it would ship silence. Output encoding, path, prototype and deserialization have not earned a
place either. M4 explicitly does not require every language or family to become eligible.
**Amendment to the amendment, same day.** Decision 1's re-run exposed something larger than the
runtime limit it was chasing. A repository holding two languages can never complete a question,
because a partition boundary is classified as a graph-integrity gap and such a gap demotes every
completed outcome; and a completed outcome is what admission requires. NodeGoat completes 299 of
299 and VAmPI 35 of 35 because each holds one language, while PMPro completes none of 520 and
still establishes 43 findings. The declared v0.1 target is WordPress plugins, and a plugin is PHP
plus its admin JavaScript, so on the target this tool was built for it can detect and never admit.
Task 13.4 carries it, and it now ranks with M2 rather than behind it: a narrow JavaScript-only
release can proceed without it, and the WordPress target cannot proceed at all.
**The risk this amendment does not remove.** Every number so far comes from two repositories used
to build the analyser, and one week of examining them surfaced seven false positives. On untouched
code the precision estimate will move, against a bar of 95% with zero false alerts on the fixed and
benign sides. Expect the first M4 evaluation to fail and budget for a second. That is the strongest
argument for decision 2.
## M1 — One real security change works end to end
Status: **NEXT — not verified**. Dependencies: existing immutable replay and scan machinery.
M1 is split into M1a (diagnose with vetoes recorded, not applied) and M1b (repair only
the vetoes M1a showed to matter). M1 is verified only when M1b passes.
Why the split: the detector already reports the NodeGoat contribution injection. The
2026-09-14 post-census bundle carries the raw `<lambda>2` witness at contributions.js
lines 32–34, and the same bundle carries thousands of `context_incomplete`,
`graph_incomplete`, `vendor_semantics_unresolved` and `ambiguous_line_correspondence`
reasons that discard it before any developer sees it. Four independent admission vetoes
are known: no context producer for dominance/config families, vendor exclusion acting as
a global completion veto, line ambiguity inherited by every question, and query identity
differing from witness identity. Requiring all four repaired before the first visible
result is a full production pass, not one detection.
### M1a — Show the detection with every veto recorded
Status: **MET 2026-09-16**, inside its one-week box. Evidence:
[M1a record](m1a-recorded-vetoes.md) and
[measurement](../../../benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json).
Raw witnesses present at contributions.js lines 32–34; fixed twin analyzed and quiet with
zero rows and no deadline reason; unchanged revision analyzed and recorded `unchanged`. The
kill criterion did not trigger. Five vetoes fired and were recorded, and M1a corrected the
diagnosis: with all five lifted the detection stops at `operation_correspondence_unresolved`,
a fifth veto that was not previously named. No defect was admitted and no capability enabled.
- Use the pinned NodeGoat contribution injection case, its fixed twin and an unchanged
vulnerable revision as development data. Record exact base/head revisions, expected
operation and outcome before running. Exercise the shipped replay path, existing ranker
and physically vendor-free graph; do not substitute a hand-selected slice or hard-coded
target function.
- Run all three revisions under an explicit experimental evaluation flag that **records**
each admission and comparison veto per finding instead of applying it. The flag is
task-local, off by default, cannot enable any hook capability, and every output it
produces is labeled experimental. A longer, explicitly named development budget applies;
it cannot satisfy the hook's latency gate.
- Deliverable: the raw finding on the vulnerable revision with location and witness, the
fixed twin genuinely analyzed and quiet, the unchanged revision not reported as a new
regression, and for every finding the exact list of vetoes that fired and why. Silence
from failure is not a pass; a fixed twin that timed out or was not analyzed fails M1a.
**Exit evidence:** a replayable three-revision record under the experimental flag with
exact revision, question and operation provenance, the per-finding veto list, and readable
source and census receipts. **Kill criterion:** if the fixed twin is not quiet under the
recorded-veto run, M1 has a detector problem, and M1b does not start until that is
diagnosed and recorded.
### M1b — Repair only the vetoes M1a showed to matter
Status: **NEXT — not verified**. M1a supplied the veto list: per-family change context,
question-owned vendor relevance, operation correspondence for an operation on a changed line,
file-wide correspondence ambiguity scope, and query-to-operation provenance. The owner
diagnosis expected the first and the last two; operation correspondence is the addition M1a
measured and is the blocker that survives every other repair.
- From the M1a veto list, repair only the context/family/identity contracts this case
actually needs. The owner diagnosis expects query-to-operation provenance and question-
owned vendor relevance to be the two that fire here; M1a confirms or corrects that.
Vetoes that did not fire on this case stay as they are and their families stay
experimental. Resolve vendor and correspondence boundaries through evidence of dependency
and ownership; unknown relevance remains unresolved.
- Rerun the three revisions **without** the recorded-veto flag through the production
evidence and comparison rules. Demonstrate a developer-readable evaluation finding with a
supported security consequence, exact location, witness, change attribution and concrete
repair direction. Verify the fixed twin is analyzed and produces no actionable defect, and
the unchanged vulnerable revision is not reported as a new regression.
- Keep qualification eligibility disabled. The demonstration uses explicit experimental
evaluation output; it cannot advertise an unqualified normal hook alert.
**Exit evidence:** a replayable vulnerable/fixed/unchanged comparison record through the
production rules, exact question and operation provenance, completed required checks,
reviewed explanation, readable source and census receipts, task-local regression
verification, and a list of which vetoes were repaired and which remain. Raw findings, rank
positions, synthetic-only controls and screenshots alone do not satisfy M1b.
Owners: contributor-scan for query/operation/context evidence; pre-push-safety-net for
comparison, admission, the recorded-veto flag and reporting. Requirements: 1.1–1.4,
2.1–2.4, 3.1–3.5, 6.3, 7.1–7.3.
## M2 — The same useful analysis fits the push budget
Status: **MEASURED AND UNMET, 2026-09-17.** Two attacks on engine startup were measured and
neither reaches the gate. A transaction-owned session is implemented and proven equivalent for
every shipped query kind, and it amortizes startup only across a long transaction: 326 s to 243 s
under a development budget, while under the 30-second deadline its own 9 to 13 second startup
means it usually declines to start. Removing JavaScript's redundant second engine start is a real
11.0 s against 21.0 s on identical graphs, but the attempt was reverted for an asymmetric census
gate that made 12 profile transactions pay two engine starts.
The profile names the actual cost: a changed-code transaction reaches its deadline inside the
build, head build 25.2 s of 30 s with query time 0.0 s. It is building a fresh whole-repository
graph for every pushed revision.
**Direction accepted 2026-09-17: do not rebuild a whole graph per push. Reuse the graph across
revisions instead of further optimising the rebuild.** The 30-second and 2-second gates are
unchanged; this milestone is not met, and the work is not scheduled here.
- Use the successful M1 case as a correctness baseline for a transaction-owned Joern
session within the existing backend. The measured census probe supports investigating
this design; it does not establish full-query equivalence or production readiness.
- Verify equivalent taint, dominance and configuration answers under matching semantics,
fresh request receipts, isolated graph identities, cache invalidation, crash handling
and process-group cancellation. Keep one deadline across the complete transaction.
- Rerun the frozen seven-class runtime profile with changed-code cases, cold starts,
dependency/configuration edits, growth and multi-ref transactions. Retain all timeouts
and unresolved checks. Identical-tip reuse cannot substitute for warm changed-code work.
**Exit evidence:** the M1 outcome remains correct; representative warm changed-code p95
is at most 30 seconds, cancellation/reporting at most two seconds, and supported-check
completion at least 95%, under the predeclared profile. A fast census alone does not pass.
Owner: contributor-scan runtime, with pre-push-safety-net deadline/cache integration.
Requirements: 4.1–4.4, 6.2–6.5, 7.3.
## M3 — The shared design transfers across languages
Status: **PARTIAL, 2026-09-17. Not met.** Python is established and PHP is blocked by runtime.
Python: all three declared VAmPI targets detected at their declared lines with their questions
answered, 35 of 35 questions complete on every case, nothing reported new on identical revisions,
zero admitted defects. That includes the SQL injection this spec names as previously unresolved,
and both broken-object-level-authorization targets, which is the first time access control has
answered on Python at all.
PHP: not established, and the blocker is now named. All PMPro comparisons exhausted every budget
tried, up to a 4500-second deadline with a 900-second query ceiling, and produced no artifact, so
there is no target evidence. Splitting a failed batch into parts did not rescue it.
A probe narrowed the cause: the evidence pass answers all 4,425 PMPro regions in one batch in
29.7 seconds with a 681 KB response, and its cost is fixed rather than per request. Neither
request count nor response size is anywhere near a limit. What fails is the execution pass, which
runs `reachableByFlows` per candidate. The pins, receipts and declared target are correct, so the
question is being asked properly and the dataflow work is what cannot finish.
The exit asks for Node, PHP and Python together. Two of three are in hand.
- Repeat the useful vulnerable/fixed comparisons with PMPro/PHP and VAmPI/Python,
including the previously unresolved VAmPI SQLI query/path evidence. Preserve explicit
limits for authorization and other families whose evidence is still unsupported.
- Keep framework knowledge in frontends/adapters and versioned facts. Equivalent normalized
evidence must retain the same core ranking, scope and admission behavior.
- Record core and adapter changes separately. Any PMPro-driven core change must be frozen
and replayed on Node/Express before claiming transfer; the initial Node-first repair
does not remove that existing requirement.
**Exit evidence:** reviewed development/regression results for Node, PHP and Python,
including positive, fixed and unchanged controls, exact provenance, coverage and runtime.
Do not label these repeatedly inspected projects independent qualification. C/C++ bounds
remain a separate unsupported abstraction; adding a new engine is not a prerequisite here.
Owners: contributor-scan shared evidence and framework adapters; pre-push-safety-net evaluation.
Requirements: 6.2–6.4, 8.1–8.4.
## M4 — Independent examples establish useful detection and low noise
Status: **PENDING M3**. Population preparation may proceed earlier, but evaluation follows
the frozen candidate. Reserve untouched cases before examining their scan results.
- Freeze reviewed vulnerable, fixed and benign changes from projects not used to develop
or tune the particular language/framework/family. Ghost remains reserved until its
actual reviewed population exists. Node evidence cannot qualify PHP or Python.
- Measure actual actionable alerts, supported recall, fixed-side silence, benign interruptions,
completed coverage and runtime together, with counts and uncertainty. Keep unlocated
targets, unanswered queries and unsupported cases visible in their declared populations.
- Apply the existing acceptance profile: at least 95% observed actionable precision,
at least 90% supported recall, at least 95% supported-check completion, the 30s/2s runtime
gates, nonzero reviewed emitted alerts and positive controls, and zero known fixed/benign
false alerts. Silence on everything fails. A core retune requires new untouched evaluation.
**Exit evidence:** reproducible, reviewed per-capability qualification with every mandatory
gate passing together. Report failures honestly and leave those capabilities experimental;
do not require every possible language or family to become eligible for a narrow release.
Owner: pre-push-safety-net task 8.3, consuming verified contributor-scan semantics.
Requirements: 6.1–6.5, 8.1–8.4 and the accepted experimental acceptance profile.
## M5 — Enable a narrow advisory rollout
Status: **PENDING M4**.
- Generate eligibility bound to the exact passing runtime, query, fact, ranking and policy
identities. Enable only independently qualified capabilities; stale or missing evidence
enables none. Advisory retains the same evidence bar as blocking.
- Revalidate the packaged vulnerable/fixed/benign push experience, compact reporting,
truthful incomplete coverage, local no-model operation and preservation/removal of hooks.
- Publish the supported scope, limitations and evidence with the advisory release.
Expand capabilities only when they earn qualification. Blocking deployment is a later,
explicit decision, even though opt-in blocking plumbing already exists.
**Exit evidence:** passing packaged acceptance validation and exact eligibility for the
declared advisory scope, with the release decision recorded. Only then mark task 8.3
and release readiness complete if their mandatory criteria are satisfied.
Owner: pre-push-safety-net admission, packaging and release validation.
Requirements: 3.1–3.5, 5.1–5.5, 6.5, 7.1–7.4.
## Execution and reporting contract
Use M1a → M1b → M2 → M3 → M4 → M5 as the critical path. Preparation may overlap, but later
milestones cannot substitute for missing earlier evidence. Necessary runtime repairs
that make M1 executable are allowed; a standalone speed benchmark cannot close M1.
Keep ranker-owned scope, vendor exclusion, language-agnostic core behavior and all
acceptance thresholds unchanged. No new source IR, second detector, differential graph
mutation, learned ranker or unrelated backlog is introduced by this amendment.
Before production edits, make the required owner design/task amendment concrete and
apply the existing Kiro approval, review and verification workflow. Prior authorizations
remain valid within their scope; this milestone record does not reset them.
Progress updates shall lead with the current milestone, observed developer outcome,
remaining blocker and next concrete action. Report implementation task counts separately.
Change a milestone to verified only with linked evidence; do not infer progress from
elapsed effort or a completed experimental release tag.
Immediate next action (M3, task 13.3): re-run the declared PHP transfer replay bounded to a
region budget whose questions complete, using the corrected census verdict, and record the
outcome either way. Then task 13.4, PHP question completion, because admission reads outcomes
and M4 measures completion at 95%. Task 15.1 (M4 population reservation) runs in parallel and
blocks on nothing. Do not begin another family vocabulary campaign: the census has said which
families can speak, and the constraint has moved to runtime and to evidence from code this
project has not seen.
@@ -2,6 +2,11 @@
Status: approved for task generation on 2026-09-12 following the maintainer's "ok, next kiro phase". The acceptance profile defines evaluation targets, not measured achievements.
Release sequencing amendment accepted 2026-09-14: [release milestones](release-milestones.md)
defines M1 usefulness → M2 speed → M3 shared-language transfer → M4 independent qualification
→ M5 narrow advisory rollout, with observable exit evidence. The requirements and numerical
acceptance thresholds below remain unchanged.
## Introduction and Boundary
OpenUltraSAST helps developers catch concrete security regressions in the resulting commits they
+77 -11
View File
@@ -1,7 +1,7 @@
{
"feature_name": "pre-push-safety-net",
"created_at": "2026-09-12T15:35:00Z",
"updated_at": "2026-09-14T06:10:56.975614+00:00",
"updated_at": "2026-09-16T17:15:00+00:00",
"language": "en",
"phase": "implementation",
"approvals": {
@@ -39,17 +39,17 @@
{
"at": "2026-09-12T16:20:40.814596Z",
"source": "approve.",
"action": "Accepted the generated task plan and authorized starting group 1 (1.1\u20131.4). Implementation started; tasks require independent review and fresh verification before completion."
"action": "Accepted the generated task plan and authorized starting group 1 (1.1–1.4). Implementation started; tasks require independent review and fresh verification before completion."
},
{
"at": "2026-09-12T18:32:19.033933Z",
"source": "ok, next kiro phase, approved; correct. we can continue with the spec",
"action": "Authorized group 2 (2.1\u20132.3), resumed after local Claude Mem repair. Independent review and fresh verification remain required."
"action": "Authorized group 2 (2.1–2.3), resumed after local Claude Mem repair. Independent review and fresh verification remain required."
},
{
"at": "2026-09-13T07:48:02.155687Z",
"source": "continue with the task",
"action": "Authorized continuing group 3 (3.1\u20133.4) after the group 3 next-step summary. Independent task review and fresh verification remain required."
"action": "Authorized continuing group 3 (3.1–3.4) after the group 3 next-step summary. Independent task review and fresh verification remain required."
},
{
"at": "2026-09-13T09:17:01.788332+00:00",
@@ -59,7 +59,7 @@
{
"at": "2026-09-13T10:26:47.307154+00:00",
"source": "move on to group 4 as a whole",
"action": "Authorized all of group 4 (4.1\u20134.3), sequential implementation, independent task review and fresh verification. Groups 5\u20138 remain pending."
"action": "Authorized all of group 4 (4.1–4.3), sequential implementation, independent task review and fresh verification. Groups 5–8 remain pending."
},
{
"at": "2026-09-13T12:40:57.961577+00:00",
@@ -69,17 +69,22 @@
{
"at": "2026-09-13T13:58:52.336506+00:00",
"source": "continue with group 6 as a whole. do we need to amend the spec to get a go for the rollout",
"action": "Authorized group 6 (6.1\u20136.4). Clarified complete preparation/discovery reuse after measured group 5 bottleneck; rollout gates remain unchanged. Manual Kiro review fallback remains necessary after a fresh agent dispatch hit the thread limit."
"action": "Authorized group 6 (6.1–6.4). Clarified complete preparation/discovery reuse after measured group 5 bottleneck; rollout gates remain unchanged. Manual Kiro review fallback remains necessary after a fresh agent dispatch hit the thread limit."
},
{
"at": "2026-09-13T15:45:15.902603+00:00",
"source": "ok, continue with group 7 as a whole",
"action": "Authorized tasks 7.1\u20137.3; manual Kiro fallback after fresh dispatch hit thread limit. Group 8 and rollout remain pending."
"action": "Authorized tasks 7.1–7.3; manual Kiro fallback after fresh dispatch hit thread limit. Group 8 and rollout remain pending."
},
{
"at": "2026-09-13T16:41:21.889949+00:00",
"source": "move on to group 8",
"action": "Authorized tasks 8.1\u20138.4 and their qualification work; preserve joint gates and record NO-GO wherever prerequisites or measurements do not pass. Fresh dispatch hit thread limit; manual Kiro fallback."
"action": "Authorized tasks 8.1–8.4 and their qualification work; preserve joint gates and record NO-GO wherever prerequisites or measurements do not pass. Fresh dispatch hit thread limit; manual Kiro fallback."
},
{
"at": "2026-09-16T13:30:00+00:00",
"source": "add the M1a task to tasks.md and start it; continue with the plan",
"action": "Added group 9 (M1a) to the approved task plan as the bounded owner amendment the milestone requires, implemented the task-local recorded-veto flag (9.1) and ran the three NodeGoat revisions (9.2). Diagnosis only: no capability enabled, no gate changed, rollout remains NO-GO."
}
],
"task_review": {
@@ -118,7 +123,18 @@
"7.3",
"8.1",
"8.2",
"8.4"
"8.4",
"9.1",
"9.2",
"10.1",
"10.2",
"10.3",
"10.4",
"10.5",
"10.6",
"10.7",
"10.8",
"10.9"
],
"remaining_tasks": 1,
"verification_record": "implementation-group-8.md",
@@ -132,11 +148,11 @@
"implementation-group-7.md",
"implementation-group-8.md"
],
"next_group": 8,
"next_group": 10,
"blocked_tasks": [
"8.3"
],
"feasibility": "NO-GO: all 21 frozen NodeGoat transactions timed out; 0/24 declared target checks completed. Cancellation/reporting remained below 2 seconds. Priming did not qualify graph reuse; named census and engine lifecycle work routed to contributor-scan.",
"feasibility": "NO-GO: post-census profile measured all 21 cases; 18 timed out, three identical-tip replays completed in 18.521–18.874s with valid reuse; 0/24 target checks completed. Maximum cancellation/report overrun 0.874s. See 2026-09-14-nodegoat-post-census-feasibility.json.",
"qualification": "NO-GO: eligibility implementation and evidence delivered; required independent populations and verified VAmPI transitive witness remain missing. Zero capabilities enabled.",
"validation": {
"decision": "NO-GO",
@@ -149,6 +165,35 @@
"status": "VERIFIED",
"record": "../contributor-scan/census-repair-verification.md",
"effect": "Node 44/44 input census restored; partial/missing census remains explicitly unresolved. Existing eligibility stale; runtime/context/independent qualification remain NO-GO."
},
"remediation_research": "../contributor-scan/release-remediation-research.md",
"m1a": {
"status": "MET",
"date": "2026-09-16",
"record": "m1a-recorded-vetoes.md",
"measurement": "../../../benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json",
"kill_criterion_triggered": false,
"result": "Raw <lambda>2 witnesses present at contributions.js 32-34; fixed twin analyzed and quiet (target question answered with zero rows, no deadline reason); unchanged revision recorded 'unchanged'. Five vetoes fired and were recorded. With all five lifted the detection stops at operation_correspondence_unresolved, a fifth veto not previously named. No defect admitted, no capability enabled."
},
"m1b": {
"status": "IN_PROGRESS",
"verified": [
"10.1",
"10.2",
"10.3",
"10.4",
"10.5",
"10.6",
"10.7",
"10.8",
"10.9"
],
"remaining": [
"10.10"
],
"measurement": "../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-exit.json",
"result": "Exit run recorded. All three pinned NodeGoat revisions classify through the production rules: vulnerable three sites new with operation_absent_from_comparable_base and admission rejecting on capability_unavailable alone; fixed twin analyzed and quiet with three base-only findings; unchanged recorded unchanged with base locations resolved. Every veto M1a listed is repaired, plus two the repairs exposed. Zero admitted defects.",
"remaining_gap": "The supported security consequence and concrete repair direction are not demonstrated. Both are owned by an evaluated capability declaration and the registry is deliberately empty. Task 10.10 renders them through an explicitly experimental declaration with eligibility still disabled; inventing the text is what admission exists to prevent."
}
},
"release": {
@@ -157,5 +202,26 @@
"channel": "alpha",
"rollout_decision": "NO-GO",
"status_record": "status.md"
},
"release_milestones": {
"record": "release-milestones.md",
"direction_approved": true,
"approval_source": "I agree. please note this into the raodmap as a spec",
"current": "M1b",
"verified": [
"M1a"
],
"sequence": [
"M1a: NodeGoat detection with admission vetoes recorded, not applied (MET 2026-09-16)",
"M1b: repair only the vetoes M1a showed to matter; rerun through production rules",
"M2: equivalent bounded runtime",
"M3: shared-language transfer",
"M4: independent qualification",
"M5: narrow advisory rollout"
],
"implementation_design_approval": "Existing approvals preserved; this records milestone order and exit criteria, not blanket approval of future design changes.",
"m1_split_accepted": "2026-09-14: maintainer asked to split M1 into M1a/M1b after review",
"m1a_outcome": "Met inside the one-week box. Corrected the owner diagnosis: all four named vetoes fire, and operation correspondence for an operation on a changed line is the blocker that survives every other repair.",
"m1b_position": "Comparison half verified and recorded on NodeGoat; the consequence and repair rendering (10.10) remains before M1b is verified."
}
}
+203 -10
View File
@@ -1,10 +1,70 @@
# Pre-push safety net status — v1.2.0-alpha.1
Updated 2026-09-14. Phase: **implementation**. Requirements, design and tasks are
approved. **26/27 executable tasks complete; task 8.3 remains blocked.**
Updated 2026-09-16. Phase: **implementation**. Requirements, design and tasks are
approved. **28/29 executable tasks complete; task 8.3 remains blocked.**
Rollout: **NO-GO**. The alpha tag publishes experimental implementation and evidence;
it does not approve normal hook capabilities or production enforcement.
## Next release milestone
**M1a is met and the M1b comparison now works through the production rules.** The other four
milestones remain unverified, and rollout remains NO-GO.
Every veto M1a recorded has been repaired. On 2026-09-16 the production comparison classified
the pinned NodeGoat revisions without any lifting: the vulnerable revision is `new` with the
three `eval` witnesses attributed to the change that introduced them, the fixed twin is
analyzed and quiet, and the unchanged revision is `unchanged`. Production and the recorded view
agree exactly, which is how we know the vetoes were repaired rather than bypassed. No capability
is enabled, no defect is admitted and no alert is emitted. See
[M1a and M1b evidence](m1a-recorded-vetoes.md).
The exit run is recorded. Every veto M1a listed is repaired, plus two the repairs exposed: the
comparison's own degradation and boundary checks, and the runner handing every candidate the
whole transaction's gaps. On the vulnerable revision admission now rejects on
`capability_unavailable` alone.
One half of M1b remains. The milestone also asks for a developer-readable finding carrying a
supported security consequence and a concrete repair direction. Both are owned by an evaluated
capability declaration and the registry is deliberately empty, so nothing renders them. Writing
that text without a declaration is what admission exists to prevent, so it is not worked around.
Task 10.10 renders it through an explicitly experimental declaration with eligibility still
disabled.
M1a ran on 2026-09-16 within its one-week box. The raw `<lambda>2` witnesses are present at
contributions.js lines 32–34, the fixed twin was genuinely analyzed and quiet, and the
unchanged revision was not reported as a new regression, so the kill criterion did not
trigger. Five vetoes fired and were recorded; with all five lifted the detection still stops
at `operation_correspondence_unresolved`, because a newly introduced operation sits on a
changed line and has no base anchor. That fifth veto was not in the earlier diagnosis and is
now M1b's central repair. See [M1a evidence](m1a-recorded-vetoes.md).
The 28/29 task count is implementation progress, not proximity to rollout. Follow the
accepted [release milestone spec](release-milestones.md): usefulness → speed → shared
PHP/Python transfer → independent qualification → narrow advisory rollout.
## Security capability, 2026-09-17
Group 12 closed three defects that limited what the tool can establish, each measured on the
pinned NodeGoat revisions with detections unchanged throughout.
- **Access control could not answer anywhere.** Dominance and configuration were never asked for
change context, because the evidence pass dispatches taint only. Each family now describes its
own scope. Head completion rose from 205 to 298 of 436, `context_projection_unavailable` fell to
zero, and access control went from 0 to 52 completed questions.
- **Admission rejected genuine findings.** The operation symbol was the text before the first
parenthesis, so `const preTax = eval(...)` was the operation `const preTax = eval`, which no
qualified declaration could ever name. It is now the trailing qualified callee, with receiver
spelling preserved.
- **Unresolved questions said why, and then there were none.** The remaining 138 each named a
function its file does not define. A route-registration module registers handlers other modules
define, and regions were scoped to methods those files do not hold. The entry record keeps the
handler name, because the authorization question is cross-file by nature, and the region is now
scoped to the file instead. **Every question the scan asks now completes on both revisions,
299 of 299**, with detections unchanged.
Evidence: [group 12 final](../../../benchmarks/measurements/2026-09-17-group-12-final.json) and
[region attribution](../../../benchmarks/measurements/2026-09-17-region-attribution.json).
## Verified
Immutable pushed snapshots, shared multi-ref deadlines, ranker-owned scope, physical
@@ -21,13 +81,16 @@ gates and a packaged native PHP/JavaScript partition smoke. See
## Rollout blockers
1. **Useful checks within the deadline are unproven.** The last full representative
runtime profile timed out on all 21 Node transactions, completing 0/24 declared target
checks. Cancellation/reporting stayed below the two-second allowance. The subsequent
census repair is verified, but the full changed-code runtime/coverage profile has not
passed under its new identities. Amortize engine startup/loading work and rerun the
frozen seven-class profile; require warm p95 <=30s and >=95% completed supported checks.
2. **Target and change-context evidence remains incomplete.** VAmPI preserves raw findings
1. **Useful checks within the deadline are unproven.** The fresh post-census profile
completed all 21 measurements: all 18 changed/cold/growth/multi-ref transactions timed
out; the three identical-tip replays took 18.521–18.874s with valid graph/query reuse.
Still 0/24 declared target checks completed. Maximum observed cancellation/report
overrun was 0.874s, below the two-second allowance. After M1 establishes the useful
comparison, amortize engine startup/loading and rerun the frozen seven-class profile; require warm p95 <=30s and >=95% completed supported checks.
2. **Target and change-context evidence remains incomplete.** M1a measured exactly which
contracts fail on the Node case: per-family change context, question-owned vendor
relevance, operation correspondence for changed lines, file-wide correspondence ambiguity
and query-to-operation provenance. VAmPI preserves raw findings
at all three known sites, but the exact SQLI function question/transitive witness and
supported authorization context are unresolved. Raw findings or rank positions cannot
establish an actionable new regression. Repair this in the shared scan boundary and
@@ -60,5 +123,135 @@ quality evaluation and packaged hook validation. Keep task 8.3 unchecked and the
in implementation until those mandatory results are established.
Evidence: [group 8 validation](validation-group-8.md),
[runtime scorecard](../../../benchmarks/measurements/2026-09-13-nodegoat-feasibility.json),
[post-census runtime scorecard](../../../benchmarks/measurements/2026-09-14-nodegoat-post-census-feasibility.json),
[census repair](../../../benchmarks/measurements/2026-09-14-node-census-repair.json).
The owner investigation now distinguishes missing family context producers, globally
propagated vendor/correspondence gaps, and query-to-operation identity. See
[remediation research](../contributor-scan/release-remediation-research.md). These are
next implementation prerequisites, not completed fixes or changed acceptance gates.
An isolated Joern session probe matched the full Node census with four warm requests in
0.221–0.467s and verified cancellation/reaping in 0.043s. It identifies a promising engine
lifecycle change; full-query equivalence and changed-code performance remain unverified.
## Family detection census, 2026-09-19
Four of seven declared families establish findings on the two development subjects:
injection, access_control, config_secrets and untrusted_destination. Output encoding, path,
prototype and deserialization have established nothing anywhere. Every question completed on
both subjects, 299 of 299 on NodeGoat and 35 of 35 on VAmPI, so the silences are answers
rather than failures — but three of them are diagnosed gaps rather than clean code:
NodeGoat's documented SSRF is invisible because `needle.get` is not in the sink vocabulary,
its XSS is invisible because output encoding declares only DOM sinks, and its two
access-control defects raise no obligation because the modelled data operations live in DAO
files while the family's scope is the handler's own file.
The census also found and fixed one false positive: NodeGoat's only prototype report was
the word `set` in a code comment, matched because the sink text clause searched a whole
assignment node. Nothing about this changes the release gates. No capability is qualified by
these subjects.
Two of the three gaps are closed. The untrusted-destination vocabulary now holds the client
surface a Node server actually calls, and NodeGoat's SSRF is reported at
`app/routes/research.js:16` with a witness naming the call. Output encoding now asks about the
server half of its own family and establishes a finding at `app/routes/research.js:25`, and it
states in its declaration that template-mediated escaping is not something it can decide. Five
of seven families now establish findings.
Access control was the third gap and its structural half is closed. The obligation now follows
the one named call a handler writes into its data-access module, attributed by the graph's own
provenance rather than by name, and located at the call a contributor changed. On NodeGoat it is
raised at six handlers where it was raised nowhere, and three reach a verdict. All three are
corroborated and none is reported without a judge, which is the honest rung: entailing a carried
obligation reported a signup handler that is public by design, because its authenticated
siblings share the same data module. VAmPI is unchanged.
The commonest collection read now carries the obligation too, so NodeGoat's documented insecure
direct object reference raises one where it never had. `find` had been left out because the name
is shared with array searches and jQuery selectors, so it is claimed only in the argument shape
of a query document. Measured on a plugin that ships jQuery, all 88 of its `find` calls stay
unclaimed. Seven NodeGoat handlers now reach a verdict, all corroborated; the census is
unchanged, because the corroborated band needs a judge it does not configure.
What a session read discharges is now decided by where its value goes. A destructuring spells
`session.userId` away, so the arbiter walks back from the names the query is given; two NodeGoat
handlers that were correct code stop being asked about, while the memo page that reads every memo
and merely renders the session's user stays reported. The sibling scope was decided by
measurement and left at the file: on this subject the file scope entails two handlers, both
public routes and both wrong, and an application-wide scope entails four true defects and the
same two wrong ones. Neither scope can tell a public route from a protected one, so the carried
rung stays capped and that question stays with the judge.
Configuration reports every setting it decides, not the first one in the file. NodeGoat's
server.js now carries both its permissive session and its disabled template auto-escaping, each
with its own site; the escaping defect had been detected on every run since task 14.3 and
reported on none, because an earlier setting in the same function won the single verdict.
The PHP record is corrected. The malformed-CPG bug this project reported upstream, a `global`
inside a closure giving a node two AST parents, was fixed in Joern 4.0.625 and the image has
pinned that version throughout; it is verified today on the real trigger file, on 4.0.625 and
4.0.631 alike. What remains under task 13.2 is runtime alone: a 2,526-question batch over a
673-file plugin does not finish in thirty minutes, and a six-fold larger ceiling only made the
failure take longer. The engine pin stays where it is, because the newer release measures the
same here.
That runtime reading was itself incomplete, and the correction matters more than the reading.
PHP is not silent: at 25 and 100 regions PMPro completes with no timeout at all and establishes
19 and 43 findings, including a true-positive SQL injection at
`adminpages/discountcodes.php:749` of exactly the class the PHP fact table predicted. It looked
silent because the census harness ordered its verdict on question completion before findings,
and because only the 500-region runs had been examined. The harness is fixed. What remains is a
scale limit of roughly one to two seconds per question, and one real gap that is bookkeeping
rather than detection: no PHP question is ever marked completed, even in the runs that find
things.
Closing the second gap surfaced four defects, three of which were false positives it would
otherwise have shipped: a permissive-value set shared across settings whose booleans have
opposite polarity, a dotted setting matched on its trailing segment alone, a flow query asked of
a call's receiver as well as its arguments, and a file's settings ordered by the spelling of
their line numbers. Template auto-escaping disabled at `server.js:135` is detected and still not
reported, because the arbiter returns one verdict per question and that file has an earlier
permissive setting; task 14.6 carries it.
Evidence: [family detection census](../../../benchmarks/measurements/2026-09-19-family-detection-census.json),
[untrusted destination vocabulary](../../../benchmarks/measurements/2026-09-19-untrusted-destination-vocabulary.json),
[output encoding family](../../../benchmarks/measurements/2026-09-19-output-encoding-family.json),
[access control across the module split](../../../benchmarks/measurements/2026-09-20-access-control-module-split.json),
[collection read obligation](../../../benchmarks/measurements/2026-09-20-collection-read-obligation.json),
[sibling scope and identity](../../../benchmarks/measurements/2026-09-20-sibling-scope-and-identity.json),
[every permissive setting](../../../benchmarks/measurements/2026-09-20-every-permissive-setting.json),
[PHP runtime after the graph fix](../../../benchmarks/measurements/2026-09-20-php-runtime-after-graph-fix.json),
[PHP detection corrected](../../../benchmarks/measurements/2026-09-20-php-detection-corrected.json).
## Strategy, 2026-09-21
Sequence amended after the family census and the PHP correction: close M3 at a completing scale
first (task 13.3), fix PHP question completion because admission reads outcomes (13.4), reserve
the untouched M4 population in parallel starting now (15.1), and give the remaining engineering to
M2 graph reuse, which cannot be scoped away because meeting the latency gate by analysing less is
explicitly not the same result. The v1 advisory scope is declared as the families whose claims can
be admitted at the entailed rung (15.2): injection, untrusted destination and configuration.
Access control is excluded because a carried obligation corroborates by design and would ship as
silence.
No acceptance threshold and no owner boundary changes. The risk that remains is that every number
so far comes from subjects used to build the analyser, and one week of examining two of them
surfaced seven false positives; the first independent evaluation should be expected to fail.
## PHP reaches its declared target, 2026-09-22
The M3 PHP target is detected: `classes/class.memberorder.php:936:getMemberOrderByCode`, a request
value concatenated into SQL with no prepare. It had never been reached, and nothing about detection
changed to reach it. The population became affordable.
Four repairs did that, and three of them are the same rule the design now carries. Tier-zero pruning
was switched off for a whole scan by any degradation, and a multi-language repository always carries
one. Batches were sized by request count when the cost is per matched sink, so a family whose sinks
are `echo` and `print` was priced like one whose sinks are rare. A per-query timeout cancelled the
entire scan, abandoning work with half an hour of budget left. And nothing was held back for
arbitration, so a scan could spend everything on queries and report none of what they answered.
On the 500-region population that previously completed zero questions at every budget tried: 271 of
406 completed, 213 findings, 2,950 seconds. The three declared comparisons are running at that
budget; detection is not transfer, and only the comparison decides M3.
@@ -0,0 +1,120 @@
# Strategy review after the independent evaluations (2026-09-28)
Status: for maintainer decision. Nothing below changes the approved specs until a direction is chosen.
## 1. What was measured
Two populations of real vulnerabilities that no rule had seen, each with its fix and a benign control, scored by
protocols committed before the first scan (`benchmarks/independent/`).
| | v1 (11 cases, 2026-09-24) | v2 (17 cases, 2026-09-28) |
|---|---|---|
| Analyzer | 13180a1 | a75b9f6 (v1 plus every retune learned from v1) |
| Recall | 1 / 11 | **0 / 17** (Wilson 95% 0–0.18) |
| Sampled precision | 1 / 12 | **0 / 17** (0–0.18) |
| Question completion | near zero | **12%** (13,206 / 108,374) |
| Benign false alerts | — | 3 |
| Run-to-run stability | not measured | 2 of 100 findings differ |
Between v1 and v2 the analyzer received every fix v1 suggested: file-owned graph gaps, `.gitignore` semantics,
Node `vm` sinks, Django sources, quoted-only SQL escapes, fixed-origin redirects, guards and allowlists. On the
v1 cases those retunes raised detection from 1 to 4 of 11. On untouched cases they transferred to nothing.
## 2. Why, per class of cause (observed in v2, from source)
1. **Entry points are framework-shaped, and our sources are token lists.** Request data arrives as
`filter_input(INPUT_GET, …)` and the WordPress REST `$request` (GoPay, Directorist), plugin helpers such as
`get_url_var()` (Groundhogg), FastAPI and NestJS typed or decorated handler parameters (qwed, ReactPress,
LMDeploy, ContextForge), and Koa's `ctx.request` (Budibase). A vocabulary grows one framework at a time, and
each new population brings frameworks it lacks.
2. **Flows cross module boundaries the frontends do not resolve.** JavaScript `require`d modules (FUXA's route
to its storage module), dependency-injected receivers (YesWiki, v1), registry dispatch (OpenCVE, v1), parser
combinators (alerta, v1).
3. **Sinks are matched by name, so precision fails where names collide.** Every sampled v2 finding was false:
a regex `.match()`/`.exec()` read as injection, `header()` with an integer cast, parameterised ORM statements,
client-SDK code, a process's own `argv`, a generated database name.
4. **Scale.** Large repositories complete a few percent of their questions inside a 40-minute budget (pgAdmin
2%, ContextForge 6%, Budibase 1%); one repository crashes the taint batch on every scan (LMDeploy).
Classes 1–3 are breadth problems: each fix is correct and local, and the next repository needs another. That is
the pattern both evaluations show, and the reason more patching on the same design is unlikely to reach M4.
## 3. What we know about the alternative
The tool hunter (a model with `read_file` / `grep_repo` / `find_refs`) was measured once, on 2026-09-06, on the
vibe-py pair holdout: **12 of 14 detected, Youden +0.857, no leaks**, against the overlay's 5 of 14 on the same
pairs (class-aware scoring; `benchmarks/measurements/2026-09-06-vibe-py-holdout-hunter-*.json`). Caveats: one
language, curated excerpts, a pair corpus rather than whole repositories, and never run on an independent
population. Classes 1–3 above are exactly where a model reading the code has an advantage over token lists;
class 4 (cost, scope) is where it has a disadvantage.
## 4. Options
**A. Continue engine-first.** Model framework entry points, fix the sink-collision classes, raise completion,
then qualify on a v3 population. Every item is sound; the evidence (v1 → v2) says the breadth does not converge
at the pace one population teaches. Highest effort, weakest evidence.
**B. Model proposes, engine proves.** The hunter supplies recall; the CPG, where it can reach, supplies the
witness and admission evidence; findings the engine cannot confirm are reported as unconfirmed rather than
dropped. This inverts `propose-adjudicate-prove`'s current out-of-scope line ("making an LLM the adjudicator")
only partly: the model proposes, it does not adjudicate. Needs a cost and scope design (which files a pre-push
scan hands the model) and a run of the **unchanged** hunter on an untouched population before any tuning.
**C. Narrow the product.** Qualify one ecosystem first (for example WordPress plugins, the declared v0.1 target),
where a framework model is finite, and ship only that capability (M5 already allows per-capability enablement).
Compatible with A or B.
## 5. Recommendation
Run the decisive, cheap experiment before choosing: **the unchanged tool hunter on the v2 pins**, scored with
protocol v2's matching, within a stated model-cost budget. v2 is spent for the engine, but the hunter was never
tuned on it; the rule is that no hunter prompt, tool or configuration may change before that measurement is
recorded, and the scope handed to it must be decided and written down first (it must not use the fix diff or the
declared sites, which would leak the answer).
- If the hunter reaches useful recall on v2 with few fixed-side and benign alerts, re-plan M2–M4 around B,
narrowed by C.
- If it does not, pursue A narrowed by C, starting with the WordPress entry-point model and the sink-collision
precision classes, and qualify on a v3 population.
## 6. Decisions needed
1. Direction: run the hunter experiment first (recommended), or choose A, B or C now.
2. For the experiment: the scope handed to the model per pin, the model, and the cost ceiling.
3. Whether the roadmap's M1b/M2 work (rendering, runtime) pauses until the direction is chosen.
## 7. Update 2026-09-29: what the experiment showed
Recorded in `benchmarks/independent/results-v2-model-pipeline.json` (exploratory; v2 is spent for tuning).
**Instrument first.** Four defects had made every model-backed result since 2026-09-10 unmeasured: the default
model id was the product name and the API rejected it (cf57995); the hunter reached the embeddings provider, not
the chat provider (cf57995); the final-answer turn used the provider's JSON mode and the answers were lost
(3ffb8b4); one grep match in a minified bundle overflowed the model's context (8794884). The "hunter 1/17" number
in section 5's experiment was produced through the broken answer turn and is void.
**Scope selection, source only.** The declared vulnerable function is among the candidates for: the ranker's
top-20 entry points 1/17; the shipped sink vocabulary 2/17; a model reading every product file 15/17 ($14). The
vocabulary route (option A) is confirmed not to converge on this population.
**Verification.** Per-file triage, one 6-step tool hunt per file, two independent passes, report only agreement:
- validation set: 16 of 20 declared sites agreed, 12 of 15 cases; the 26 pgAdmin candidates that flickered
between single runs reduce to 9 stable, 4 disputed, 13 quiet;
- full run, 7 of 17 cases before the account emptied ($23): detected 4/7 (pgAdmin, qwed, LMDeploy,
ReactPress); ContextForge and FUXA found in one pass of two; winml missed because a CORS literal is triaged
"constant" -- the config family needs its own question, not the reachability one;
- cost: $0.009-0.022 per candidate for two passes (old single-pass design: $0.017); a full two-pass run of the
population is roughly $45-50.
**Precision is the open problem.** 196 agreed findings across 7 cases (pgAdmin alone 126 of 875 verified). Five
pgAdmin findings read against source: three true (privileged; raw `{{ data.* }}` in SQL templates, unescaped
stored `schema_res`), two false with rationales that contradict their own verdict (an ORM call the model itself
calls parameterized; a pickle from the signed session). Agreement cannot catch a mistake both passes make. The
proposed next filter is a no-tools judge over each agreed finding's rationale and code (~$0.002 per finding),
which was not run. Two-pass agreement is itself not stable between runs (ContextForge, FUXA).
**Standing.** Recall: the model route reaches 12/15 candidate-site cases where the engine reached 0/17 and the
vocabulary 2/17; that decides between options A and B in the model's favour for recall. Precision, fixed-pin
behaviour and cost at product scale are unmeasured. No gate is met. Any qualification needs population v3 and
a pre-registered protocol that also states the judge stage, the pass rule and the config-family question.
+385 -7
View File
@@ -1,12 +1,16 @@
# Implementation Plan: pre-push-safety-net
Status: task plan approved by the maintainer on 2026-09-12. Groups 1–6 (tasks 1.1–6.4)
are implemented, reviewed and verified (20/27 subtasks). Tasks 4.3 and groups 5–6 used the
explicit Kiro manual review fallback after fresh agent dispatch reached its thread limit.
Group 5's six cold PMPro/Node timeouts remain historical NO-GO evidence. Group 6 verifies
compatible reuse with equal evidence/admission in a packaged authored JavaScript control;
its identical-revision run completed in 2.00s. This is not representative warm p95 or
independent capability qualification. Groups 7–8 remain pending; rollout remains NO-GO.
Status: the existing task plan remains approved. Groups 1–7 and tasks 8.1, 8.2 and 8.4
are verified (28/29 executable subtasks, including group 9); task 8.3 and rollout remain NO-GO. Later work
used the documented manual review fallback where agent dispatch reached its limit.
The accepted [release milestone spec](release-milestones.md) now orders remaining work:
M1a diagnose → M1b repair → M2 speed → M3 shared-language transfer → M4 independent
qualification → M5 narrow advisory rollout. All milestones remain unverified.
Milestones are release outcomes, not checked implementation tasks. Group 9 below is the
bounded owner amendment M1a requires; it adds the task-local recorded-veto evaluation and
its three-revision development run. It enables no capability and changes no gate.
Preserve all completed task evidence below.
## Execution contract
@@ -243,9 +247,383 @@ independent capability qualification. Groups 7–8 remain pending; rollout remai
- _Requirements: 3.1, 3.4, 5.1, 5.2, 5.3, 5.4, 5.5, 6.5, 7.1, 7.3, 7.4, 8.3, 8.4_
- [x] 9. Diagnose the first useful detection with vetoes recorded (release milestone M1a)
- [x] 9.1 Add the task-local recorded-veto evaluation flag
- Add an explicit experimental evaluation path that **records** each comparison and admission veto per raw finding instead of applying it. Keep it task-local, off by default, and available only on explicit `--base`/`--head` replay; refuse it on the Git stdin path so it can never become a hook capability.
- Lift only recognized completeness vetoes over a copy of the evidence: answered-then-demoted question outcomes, the vendor exclusion completion veto, and the diagnosed context/correspondence boundary kinds. Never lift failures: deadline exhaustion, unanswered questions, unreadable input, empty graphs and unparsed files stay applied, because silence from failure is not a quiet fixed twin.
- Retain the production novelty, reason and admission reasons beside the recorded ones for every finding, plus the query-to-witness identity difference, base-only findings and readable census receipts. Every payload carries an explicit experimental label; the recorded path admits no defect and enables no capability.
- Done when the flag is off by default, produces a labeled artifact section and terminal line under explicit replay, is refused in Git stdin mode, and unit controls prove recognized vetoes are recorded while failures and unrecognized boundaries remain applied.
- _Boundary: Delta and Actionability Policy, Push runner, Result and Hook Adapter, CLI integration_
- _Requirements: 3.1, 3.4, 5.5, 6.3, 7.3_
- _Verified 2026-09-16:_ 1,330 tests passed, nine skipped; global Ruff/format and touched mypy passed. Flag off by default, refused in Git stdin mode, labeled artifact section and terminal line under explicit replay. Controls prove recognized vetoes are recorded while deadline exhaustion, unanswered questions, empty graphs and unrecognized boundaries stay applied, and that the recorded path emits no defect even when a lifted candidate would otherwise admit. Regenerated the committed module audit for the added module.
- [x] 9.2 Run the three NodeGoat revisions and produce the per-finding veto list
- Record exact base/head revisions, expected operation and expected outcome before running. Exercise the shipped replay path, existing ranker and physically vendor-free graph; do not substitute a hand-selected slice or hard-coded target function.
- Replay the pinned vulnerable, fixed and unchanged revisions under the flag with an explicitly named development budget. That budget cannot satisfy or measure the hook latency gate.
- Deliverable: the raw finding on the vulnerable revision with location and witness, the fixed twin genuinely analyzed and quiet, the unchanged revision not reported as a new regression, and for every finding the exact list of vetoes that fired and why, with source and census receipts.
- Done when one replayable three-revision record states each expectation against its observed outcome and names the vetoes that actually fired. A fixed twin that timed out or was not analyzed fails M1a and blocks M1b until diagnosed.
- _Boundary: Evaluation harness, full-run integration_
- _Depends: 9.1_
- _Requirements: 1.1, 2.1, 2.4, 3.4, 6.3, 7.1, 7.3_
- _Verified 2026-09-16:_ all three pinned revisions replayed through the production CLI under a named 900-second development budget; expectations frozen on disk first. Raw `<lambda>2` witnesses present at lines 32–34; fixed twin analyzed and quiet (target question answered with zero rows, no deadline reason, three base-only findings); unchanged revision analyzed and recorded `unchanged`. Five vetoes fired and were recorded; the remaining blocker is `operation_correspondence_unresolved` for an operation on a changed line. Zero admitted defects. Evidence: [M1a record](m1a-recorded-vetoes.md) and `benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json`. Diagnosis only; no capability enabled and no latency claim.
- [x] 10. Repair only the vetoes M1a recorded (release milestone M1b)
- [x] 10.1 Establish operation correspondence for an operation on a changed line
- M1a measured this as the blocker that survives every other repair: a newly introduced operation sits on a changed line, has no base line anchor, and therefore cannot be classified even when every completeness veto is lifted.
- Bind the required base/head correspondence to the specific operation identity rather than to a line anchor. A complete base answer for the counterpart question that contains no operation with the same operation and source establishes absence; a matching operation elsewhere in that answer is movement, not novelty.
- Keep the existing conservative rules: an incomplete or failed base stays unknown, benign movement and renames stay unchanged, and lexical alignment is never semantic proof.
- Add absent, renamed, removed-guard, moved-operation and repeated-identical-operation controls. Done when a newly introduced operation on a changed line is classified new, while movement, renames and an incomplete base remain unchanged or unknown.
- _Boundary: Delta and Actionability Policy_
- _Requirements: 2.4, 3.1, 3.4_
- _Verified 2026-09-16:_ 1,337 tests passed, nine skipped; Ruff/format/mypy passed. Correspondence now reports `anchored`, `ambiguous`, `changed_line`, `unmapped` or `unchanged_path`, and only `changed_line` supports an introduced operation. Movement keys on the operation spelling, which the existing benign-rename control proved necessary. One superseded control was updated with its reason; movement, ambiguity, unmapped operations, discharged or unentailed claims and an incomplete base all remain unknown.
- [x] 10.2 Establish the required comparison scope explicitly
- Comparison currently requires every selected base question to complete with no base degradation and no non-tier-zero deferred question, so one unsupported family anywhere prevents an otherwise complete target comparison. The owner research names this and requires the amendment to state the required scope rather than relax the bar.
- Define the comparison's required scope as the counterpart question plus the questions its evidence actually depends on, and require completeness over exactly that scope. Retain every other gap in aggregate coverage; a question outside the required scope stays reported and unresolved, never silently discharged.
- A missing raw finding in an incomplete base still cannot establish novelty. Unknown dependency remains inside the required scope.
- Done when an unrelated unsupported family no longer prevents a complete target comparison, while an incomplete counterpart, an incomplete dependency of it, or unknown dependency ownership still yields unknown.
- _Boundary: Delta and Actionability Policy_
- _Depends: 10.1_
- _Requirements: 2.2, 2.4, 3.4, 6.4_
- _Verified 2026-09-16:_ 1,342 tests passed, nine skipped; Ruff/format/mypy passed. Completeness is now demanded over the counterpart plus its recorded dependencies. Controls prove an unrelated unsupported family no longer blocks a complete target comparison, while an incomplete counterpart, an incomplete recorded dependency, an unknown-ownership boundary and a truncated base scan all still yield unknown. Every other gap stays in aggregate coverage.
- [x] 10.3 Scope correspondence ambiguity to the operations it actually affects
- `ambiguous_line_correspondence` is recorded file-wide and inherited by every question, so repeated unchanged lines invalidate distinct, uniquely mapped operations elsewhere.
- Retain ambiguous mappings as unavailable and record which regions are ambiguous, so only an operation whose required correspondence needs an ambiguous line is vetoed. Do not delete the ambiguity check.
- Done when a uniquely mapped operation is comparable in a file that also contains repeated unchanged lines, while an operation needing an ambiguous line stays unresolved.
- _Boundary: Snapshot Adapter, Delta and Actionability Policy_
- _Depends: 10.2_
- _Requirements: 2.4, 3.4_
- _Verified 2026-09-16:_ 1,347 tests passed, nine skipped; Ruff/format/mypy passed. Ambiguity is no longer inherited as a context gap by every question, and no longer blocks the transaction comparison; it is decided per operation by the correspondence classifier. Controls prove a uniquely mapped or newly introduced operation stays comparable in a file with repeated unchanged lines, an operation needing an ambiguous anchor stays unresolved, and any other transaction gap still blocks. The mapping is still withheld and still reported in aggregate coverage.
- [x] 10.4 Export the enumerated operation inventory so absence is provable
- **Prerequisite discovered 2026-09-16, not in the original plan.** The 10.1-10.3 rerun showed the remaining two vetoes cannot be scoped or relaxed soundly as written. A taint answer reports flows, so an absent row means either that the operation does not exist in the base or that no traced flow reached it. `dynamic_external_or_depth_context_unresolved` exists precisely to stop the second being read as the first, and on a real Express application it is always true, because ordinary framework and dependency calls have unresolved destinations. It therefore carries no discrimination and can never be discharged, while removing it would make an unreached operation look absent.
- Export the operations the query actually enumerated in the question's scope, separately from the flows it traced, so absence of an operation is established by the inventory rather than by the absence of a flow. Sanitizer and source facts are unchanged; this adds no detector, alias resolver or second IR.
- Done when a base answer distinguishes "this operation does not exist here" from "no traced flow reached it", with an unenumerated scope remaining explicitly unresolved.
- _Boundary: contributor-scan query/evidence contract, Delta and Actionability Policy_
- _Depends: 10.1_
- _Requirements: 2.1, 2.2, 3.4, 8.1, 8.2_
- _Verified 2026-09-16:_ 1,354 tests passed, nine skipped; Ruff/format/mypy passed. The taint query exports one row per enumerated operation plus a completion marker, which is what makes an empty inventory meaningful. Comparison now separates three outcomes: operation present in the base without a traced flow, operation absent from an enumerated base, and a base scope never enumerated, which stays explicitly unresolved along with any malformed inventory. The arbiter skips every row carrying a kind, so an inventory row cannot become a flow with an empty source. Real-engine smoke on an authored two-revision JavaScript control confirmed emission, the zero-operation marker and unchanged answer integrity: `benchmarks/measurements/2026-09-16-operation-inventory-smoke.json`. Pre-inventory fixtures were given an enumerated base, since an unenumerated one can no longer establish absence.
- [x] 10.5 Make the external and dynamic context boundary question-owned
- M1a recorded `dynamic_external_or_depth_context_unresolved` on the target's own question. The flag is computed over every in-scope method, so one unresolved call anywhere vetoes every question.
- With 10.4 supplying inventory evidence, bind the boundary to what it actually limits: reachability, not enumeration. A question establishing absence from the inventory no longer needs complete reachability; a question asserting a traced flow still does. Unknown ownership remains a blocker; absent evidence of independence is not proof of independence.
- Done when an absence established from the inventory survives an unrelated unresolved destination, while a reachability claim under a bounded scope stays unresolved.
- _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
- _Depends: 10.4_
- _Requirements: 2.1, 2.2, 8.1, 8.2_
- _Verified 2026-09-16:_ 1,362 tests passed, nine skipped; Ruff/format/mypy passed. The boundary is now bound to what it limits. At scan level it is recorded against its owning question without demoting that question's answer, while an unavailable projection, an empty scope and an unreadable extent still demote. In comparison it no longer blocks the head's own traced flow, and no longer blocks an absence read off a structural enumeration, but still blocks every claim resting on a flow not having been traced. The query declares its enumeration basis, because a scope followed through the call graph can be shrunk by the same unresolved destination; such an inventory cannot establish absence while reachability is bounded. Controls cover both directions and both bases; every other gap kind still blocks.
- [x] 10.6 Stop vendor exclusion acting as a global completion veto
- `vendor_semantics_unresolved` degrades every otherwise complete outcome to `graph_incomplete`, including on a complete 44-file first-party census with successful query receipts.
- Keep the exclusion physical and keep the coverage note. Let the inventory and reachability evidence from 10.4 and 10.5 carry whether a specific answer needed excluded semantics; a recorded dependency on an excluded path still blocks that question.
- Done when an unrelated vendor exclusion no longer invalidates a first-party answer, while a question with evidence of dependency on excluded code remains unresolved and the exclusion stays out of targets and graphs.
- _Boundary: contributor-scan graph/scope evidence, Cache and Engine Boundary_
- _Depends: 10.5_
- _Requirements: 2.2, 2.3, 8.2, 8.4_
- _Verified 2026-09-16:_ 1,365 tests passed, nine skipped; Ruff/format/mypy passed. Graph boundaries are split into integrity failures, which still demote a completed answer, and declared exclusions, which do not. Whether a particular answer needed the excluded semantics is carried by that question's own unresolved call destinations from 10.5, not assumed for every answer. Controls exercise the real partition logic rather than an injected reason: a vendored directory leaves the answer completed with the exclusion still reported, an unresolved symlink still demotes, and the two together still demote. The exclusion stays physical and out of targets and graphs. The NodeGoat rerun showed the same distinction was still missing inside comparison, where any degradation and any boundary blocked: comparison now filters declared exclusions and correspondence ambiguity on both sides, keeping integrity degradations and every other boundary blocking.
- [x] 10.7 Bind candidate dependency gaps to the candidate
- **Discovered 2026-09-16 in the M1b production rerun.** The runner passes every coverage reason as every candidate's `dependency_gaps`, so admission always answers `dependency_unresolved`. That is not "an unresolved dependency that undermines the claim"; it is every gap anywhere in the transaction.
- Supply the gaps that pertain to this candidate's required comparison scope, the same scope 10.2 established. Unknown ownership still counts against the candidate.
- Not an M1b exit blocker, since eligibility stays disabled and no alert is emitted either way. It would prevent admission from ever passing after qualification, so it belongs before M4.
- Done when an unrelated transaction gap no longer appears as this candidate's unresolved dependency, while a gap in its required scope still does.
- _Boundary: Push runner, Delta and Actionability Policy_
- _Depends: 10.2_
- _Requirements: 3.1, 3.3, 3.4_
- _Verified 2026-09-16:_ 1,371 tests passed, nine skipped; Ruff/format/mypy passed. Gaps are now attributed: a boundary carrying a question id counts only when that question is in this candidate's required scope, an unattributed gap always counts because unknown ownership is not evidence of independence, and integrity degradations count. A declared exclusion, correspondence ambiguity and bounded reachability do not, because each was already weighed against this specific claim. The runner no longer hands every candidate the whole transaction's reason list, which had included other candidates' comparison outcomes.
- [x] 10.8 Carry query-to-operation provenance into the reported evidence
- The target is selected through a file-fallback question with `function=None` while rows report method `<lambda>2`. M1a showed this did not block the comparison once the question completed, so it is provenance quality rather than the blocker.
- Record verified query-to-operation provenance on the evidence; do not inject benchmark function names into regions or count coincident sites.
- Done when a reported operation names the question that produced it and the method the engine reported, with unresolved provenance explicit.
- _Boundary: Delta and Actionability Policy, contributor-scan normalization_
- _Requirements: 3.1, 3.4, 6.3_
- _Verified 2026-09-16:_ 1,371 tests passed, nine skipped; Ruff/format/mypy passed. Evidence now carries the method the engine reported beside the question that produced it, so the link is inspectable rather than inferred or dug out of a detail blob. Provenance is recorded as `exact`, `file_scope` when the question named no function and the engine supplied one, `unresolved` when the two disagree, and `method_unreported` when the engine named none. Controls cover all four. No region is renamed, no benchmark function name is injected and no coincident site is counted; the state is recorded, not applied.
- [x] 10.9 Rerun the three revisions through the production rules
- Replay the same pinned vulnerable, fixed and unchanged revisions **without** the recorded-veto flag under the named development budget. Demonstrate a developer-readable evaluation finding with a supported consequence, exact location, witness, change attribution and repair direction.
- Verify the fixed twin is analyzed and produces no actionable defect and the unchanged revision is not reported as a new regression. Keep qualification eligibility disabled; experimental evaluation output cannot advertise an unqualified hook alert.
- Done when one replayable record states which vetoes were repaired and which remain, with completed required checks and readable source and census receipts.
- _Boundary: Evaluation harness, full-run integration_
- _Depends: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8_
- _Requirements: 1.1, 2.1, 2.4, 3.1, 3.4, 6.3, 7.1, 7.3_
- _Verified 2026-09-16 with one named exception:_ all three revisions replayed through the production rules with exact pins and reopened source receipts. Vulnerable: three sites new with `operation_absent_from_comparable_base`, admission rejecting on `capability_unavailable` alone. Fixed twin: analyzed and quiet with three base-only findings. Unchanged: three sites `unchanged` with base locations resolved to the same lines. Every finding carries its question, the engine-reported method and its witness; the head census is the complete 44 files; all 436 base questions complete. Zero admitted defects. The repaired and remaining veto lists are in [the M1b record](m1a-recorded-vetoes.md). **Exception:** the supported security consequence and concrete repair direction are not demonstrated, because both are owned by an evaluated capability declaration and the registry is deliberately empty. M1b's comparison half is verified; its explanation half waits on M4.
- [x] 10.10 Render the experimental evaluation finding with its consequence and repair
- **The remaining half of M1b's exit, named by the 10.9 run.** The comparison is verified, but the milestone also asks for a developer-readable finding carrying a supported security consequence and a concrete repair direction. Both come from an evaluated capability declaration and the registry is deliberately empty, so nothing renders them.
- Generate an explicitly unreviewed, experimental declaration for this capability with the existing registry tool, and render the finding through the same grounded-template path the admitted case uses. Keep eligibility disabled: an experimental declaration is not enabled and not PASS, so admission must still reject it and no normal hook alert may be advertised.
- Do not write consequence or repair text that no declaration supports. Refusing generic advice without a supported context and operation is the rule admission exists to enforce, and the experimental path must not become a way around it.
- Done when the experimental output shows family, exact location, witness, change attribution, consequence and repair direction for the vulnerable revision, is labeled experimental throughout, and the production result still reports no actionable defect with zero admitted defects.
- _Boundary: Result and Hook Adapter, Delta and Actionability Policy, Evaluation harness_
- _Depends: 10.9_
- _Requirements: 3.1, 3.2, 3.3, 5.1, 5.5, 6.3_
- _Verified 2026-09-16:_ 1,386 tests passed, nine skipped; Ruff/format/mypy passed. Three evaluation findings rendered on the vulnerable revision with exact location, witness, change attribution, provenance, consequence and repair, all from the declaration's templates through the grounded-template path. The declaration reports verdict experimental and enabled false, the production result still reports no actionable defect, and all three artifacts carry zero admitted defects. Controls prove an ungrounded template is refused, an uncovered operation symbol is stated, no matching declaration leaves both sentences absent with the reason, and a declaration claiming PASS still admits nothing because the rendering path is outside admission. Evidence: `benchmarks/measurements/2026-09-16-nodegoat-m1b-explained.json`.
- [x] 11. Fit the same useful analysis into the push budget (release milestone M2)
- [x] 11.1 Give the backend a transaction-owned engine session
- The measured cost is JVM startup, not analysis. In the M1b runs each disposable `joern --script` invocation cost 13 to 23 seconds while the work itself took under a second, and the 2026-09-14 lifecycle probe measured one session at 12.8 s to start, 3.6 s to load a graph and 0.22 to 0.47 s per warm request.
- Own the session inside the existing backend for the length of one transaction: loopback only on an ephemeral port with no published port, a private working directory, one shared deadline, serialized requests, explicit graph identity and epoch, and kill-and-reap of the whole process group on cancellation, failure or exit.
- An HTTP status is not an answer contract. Require a per-request receipt written to a fresh private file and verified before the answer is accepted, the way the probe's repaired instrument did after its first attempt read an empty stdout as a census.
- Poison the session on any crash or malformed response so a failed session can never answer a later request, and fall back to the disposable path rather than reporting a zero.
- Done when a controlled hanging in-session request is cancelled inside the allowance with no surviving process group, a crashed session answers nothing afterwards, and the disposable path still works unchanged.
- _Boundary: contributor-scan runtime, Cache and Engine Boundary_
- _Requirements: 4.1, 4.2, 7.3_
- _Verified 2026-09-16:_ 1,391 tests passed, nine skipped; Ruff/format/mypy passed. Sixteen session tests drive a real fake-server process: receipt verified before an answer is accepted, a mismatched receipt or crash poisons the session so nothing answers afterwards, an empty body with its receipt is a valid empty answer, a hanging request is cancelled and the process group is gone inside the allowance, and a launch failure is recorded rather than raised. The probe took four iterations, each failing loudly: `runScript` is absent from the server REPL; the server answers success even for a compile error, so the receipt is the only proof; a shipped `@main` script cannot be nested in a block, so definitions go to the top level unwrapped; and the REPL shadows `Console`, so capture must qualify `scala.Console.withOut`. Evidence: `benchmarks/measurements/2026-09-16-session-script-execution.json`.
- [x] 11.2 Prove every shipped query kind answers identically through both transports
- Run taint, dominance, configuration and census over the same graph through the session and through the disposable path and require identical payloads, including empty answers and the distinction between an empty answer and an unavailable one.
- Cover a malformed or absent response, a switched graph, a second graph loaded in the same session, and a request arriving after cancellation. Equivalence on a census alone does not establish equivalence for security queries.
- Done when every shipped query kind matches byte for byte on a real graph, and every failure shape is reported as unavailable rather than empty.
- _Boundary: contributor-scan runtime, engine integration tests_
- _Depends: 11.1_
- _Requirements: 4.1, 4.2, 6.4, 7.3_
- _Verified 2026-09-16:_ taint, dominance and configuration all return byte-identical payloads through both transports on a real NodeGoat graph, with parameters built by each arbiter's own request function through the driver's family lookup. Failure shapes are correct: an empty answer stays present and distinguishable from an unavailable one, an unknown query and an absent graph are both unavailable, and two graphs in one session each answer about the graph the request names. Amortized over five taint requests the session took 22.48 s against 71.96 s disposable, all answered and identical; a single request per session is its worst case and is recorded as such. Evidence: `benchmarks/measurements/2026-09-16-session-transport-equivalence.json`.
- [ ] 11.3 Route the batch path through the session behind an explicit setting
- Consume the session from the existing batch and census paths, keeping the disposable path as the default and as the fallback. No capability, ranking decision or evidence semantics changes; only the transport does.
- Account for session startup, load, requests and teardown inside the one shared deadline, and record them as separate stage costs so the next decision is made on measurements rather than estimates.
- Done when a replay produces the same evidence and admission through either transport under a sufficient budget, with per-stage costs recorded for both.
- _Boundary: Cache and Engine Boundary, Push runner_
- _Depends: 11.2_
- _Requirements: 4.1, 4.3, 6.4, 7.3_
- [x] 11.4 Remove the startup cost from the critical path where the measurement says it pays
- With 11.3's stage costs in hand, decide what else to change. The candidates the M1b runs point at are the overlay pass, 15.8 s in a separate invocation, and session startup, 12.8 s, which may overlap the frontend build rather than follow it.
- Change only what the measurement implicates, and keep each change's correctness evidence separate from its timing evidence. Do not substitute a faster census for a completed security check.
- Done when the recorded stage costs after the change are compared against the same profile before it, with any regression in completed checks reported rather than traded away.
- _Boundary: contributor-scan runtime_
- _Depends: 11.3_
- _Requirements: 4.1, 4.2, 6.2, 6.5_
- _Verified 2026-09-17:_ 1,399 tests passed, nine skipped; Ruff/format/mypy passed. The overlay, the last fixed cost bypassing the session, now runs in-session and the saved graph is verified rather than assumed. The first session-enabled replay then showed correct detections and no speedup, and a targeted probe disproved the sequence hypothesis before finding the real defect: a build runs on a copy of the backend, so the session it started was never closed, leaking a JVM holding the configured heap, twice per transaction, enough to exhaust the container. With that fixed the session stays alive for the whole transaction and no `session_query_failed` appears. Measured on the same pins and budget, the vulnerable revision went from 326.4 s to 243.3 s, head query time from 63.6 s to 32.0 s and base from 85.1 s to 51.2 s, with dominance and configuration each falling from about 14 s to about 4 s. Detections unchanged. Evidence: `benchmarks/measurements/2026-09-17-nodegoat-session-transport.json`.
- [x] 11.5 Rerun the frozen seven-class runtime profile and decide
- Rerun the unchanged seven-class profile: identical-tip, function edit, dependency edit, configuration edit, cold, growth and multi-ref, three samples each, retaining every timeout and unresolved check. Identical-tip reuse cannot substitute for warm changed-code work.
- Verify the M1 outcome still holds on the same three revisions through the production rules, so speed is not bought with a lost detection.
- Apply the existing gates without alteration: representative warm changed-code p95 at most 30 s, cancellation and reporting at most 2 s, supported-check completion at least 95%. A failed gate stays NO-GO and names the dominant remaining cost.
- Done when one versioned scorecard reports p50 and p95 with sample counts beside the M1 correctness outcome, and states GO or NO-GO against the unchanged gates.
- _Boundary: Evaluation harness, runtime integration_
- _Depends: 11.4_
- _Requirements: 4.1, 4.2, 6.2, 6.3, 6.5, 7.3_
- _Measured 2026-09-17, verdict NO-GO:_ the unchanged seven-class profile, three samples each, session enabled. Every changed-code workload still times out at the 30-second deadline, and identical-tip got worse, p50 rising from 18.5 s to 24.2 s. The cause is recorded in the artifacts rather than inferred: 13 transactions report `session_unavailable:session_startup_timeout`, so under the real deadline the session usually never starts and everything falls back. Startup is 9 to 13 seconds, about a third of the whole budget, spent before any analysis. Where a session did start, on cached-graph transactions with little remaining work, it cost more than it saved. Correctness is unchanged: the same three revisions through the production rules with the session on reproduce the M1 verdict. Evidence: `benchmarks/measurements/2026-09-17-session-runtime-profile.json`. **Dominant remaining cost: engine startup itself, which a transaction-owned session relocates but cannot remove.**
- [x] 12. Security capability work found during M1a and M1b
- Group 11 established that M2 needs graph reuse rather than more runtime tuning, and that decision is recorded there. This group returns to what the tool can actually establish about a codebase, which is where the remaining release risk sits: two of the three shipped families cannot complete their change context at all, and admission rejects genuine findings on a mis-extracted operation symbol.
- [x] 12.1 Extract the operation symbol from the operation, not from the statement
- **Found 2026-09-16 in the M1b explanation run.** Admission takes the text before the first parenthesis as the operation symbol, so `const preTax = eval(req.body.preTax)` yields `const preTax = eval`. The demonstration declaration had to enumerate each assignment spelling for its coverage check to pass.
- A qualified declaration cannot enumerate every assignment a codebase might write, so admission would reject genuine findings on `operation_semantics_mismatch`. Take the symbol from the called operation the engine reported, preserving receiver spelling, and leave an unresolved or dynamic spelling ineligible as it is today.
- Done when the symbol for an assigned call is the call's own name, a receiver-qualified call keeps its receiver, a dynamic spelling stays ineligible, and a declaration naming the plain symbol covers the assigned form.
- _Boundary: Delta and Actionability Policy_
- _Requirements: 3.1, 3.2, 3.4_
- _Verified 2026-09-17:_ 1,403 tests passed, nine skipped; Ruff/format/mypy passed. The symbol is now the trailing qualified callee rather than everything before the first parenthesis, so `const preTax = eval(...)` yields `eval` and a declaration naming the call covers the assigned form. Receiver spelling is preserved, so `$wpdb->prepare` and `prepare` remain different operations. A spelling that is not a qualified name, such as a dynamic `handlers[name](x)`, yields the statement text, matches no declaration and stays ineligible exactly as before. The recorded-veto report reads the same helper rather than repeating the extraction.
- [x] 12.2 Supply per-family context producers for dominance and configuration
- The M1b exit run left 231 of 436 head questions reporting `change_context_incomplete` from `context_projection_unavailable` and `context_scope_empty`, because those families' queries export no context. They did not block the Node injection case, so M1b correctly left them alone, but their families stay experimental until they can answer.
- Give dominance and configuration their own context producers describing the actual scope of the query they accompany. Do not copy taint context into them: dominance evaluates sibling operations across a file and configuration has different scope and obligations.
- Done when a dominance and a configuration question complete their change context on a real repository, with missing extents and unsupported reasoning still explicit.
- _Boundary: contributor-scan query/context evidence_
- _Requirements: 2.1, 2.2, 8.1, 8.2_
- _Verified 2026-09-17:_ 1,405 tests passed, nine skipped; Ruff/format/mypy passed. The gap was larger than a missing query block: the evidence pass dispatches taint only, because only taint has an evidence vector, so these families were never asked for context at all. Each query now describes its own scope, dominance the file's methods labelled operation, guard or sibling because its claim is about a file contradicting itself, and configuration the file's methods labelled setting or sibling. Neither copies taint's reachability projection, which would describe work they never do, and both report an empty scope explicitly. A new context pass asks each family with the flag set; a family whose query answers nothing keeps no context and stays unresolved, which a control asserts so no family can complete on another's context. Confirmed on a real NodeGoat graph: both produce a context summary and seven scoped methods. Evidence: `benchmarks/measurements/2026-09-17-per-family-context.json`.
- [x] 11.6 Decide what to do about engine startup, which the session cannot remove
- **Named by the 11.5 measurement.** A transaction-owned session amortizes engine startup across the invocations inside one transaction, and a warm changed-code push does not have enough of them: startup is 9 to 13 seconds of a 30-second budget, so the session either fails to start or costs more than it saves. Group 11 as designed cannot reach the gate.
- The options are a decision for the maintainer, not an implementation detail, because two of them change the product's shape. A session shared across transactions is a resident local service, which the accepted milestone spec explicitly does not imply. A smaller budget for what a push analyses is a scope change, not a speed change. A faster cold path would mean attacking the frontend and overlay costs directly, which the session does not touch.
- Record the measured cost of each candidate before choosing, and keep the 30-second and 2-second gates unchanged; a gate met by analysing less is not the same result.
- Done when one record states the candidates, their measured or estimated costs, their effect on the product's shape, and the maintainer's choice.
- _Boundary: contributor-scan runtime, release direction_
- _Depends: 11.5_
- _Requirements: 4.1, 4.2, 6.5_
- _Decided 2026-09-17 by the maintainer: do not rebuild a whole graph per push._ Both startup attacks were measured and neither reaches the gate. The session helps a long transaction, 326 s to 243 s, and cannot help a 30-second one: startup is 9 to 13 seconds of the budget, so it usually declines. Removing JavaScript's second engine start is real, 11.0 s against 21.0 s on identical graphs, but its census gate was asymmetric and it was reverted after costing two engine starts on 12 profile transactions. What the profile names is not startup at all: a changed-code transaction reaches its deadline during the build, head build 25.2 s of 30 s with query time 0.0 s. The remaining cost is building a fresh whole-repository graph for every pushed revision. **Direction: reuse the graph across revisions instead of rebuilding it, rather than further optimising the rebuild.** Not scheduled here; M2 stays unmet and the 30-second and 2-second gates stay unchanged.
- [x] 12.3 Resolve function-scoped questions whose function the query cannot locate
- **Named by the 12.2 measurement.** With per-family context in place, head completion rose from 205 to 298 of 436 and `context_projection_unavailable` fell to zero. Every remaining unresolved question reports `context_scope_empty`, and they sit in three files: `app/routes/index.js` (114), `app/routes/tutorial.js` (12) and `server.js` (12). None is access control.
- These are function-scoped questions whose named function the query cannot locate in the graph, so the scope it describes is honestly empty. It is the same query-to-operation identity problem M1a recorded at the witness end, appearing at the scope end.
- Carry verified provenance from the region's declared function to the method the engine actually holds, and leave an unresolvable name explicit. Do not widen a question to the whole file to make it complete, and do not inject a declared name into a region.
- Done when a question whose function the graph holds completes, a question whose function it does not hold stays unresolved with the name recorded, and no question completes by widening its scope.
- _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
- _Depends: 12.2_
- _Requirements: 2.1, 2.2, 6.3, 8.1_
- _Verified 2026-09-17:_ 1,406 tests passed, nine skipped; Ruff/format/mypy passed. The cause is recorded rather than guessed: `app/routes/index.js` requires seven handler modules and defines none of them, so a question naming `handleContributionsUpdate` there names a function that file genuinely does not hold and the empty scope is correct. Taint and configuration now separate the two causes, reporting `context_function_unresolved` with the name when a question names a function the file does not define, and `context_scope_empty` only when the file has no methods at all. The driver no longer adds its generic empty-scope gap on top of a query that already explained itself, and the new gap is question-owned so it cannot block another question's comparison. Completion is unchanged by design: a question whose function the graph does not hold stays unresolved, and nothing completes by widening its scope to the whole file.
- [x] 12.4 Attribute a region to the file that defines its handler
- **The region-attribution half of 12.3.** A route-registration module registers handlers other modules define. `app/routes/index.js` requires seven handler modules and defines none, yet regions there were scoped to `handleContributionsUpdate` and its siblings, so the graph rightly held no such method and every question in those regions was unresolvable: 138 across six families.
- The entry record still names the handler, because the authorization question is which middleware guards which handler and that is cross-file by nature. It now also records whether the file defines what it names, and a region built from an entry that does not is scoped to the file rather than to a method the file lacks.
- _Verified 2026-09-17:_ 1,407 tests passed, nine skipped; Ruff/format/mypy passed. A first attempt changed the mapper to stop naming cross-file handlers and was reverted: an existing control proved the name carries the authorization attribution, so the fix belongs in region construction, not in the mapper. Controls assert the registering file asks about itself with no function-scoped region, while the defining module keeps its function-scoped region.
- _Boundary: Ranker Scope Contract, contributor-scan entry mapping_
- _Depends: 12.3_
- _Requirements: 2.1, 2.2, 8.1_
- [ ] 13. Show the shared design transfers across languages (release milestone M3)
- Status 2026-09-17: Python established, PHP blocked by runtime on a large repository. M3 is not met. Its exit asks for Node, PHP and Python together, and two of the three are in hand.
- [x] 13.1 Replay the declared Python family through the shared core
- Generalize the replay instrument so a case family is an argument rather than a hard-coded target: a harness that only knows one language cannot say whether the design transfers. Targets, pins and expectations come from the committed inputs.
- VAmPI is a regression snapshot with the same revision on both sides, so it can establish detection and completion but never novelty. The mode asserts that too: with nothing changed, a target reported new would be a comparison defect rather than a finding.
- _Verified 2026-09-17:_ all three declared targets detected at their declared lines with their questions answered, 35 of 35 questions complete on every case, nothing reported new, zero admitted defects. The SQL injection at `models/user_model.py:73` is the evidence the milestone names as previously unresolved. Two of the three are access control, the family that could not complete a question anywhere before group 12, so this is the first time broken object-level authorization has been asked on Python. Evidence: `benchmarks/measurements/2026-09-17-m3-python-transfer.json`.
- _Limits:_ identical revisions establish detection, not novelty; VAmPI is inspected teaching data and cannot qualify production Python support.
- _Boundary: Evaluation harness, full-run integration_
- _Requirements: 6.2, 6.3, 8.1, 8.2_
- [ ] 13.2 Replay the declared PHP pair through the shared core
- PMPro 2.9.7 against 2.9.8 with its upstream prepare fix, plus a comment-only benign variant. A different language, a different mechanism and a fix this project did not author.
- The first attempt selected one case of three, because the manifest labels a case by what it is while the case id says what the comparison does; the harness now maps label to role explicitly. That run also exhausted a 900-second budget on the single case it ran.
- Done when all three comparisons complete with their declared target evidence recorded, or the budget they need is stated as a finding rather than reported as a pass.
- _Measured 2026-09-17, PHP transfer NOT established._ All three comparisons exhausted a 3000-second deadline with a 1200-second query ceiling and produced no artifact, so there is no target evidence at all. The engine reports `cpg batch taint failed` twice per case on the 1,274-file PMPro tree: first an exception whose trace runs through ujson array transformation, then a timeout on the retry. Budgets tried: 900 s, 1200 s and 3000 s deadlines against 300 s and 1200 s query ceilings. Pins, source receipts and the declared target are all correct, so the question was asked properly and the engine could not answer it. This is a runtime blocker on a large repository, not evidence about whether the design transfers to PHP. Evidence: `benchmarks/measurements/2026-09-17-m3-php-transfer.json`.
- _Two harness defects found and fixed on the way, each of which had made an earlier run look like a transfer failure:_ the manifest labels a case by what it is while the id says what the comparison does, and filtering on one while keying expectations by the other ran a single case; and the per-query ceiling is taken as the minimum with the remaining transaction time, so raising only the deadline could never have helped.
- _Retried 2026-09-18 with batch splitting, still not established._ A batch that fails whole is now retried in eight parts, which a probe made look promising: the evidence pass answers all 4,425 PMPro regions in one batch in 29.7 s with a 681 KB response, and cost is fixed rather than per request, so neither count nor size is near a limit. That probe sent `evidenceOnly=true` and therefore measured the structural pass, not the failing one; recorded because those numbers would otherwise read as evidence that PMPro's taint work is cheap. With a 4500-second deadline and a 900-second query ceiling the run was stopped after 26 minutes with no case completed. Splitting also makes an unrecoverable batch slower to abandon, since each part burns its own share of the ceiling.
- _Correction 2026-09-20: the malformed-CPG bug is fixed and was never what these runs were hitting._ joernio/joern#6281, this project's own report, is a PHP `global` inside a closure producing a node with two AST parents, which made the dataflow overlay throw and cost the WHOLE graph. It was filed against 4.0.623 and fixed by #6270 in **v4.0.625**, which is the version the image has pinned since before any measurement here. Verified today on the real trigger: PMPro's `includes/fields.php`, the file the original bisection found, builds and applies the default overlays cleanly on 4.0.625 and on 4.0.631. So what remains under this task is a RUNTIME failure and must not be re-diagnosed as the graph bug.
- _Quantified 2026-09-20, and it is a runtime failure._ A taint batch of 2,526 questions over the 673-file checkout does not complete in thirty minutes. Raising the query ceiling six-fold, from 300 to 1,800 seconds, changed nothing but how long the failure took: query time went from 363 s to 1,874 s and zero questions completed in both. The eight-way split retry did not rescue it, finishing its parts in about three minutes against a deadline with 3,300 seconds still on it, which is worth its own look. Joern 4.0.625 and 4.0.631 are indistinguishable here, 627.7 s against 619.1 s. The engine pin stays at 4.0.625: the newer release was built and measured and is equivalent on this subject, so there is no benefit to weigh against re-verifying every JavaScript and Python measurement. Evidence: `benchmarks/measurements/2026-09-20-php-runtime-after-graph-fix.json`.
- _Corrected 2026-09-20: PHP is not silent, and this project's own instrument is why it looked silent._ The census harness ordered its verdict completion-first, so a family with eighteen findings and no completed question reported "asked but no question completed" — and only the 500-region runs, the ones that time out, had been looked at. Fixed: findings are checked first, because a question-outcome count is bookkeeping and cannot hide the thing being censused. NodeGoat and VAmPI were never affected, since they complete every question.
- _What PHP actually detects on PMPro._ At 25 regions, 19 findings in seven minutes with no timeout; at 100 regions, 43 findings in twelve minutes with no timeout. Among them a true positive at `adminpages/discountcodes.php:749`: `sanitize_text_field($_REQUEST["s"])` reaching `$wpdb->get_results($sqlQuery, OBJECT)` through `$sqlQuery .= "WHERE code LIKE '%$s%' "` with no `prepare`. That is the class of CVE-2023-23488, which `php.toml`'s own comment names as the reason those WordPress sanitizers are excluded — the prediction and the measurement agree.
- _And a measured precision cost beside it._ Output encoding reports eighteen findings at 25 regions and the three sampled are all properly escaped, two with `esc_attr` and one coerced with `intval`. Those escapers are deliberately absent from a sanitizer list that is flat across families, because none of them stops SQL injection; keeping the injection finding above costs this family its precision, and that is the documented tension's price.
- _The scale limit, quantified._ Roughly one to two seconds per question. 25 and 100 regions complete cleanly; 500 regions asks 2,526 questions, needs more than the 1,800-second ceiling and is killed. Nothing pathological, and the retry's fast failure is still unexplained.
- _What is NOT the problem, each measured rather than assumed:_ the malformed-CPG bug (fixed in 4.0.625, verified on the real trigger file); the dataflow overlay (42 s on the full 637-file graph); the engine version (4.0.625 and 4.0.631 indistinguishable); and the bundled Stripe, Braintree and Twocheckout SDKs under `includes/lib`, which the vendored list does not cover — that was the hypothesis and the overlay costs 42 s with them and 37 s without, so it is refuted.
- **The remaining real gap is bookkeeping, not detection:** `questions_completed` is zero in every PHP run including the ones that find things, with `cross_partition_semantics_unresolved` and `partition_file_census_incomplete` present throughout. Findings and question outcomes disagree in public, and the admission policy reads the outcomes. Group 12 raised completion for JavaScript and Python and PHP was never measured. Evidence: `benchmarks/measurements/2026-09-20-php-detection-corrected.json`.
- **IDENTIFIED 2026-09-22.** The cost is the number of CALL NODES a family's sink names match in the repository. Not the request count, not the declared sink count, not any request parameter. `reachableByFlows` is asked per matched sink, and PHP's output-encoding sinks are `echo` and `print`, which a WordPress plugin writes thousands of times.
- _Three controlled sweeps on one graph, built once._ Batch size is exonerated: 400 requests answer in 170 s and cost per request FALLS with size, because roughly 70 s is fixed JVM start and graph load. Request parameters are exonerated: at 200 requests the full production shape, call depth 3 with parameter sources, costs 221 s against a 150 s baseline. The family is the variable: at the same 100 requests, SQL injection costs 147 s with six declared sinks, path costs 194 s with sixteen, and output encoding costs 704 s with seven.
- _And the driver named its own expensive step, once it was asked to._ `taint: 2435 request(s) in 45.5s, answered` for the evidence pass, then `taint: 315 request(s) in 1200.1s, unanswered` for arbitration across all families, then a split abandoned for want of time.
- _Why it survived since September:_ every reproduction outside the driver carried the SQL sinks, because that is the family the declared target belongs to. The one family that is cheap on this subject was the one being re-measured, and nine hypotheses were eliminated that way.
- _Why splitting never rescued it:_ a failed batch is split into eight parts BY REQUEST COUNT, and each part carries the same everywhere-sinks, so the whole is no cheaper. Splitting by request count cannot help when the cost is per matched sink. Evidence: `benchmarks/measurements/2026-09-22-php-query-cost-identified.json`.
- **Blocker identified; the fix is contributor-scan 7.7.** Six hypotheses were measured and eliminated: batch size (17,700 requests answer in 44 s with a 2.6 MB response), dataflow cost (50 execution requests in 24 s), call depth 3 with parameter sources (24.7 s for 20), argument length (a 16 KB hook table against a 128 KB cap), hook callback resolution (1 s difference with 233 hooks), and build or sharding (79 s, no shards). Preparation measures 273.5 s. Every reproduction of the failing 22,125-request batch outside the push path answers in under a minute, and the difference from the production invocation is not known. The exception has never been read, because the backend logged a stack trace's tail and the corrected extractor reports its head, which on this engine is JVM startup noise. Evidence: `benchmarks/measurements/2026-09-19-php-blocker-open.json`.
- _Boundary: Evaluation harness, full-run integration_
- _Depends: 13.1_
- _Requirements: 6.2, 6.3, 8.1, 8.2_
- [ ] 14. Ask every declared family whether it establishes anything (precondition for M4)
- Seven families are declared and two were proven end to end. A declared family that has never been asked whether it finds anything is advertised coverage, and M4 cannot qualify a capability that establishes nothing.
- [x] 14.1 Census the seven declared families on NodeGoat and VAmPI
- Run production discovery and scan over each whole checkout and report, per family, the regions that offered it, the questions asked, the questions completed, and what was found with the witness and rung. No comparison, no novelty, no admission: detection only.
- A family that answered and found nothing must be reported differently from one that was never asked, and neither may read as clean.
- _Verified 2026-09-19:_ 1,411 tests passed, ten skipped; the six Joern-gated query tests pass in the image. Four families establish findings across the two subjects: injection (three `eval` flows on NodeGoat, one SQL flow on VAmPI), access_control (two sibling contradictions on VAmPI), config_secrets (one on each) and untrusted_destination (one open redirect on NodeGoat). All 299 NodeGoat and 35 VAmPI questions completed. Evidence: `benchmarks/measurements/2026-09-19-family-detection-census.json`.
- _The census found one false positive and it is fixed._ NodeGoat's only prototype report was the word `set` inside the comment `// set these up in case we have an error case`: the matched node is the `<operator>.assignment` holding the whole 1,000-character signup handler, and its text was searched entire. `sinkMatches` now confines its text clause to the callee and excludes operator nodes, which is the fourth instance of this bug class after `fgets`/`gets`, `resolve`/`resolveUrl` and `user`/`users`. Measured on NodeGoat's non-vendor calls: injection sinks 28 to 3, prototype 13 to 2, untrusted_destination 26 to 7, with no true positive lost. `req.query` had been an injection sink, because `query` is word-bounded inside it.
- _Boundary: contributor-scan detection, Evaluation harness_
- _Requirements: 6.2, 6.3, 8.1_
- [x] 14.2 Give untrusted destination the sinks a Node server actually calls
- **Named by the 14.1 census.** NodeGoat's documented SSRF is `needle.get(req.query.url + req.query.symbol)` at `app/routes/research.js:16`, and `needle.get` is absent from a vocabulary holding axios, fetch, got, `http.get` and request. The family works and found the open redirect in the same repository; it could not ask about the request.
- This is a vocabulary gap, so fix it as data rather than as a query change, and state which client libraries the declared set now covers.
- Done when the research handler's request is asked about, with a witness naming the call, and no benign client call is reported.
- _Verified 2026-09-19:_ 1,411 tests passed, eleven skipped; seven Joern-gated query tests pass in the image; Ruff/format/mypy passed. One data change, no query change. The declared set now covers axios, got, needle, request, superagent, unirest, undici and fetch as bare module tokens, plus `http.get`, `http.request`, `https.get` and `https.request` dotted. Bare is the documented call form of each module and also covers its verb methods, because a token matches word-bounded against the callee and `axios` is bounded inside `axios.get`; `axios.get` and `axios.post` were therefore dropped as subsumed. `http` and `https` stay dotted because their namespace also holds `createServer`, and a bare token would have claimed every server a repository starts.
- The research handler's request is now reported at `app/routes/research.js:16`, model_entailed, witness `req.query.url + req.query.symbol -> needle.get(url, ...)`. This family goes from one finding to two on NodeGoat and every other family is unchanged, with 299 of 299 questions still completing. Measured on that graph, `needle` matches exactly one call and the eleven other clients match none, in first-party or vendored code, so NodeGoat shows the gap closing and cannot show those clients' noise.
- A control asserts both halves rather than only the recall half: a handler calling `axios.get` with a literal and `needle.get` with `req.query.url` reports the tainted client alone, and the control reads the shipped vocabulary rather than a copy so removing the client fails it. The CWE-601 redirect entry is untouched; a Location-header redirect is value-shaped and stays outside what this family asks. Evidence: `benchmarks/measurements/2026-09-19-untrusted-destination-vocabulary.json`.
- _Boundary: shipped semantic facts_
- _Depends: 14.1_
- _Requirements: 6.2, 8.1_
- [x] 14.3 Decide what output encoding claims for server-rendered JavaScript
- **Named by the 14.1 census.** All six declared JavaScript sinks are DOM sinks. NodeGoat's XSS is server-side, rendered into Swig templates and written with `res.write`, so the 44 questions this family completed could not have found it whatever the code did.
- Either the family gains server-side sinks and template-rendering sinks, or it declares itself client-side only. Both are honest; silence while declaring XSS coverage is not.
- Done when one record states the decision, and the family either asks about `res.write` and template rendering or says in its declaration what it does not cover.
- _Verified 2026-09-19:_ 1,415 tests passed, thirteen skipped; nine Joern-gated query tests pass in the image; Ruff/format/mypy passed. **Decision: the family asks about a value reaching a rendering call and states in its own declaration that it cannot decide template-mediated escaping.** Whether `res.render` is safe depends on the engine's autoescape setting and on the template's markup, neither of which is at the call, so `res.render` is not a sink and the family's `limits` now says so. The setting is claimed where its evidence is instead, as a configuration defect.
- It now establishes a finding: `app/routes/research.js:25`, model_entailed, `req.query.url + req.query.symbol -> res.write(body)`. The handler fetches a URL the caller chooses and writes the response body into a text/html response, so the document served from this origin is the attacker's. The record states how coarse the step is: the path from `url` to `body` runs through the client's callback parameter, which the graph treats as reachable from the call's arguments, so the conclusion rests on the call's semantics rather than on what the graph established. Five of seven families now establish findings. Evidence: `benchmarks/measurements/2026-09-19-output-encoding-family.json`.
- _Four defects found on the way, three of them false positives this change would otherwise have shipped._ The permissive-value set was shared across every setting while a boolean's polarity belongs to the setting, so `swig.setDefaults({autoescape: true})` would have been reported as the defect on the line that repairs it; `ConfigSpec` now carries per-setting values and Jinja2's `Environment` was moved out of Python's shared list for the same latent reason. `config.sc` matched a dotted setting on its trailing segment alone, so `marked.setOptions` would have claimed an editor's `setOptions`. `taint.sc` asked its flow query of every argument including the receiver, so `res.end()` — which takes no data — was reported exactly like the `res.write(body)` two lines above it. And the arbiter sorted a file's settings by line number as text, so which setting a file reported depended on the spelling of its line numbers.
- _Boundary: shipped semantic facts, family taxonomy_
- _Depends: 14.1_
- _Requirements: 6.2, 8.1_
- [x] 14.6 Report every permissive setting in a file, not the first one
- **Named by the 14.3 measurement.** The configuration arbiter returns one verdict per question. NodeGoat's `server.js` holds a permissive session at line 78 and template auto-escaping disabled at line 135 in the same function, so the escaping defect is detected — measured model_entailed against the real query and the real arbiter — and never reported. A contributor is shown one of two defects and told nothing about the other.
- This is the arbiter's contract, not a data gap: `verdict()` returns `Verdict | None` and the scan collects one per question. Changing it touches every family that returns a verdict, so it is scheduled rather than folded into a vocabulary change.
- Done when a file with two permissive settings reports both, with their own sites and witnesses, and no family reports a defect twice.
- _Verified 2026-09-20:_ 1,417 tests passed, seventeen skipped; thirteen Joern-gated query tests pass in the image; Ruff/format/mypy passed. Configuration now returns one verdict per setting whose value the model could read. The pipeline already emitted one finding per entailed answer, so only this arbiter was still folding them to one. NodeGoat's `server.js` reports both defects with their own sites, `server.js:78` for the permissive session and `server.js:135` for the disabled template auto-escaping, which is the defect task 14.3 could detect and never report. Its applied fix at line 126 stays silent. VAmPI is unchanged and no other family moved. Evidence: `benchmarks/measurements/2026-09-20-every-permissive-setting.json`.
- _Three boundaries drawn rather than assumed._ A setting whose value is COMPUTED stays one verdict per region: that band is a question for the judge, and returning every undecided setting would multiply model calls for claims the graph did not establish. A call carrying two permissive literals is one finding, because `cors({origin: true, credentials: true})` is one misconfiguration. And config verdicts now carry a `path:line:function` location, without which the plural answer collapsed straight back to one in the deduplicator, which is the same gap dominance carried until its findings stopped arriving with no line.
- _Dominance is left answering once per region, and that is settled rather than pending._ Its claim is that a region contradicts itself, so the region is the unit and a second verdict would restate the same inconsistency. The pipeline says so where it used to call this a known gap.
- _Boundary: contributor-scan arbitration, Rung ladder_
- _Depends: 14.3_
- _Requirements: 6.3, 8.1_
- [x] 14.4 Raise the access-control obligation where the application splits route from query
- **Named by the 14.1 census and measured with the real dominance query.** Every modelled data operation in NodeGoat lives in `app/data/*-dao.js`; none lives in a route handler file. Operations are scoped to the region's own file and access_control regions are the handler files, so no obligation is raised where the question is asked, and inside the DAO files every operation is equally unguarded so the sibling rung has nothing to contradict. The family answers correctly on VAmPI, which keeps handler and query in one file.
- The file scope was chosen deliberately, because unscoped rows attributed a model's unguarded query to a handler of the same name. Widening it is not the fix; carrying the obligation along the call the handler actually makes might be.
- NodeGoat's two documented defects are also outside what the model can claim: an authenticated handler reading `userId` from `req.params` (`app/routes/allocations.js:17`), and a commented-out `isAdmin` in the route table (`app/routes/index.js:58`). Record which of the two this task is for, and do not silently widen the claim to the other.
- Done when a route-and-DAO split raises the obligation at the handler with a witness naming both sites, and a same-name model function is still not attributed to a handler.
- _Verified 2026-09-20:_ 1,415 tests passed, fifteen skipped; eleven Joern-gated query tests pass in the image; Ruff/format/mypy passed. **This task is for the first defect, the route-and-DAO split itself.** The obligation now follows the one named call the handler writes, out of the region's file and no further. Attribution is by the graph's own provenance, not by name: jssrc2cpg leaves the callee an external stub, and that stub's fullName records the file and member where the implementation was written, so a module the handler never imports is never consulted. A row is attributed to the region's DECLARED function, which is what lets a question about `displayAllocations` match a graph that holds a `<lambda>`. A carried row is also checked for a guard at the CALL, because that is where a handler establishes who is asking, and the method holding the carried operation joins the question's context scope so the query's own module cannot change unattributed.
- Measured on NodeGoat: 15 handlers asked, obligations raised at zero before and at six now, three of which reach a verdict. All three are corroborated and none is reported without a judge. VAmPI is unchanged at two entailed findings, because it keeps handler and query in one file and carries nothing. Evidence: `benchmarks/measurements/2026-09-20-access-control-module-split.json`.
- _A carried obligation corroborates, it does not entail, and that was measured rather than assumed._ Entailing it reported `handleSignup`, which looks a user up by name to check the name is free during registration and is public by design, because two authenticated siblings in the same file pass `req.session.userId` into the same module. The entailed rung's justification is that one FILE contradicts itself, same file and same trust context; carried, the compared operations live in a data-access module that handlers of different trust contexts share. The corroborated band already asks the right residual: does this operation require a check, or is the route legitimately public?
- _What this does not reach, recorded rather than implied:_ NodeGoat's documented IDOR at `app/routes/allocations.js:17` still raises nothing, because its module reads with `allocationsCol.find(...)` and `find` is not a modelled operation (task 14.8). The commented-out `isAdmin` at `app/routes/index.js:58` is not claimable by this abstraction at all, since the evidence that `/benefits` needs a stronger guard than its siblings exists only in a comment.
- _Boundary: contributor-scan dominance, Ranker Scope Contract_
- _Depends: 14.1_
- _Requirements: 6.2, 6.3, 8.1_
- [x] 14.7 Decide the sibling scope, and see identity that a destructuring spelled away
- **Named by the 14.4 measurement.** Two of NodeGoat's three corroborated claims are correct code: `const { userId } = req.session` followed by `dao.getByUserId(userId, ...)` is identity-constrained, and the destructuring erases the `session.userId` spelling the token list holds, so the guard is invisible at the call. A judge would be asked about handlers that are not defects.
- The contradiction that would make NodeGoat's IDOR entailed is cross-file: `allocations` passes a URL parameter while `contributions` and `profile` pass session identity. The sibling set is one file, and widening it is a decision, not an implementation detail: measured, a route file mixes public with authenticated handlers, so application-wide siblings would contradict every login and signup handler there is.
- Record the measured cost of each candidate before choosing, and do not widen the scope to make a finding appear.
- Done when one record states the candidates and the choice, a destructured identity read discharges the obligation it actually discharges, and no public route is reported.
- _Verified 2026-09-20:_ 1,415 tests passed, sixteen skipped; thirteen Joern-gated query tests pass in the image; Ruff/format/mypy passed. A destructured identity read now discharges what it reaches, through a bounded three-hop walk back over the method's own assignments from the names the operation is GIVEN. `req.session` joins the identity sources and is only safe because of that requirement: what discharges an object-level obligation is that the value PASSED to the operation comes from the authenticated context, not that a session is mentioned nearby. On NodeGoat `displayContributions` and `displayProfile` go silent, which is correct code no longer put to a judge, and VAmPI is unchanged.
- _Two exclusions the measurement forced, each caught by a page whose whole purpose is the leak it demonstrates._ Taking identifiers from the whole argument subtree credited NodeGoat's memo listing with a discharge, because `memosDAO.getAllMemos((err, docs) => ... userId ...)` reads every memo and only RENDERS the session's user; a callback is not a constraint, so method references are excluded. Then the session read still dominated the query and the dominating-guard clause credited that instead; an assignment is a read of the authenticated context rather than a check applied to it, so operator calls are excluded from that clause, which is the same exclusion the sink matcher needed. The leak stays reported.
- _The sibling scope is decided by measurement, and the choice is to keep the file._ On NodeGoat the region's own file entails two handlers, both public routes and both false, and no true defect at all. Every handler in the application entails six: four true, and the same two public routes. Both scopes carry the same false positives, because neither can tell a public route from a protected one — which is a semantic question and already the corroborated band's residual. With the carried rung capped at corroborated the scope changes no verdict, only the witness, so widening it would buy a repository-wide walk per question and no precision. What would change the decision is a discriminator for a public route; with one, the wider scope is worth four true entailments on this subject. Evidence: `benchmarks/measurements/2026-09-20-sibling-scope-and-identity.json`.
- _Boundary: contributor-scan dominance, shipped obligation facts_
- _Depends: 14.4_
- _Requirements: 6.2, 6.3, 8.1_
- [x] 14.8 Model the collection read an application actually writes
- **Named by the 14.4 measurement.** NodeGoat's documented insecure direct object reference reads with `allocationsCol.find(searchCriteria())`, and `find` is absent from an operation vocabulary holding `findOne`, `findAll` and `findById`. It is the commonest read there is, so the family's obligation is missing at exactly the shape it exists for.
- It is absent for a reason: `Array.prototype.find` is everywhere in JavaScript and jQuery's `.find` is everywhere in a vendored bundle. A collection read takes a query document where those take a closure or a selector, so the discriminator is the argument's shape rather than the name.
- Done when a collection read raises the obligation, an array search and a selector do not, and the false-positive count is measured on a repository with vendored bundles rather than argued from the shape alone.
- _Verified 2026-09-20:_ 1,415 tests passed, sixteen skipped; twelve Joern-gated query tests pass in the image; Ruff/format/mypy passed. The obligation is claimed where the first argument has the shape of a query document, which the operation fact states as `query_document_arg` and the arbiter decides: no arguments at all, an object literal, or a builder call. Not claimed: a method reference, a literal, an operator call, and a bare identifier.
- _The false-positive count is measured, on wp-statistics, a WordPress plugin that ships jQuery._ Its assets hold 88 calls named `find` — 70 jQuery selectors, 13 array predicates and 5 selectors built at run time by a template literal or a concatenation, and no collection read at all. Zero became an obligated operation. The zero is a measured one and not an empty graph: the same run's census reports 60 files and 4,552 methods, and the files holding those 88 calls are in it.
- On NodeGoat all three `find` calls are collection reads and all three are claimed, which raises the obligation at four more handlers including `app/routes/allocations.js:displayAllocations` — the documented insecure direct object reference, which had raised nothing at any point in the pipeline before. Seven of fifteen handlers now reach a verdict, all corroborated, none reported without a judge, and the census is unchanged on both subjects. Evidence: `benchmarks/measurements/2026-09-20-collection-read-obligation.json`.
- _A bare identifier is deliberately not claimed._ `col.find(query)` and `arr.find(pred)` are the same shape and neither repository measured holds one, so this is a stated false negative rather than an unmeasured false-positive risk.
- _Boundary: shipped obligation facts, contributor-scan dominance_
- _Depends: 14.4_
- _Requirements: 6.2, 8.1_
- [x] 14.5 Ask the prototype and deserialization families somewhere they can answer
- **Named by the 14.1 census.** Prototype pollution has been asked on one repository and its only report was a false positive; deserialization was never asked at all, because VAmPI holds no such sink and NodeGoat is not a language where the family is declared. Neither has established anything anywhere.
- Find or record a subject where the family has a sink, and report what it does. A negative on a subject that contains the mechanism is a result; silence on a subject that does not contain it is not.
- _Verified 2026-09-20:_ 1,417 tests passed, eighteen skipped; Ruff/format/mypy passed. **Neither family has a subject that can answer, and each for its own reason, which is the result rather than a failure to produce one.**
- _Deserialization: the mechanism exists on exactly one subject and that subject cannot be queried._ Across four WordPress plugins and 1,164 PHP files there is one deserialization site, `maybe_unserialize` at `includes/fields.php:1111` in Paid Memberships Pro. The vocabulary held only the bare `unserialize`, which never reached it and correctly so: the matcher is word-bounded and the character before the token inside `maybe_unserialize` is a word character. That spelling is now declared, so the family has a sink to ask about — and the census asked 487 deserialization questions on that plugin and completed none, because no query over that repository completes at all. That is task 13.2's runtime failure, now quantified there, not a statement about this family.
- _Prototype pollution: no available subject contains the mechanism._ The only first-party matches on the Node subject are `app.set("view engine", "html")` and its sibling, Express settings whose arguments are literals, and the other JavaScript available is minified browser bundles with no request source. So the family's ability to answer is established on a control instead: a merge of the request body into an object is reported, and the Express setting beside it is not. Declared and never demonstrated has become demonstrated on a control and never on a real subject, which is a different and honest claim.
- _Boundary: Evaluation harness_
- _Depends: 14.1_
- _Requirements: 6.2, 8.1_
- [ ] 13.3 Re-run the declared PHP transfer at a scale whose questions complete
- **Named by the 2026-09-20 correction.** The not-established record came from two instrument defects, not from the analyser: the census verdict checked question completion before findings, and a 500-region budget asks 2,526 questions, which no ceiling this project sets can hold. Bounded to 25 and 100 regions, PMPro completes with no timeout and establishes a true-positive SQL injection at `adminpages/discountcodes.php:749`.
- Replay the three declared comparisons with the corrected harness at a region budget that completes, and record the declared target's evidence or its absence. State the budget as a limit of the run, not as a property of the repository: a bounded population establishes transfer, never coverage.
- Done when all three comparisons complete with their declared target evidence recorded, or the budget they need is stated as a finding.
- _Attempted 2026-09-21, and the attempt named its own blocker._ At 25 regions the first case now runs to completion and writes a full artifact, in 1,706 s, where at 100 it had exhausted a 2,400 s deadline with nothing. But 25 regions of 4,461 is 0.6% of the population, and the declared target at `classes/class.memberorder.php:936` was not among them: zero findings at the target, because the question was never asked. A bounded population establishes transfer only if it CONTAINS the target, and this one cannot.
- **So this task depends on contributor-scan 7.3, not on a larger budget.** The population that holds the target is affordable only if arbitration stops being paid for questions whose evidence says there is nothing to find. Raising the budget is the thing already measured to fail; making most of the budget unnecessary is the thing that has not been tried.
- _Two further observations from that run, recorded because they cost nothing to keep._ The head scan cost 1,178 s against the base's 219 s, which is worth explaining before M2 designs around either number. And every coverage reason was `ambiguous_line_correspondence`, one per file — so in the PUSH path, unlike the census, correspondence is what stands between this subject and a completed question. The partition work removed one blocker and revealed the next.
- _Sized and attempted 2026-09-22._ One whole-repository scan at 500 regions with a 3,600 s deadline puts the declared target in scope and finds it: 271 of 406 questions completed, 213 findings, 2,950 s. The three-case replay at that budget then exhausted a 7,200 s case deadline with no artifact, and the failure is mine to own twice over: the container mounts `src` live and I edited the backend heavily during the run, including a version that could hang on a timed-out pipe; and the push's EVIDENCE pass is one unportioned 22,125-request batch protected only by the ceiling, which it exceeded under the CPU and memory contention my concurrent test runs created. Neither is the algorithm. The evidence pass is now portioned, streaming keeps what a killed portion finished, and the rerun uses a frozen export of the committed source with per-invocation logging so the record names its own expensive step.
- _Attempted again 2026-09-23 on frozen `cda679c`, after contributor-scan 7.9, and it failed at a new wall._ Case 1 (introduce) spent its whole 7,200 s in the HEAD side's evidence pass and saved no artifact; cases 2 and 3 were stopped on the same path. The evidence portions took 13, 31 and 51 minutes each. A push asks evidence of the WHOLE population with change context, 22,125 requests, where the census asks about 2,400 without it.
- _Diagnosed by probe, not by timestamp._ On one PMPro graph from the production request builder, context costs ~20% more engine time per request -- not the wall. The wall is OUTPUT: each request itemised every method its scope reached, up to 1,845 rows; 200 requests into the bundled Braintree SDK returned 52,831 rows, 9.4 MB, against 211 without context. `affected_context` only ever attaches a location in a CHANGED file and discards every other row after checking it, so nearly all of it was shipped to be thrown away. Fixed in `3c195c9`: the engine itemises the changed files and summarises the rest into one row carrying what the check reads. On PMPro the same 2,440 relationships and zero gaps are decided from both forms, 52,831 rows down to 3,241 with the most-referenced file as the change. Probe: `benchmarks/push/context_cost_probe.py`. The replay is re-running on frozen `3c195c9`.
- _Corrected the same day: that was real but it was not the wall._ The re-run on `3c195c9` showed the same 13 → 31 → 51-minute portions. Sampled in flight, no engine was running for minutes at a time while the `pre-push` Python process sat at 83% CPU. The cause was the reuse cache: every publication listed and stat'ed the WHOLE cache to total its size, and a push publishes one answer per question, so the pass was quadratic. Measured on the live cache, one scan took 0.68 s at 12,238 entries, ~70 minutes of scanning by then and ~2.3 hours to finish the pass. The growing portion times were the quadratic, not the engine. Fixed in `7cf256b`: the scan is remembered, keyed by the cache root's modification stamp, so another writer still forces a rescan and the size limit stays exact. The probe did not catch it because it called the engine directly, without the cache -- the instrument lesson again: a probe that skips a layer cannot clear that layer. The output fix stays: it is still 16× less data per push. The replay is re-running on frozen `7cf256b`.
- _The loop, named and broken, 2026-09-23._ Walls kept appearing one per two-hour run -- the cache rescan, then the harness losing a killed case's output (`982cb14`), then a linear-list relationship check that also overran the deadline by >60 s (`e7e2941`) -- because every one was Python bookkeeping between engine calls, every one grew with the repository, and the only full-scale instrument was a Joern replay. Two instruments replace that: an in-process sampling profiler (`OUSAST_SAMPLE_PROFILE`, on for every replay case) and `benchmarks/push/python_overhead.py`, which runs the real `replay()` on the real PMPro pins with an instant stub engine answering real recorded rows. Its first pass found the remaining two walls at once: discovery re-parsed the ruleset per region (89,127 TOML parses, 339 s → 7 s, output digest identical) and change attribution decoded every line anchor per row (78% of a stubbed push; now indexed). The whole stubbed PMPro push now costs 58.5 s of Python; per-entry cache fsync is gone too (`b4b9308`, `6b85a60`).
- _The budget, stated correctly._ The push's head scan can select 2,500 questions (500 regions × 5 families) where change context leaves them incomplete, because tier-zero pruning never prunes an incomplete question -- ~6 hours of engine at ~9 s per PHP question. That does NOT make a 7,200 s case infeasible: contributor-scan 7.7–7.9 portion the arbitration phase in ranked order and hold time back for arbitration, so a case answers what fits and reports the rest as `query_budget_reserved`. What defeated every earlier case was the phases BEFORE portioning -- discovery, the cache, change attribution -- each spending the deadline before arbitration began. With those fixed (preparation 290 s → 19.5 s in the real run), the three cases re-run at 7,200 s on frozen `6b85a60`; completion is expected to be partial and stated, and the declared target's file ranks 19th of 86.
- _Boundary: Evaluation harness, full-run integration_
- _Depends: 13.2, contributor-scan 7.3, contributor-scan 7.8_
- _Requirements: 6.2, 6.3, 8.1, 8.2_
- [ ] 13.4 Let a question inside one partition complete, whatever else the repository holds
- **Diagnosed 2026-09-21, and PHP was never the variable.** A repository holding TWO languages can never complete a question, so it can never admit a finding. `partitions.py` raises `cross_partition_semantics_unresolved` whenever more than one language partition exists; that reason is a member of `GRAPH_INTEGRITY_GAPS`; and a graph-integrity gap demotes every completed outcome in the scan.
- The evidence is one column wide: NodeGoat is JavaScript alone and completes 299 of 299, VAmPI is Python alone and completes 35 of 35, PMPro is PHP with 36 regions of bundled JavaScript and completes 0 of 127 and 0 of 520 while establishing 19 and 43 findings. NodeGoat's own `vendor_semantics_unresolved` is a DECLARED exclusion rather than an integrity gap, which is why it does not demote.
- **This is larger than the PHP runtime limit and it lands on the declared v0.1 target.** A WordPress plugin is PHP plus its admin JavaScript, so every one of them is multi-language, so on the target this tool was built for it can detect defects and never admit one. Neither subject that currently passes could reveal it.
- Recording the boundary is right: a flow from PHP into JavaScript is semantics the partitioned graph cannot follow. Demoting every question is not, including one wholly inside a single PHP file. Task 12.3 already fixed this bug class one level down, making an empty-scope gap question-owned "so it cannot block another question's comparison"; the same principle applies to a partition boundary.
- Do not reclassify the reason as a declared exclusion. That would claim cross-language flows are out of scope when they are unresolved, which is the opposite of what this project's degradation vocabulary exists to say.
- Done when a question whose scope lies inside one partition completes, a question whose scope could cross a partition boundary stays unresolved with the boundary named, and no question completes by widening its scope. Evidence: `benchmarks/measurements/2026-09-21-two-languages-defeat-completion.json`.
- _Both halves implemented 2026-09-21._ A language boundary is now a declared limit: it can hide a finding and never invent one, so it stays published and stops demoting. And an integrity gap raised by a partition census now carries the partition it came from and reaches only that partition's questions, because a missing JavaScript file cannot make a PHP answer wrong. Two controls exercise the real boundaries rather than injecting a reason, and both were confirmed to FAIL with the classification restored. The rule behind both is now written in the design as an invariant, with all five instances listed, because this is the fifth time it has been found and fixed one instance at a time.
- _Verified on the subject 2026-09-21:_ PMPro completes **120 of 127** questions where it completed none, with its 19 findings unchanged. The seven still unresolved are the JavaScript partition's own questions, which is correct: the four missing files are its files, so its answers are the ones that could be wrong. 1,419 tests pass, eighteen skipped; Ruff/format/mypy pass.
- _Three causes, not one, and the third was hidden by the record rather than by the code._ The same census gap is raised in TWO passes, arbitration and evidence, and carrying the partition in one but not the other left an unscoped copy that kept demoting everything, so the fix looked applied and was not. It was found by making the census print each outcome's reason instead of only its status, after which three separate causes stopped looking identical.
- _Admission is scoped too._ `_blocking_degradations` already exempted declared exclusions; it now also ignores a gap another language partition raised, so a JavaScript build artefact cannot block a PHP finding. A gap naming no partition still reaches everything, and so does every gap when the claim's own language is unknown, because unknown scope is not evidence of independence. Evidence: `benchmarks/measurements/2026-09-21-partition-gap-blast-radius.json`.
- _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
- _Depends: 13.3_
- _Requirements: 2.1, 2.2, 6.3, 8.1_
- [ ] 15. Prepare the independent population and declare the shippable scope (M4 preparation)
- Runs in parallel with M2 and M3. M4 already permits population preparation before the frozen candidate exists, and it is the longest lead-time item on the path: nothing can be qualified without it however good the analyser gets.
- [ ] 15.1 Reserve and review an untouched evaluation population
- Freeze reviewed vulnerable, fixed and benign changes from projects not used to develop or tune this analyser. NodeGoat, VAmPI, PMPro, WP Statistics, MW WP Form and libpng are all development subjects and none of them may qualify anything.
- Reserve the cases untouched: do not examine their scan results before the population is frozen, and record the reservation so a later run cannot be mistaken for a blind one.
- **The population MUST hold a multi-language repository.** Task 13.4 found that a second language in a repository cost every question its completion, and so cost every finding its admission — on the declared v0.1 target, which is PHP plus admin JavaScript. Neither subject that passes today could reveal it, because both hold exactly one language. A population of single-language repositories would qualify a capability that fails on the product's own target.
- Done when a frozen population exists with its provenance and review recorded, at least one member holds more than one language, and nothing in it has been scanned.
- _Reserved 2026-09-24, not yet frozen._ Eleven cases selected by the maintainer from fifteen researched by advisory and fix diff alone (no clone, no scan), none among the 262 repositories this project's benchmarks reference, plus the earlier Ghost reservation: 4 PHP with JavaScript (Ultimate Member and LearnPress WordPress plugins, phpMyFAQ, YesWiki; all SQL injection), 4 Node/TypeScript (Signal K SSRF, Saltcorn SQLi, mongo-express code injection, Mockoon permissive CORS), 3 Python (alerta SQLi, wger open redirect, OpenCVE SSRF). Record: `benchmarks/independent/population-v1.toml`; research and rejected alternates: `research-2026-09-24.md`. A guard test fails if any reserved repository is referenced elsewhere in the tree. Remaining to FREEZE: per case, verify the pins and the sink by reading source, and choose a benign change from the same history without scanning; the OpenCVE licence must be checked before any result is published.
- _Frozen 2026-09-24._ `benchmarks/independent/freeze.py` verified every case with git and file reads only -- no analyzer, query or tooling: both pins present and ordered, the sink text present at the vulnerable pin (LearnPress's fix is source-side and recorded as `fix_site`; Mockoon's sink is recorded as `sink_evidence`), the fix touching the sink or its declared fix site, and SHA-256 of the sink file at both pins. Benign controls were proposed mechanically and REVIEWED by diff: the keyword filter alone let through an access-restriction, a filter check, API-token scopes, a login change and a TLS-verification option, all rejected; Ultimate Member's and alerta's controls were replaced by hand (a coding-standards reformat; a dependency-plumbing change with both sides still vulnerable). Record: `freeze-v1.json`. The guard test now requires a benign pair for every case of a frozen population. Nothing in it has been scanned; the first scan is the evaluation.
- _Boundary: Evaluation harness, corpus provenance_
- _Requirements: 8.1, 8.2_
- [ ] 15.2 Declare the v1 advisory scope as the families whose claims can be admitted
- **Admission requires the entailed rung.** Injection, untrusted destination and configuration entail with witnesses on the measured subjects. Access control does not: on an application that splits its route from its data access the obligation is carried, and a carried obligation corroborates by design, so shipping the family would ship silence. Output encoding, path, prototype and deserialization have not earned a place.
- Record the scope, the reason each excluded family is excluded, and what a contributor is therefore NOT told. M4 does not require every language or family to become eligible, and a published limitation is the honest form of a narrow release.
- Done when one record states the declared scope and every exclusion's reason, and the packaged advisory output names the limits it ships with.
- _Boundary: pre-push-safety-net admission, release validation_
- _Depends: 15.1_
- _Requirements: 3.1, 5.1, 5.6, 7.1_
- [ ] 16. Make a harness report the cause of its own failure (robustness)
- **Named by a week in which three separate causes looked identical.** The census reported 19 findings and 0 completed questions side by side for days without complaint; a stale image measured the engine a bug had been diagnosed against; and a partition fix looked applied while an unscoped copy of the same gap kept demoting everything. Each was found by reading code, and each would have been visible in a record that carried its own causes.
- Every harness result should carry the counts that disambiguate its failure modes, the degradations whole rather than by reason, and the engine and ruleset identities actually used. Two of these were added reactively during that week and should be the standard shape rather than a reaction.
- Add a self-consistency assertion to each harness. Findings above zero with completion at zero is a contradiction, and an instrument that says so loudly is the difference between an hour and a week.
- Done when each harness record carries its provenance and per-cause counts, and a deliberately contradictory result is reported as a contradiction rather than as a number.
- _Boundary: Evaluation harness_
- _Requirements: 8.1, 8.2_
## Implementation Notes
- Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.
- Cancellation must cover semantic planning and reporting, not only engine subprocesses: `_collect` and `_scope_work` now consume the shared deadline, and replay instantiates the budget-aware backend once. Preserve the 400-question failure-census and real Node artifact-retention regressions.
- Group 9 (M1a) corrected the owner diagnosis: all four named vetoes fire on the NodeGoat case, and a fifth, `operation_correspondence_unresolved`, is the actual remaining blocker because a newly introduced operation sits on a changed line and has no base anchor. The unchanged revision resolves its base location on the same lines and is classified `unchanged`, which isolates correspondence as the mechanism. M1b repairs context producers, question-owned vendor relevance, operation correspondence, file-wide ambiguity scope and query-to-operation provenance; vetoes that did not fire stay unchanged.
- Group 6: complete discovery/graph/query reuse is explicit via `--cache-dir`; source/declaration edits invalidate affected units. Installed frontend/adaptation bytes now bind graph and admission provenance. A valid cached census cannot qualify a fresh answer that omitted its own census. Identical-revision timing is not the changed-code rollout gate.
@@ -1,5 +1,7 @@
# Requirements Document
> **Status 2026-10-02:** complete, 22 of 22 tasks (spec.json phase corrected to `implementation-complete`).
## Introduction
OpenUltraSAST users scan **working trees they are writing**: mixed files, incomplete edits, generated code, framework wrappers, and no labels. Today the harness still *decides* with pattern inventory. The complexity map ranks; the sandbox can run; neither adjudicates “this sink is fed by a source” versus “this sink is hardcoded, sanitized, or unreadable.”
@@ -1,9 +1,9 @@
{
"feature_name": "propose-adjudicate-prove",
"created_at": "2026-09-04T10:58:28Z",
"updated_at": "2026-09-04T12:00:00Z",
"updated_at": "2026-10-02T00:00:00Z",
"language": "en",
"phase": "implementation",
"phase": "implementation-complete",
"approvals": {
"requirements": {
"generated": true,
@@ -1,5 +1,7 @@
# Requirements Document
> **Status 2026-10-02:** dormant. 23 tasks generated, nothing approved; adopting Joern (`cpg/`, `model/taint.py`) delivered most of what it asked for. Reopen it only for a reachability gap Joern's taint does not cover.
## Introduction
OpenUltraSAST cannot yet steer effort where risk is reduced because it has no model of how an entry point reaches a sink. Reachability today means "inside a handler," taint is one hop inside one file, the LLM hunter cannot create a sandbox candidate, and the improve loop cannot add the facts that would raise recall. This spec adds a call graph and composed taint summaries over the existing `FunctionIR` records, attaches path-based reachability to findings and ranking, puts the hunter on paths with one checkable question per gap, turns model answers into gated facts, and ranks fix points by reachable risk removed. Concolic execution is out of scope; the sandbox remains the only place target code runs.
@@ -1,5 +1,7 @@
# Requirements Document
> **Status 2026-10-02:** superseded by `pair-corpus-honesty` (spec.json `superseded_by`); its 12 tasks were done and the vfc pairs live on in that spec's corpus. Kept as decision history; nothing in src, tests or CI points here.
## Introduction
AppSec engineers cannot tell whether OpenUltraSAST improved on real product code. The labeled pair slices today are in-tree fixtures (`local`), small GitHub/SVEN functions (`github`), and textbook OWASP/Juliet files (`sast`). Operators care whether the harness fires on a real buggy snapshot of OpenSSL, Firefox, or Chromium and stays quiet after the patch. This spec adds a reviewed, isolated-function **vfc** slice drawn from public vulnerability-fixing commits in those projects, scored as an honesty dashboard (not a merge gate), so the tool can be evolved against labeled product code instead of cheat-sheet recall.
+3
View File
@@ -1,5 +1,8 @@
# Brief: three-stage-scan
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
## Problem
AppSec engineers and DevOps teams running OpenUltraSAST in CI get a regex static scan scored against cheat-sheet fixtures. They cannot tell what actually exists in a real repository, where complexity concentrates, how to aim tests, or whether a finding is worth a CI failure. The previous HarnessX spec added governance around those regexes (CWE policy, score, shadow status) without making the hunter evolutionary or the evaluation realistic.
+3
View File
@@ -1,5 +1,8 @@
# Design Document
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
## Overview
**Purpose.** This feature turns an OpenUltraSAST scan into three operator questions: what exists (static inventory), where problems are most likely and how to aim tests (complexity map), and whether a finding is worth fixing (isolated regression). It is a follow-up to the HarnessX governance spec, not a replacement of CWE policy, scoring, or rules-as-data.
@@ -1,5 +1,10 @@
# Requirements Document
> **Status 2026-10-02:** retired (see the note below); all 31 tasks were done before HarnessX was removed. Kept as decision history; nothing in src, tests or CI points here.
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
## Introduction
OpenUltraSAST today answers one question well: which regexes fired. It does not answer the three questions a rugged DevOps CI workflow actually asks: what exists in this repository, where problems are most likely and how to aim tests, and whether a finding is worth fixing. This feature turns the scan into those three stages. Stage 1 is a static inventory of what exists. Stage 2 is a complexity map of likely problem areas plus test-tuning hints. Stage 3 is isolated regression that proves triggerability before CI treats a finding as blocking.
@@ -1,5 +1,7 @@
# Requirements Document
> **Status 2026-10-02:** complete, 13 of 13 tasks (spec.json phase corrected from `tasks-generated` to `implementation-complete`).
## Introduction
AppSec engineers scanning mixed working trees need overlay adjudication on Python, JavaScript, C/C++, and Java, not only on files a Python stdlib parse can read. Today a `standard` or `deep` scan leaves JS/C/Java inventory as `language_unsupported` even when a host parser CLI is present, so those proposals never promote or demote. This spec makes an **optional semantic extra** the way to turn on a real structured parse for those languages, while `quick` stays usable with no extra packages and a missing extra stays fail-closed.
@@ -1,9 +1,9 @@
{
"feature_name": "tree-sitter-overlay-extra",
"created_at": "2026-09-04T14:00:00Z",
"updated_at": "2026-09-04T15:00:00Z",
"updated_at": "2026-10-02T00:00:00Z",
"language": "en",
"phase": "tasks-generated",
"phase": "implementation-complete",
"approvals": {
"requirements": {
"generated": true,
+2
View File
@@ -1,5 +1,7 @@
# Brief: zero-with-a-reason — a finding count is meaningless without a denominator
> **Status 2026-10-02:** dormant. Brief only and unapproved; not touched since 2026-09-10.
**Status:** brief only, unapproved.
## Why now
+51 -9
View File
@@ -1,6 +1,37 @@
# Overview — what this is, and what is actually true of it
**Current direction, 2026-09-12:** an actionable pre-push security safety net for AI-accelerated
## Current state, 2026-10-02
Everything below this section is dated history; where it disagrees with this section, this section wins.
User-facing documentation: `README.md`, `docs/architecture.md`, `docs/evaluation.md`, `docs/decision-engine.md`,
`ops/ax/README.md`.
- **Scanning.** `ousast scan --mode quick|standard|deep`. Quick rules for C/C++, Python, JavaScript/TypeScript,
Java/Groovy and PHP (PHP numbers are in-sample only: `benchmarks/measurements/2026-09-30-php-quick-rules/`).
Standard adds the MAP stage with the Joern model layer (shipped in the Docker image); deep adds sandboxed
regression snippets when Docker is available. Framework knowledge is tagged as optional priors
(`ruleset/frameworks.toml`, `priors=`). `ousast pre-push` compares head against base; the capability registry
is empty, rollout NO-GO.
- **Agentic plane.** google/ax over Agent Substrate on kind is the only executor of model work over whole
repositories (`ousast plane run|status|doctor|workspaces|remember|alerts-engine|harvest`). Credentials travel
only in the start request through `atenet-router`; egress is per task; budgets are per task; rows go to the
memory store (`OUSAST_MEMORY`: FileStore, or S3Store on an S3-compatible bucket; RustFS is the tested server).
- **Removed.** HarnessX (retired 2026-09-30): its config keys fail, a `[harnessx]` section warns. The LLM judge
of the verifier and the LLM fusion panels are gone; fusion is deterministic.
- **Improvement.** `ousast improve --memory` proposes rule-status edits from plane memory through the unchanged
validator and gate; the loop also runs as a plane Run.
- **Decision engine** (`learned-decision-engine`, implementation, not adopted). First slice, injection only:
ADVISORY recall 0.90, precision 0.54, BLOCK not offered
(`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`).
- **Evaluation.** Populations v1 and v2 are spent and failed every M4 gate (`benchmarks/independent/results-v1.json`,
`results-v2.json`); v3 (PHP) is frozen and untouched for the one-time final check. The plane increment met its
gate on the second measurement with a tie-break chosen after the first (`benchmarks/independent/plane-increment-2.json`).
- **Providers.** DeepSeek for LLM calls, OpenRouter only for OpenAI embeddings; keys in `.env`, which never
overrides an exported shell variable.
## History
**Direction, 2026-09-12:** an actionable pre-push security safety net for AI-accelerated
development across supported ecosystems. See `safety-net.md` and the approved `pre-push-safety-net`
spec. The ranker remains the scope mechanism; third-party vendor code stays outside targets and
graphs. Advisory must meet the same actionability bar as blocking.
@@ -163,15 +194,26 @@ function-level pairs could not see any of them, and a repository showed all nine
## Where the specs stand
Counted 2026-10-02 from each spec's `tasks.md` checkboxes and `spec.json`.
| spec | phase | state |
|---|---|---|
| `contributor-scan` | implementation | 14 done, 15 open — **the active one** |
| `learned-decision-engine` | implementation | 11 of 17 tasks done |
| `pre-push-safety-net` | implementation | 67 of 79 tasks done |
| `contributor-scan` | implementation | 27 of 51 tasks done |
| `ai-service-plane` | implementation | 11 of 11 tasks done; the ax plane is the agentic path |
| `harnessx-removal` | complete | 8 of 8; HarnessX removed 2026-09-30 |
| `finding-feedback-loop` | design | requirements approved, design awaiting approval |
| `model-grounded-detection` | complete | 16/16; built the arbiter this all rests on |
| `authorization-obligations` | tasks approved | 24 open — superseded in practice by the dominance arbiter |
| `constrained-detector` | tasks | 4 of 20, tasks unapproved |
| `reachability-flow-model` | tasks generated | 23 open, nothing approved — largely delivered by adopting Joern |
| `learning-harness` | implementation | 30 done, 3 open |
| six others | complete | `pair-corpus-honesty`, `corpus-seeded-mechanisms`, `propose-adjudicate-prove`, `three-stage-scan`, `real-world-vfc-slice`, `harnessx-self-improving-rulesets`, `tree-sitter-overlay-extra` |
| `model-grounded-detection` | complete | 16 of 16; built the arbiter this all rests on |
| `authorization-obligations` | dormant | tasks approved, 0 of 24 started; superseded in practice by the dominance arbiter |
| `reachability-flow-model` | dormant | 23 tasks generated, nothing approved; largely delivered by adopting Joern |
| `flow-aware-ranking`, `zero-with-a-reason`, `ir-consolidation` | dormant | brief only, unapproved |
| four others | complete | `pair-corpus-honesty`, `corpus-seeded-mechanisms`, `propose-adjudicate-prove`, `tree-sitter-overlay-extra` |
| six others | retired or superseded | `openrouter-sast-harness`, `three-stage-scan`, `harnessx-self-improving-rulesets`, `learning-harness` (retired with HarnessX), `constrained-detector`, `real-world-vfc-slice` (superseded) |
97 source modules, 90 load-bearing and 7 standalone capabilities; 814 tests.
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax (`ai-service-plane`),
> and evolutionary proposals come from the plane memory. The HarnessX specs in the table above are history.
As of 2026-09-09: 97 source modules, 90 load-bearing and 7 standalone capabilities; 814 tests.
150 source modules, 145 load-bearing and 5 standalone capabilities, 0 orphaned
(`benchmarks/measurements/2026-09-08-module-audit.json`, regenerated 2026-10-02); 1,964 tests collected.
+72 -67
View File
@@ -1,91 +1,96 @@
# Roadmap
## Current direction and next review — 2026-09-12
## Status, 2026-10-02
The product is an **actionable pre-push security safety net for AI-accelerated development**, across
supported languages/frameworks. WordPress is one regression workload. Advisory must be useful and
low noise; moving speculative SAST output into a nonblocking hook does not meet the goal. See
`safety-net.md` for current steering. The earlier narrative below is retained as dated history and
does not override this direction or the latest measurements.
The milestone table below dates from 2026-09-14. Since then (`.kiro/specs/pre-push-safety-net/release-milestones.md`):
M1a and M1b are met, M2 is measured NO-GO, and M3 is next, with M4 population preparation in parallel; none of
M1-M5 is verified and rollout remains NO-GO. Populations v1 and v2 were evaluated and failed every M4 gate
(`benchmarks/independent/results-v1.json`, `results-v2.json`); v3 (PHP) is frozen for the one-time final check.
### Approach decision
Work since then moved to: the ax plane as the only agentic executor (`ai-service-plane`, HarnessX retired
2026-09-30 by `harnessx-removal`), and the learned decision engine (`learned-decision-engine`, task 6 in
progress, not adopted). See `overview.md` for the current-state summary.
Keep the existing ranker as the scope mechanism. Supply it with pushed-change and affected
first-party context. Third-party vendor code remains outside both targets and graphs. Reuse
compatible graph/results artifacts and apply a total deadline; treat unknown external semantics
as a coverage boundary. Require change attribution, witness and repair direction before a finding
earns normal hook visibility. A second heuristic scope selector and an assumed incremental Joern
engine are out of scope.
## Release path as accepted 2026-09-14
### Current evidence and limitations
Build an actionable pre-push security safety net for AI-accelerated development.
The maintainer accepted the following order: **prove one useful detection, make it
fast, prove cross-language transfer, qualify independently, then enable narrow advisory
rollout**. The executable outcome contract lives in the existing
[pre-push-safety-net release milestone spec](../specs/pre-push-safety-net/release-milestones.md).
- Whole-PMPro analysis has completed with its known CVE; the earlier "does not finish" diagnosis
is superseded. The recorded 72-minute scan is not a pre-push performance result.
- Historical Joern 4.0.625 + layout position budgets are PMPro 82, WP Statistics 91, MW WP Form 54.
`<50` remains unmet. VAmPI's old 16 omitted SQLI; the corrected instrument retains the unmatched
target and reports no full-target budget or detection recall without the required evidence.
- Equal-score tie-breaking alone cannot bring PMPro or WP Statistics under 50. The existing
ranker is useful, but its measured ordering is not enabled by the ordinary CLI caller.
- Docker/Compose/ops now default to 4.0.625; task 1.1 passed local PHP/JavaScript image smoke and independent review. No host deployment or hook latency claim follows from that runtime evidence.
- `contributor-scan` has 25/41 checked tasks, including the approved and verified frontend-retention prerequisite 2.18.
`flow-aware-ranking` is still an unapproved brief despite implementation history.
**We are at M1b. M1a is met; no other release milestone is verified yet.** The published
`v1.2.0-alpha.1` and 28/29 completed implementation tasks describe delivered machinery,
not a demonstrated useful safety net. Task 8.3 and rollout remain NO-GO.
| Order | Milestone | Observable exit | Current state |
|---|---|---|---|
| M1a | Show the NodeGoat detection with every admission veto recorded, not applied | Raw finding with witness on the vulnerable revision; analyzed fixed twin and unchanged control quiet; per-finding veto list. One-week time box | MET 2026-09-16 |
| M1b | Repair only the vetoes M1a showed to matter | Supported evaluation finding through production rules, exact change/base evidence and repair direction; fixed twin and unchanged control stay quiet | IN PROGRESS: comparison half verified on all three revisions; the consequence and repair rendering (10.10) remains |
| M2 | Make that same analysis fast | Equivalent evidence; representative warm changed-code p95 <=30s, cancellation/report <=2s, >=95% supported completion | Pending M1 |
| M3 | Prove the shared design transfers | PMPro/PHP and VAmPI/Python vulnerable/fixed evidence; unchanged core rules; Node replay after any PMPro core change | Pending M2 |
| M4 | Qualify on independent cases | Reviewed untouched vulnerable/fixed/benign populations pass joint quality, coverage and latency gates per capability | Pending M3 |
| M5 | Enable a narrow advisory rollout | Exact passing eligibility and packaged acceptance; only qualified capabilities enabled | Pending M4 |
### Immediate next action
Finish M1b by rendering the experimental evaluation finding with its consequence and repair
direction (10.10), through an explicitly unreviewed declaration and with eligibility still
disabled. The comparison half is verified: the production rules classify the vulnerable, fixed
and unchanged NodeGoat revisions correctly, and admission rejects the vulnerable one on the
eligibility gate alone. What is missing is the explanation, which no declaration currently
supports, and inventing it is exactly what admission exists to prevent. Then M2 runtime.
Do not start another broad speed campaign before this outcome exists. See the
[M1a evidence](../specs/pre-push-safety-net/m1a-recorded-vetoes.md) and the
[owner diagnosis](../specs/contributor-scan/release-remediation-research.md).
### Approach and boundaries
The existing ranker owns scope. Vendor code stays out of targets and graphs. Framework
facts and frontends supply evidence to a language-agnostic core. Unknown dependencies
remain explicit; removing global completeness vetoes requires question-owned evidence,
not dropped safety checks. Advisory and blocking share the same actionability bar.
Graph persistence now works: identical-tip replays take 18.5–18.9s. All 18 other measured
transactions still time out and 0/24 target checks complete. A local Joern session census
probe is promising, but full-query equivalence and hook performance are unverified.
These findings inform M2; they do not complete M1. No second IR/engine, new heuristic
scope selector, differential graph mutation or unrelated backlog is part of this path.
## Existing Spec Updates
- [ ] `contributor-scan` — review the draft driver/backend integration amendment; reconcile packaging
pins, family-specific sanitizer task 5.6, exact scope reporting and repository baseline honesty.
Preserve previously approved work; the new amendment is not implicitly approved.
- [ ] `flow-aware-ranking` — review the amended experiment contract: ranker owns scope, vendor
boundary preserved, missing targets stay unresolved, actual detection and cost accompany rank,
untouched evaluation and exploration precede learned-ranking claims.
- [ ] `zero-with-a-reason` — reconcile the broader funnel proposal with the hook's consumed coverage
contract; it is adjacent work, not a requirement to finish every diagnostics mechanism first.
- [ ] pre-push-safety-net — Execute the accepted M1–M5 release milestone amendment.
Owns immutable replay, comparison, admission, evaluation and reporting. Preserve the
existing completed tasks; task 8.3 remains blocked until qualification passes.
- [ ] contributor-scan — Supply the required query/operation/context fixes for M1,
followed by equivalent bounded session execution for M2 and shared transfer evidence.
Frontend retention 2.18 and named census repair 2.19 are already verified.
- [ ] flow-aware-ranking — Preserve ranker-owned scope and normalized cross-language
behavior. New ranking research is not a prerequisite for the first useful comparison.
- [ ] zero-with-a-reason — Adjacent diagnostics work; finishing its whole backlog is
not required before M1.
## Specs (dependency order)
- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
reuse, total deadline, change attribution, actionability admission and compact hook behavior.
Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–7 are implemented and verified (23/27 tasks), including the resolved frontend-retention prerequisite.
- [ ] pre-push-safety-net — Existing approved implementation, dependent on contributor-scan.
Groups 1–7 and tasks 8.1, 8.2, 8.4 are verified; 26/27 executable tasks complete.
Remaining release work follows [M1–M5](../specs/pre-push-safety-net/release-milestones.md).
This amendment creates no duplicate implementation spec and resets no prior approvals.
### Next concrete step
### Progress reporting
Groups 1–5 now connect immutable replay to the existing ranker, detector, comparison,
admission and compact artifacts. The first frozen PMPro/Node experiment is **NO-GO**:
all six cold security/fixed/benign cases timed out without completed target checks.
After repairing post-deadline semantic planning and a repeated availability probe,
every final CLI run saved its artifact and returned in 30.55–30.64 seconds. PMPro spent
its deadline in preparation/discovery after materializing both trees; Node reached the
driver but did not complete checks. The exact 11-case population remains unresolved.
No normal capability is enabled and these six cold samples establish no warm p95 claim.
Group 6 now provides complete preparation/discovery, graph and query reuse through explicit
`--cache-dir`, bounded publication and isolated graph leases. A packaged authored JavaScript
control preserved the same evidence/admission as an uncached run; its identical-revision
replay took 2.00s. Source/declaration changes invalidate affected units. This correctness
control does not establish representative changed-code p95, independent alert precision or
eligibility. Rollout remains NO-GO and normal capabilities remain disabled.
Group 7 now provides the real multi-ref push interface, opt-in hook integration and bounded
optional selection among already-admitted witnesses. Group 8 is next and
must still pass the joint gates on declared populations. Keep the existing thresholds and
profile preparation, graph construction and queries separately when investigating misses;
do not substitute identical-tip hits for warm changed-code performance.
Tasks 4.3 and groups 5–6 used the documented Kiro manual review fallback after the agent
thread limit; this is not independent review. See `pre-push-safety-net/implementation-group-5.md`
and `benchmarks/measurements/2026-09-13-first-replay-experiment.json` for provenance,
initial instrument failures, their repairs and the newly frozen final trial. The NodeGoat
teaching workload did not cause a framework/fact adaptation or ranking retune. Ghost remains
untouched; independent PHP/Python eligibility populations and C/C++ property support remain
separate gaps. Automatic dominance-only context projection is still an upstream limit.
Lead with the current milestone, the demonstrated developer outcome, the blocker and
the next action. Keep implementation counts separate from release readiness. Preserve
prior evidence and review limits in the [implementation records](../specs/pre-push-safety-net/status.md).
The earlier PHP-first and runtime-first narrative below is historical and does not
supersede this accepted sequence. Blocking deployment follows a later explicit decision.
## Historical roadmap narrative — September 9–11
The sections below preserve prior plans and measurements. Statements about the current bottleneck,
PHP coverage, rollout order or feature readiness are superseded by the current section above.
**Rewritten 2026-09-09.** The previous version (kept as `roadmap-2026-09-04.md`) described a pre-Joern world —
**Rewritten 2026-09-09.** The previous version (kept as `.kiro/archive/steering/roadmap-2026-09-04.md`, outside steering) described a pre-Joern world —
"adjudication is still Python-`ast` plus a tree-sitter CLI stub", "Joern-as-required deferred". Joern was
adopted, the model layer was built, wired and measured, and two repositories have been scanned. See
`overview.md` for what is true today; this is what happens next and in what order.
+22
View File
@@ -3,6 +3,19 @@
Updated 2026-09-12 from the maintainer's explicit direction. This direction supersedes the
WordPress-first release framing in older steering; historical measurements remain historical.
Release order accepted 2026-09-14: first prove one useful Node changed-code comparison
with an analyzed quiet fixed twin, then make the same analysis fast, verify PHP/Python
transfer, qualify independently, and enable only passing advisory capabilities. See
[the milestone spec](../specs/pre-push-safety-net/release-milestones.md). M1a is met
(2026-09-16: witness shown, fixed twin analyzed and quiet, five vetoes recorded), and the M1b
comparison now classifies all three NodeGoat revisions through the production rules with no
lifting. M1a added a fifth veto the earlier diagnosis missed, operation correspondence for an
operation on a changed line, and M1b added a prerequisite it did not anticipate: absence must
be provable from an enumerated operation inventory, not from the absence of a traced flow.
Task counts and an experimental alpha tag do not establish release readiness. PMPro
remains a core regression/development case; any PMPro core retune still requires frozen
Node transfer evidence. This sequence supersedes older runtime-first next-step wording.
## Purpose
AI accelerates code production. OpenUltraSAST should help developers catch concrete security
@@ -120,3 +133,12 @@ questions. Native PHP/JavaScript partition smoke and 1321 tests pass. Runtime/co
identity changed, so prior eligibility is stale and remains disabled. This repairs one
shared prerequisite; pre-push 8.3, runtime amortization, VAmPI context/transitive
evidence and independent qualification remain open. No ranker retune or rollout GO.
The 2026-09-14 post-census profile now verifies identical-tip reuse (18.521–18.874s,
two graph hits and 1,512 query hits per repeat), while all 18 other transactions time
out and 0/24 target checks complete. The separate local-session census experiment is
promising but does not qualify security queries or hook latency. The contributor-scan
`release-remediation-research.md` records missing family context producers, global
vendor/correspondence completeness vetoes, and query-to-operation identity as the next
shared-contract work. Resolving a boundary requires evidence of its relevance and scope;
do not simply drop completeness checks to increase admissions. Rollout remains NO-GO.
@@ -1,26 +0,0 @@
---
name: openultrasast-kiro-impl
description: OpenUltraSAST Kiro implementation steering. Use when continuing `.kiro/specs/openrouter-sast-harness/tasks.md`, choosing the next phase, updating checkboxes, or keeping implementation aligned with the spec.
---
# OpenUltraSAST Kiro Implementation
Use this skill when implementing the OpenUltraSAST Kiro spec from opencode.
## Rules
1. Read `.kiro/specs/openrouter-sast-harness/tasks.md` before choosing work.
2. Implement the next unchecked task slice unless the user names a different phase.
3. Keep each slice independently verifiable with `uv run pytest`.
4. Update task checkboxes only after implementation and tests pass.
5. Prefer minimal harness-compatible primitives over large framework imports.
6. Keep OpenUltraSAST independent of Clearwing; use Clearwing only as an oracle for design patterns.
7. Use project-local opencode skills to steer scan, triage, and fix workflows.
## Current Phase Order
- Phase 5: Harness runtime events, processors, contracts, and traces.
- Phase 5A: Static mapping disciplines for SARIF, Semgrep, CodeQL, differential review, and sharp edges.
- Phase 6: Evidence ladder verifier.
- Phase 7 and 7A: Docker sandbox and dynamic evidence.
- Phase 8 and later: Hunter pool, SARIF, deep mode, patch oracle, MCP, fusion, skill index, hardening.
+191 -367
View File
@@ -1,159 +1,78 @@
# OpenUltraSAST
**v1.2.0-alpha.1:** experimental pre-push safety net. Integration and census repairs are
verified; useful hook coverage, latency and independent capability qualification remain
**NO-GO**. No real hook capability is enabled. See [release notes](RELEASE_NOTES.md) and
[spec status](.kiro/specs/pre-push-safety-net/status.md).
**v2.0.0 (Python package `2.0.0`), state as of 2026-10-02.** An experimental static
security analyser. The pre-push safety net is **NO-GO** for rollout: no hook capability is
enabled and no independent population has passed the qualification gates (see
[docs/evaluation.md](docs/evaluation.md)). See the [release notes](RELEASE_NOTES.md).
OpenUltraSAST is a command-line security analysis harness built around one
goal: eliminating false positives by turning a suspicion into an
evidence-backed finding, and learning from the claims that were rejected. A
scan combines static rules, a semantic overlay, optional model-driven hunting
and Docker-isolated proof, and every claim carries the evidence that earned its
place.
OpenUltraSAST separates what it can prove from what it merely suspects, and says which is which.
A scan combines language-scoped pattern rules, a semantic overlay, a Joern-based taint engine
and optional model calls; every finding carries the evidence that earned its place, and a scan
that could not look at something records that as a degradation instead of reporting a clean
result.
The project began as an editor-agent skill and is now a CLI first. Agent
integrations remain, but nothing requires one: `ousast` is the interface, and
the model-driven parts are optional extras that degrade visibly when absent.
> **Maturity legend:** ✅ implemented and tested · 🟡 primitive exists, not yet
> auto-wired into the scan loop · 🧭 designed, on the roadmap. The current
> verified gate is `usable_harness_mvp`.
`ousast` is the interface. Model-driven work over whole repositories runs on a separate
agentic plane (google/ax), and a learned decision engine that weighs all the signals is in
development; neither is needed for a scan.
## Install and quickstart
```bash
uv sync --group dev
# Scan a local repository (quick mode = deterministic, no model calls)
# Deterministic scan, no model calls
uv run ousast scan /path/to/repo --mode quick
# Build an embedding-ready chunk index
uv run ousast index /path/to/repo
# Score the detector against a benchmark manifest
uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
# Differential pair eval: fire on vuln, stay silent on the fix (local + real GitHub VFCs)
uv run ousast pairs
uv run ousast pairs --slice sast # OWASP Benchmark Java/Python + Juliet
uv run ousast pairs --slice vibe-py --pointers # nightly: also harvest non-vendored pointer pairs into ~/.cache
uv run ousast mechanisms export --slice vibe-py # seed mechanism candidates from trusted pairs (offline); `ousast improve` admits them under the holdout gate
uv run ousast pairs --slice vibe-py --loo # leave-one-out recall of the corpus as a teacher
uv run python -m openultrasast.pair_gate
# Run the bounded self-improvement loop over the ruleset (benchmark-driven)
uv run ousast improve benchmarks/manifests/python-vulnerable.toml --dry-run
# CI: exit 1 on any evidence-verified finding
uv run ousast scan . --mode quick --fail-on verified
```
Every run writes auditable artifacts under `.openultrasast/runs/<scan-id>/`
(`manifest.json`, `findings.json`, `verification.json`, `report.md`,
`report.sarif`, `trace/events.jsonl`, ranking/preprocess/mapping JSON).
`--fail-on findings|verified` gives CI-friendly exit codes.
The core install depends only on PyYAML. Optional extras: `semantic` (tree-sitter grammars
for the overlay) and `s3` (the plane's memory store on an S3-compatible bucket, boto3).
## How a scan works
For `standard` and `deep` scans, use the Docker image, which ships Joern (with `php-cli` for
the PHP frontend) next to the tool. Source the shell wrapper once and `ousast` runs in the
container with your directory mounted read-only:
The CLI is thin; the `HarnessRuntime` owns the lifecycle and emits an event
trace for every stage (`task_start … stage_start/stage_end … task_end`).
```
repo ──▶ preprocess (language, LOC, tags, fuzz entry points)
──▶ static mapping (Semgrep/CodeQL SARIF ingest, normalized hints)
──▶ entry-point + reachability mapping (routes, CLI, parsers, contracts)
──▶ rank (surface·0.5 + influence·0.2 + reachability·0.3)
──▶ findings ─ quick: language-scoped pattern rules
│ └ standard: tiered hunter pool (retrieval + skill snippets)
──▶ independent verifier (evidence ladder enforced)
──▶ report.md + report.sarif + manifest.json (shared finding IDs)
```bash
source /path/to/OpenUltraSAST/ops/shell/ousast.sh # PowerShell: ops/shell/ousast.ps1
ousast scan . --mode standard
```
- **quick** ✅: deterministic, language-scoped pattern rules + reachability +
verification. No model calls, fully reproducible.
- **standard** ✅ scheduling / 🧭 LLM hunters: runs the tiered hunter pool
(A/B/C budgets, retrieval context, selected Trail of Bits skill snippets,
JSONL trajectories) and verification; language-aware LLM hunters are the next
upgrade (today standard reuses the deterministic findings per target).
- **deep** 🧭: sandboxed build/fuzz/dynamic reproduction (not yet implemented).
Details, including a pinned host install of Joern, are in [ops/README.md](ops/README.md).
## Detection coverage
Every run writes auditable artifacts under `<target>/.openultrasast/runs/<scan-id>/`:
`manifest.json`, `findings.json`, `verification.json`, `score.json`, `report.md`,
`report.sarif` and `trace/events.jsonl`, plus the stage artifacts of the mode.
Language-scoped rule families (rules only run against matching languages, which
removes cross-language false positives):
## Scan modes
| Mode | What runs | Needs |
| --- | --- | --- |
| `quick` | Language-scoped pattern rules, entry-point reachability hints, ranking, verification, scoring. Deterministic and reproducible. | nothing |
| `standard` | `quick` plus the MAP stage: complexity map, semantic overlay, authorization obligations, and the model layer, which builds one Joern code property graph per repository and decides taint, guard-dominance and configuration questions on it. An LLM is asked only the residual question the graph cannot settle (a `suspicion`); `[models] hunter` adds the tool hunter. | Joern for the model layer (else a `cpg_unavailable` degradation); a provider key for the LLM parts (else they are skipped and recorded) |
| `deep` | The MAP stage of `standard` plus REGRESS: promoted candidates are loaded by a small snippet inside a Docker sandbox (no network, read-only source, non-root, memory/pid/time limits) and get a `triggerable` / `not_triggerable` / ... verdict. | a working `docker` (else a `sandbox_unavailable` degradation) |
`--fail-on never|findings|verified|worth-fixing` sets the exit code: `findings` fails on any
reported finding, `verified` on any evidence-verified one, `worth-fixing` on a deep-mode
verdict that is worth fixing. `--config openultrasast.toml` loads settings.
## Detection coverage (quick rules)
Rules run only against files of their language. The bundled ruleset lives in
`src/openultrasast/ruleset/<language>/rules.toml`:
| Language | Classes (CWE) |
| --- | --- |
| C / C++ | buffer overflow (120), format string (134), command injection (78) |
| Python | command injection (78), code injection (95), deserialization (502), SQLi (89), path traversal (22), SSRF (918), weak hash (327), SSTI (94), insecure default (489) |
| JavaScript / TS | command injection (78), code injection (95), reflected/DOM XSS (79), SQLi (89), path traversal (22), SSRF (918), weak hash (327), deserialization (502) |
| Java + Groovy templates | command injection (78), SQLi via concatenation (89), weak hash (327), deserialization (502), unescaped template XSS (79) |
| C / C++ | stack buffer overflow (121), format string (134), command injection (78) |
| Python | command injection (78), code injection (95), SSTI (94), deserialization (502), SQLi (89), path traversal (22), SSRF (918), weak hash (327), insecure default (489) |
| JavaScript / TS | command injection (78), code injection (95), XSS (79), SQLi (89), path traversal (22), SSRF (918), weak hash (327), deserialization (502) |
| Java + Groovy templates | command injection (78), SQLi (89), weak hash (327), deserialization (502), unescaped template XSS (79) |
| PHP | SQLi (89), command injection (78), file inclusion (98), path (22), open redirect (601), SSRF (918); code eval (95), unserialize (502) and echo XSS (79) ship as `shadow` rules |
Cheat-sheet fixtures live under `benchmarks/fixtures/` and feed the 90/10 smoke
gate. **Efficiency** is measured separately on isolated vuln-vs-fixed pairs
(`ousast pairs`, `python -m openultrasast.pair_gate`): the harness must fire on
the vulnerable snapshot and stay silent after the patch. Local fixture
counterparts are a CI gate; vendored GitHub VFCs (SVEN, CWE-Bench-Java, NVD
patterns) are the honesty dashboard and the miss/fp signals for `ousast improve`.
Dataset pointers (CVEfixes, DiverseVul, PrimeVul, MegaVul, MoreFixes, Vul4J,
GitHub Advisory Database, OSV) are in `benchmarks/pairs/datasets.toml`.
## Project scores and central CWE policy
Severity used to be whatever string a rule set on itself. It is now governed centrally: one CWE, one severity, decided by policy — not by the rule that fired.
**Central CWE policy** ✅ (`policy/verycode.py`): a vendored `CWE_Score.tsv` (verycode-policies, ~167 CWEs) is the single source of truth. Each CWE carries a flaw category, a `severity` (0-5), and `static` / `dynamic` scope flags. `load_policy()` reads it positionally (the upstream file ships CRLF and a trailing space in the `Flaw Severity ` header), and `resolve_severity()` resolves a finding's severity *exclusively* from policy, keyed on CWE — any legacy rule-local severity is discarded. CWEs that are not `static` resolve to `0` and are report-only, never scored.
**Fail-loud startup** ✅: `assert_rules_resolve(PATTERN_RULES, policy)` runs as the `policy_check` stage right after `policy_load`. If any *enabled* rule names a CWE the policy does not govern, it raises `PolicyError` and the scan aborts before doing work — you cannot ship a rule whose severity nobody decided.
**0-100 project score** ✅ (`scoring/project_score.py`): each finding's penalty is `severity weight × reachability multiplier`, and the score is an exponential decay of the total (`100 · e^(−total/k)`, `k=60`).
| Severity | Weight (`SEV_WEIGHT`) | | Reachability | Multiplier (`REACH_MULT`) |
| --- | --- | --- | --- | --- |
| 5 | 50 | | `reachable` | 1.0 |
| 4 | 25 | | `inferred-file-surface` | 0.6 |
| 3 | 10 | | `unknown` | 0.4 |
| 2 / 1 / 0 | 2 / 1 / 0 | | | |
No penalty → 100; one reachable severity-5 finding → ~43. The **reachability multiplier is the false-positive calibration knob**: a confirmed FP lowers a finding's effective reachability instead of deleting the rule, so the score moves without losing the detection.
**Two-condition gate** ✅: a finding that is both severity-5 *and* `reachable` **always** fails the gate. The `score < min_score` threshold (default `min_score=80`) only fails when `blocking` is enabled — so scoring is **advisory-first by default** and turns into a CI gate when you opt in.
**Artifacts** ✅: the `score` stage writes `score.json` (project score, `max_severity`, `penalty_total`, `by_category`, `out_of_scope_dynamic_only`, `unmapped_cwe`, gate verdict) and merges the same block into `manifest.json`. Scoring is zero-dependency (stdlib only). One mapping detail: verycode has no CWE-120 (generic buffer copy), so the C/C++ memory-unsafe rules listed under [Detection coverage](#detection-coverage) carry **CWE-121** (Stack-Based Buffer Overflow, severity 5), which the policy does govern.
🧭 This is the first implemented slice of the self-improving ruleset work: a central, policy-governed severity model and a project score. Rules-as-data and the full self-improvement loop over rulesets are later phases.
## Running the benchmarks
Benchmarks wrap the same scan pipeline and score it against ground-truth
manifests in `benchmarks/manifests/`. Each manifest lists the *known*
vulnerabilities (CWE, file, line, sink, evidence) plus optional external
baselines; fixtures live in `benchmarks/fixtures/` and include safe-API files so
precision is measured honestly.
```bash
# One language
uv run ousast benchmark benchmarks/manifests/cpp-damn-vulnerable.toml --mode quick
# All in-scope corpora
for m in c-cpp-smoke cpp-damn-vulnerable \
python-web-smoke python-vulnerable \
javascript-node-web-smoke javascript-vulnerable \
java-web-smoke java-spring-boot-vulnerable; do
uv run ousast benchmark "benchmarks/manifests/$m.toml" --mode quick
done
```
Each run writes `.openultrasast/benchmarks/<id>/`:
`benchmark_result.json` (recall, precision, misses, false positives, runtime),
`calibration_records.json` (every miss as a next-improvement candidate), and
`external_baseline_deltas.json` (tool-vs-tool comparison).
### Latest results (`--mode quick`)
The corpora are modeled on Damn Vulnerable C/C++, PyGoat/DVPWA and NodeGoat,
plus the real [`kiview/damn-vulnerable-spring-boot-app`](https://github.com/kiview/damn-vulnerable-spring-boot-app)
vendored verbatim (MIT). Three vulnerabilities are deliberately left in the
ground truth that pattern matching *cannot* reach (integer overflow, second-order
SQLi, prototype pollution) so recall is not self-fulfilling.
The detection gate (`uv run python -m openultrasast.gate`, also `tests/test_detection_benchmarks.py`)
enforces the project goal of **at least 90% recall and under 10% false positives** per language
on the bundled corpora in `benchmarks/manifests/`. Its output on this tree:
| Language | Recall | False-positive rate |
| --- | --- | --- |
@@ -163,255 +82,156 @@ SQLi, prototype pollution) so recall is not self-fulfilling.
| Java | 100% (4/4) | 0.0% |
| **Overall** | **93.6% (44/47)** | **0.0%** |
The project goal is **≥90% recall and <10% false positives** per language. The
gate lives in `tests/test_detection_benchmarks.py`, so a rule change that drops
recall or raises false positives fails CI.
These corpora are cheat-sheet fixtures modelled on deliberately vulnerable applications. They
show that a rule change did not break a known detection; they are not a recall estimate on
real code. Real-code results are in [docs/evaluation.md](docs/evaluation.md).
## Agent integrations, fusion, and ultra workflows
**PHP is not in the gate, and its numbers are in-sample only.** The PHP rules were written and
tightened while looking at the development corpus they were measured on
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`): 13 of 14 expected
findings on the `php-vulnerable` fixture with no false positive, 2 false positives on
`php-benign`, and per-rule precision lower bounds from 0.03 to 1.0 on the real pins. Treat
them as development numbers, not as a holdout result.
The `ousast` CLI is the interface. An agent that can run shell commands needs
nothing else:
**Framework knowledge is optional.** Rule and taint-fact entries that know a framework or
library (WordPress, Flask, Django, Express, Spring, ...) carry a `framework` or `library` tag
naming a row of `src/openultrasast/ruleset/frameworks.toml`. The loaders take `priors=`
(`"all"`, the default and today's behaviour; `"off"`, language-level knowledge only; or a set
of ids), so their contribution can be measured and switched off.
## Pre-push checks (experimental)
`ousast pre-push` analyses the commits a `git push` would publish, or an explicit
`--base`/`--head` pair, and compares the head against its base so that only new or worsened
defects are candidates. `--mode advisory` (the default) never blocks; `--mode blocking` and
`--incomplete-coverage block` opt into enforcement. The artifact must be written outside the
analysed repository.
```bash
uv run ousast scan . --mode quick --fail-on verified
uv run ousast pre-push /path/to/repo --base BASE --head HEAD \
--artifact /outside/the/repo/result.json
```
For [OpenCode](https://opencode.ai) there are optional project skills that
describe the workflows above, so the agent drives the same CLI
(`.opencode/skills/`):
The default capability registry is empty, so no normal alert is emitted today; every
candidate stays diagnostic in the artifact. Installation into a repository's hooks
(`ops/install-pre-push`), caching and the optional witness model are documented in
[ops/README.md](ops/README.md).
| Skill | Purpose |
| --- | --- |
| `openultrasast-scan` ✅ | run/plan `ousast scan`, indexing, evidence-gated analysis |
| `openultrasast-triage` ✅ | false-positive elimination, ranking calibration, verifier adjudication |
| `openultrasast-fix-audit` ✅ | OpenUltraCode fix lifecycle and adversarial fix review |
## Model providers and keys
### MCP server (narrow surface) ✅
Keys live in `.env` in the working directory (gitignored, never committed):
`ousast mcp` runs a zero-dependency MCP server over stdio (newline-delimited
JSON-RPC) exposing only stable, project-level operations — **never** arbitrary
shell, Docker, or internal hunter tools. Point an MCP client (OpenCode, Claude
Desktop, an IDE) at it:
```bash
DEEPSEEK_API_KEY=... # every LLM call: model layer, tool hunter, plane tasks
OPENROUTER_API_KEY=... # only OpenAI embeddings (openai/text-embedding-3-small)
```
The project uses DeepSeek (`deepseek-flash`) for LLM calls and OpenRouter only for embeddings.
Set `DEEPSEEK_API_KEY`: without it, chat calls fall back to OpenRouter when that key is set.
**`.env` never overrides a variable already exported in your shell.** A stale
`export DEEPSEEK_API_KEY=...` in your profile wins over the `.env` file silently; `unset` it
when a key change in `.env` seems to have no effect.
## Benchmarks and pair evaluation
```bash
# Score the scan against a ground-truth manifest
uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
# Vulnerable-vs-fixed pairs: fire on the vulnerable side, stay silent on the fix
uv run ousast pairs
uv run ousast pairs --slice sast # OWASP Benchmark + Juliet
uv run ousast pairs --slice vibe-py --pointers # also fetch non-vendored pointer pairs (network)
uv run python -m openultrasast.pair_gate # CI: local pairs must all pass
```
Each benchmark run writes `.openultrasast/benchmarks/<id>/` (`benchmark_result.json`,
`calibration_records.json` with every miss, `external_baseline_deltas.json`). The pair
catalogs live in `benchmarks/pairs/`.
## Self-improvement
`ousast improve` runs a bounded loop over the governed ruleset: benchmark, propose edits to
rule status and score constants only, validate, re-benchmark, and accept a round only if
recall stays at least 90%, false positives under 10%, the project score does not regress and
no matched finding is lost. Accepted rounds land in
`<target>/.openultrasast/calibration/rule_policy.json`, which the next scan of that target loads.
```bash
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory
```
With `--memory`, proposals also come from the rows plane runs stored (see below), through the
same validator and gate; rows from holdout pairs, from the gated manifest's own cases and from
any `--qualify-population` are dropped first. Details:
[docs/architecture.md](docs/architecture.md#self-improvement).
## The agentic plane (maintainers)
Model work over whole repositories runs as a Run manifest on [google/ax](ops/ax/README.md)
over Agent Substrate (a kind cluster on the maintainer's host), one isolated actor per task.
Each task binds its own Model and its own `usd`/`calls` budget, egress is deny-by-default per
task, and the provider key travels only in the task's start request.
```bash
uv run ousast plane doctor # kind, Agent Substrate, ax controller, runner image
uv run ousast plane run plane/runs/validation-46.yaml # a rerun skips tasks already done
uv run ousast plane status validation-46 # per-task status and token attribution
uv run ousast plane remember validation-46 # ingest the run's rows into the memory store
```
`ousast plane workspaces`, `alerts-engine` and `harvest` generate Runs and inputs. Bring-up,
the task catalogue, the memory store (`OUSAST_MEMORY`) and the checklist for moving the plane
to another Kubernetes cluster are in [ops/ax/README.md](ops/ax/README.md).
> HarnessX, the earlier optional agentic extra, was retired 2026-09-30, and a leftover
> `[harnessx]` section (retired 2026-09-30) is ignored with one warning. `[models] verifier`,
> `[fusion] panel_model` and `[fusion] decider_model` fail with a message naming the plane
> replacement. The LLM judge and the LLM fusion panels were retired with it; fusion is
> deterministic. See [RELEASE_NOTES.md](RELEASE_NOTES.md).
## Decision engine (in development, not adopted)
A learned decision engine is being built to replace hand-tuned detection edits: each
instrument (quick rules, the Joern engine, model sink classification, verify passes, repository
facts) produces signals, never verdicts, and a compiled AI classifier with local memory turns
them into a calibrated probability per candidate with BLOCK and ADVISORY operating points.
No scan, pre-push or report uses it yet. Status and the first measured slice:
[docs/decision-engine.md](docs/decision-engine.md).
## Agent integrations
An agent that can run shell commands needs nothing but the CLI. For
[OpenCode](https://opencode.ai) there are optional project skills in `.opencode/skills/`:
`openultrasast-scan`, `openultrasast-triage` and `openultrasast-fix-audit`.
`ousast mcp` runs a narrow MCP server over stdio (newline-delimited JSON-RPC) with ten tools:
`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`,
`benchmark`, `explain`, `propose_patch` (degrades visibly) and `export_report`. No tool runs a
shell, Docker or a free-form command.
```jsonc
// e.g. an MCP client config entry
{ "command": "uv", "args": ["run", "ousast", "mcp"] }
```
Tools: `openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`,
`artifacts`, `benchmark`, `explain`, `propose_patch` (degrades visibly — the patch
oracle is a later phase), and `export_report`. The `scan`/`benchmark` tools run the
bounded analysis pipeline; no tool accepts a free-form command. A typical flow:
`scan {path}` → returns a `run_dir` → `findings {run_dir}` → `explain {run_dir, finding_id}`.
> The `kiro-*` skills, `AGENTS.md` and `.kiro/` are the maintainer's development tooling and
> can be ignored by users.
**Ultra workflows (OpenUltraCode discipline):** fixes are never a one-shot patch
prompt. `openultrasast-fix-audit` runs the bounded lifecycle
`intake → plan → minimal patch → adversarial review → reconcile → fresh
verification → ready`, and a patch can only reach `patch_validated` after
sandboxed validation passes with no accepted blocking findings.
## Further reading
**Fusion (deepening) ✅:** when a finding needs more reasoning than the normal
ranker → hunter → verifier → mapping loop provides (critical/high severity,
verifier disagreement, conflicting static vs semantic evidence, risky fixes, or
an explicit high-assurance request), fusion runs two independent panels that
steel-man the vulnerability and false-positive cases, vote, and a decider issues
the disposition (accepted / rejected / mitigated / deferred / blocked) with votes,
model IDs, and degradations disclosed. It runs automatically on triggered findings
in `--mode standard`, writing `fusion.json` and a `fusion` block in the manifest.
Deterministic by default (`fusion.py`); set `[fusion] panel_model` to route the
panels through the configured provider:
Documentation site with flow diagrams (`docs/`, `mkdocs.yml`): `uv sync --extra docs && uv run mkdocs serve`.
```toml
[fusion]
enabled = true # default; runs on triggered findings in standard mode
panel_model = "gpt-4o" # optional: LLM panels via the [harnessx] provider
decider_model = "gpt-4o" # optional: disclosed in the decision's model IDs
high_assurance = false # true forces fusion on every finding
```
> The `kiro-*` skills, `AGENTS.md` and the `.kiro/` directory are the
> maintainer's spec-driven development tooling. They are not part of using
> OpenUltraSAST and can be ignored by users.
## Evidence ladder: how a false positive is eliminated
Evidence level is a state machine, not a label a model may assign. The verifier
runs on independent context (it never sees hunter reasoning) and enforces:
```
suspicion ─▶ static_corroboration ─▶ crash_reproduced ─▶ root_cause_explained
─▶ exploit_demonstrated ─▶ patch_validated
```
A finding is rejected or held back unless the *artifact* for the next level
actually exists:
- below `static_corroboration` → `NEEDS_EVIDENCE` (a model/heuristic suspicion is
never reported as verified);
- `static_corroboration` but **reachability unknown** → `REJECTED` with a
tie-breaker demanding call-graph / route / CLI / parser / dynamic evidence
(this is what kills "pattern matched but nothing attacker-controlled reaches
it" findings);
- `static_corroboration` **and** function-level reachability → `ACCEPTED`.
When a finding is still rejected after review, `openultrasast-triage` records a
**scoped false-positive learning** (`calibration.py`): a reason from a fixed
taxonomy (`unreachable_path`, `missing_attacker_control`, `sanitizer_disproved`,
`static_rule_mismatch`, `incorrect_model_assumption`, `duplicate`,
`insufficient_impact`, `unsupported`, `contradicted`, `unverified`), evidence,
and a **scope**. Scoping keeps the learning narrow: a rejected `syscall_entry`
claim in `auth/` demotes only `auth/…`, never the whole vulnerability class. This
is covered by `tests/test_calibration.py::test_scoped_false_positive_demotion_does_not_suppress_class_globally`.
Demotion (with an audit trail) is preferred over deletion, so an analyst can
always see *why* something was downgraded.
## How a false negative is eliminated
Misses are first-class signals, surfaced by the benchmark layer rather than
hidden. Every expected vulnerability that no finding matched becomes a
`BenchmarkCalibrationRecord` in `calibration_records.json`, e.g. the integer
overflow the regex engine cannot see:
```json
{
"cwe": "CWE-190",
"vulnerability_class": "integer overflow",
"path": "src/buffer.cpp",
"failed_stage": "benchmark_ground_truth_matching",
"next_improvement_candidate": "rules_or_language_hunter",
"reason": "no OpenUltraSAST finding matched the expected benchmark vulnerability"
}
```
The `next_improvement_candidate` routes the gap to the stage that should close
it: a static rule, SARIF source, entry-point mapping, retrieval package, hunter
prompt, dynamic reproducer, or skill route. Once a miss is
addressed, it stays closed because the recall/precision gate runs in CI.
`external_baseline_deltas.json` additionally shows where another tool found a
vulnerability OpenUltraSAST missed (or vice-versa).
## Configuring the HarnessX agentic plane
The LLM hunter pool and the llm-judge verifier are optional. The core install is
zero-dependency and `--mode quick` never calls a model. To enable the agentic
plane, install the extra, name the models, and pick a provider:
```bash
uv sync --extra harnessx # SHA-pinned optional dependency
export OPENAI_API_KEY=sk-... # or ANTHROPIC_API_KEY, matching the provider
uv run ousast scan /path/to/code --mode standard --config openultrasast.toml
```
```toml
# openultrasast.toml
[models]
hunter = "gpt-4o" # LLM hunter pool (empty -> deterministic hunter)
verifier = "gpt-4o" # llm-judge verifier (empty -> structural verifier)
[harnessx]
provider = "openai" # "anthropic" (default) | "openai" | "litellm"
max_cost_usd = 2.0 # per-task spend cap
token_threshold = 120000 # per-task token budget
```
The provider selects the HarnessX backend for **both** the hunter and the
verifier; each reads its own standard key env var (`openai` → `OPENAI_API_KEY`,
`anthropic` → `ANTHROPIC_API_KEY`). The HarnessX path activates only when the
extra is installed, a model is configured, and the mode is `standard` — all three
gated through one capability seam (`harness_ext.has_harnessx`). If any condition
fails, the scan transparently falls back to the deterministic hunter / structural
verifier and records a `degradations` entry in `manifest.json`, so a run is never
silently downgraded:
```bash
jq '.degradations' "$run/manifest.json" # null = HarnessX ran; entries = fell back
```
## The HarnessX self-improving cycle
Each scan is a composed harness of typed processors with read/write **state
contracts**. Strict mode fails the scan on a violation; warn mode records a
degradation. The loop now runs inside the pipeline without an agent: every scan
persists verifier rejections as scoped learnings, and the next scan loads them
and demotes those scopes before findings are produced.
```
┌──────────── automatic ledger (no agent in the loop) ───────────┐
▼ │
scan ──▶ verify ──▶ non-accepted findings ──▶ scoped learnings ─────────┤
│ (.openultrasast/calibration/ │ loaded by
│ false_positive_learnings.json) │ the next
next scan ──▶ rank ──▶ calibrate (demote rejected scopes) ──▶ findings ──┘ scan
```
What runs automatically each scan (✅, `calibration.py` + `cli.py`):
1. **`calibrate` stage** (after `rank`, before findings) loads the persistent
ledger and demotes the priority of every scope that previously produced a
rejected finding (`calibrate_rankings`), writing `applied_calibrations.json`.
2. **`record_calibration` stage** (after `verify`) turns this run's non-accepted
verifier outcomes into scoped `FalsePositiveLearning` records
(`learnings_from_verifications`) and merges them into the ledger, de-duplicated
by finding ID so a repeat rejection does not compound without bound.
Demotion is **scoped and reversible**: a rejection in `lib.py` demotes only that
scope; an accepted (reachable) finding in `app.py` is never touched; and because
priority is demoted rather than the finding deleted, the audit trail
(`applied_calibrations.json`, the ledger) shows exactly why a surface lost
attention. Covered by `tests/test_pipeline_calibration.py`.
### Rule-level loop: `ousast improve`
The ranking loop above adjusts *attention*. A second, bounded loop adjusts the
**governed ruleset itself** from benchmark feedback — and writes its result to the
same loop-owned ledger a scan reads, so the loop closes end-to-end:
```bash
# Preview what the loop would change (writes nothing)
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run
# Apply: accepted rounds land in <target>/.openultrasast/calibration/rule_policy.json,
# which the next `ousast scan`/`benchmark` of that target loads automatically.
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml
```
Each round (`improve/evolve.py`): benchmark with the current ledger → per-rule
signals (`rule_signals.json`) → propose **bounded** edits → validate → replay
smoke → re-benchmark → a **hard acceptance gate** → accept or revert
byte-for-byte. The loop may only pull two levers (rule status, score constants);
it can never edit a rule's pattern text or the authoritative 0–5 CWE severity, and
must stage `enabled → shadow` before `disabled`. A round is accepted only if
**recall ≥ 90% AND FP < 10% AND the project score did not regress AND no matched
finding was lost** — otherwise the ledger is never written. Auto-shadowing removes
a precision-dragger's false-positive cost without deleting it (recall is
preserved); persistent misses are *nominated* through the signal file, leaving
pattern authoring to a human pull request. The project score is the optimization
reward; the recall/FP gate is a separate hard constraint that is never folded into
it — and the same gate runs in CI (`python -m openultrasast.gate`), so a loop
result that breached it could never merge. Covered by `tests/test_improve.py` and
`tests/test_cli_improve.py`.
This is the deterministic substrate of the self-improvement loop. When the
optional `openultrasast[harnessx]` extra is present, the HarnessX `MetaAgent`
becomes the richer *proposer* that plugs into this same validated, gated
machinery — the safety contract (bounded levers, replay/novelty/journal gates,
hard acceptance gate, byte-for-byte revert) is identical either way.
Beyond these loops, the `openultrasast-triage` skill lets an agent adjust
prompt constraints, retrieval filters, skill routing and benchmark-miss triage.
Those `RankingCalibration` fields are already produced and will be consumed
directly once the per-family detectors land.
## Security & hardening
OpenUltraSAST treats scanned code as **untrusted input and never executes it** (quick
and standard modes). Scan artifacts are scrubbed of credentials before they are written
(`[hardening] redact_secrets`, on by default), agentic spend and output size are bounded
(`[harnessx] max_cost_usd`/`token_threshold`, `[hardening] max_findings`), provider calls
retry transient failures with backoff, and missing capabilities degrade visibly in the
manifest rather than silently. See [docs/threat-model.md](docs/threat-model.md) for the
full trust boundaries and sandbox limits, and [docs/examples.md](docs/examples.md) for
end-to-end walkthroughs.
- [docs/architecture.md](docs/architecture.md): the scan pipeline, evidence ladder, CWE policy
and project score, and the self-improving loops.
- [docs/evaluation.md](docs/evaluation.md): independent populations, the qualification gates,
pair corpora and the plane increment results.
- [docs/decision-engine.md](docs/decision-engine.md): the learned decision engine.
- [docs/threat-model.md](docs/threat-model.md): trust boundaries, sandbox and hardening.
- [docs/token-ergonomics.md](docs/token-ergonomics.md): model spend at runtime (budgets, reuse,
measured costs) and in coding sessions.
- [docs/examples.md](docs/examples.md): end-to-end walkthroughs.
## Development
@@ -420,7 +240,11 @@ uv run pytest # tests, incl. the 90/10 detection gate
uv run ruff check . # lint
uv run ruff format --check . # format
uv run mypy src/openultrasast # types
uv run python -m openultrasast.gate # standalone detection gate
uv run python -m openultrasast.pair_gate # local vuln-vs-fix efficiency + GitHub honesty
uv run python dagger/ci.py # full containerized CI pipeline
uv run python -m openultrasast.gate # standalone detection gate
uv run python -m openultrasast.pair_gate # local vuln-vs-fix pairs
uv run python dagger/ci.py # containerized CI pipeline
```
Maintainer commands not covered above: `ousast repos` (pinned known-vulnerable checkouts),
`ousast model candidates` (what the candidate enumerator can reach) and `ousast learn`
(the decision engine's data; see [docs/decision-engine.md](docs/decision-engine.md)).
+106
View File
@@ -1,3 +1,109 @@
# v2.0.0 (2026-10-02)
A new major release: the agentic plane runs on google/ax, HarnessX is gone, detection decisions move
toward a trained-by-memory AI classifier, and the plane memory lives in an S3-compatible store.
- **Agentic plane on ax** (`ousast plane run|status|doctor|workspaces|remember|alerts-engine|harvest`): tasks
run in gVisor sandboxes on Agent Substrate; one Model and one budget per task; token attribution per task
and model; credentials only in the start request; per-task egress policies. First increment measured:
16/20 declared sites agreed at $0.0205 per candidate (`benchmarks/independent/plane-increment-2.json`).
- **HarnessX removed.** Retired config keys fail with a reason; `[harnessx]` warns. Deterministic outputs
proven byte-identical (`benchmarks/measurements/2026-09-30-harnessx-removal-equality/`).
- **Memory store** (`OUSAST_MEMORY`): local files or any S3-compatible store with versioning, lifecycle,
tags and S3 Select; RustFS is the tested server (`docs/rustfs.md`). The store verifies its bucket at
startup and never falls back; every row carries its kind's queryable fields. New extra `s3` (boto3).
- **Decision engine (in development, not adopted):** an in-house DSPy-style AI classifier with local
memory; labels from ground truth; repository-grouped evaluation. Slice 2 on six families: out-of-repo
AUC 0.77-0.96 (within-pair 0.72-0.93); BLOCK not offered anywhere; see
`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json` and `docs/memory-and-detection.md`.
- **Improvement from memory:** `ousast improve --memory`; the loop as a plane Run.
- **PHP quick-mode rules** (in-sample measurement only) beside the Joern engine; framework knowledge
tagged as optional priors (`ruleset/frameworks.toml`).
- **Populations:** v3 (PHP, 15 cases) frozen and untouched as the one-time final check; 97 advisory-fix
pairs added for negatives; licences of published cases verified (v1's OpenCVE case is BUSL-1.1).
- **Removed legacy:** dead modules (`model.judge`, `model.execution`, `model.calibrate`,
`stage_processors`, `slot_contract`); unread config keys `[dynamic]`, `[evidence]`, `[variants]`,
`[models] patcher` retired (`resolved_config.json` loses those sections); reference-only scripts under
`benchmarks/archive/`; the module audit now classifies by real imports.
- **Docs:** MkDocs site (`mkdocs serve`), including token ergonomics and the RustFS setup.
Detailed notes for this release follow.
## PHP quick-mode rules (2026-09-30)
`ruleset/php/rules.toml` adds 18 PHP rules (SQL injection, command injection, file inclusion,
path, open redirect, SSRF; code eval, unserialize and echo XSS as `shadow`). They were written
on the development corpus, so their numbers are in-sample only
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`); PHP is not part of
the detection gate. `ousast plane alerts-engine` produces a Run's `alerts` for PHP from the
Joern engine image on the host.
## Framework knowledge as optional priors (2026-09-30)
Rule and taint-fact entries that know a framework or library carry a `framework`/`library`
tag naming a row of `ruleset/frameworks.toml`; the loaders take `priors` (`"all"` by default,
unchanged behaviour; `"off"`; or a set of ids). Scan output is unchanged.
## Decision engine, in development (2026-10-01)
The learned decision engine (signals from every instrument, labels from ground truth, an AI
classifier with local memory compiled in house) is in development and **not adopted**: no scan,
pre-push check or report uses it. New maintainer commands: `ousast learn labels|memory|compile|
evaluate|curve|audit-leaks` and `ousast plane harvest`; new plane tasks `features` and `roles`;
the `advisory-fixes` pair source (97 advisory fix commits). First slice (injection, 110
out-of-repository candidates): ADVISORY recall 0.90, precision 0.54, BLOCK not offered
(`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`;
[docs/decision-engine.md](docs/decision-engine.md)).
## Documentation (2026-10-02)
The README is reduced to user-facing material; scan internals moved to
[docs/architecture.md](docs/architecture.md) and measured results to
[docs/evaluation.md](docs/evaluation.md). Removed references to commands that no longer exist
(`ousast mechanisms export`, `ousast pairs --loo`) and corrected the description of `deep`
mode, which runs sandboxed regression snippets when Docker is available.
## Unread config keys retired (2026-10-02)
These keys loaded but nothing ever read them. They now fail with `RetiredConfigError` (the
CLI prints it and exits 2), naming why. Remove them from your config:
- `[models] patcher`: no patching stage calls a model.
- `[dynamic] enabled`, `[dynamic] network_scope`: no dynamic or deep tier exists yet.
- `[evidence] minimum_report_verified`, `minimum_exploit`, `minimum_patch`: evidence tiers
come from the program model's ladder, not from configuration.
- `[variants] enabled`, `[variants] max_mechanisms`: the variant search they configured was
deleted earlier.
**Output shape.** `resolved_config.json` no longer has the `dynamic`, `evidence` and
`variants` sections or `models.patcher`. `harness.json` keeps its shape:
`model_roles.patcher` is always `null`, like `model_roles.verifier`.
## HarnessX removed; the ax plane is the agentic path (2026-09-30)
HarnessX, the optional agentic extra (`openultrasast[harnessx]`), is removed with its code,
packaging extra and lock entries (retired 2026-09-30). Agentic work runs on the ax plane:
`ousast plane run|status|doctor|remember`, with one Model and one `usd`/`calls` budget per
task, deny-by-default egress per task, and the provider key sent only in the task's start
request. See [ops/ax/README.md](ops/ax/README.md). Deterministic outputs (pair replay,
quick benchmark, standard scan, `improve --dry-run`) are byte-identical before and after.
Retired config keys (retired 2026-09-30):
- `[models] verifier` fails with `RetiredConfigError` (the CLI prints it and exits 2): LLM
verification runs on the plane (`verify` + `agree`). Remove the key.
- `[fusion] panel_model` and `[fusion] decider_model` fail the same way: fusion is
deterministic; independent LLM agreement is the plane's `agree` task. Remove the keys.
- A `[harnessx]` section (retired 2026-09-30) loads with one warning naming the plane and is
otherwise ignored.
**Silent change for `[models] hunter`.** If you set `[models] hunter` and had the extra
installed, a standard scan ran two LLM hunters: the MAP-stage tool hunter and the
HarnessX hunter pool (findings with ids `hx-hunter:<path>:<line>`). The key is still
valid and still enables the tool hunter, so nothing fails and nothing warns, but the
second hunter's findings no longer appear. Repository-wide LLM hunting runs on the plane
(`verify` passes and `agree`).
The LLM judge, the LLM fusion panels and the in-loop hosting of deterministic stages are
retired without a local replacement. `ousast improve --memory` now proposes rule-status
edits from the plane memory, through the unchanged validator and gate.
# v1.2.0-alpha.1
Experimental pre-push security safety net for AI-accelerated development. Python package
+26
View File
@@ -0,0 +1,26 @@
# Archived benchmark scripts
Reference-only scripts, moved here with `git mv` on 2026-10-02 (legacy cleanup) so their history
follows them. Nothing in `src/` imports them and no current workflow runs them. They stay because
committed records cite them and their numbers: to re-run a record, check out the commit the record
names rather than running these copies against today's package.
The old paths still appear inside committed records and in some docstrings. [paths.md](paths.md)
maps each old path to the new one. The records themselves are not rewritten.
| script | what it was | records that cite it | last recorded run |
|---|---|---|---|
| `verify_batched.py` | Stage-2 verifier, second design: triage per file, verify per file, report what two passes agree on. Replaced by the plane's `verify` + `agree` tasks. | `independent/results-v2-model-pipeline.json`, `independent/verifier-batched-check-2026-09-29.json`, `independent/plane-increment-2.json` (parity baseline) | `072a102` (named in `results-v2-model-pipeline.json`) |
| `score_batched.py` | Exploratory scorer for the batched verifier: sites anchored to the candidate, matched by `evaluate.matches_v2`. | `independent/verifier-batched-check-2026-09-29.json`, `independent/plane-increment-2.json`; `tests/test_plane_scoring_parity.py` still loads it as the reference | added in `6acd354` (2026-09-29); the parity test replays it on hosts with the recorded scans |
| `verify_sinks.py` | The first stage-2 verifier, one candidate at a time. `verify_batched.py` imports `ACCOUNT_ERRORS` and `_Recording` from it. | `independent/verifier-check-2026-09-29.json` | the known-answer check after `041d871` |
| `model_sinks.py` | Stage-1 source-only model classification of sinks. Moved into the package as `plane/tasks/roles.py`. | `independent/selection-v2-model-sinks.json`, `measurements/2026-10-01-decision-engine-harvest/record.json`, both `2026-10-01-decision-engine-leak-audit` records | `05d5699` (named in `selection-v2-model-sinks.json`) |
| `sink_candidates.py` | Diagnostic: vocabulary-declared sink candidates per case. Its declaration patterns live on in `plane/tasks/repo_facts.py`. | `independent/results-v2-model-pipeline.json` ("vocabulary sink-first: 2/17") | no commit recorded |
| `hunt.py` | The v2 tool-hunter run over the population (protocol `hunter-protocol-v2.md`, including its `FAMILY_WORDS` table). Replaced by the plane's `verify` / `agree` tasks. | `independent/hunter-protocol-v2.md`, `independent/results-v2-hunter.json` (a label source in `learn/sources.toml`) | `6bfb05f`, the commit that introduced the protocol, which pins the runner |
| `harnessx_removal_equality.py` | One-off equality proof for `harnessx-removal` (record and compare deterministic outputs). The spec is closed. `tests/test_harnessx_removal_equality.py` still checks that the comparison detects differences. | `measurements/2026-09-30-harnessx-removal-baseline/`, `measurements/2026-09-30-harnessx-removal-equality/` | baseline `ca5caf7`, candidate `b9afb66` |
The six `independent` scripts still read the population, `evaluate.py` and the results from
`benchmarks/independent/`. Their `HERE` points there, so outputs land where they always did.
`evaluate.py` and `freeze.py` are current and were not moved.
`verify_sinks.py --help` fails with an argparse `%` formatting error. That bug was already there
before the move. The script still imports cleanly.
@@ -0,0 +1,396 @@
"""Equality proof for the HarnessX removal (spec harnessx-removal, design section 3, Requirement 3).
``record`` exports a frozen source tree, runs the deterministic commands with no model reachable and the
``harnessx`` extra absent, and stores their normalized outputs. ``compare`` diffs two such records and exits
non-zero on the first difference outside the exclusion list, naming the file and key.
Usage:
python benchmarks/archive/harnessx_removal_equality.py record <out-dir> [--tree-ish HEAD] [--workdir DIR]
python benchmarks/archive/harnessx_removal_equality.py compare <baseline-dir> <candidate-dir> \\
[--suite-map map.json] [--record comparison.json]
The instrument checks come first and fail loudly: ``harnessx`` must not be importable, ``tree_sitter`` must be,
every child must import ``openultrasast`` from the export, every input manifest must be non-empty, every
command must exit 0, and every command must produce non-zero counts (pairs, TPs, expected findings, suite tests).
"""
from __future__ import annotations
import argparse
import hashlib
import importlib.util
import json
import os
import re
import shutil
import subprocess
import sys
import tempfile
import xml.etree.ElementTree as ET
from pathlib import Path
from typing import Any
REPO = Path(__file__).resolve().parents[2] # archived under benchmarks/archive/
# Design section 3 "Comparison": timestamps, durations and run identifiers are excluded; nothing else is.
EXCLUDED_KEYS = frozenset({"runtime_seconds", "scan_id", "timestamp", "started", "finished", "created_at", "generated_at"})
EXCLUDED_SUFFIXES = ("_seconds", "_ms")
RUN_ID = re.compile(r"\d{8}T\d{6}Z-[0-9a-f]{8}")
EXCLUSIONS = [
f"JSON keys: {', '.join(sorted(EXCLUDED_KEYS))}",
f"JSON keys ending in: {', '.join(EXCLUDED_SUFFIXES)}",
"run directory names / run ids (<timestamp>-<8 hex>, benchmark.py create_benchmark_run, run.py create_scan_run) -> <RUN_ID>",
"trace/events.jsonl as a whole (timings); never copied",
"the export's absolute root path -> <ROOT>; the children's TMPDIR -> <TMP>",
"suite: only the outcome per test id is kept (junit durations dropped)",
"env.json is a provenance record (commit, freeze hash, instrument checks), not compared",
]
INPUTS = (
"benchmarks/pairs/catalog.toml",
"benchmarks/manifests/python-vulnerable.toml",
"benchmarks/manifests/java-spring-boot-vulnerable.toml",
)
SCAN_KEEP = ("findings.json", "verification.json", "fusion.json", "report.sarif", "harness.json", "manifest.json")
BENCH_KEEP = ("benchmark_result.json", "calibration_records.json", "external_baseline_deltas.json")
class InstrumentError(SystemExit):
def __init__(self, message: str) -> None:
super().__init__(f"instrument check failed: {message}")
def normalize(value: Any, root: str, tmp: str) -> Any:
"""Drop excluded keys recursively and scrub the export root, TMPDIR and run ids from strings."""
if isinstance(value, dict):
return {k: normalize(v, root, tmp) for k, v in value.items() if not _excluded(k)}
if isinstance(value, list):
return [normalize(v, root, tmp) for v in value]
if isinstance(value, str):
return scrub(value, root, tmp)
return value
def _excluded(key: str) -> bool:
return key in EXCLUDED_KEYS or key.endswith(EXCLUDED_SUFFIXES)
def scrub(text: str, root: str, tmp: str) -> str:
text = text.replace(tmp, "<TMP>").replace(root, "<ROOT>")
return RUN_ID.sub("<RUN_ID>", text)
def dump(value: Any) -> str:
return json.dumps(value, indent=1, sort_keys=True) + "\n"
def _child_env(root: Path, tmp: Path) -> dict[str, str]:
env = {k: v for k, v in os.environ.items() if not k.endswith("_API_KEY") and k not in {"PYTHONPATH", "VIRTUAL_ENV"}}
env.update(OPENULTRASAST_SKIP_DOTENV="1", PYTHONPATH=str(root / "src"), TMPDIR=str(tmp), PYTHONHASHSEED="0")
return env
def _run(cmd: list[str], root: Path, env: dict[str, str], label: str) -> str:
proc = subprocess.run(cmd, cwd=root, env=env, capture_output=True, text=True, check=False)
if proc.returncode != 0:
tail = (proc.stdout + proc.stderr)[-2000:]
raise InstrumentError(f"{label} exited {proc.returncode}: {' '.join(cmd)}\n{tail}")
return proc.stdout
def _field(stdout: str, name: str) -> str:
for line in stdout.splitlines():
if line.startswith(f"{name}="):
return line.partition("=")[2]
raise InstrumentError(f"no {name}= line in output")
def _freeze_hash() -> str:
uv = shutil.which("uv")
if uv is None:
raise InstrumentError("uv not found; cannot hash the environment")
out = subprocess.run([uv, "pip", "freeze", "--python", sys.executable], capture_output=True, text=True, check=True).stdout
return hashlib.sha256(out.encode()).hexdigest()
def _suite(xml_path: Path) -> dict[str, str]:
outcomes: dict[str, str] = {}
for case in ET.parse(xml_path).getroot().iter("testcase"):
test_id = f"{case.get('classname')}::{case.get('name')}"
kinds = {child.tag for child in case}
outcome = next((k for k in ("error", "failure", "skipped") if k in kinds), "passed")
outcomes[test_id] = outcome
return dict(sorted(outcomes.items()))
def record(out: Path, tree_ish: str, workdir: Path) -> None:
if importlib.util.find_spec("harnessx") is not None:
raise InstrumentError("harnessx is importable; re-sync the venv without the extra")
if importlib.util.find_spec("tree_sitter") is None:
raise InstrumentError("tree_sitter is not importable; the semantic extra is missing")
workdir.mkdir(parents=True, exist_ok=True)
root = Path(tempfile.mkdtemp(prefix="export-", dir=workdir)).resolve()
tmp = Path(tempfile.mkdtemp(prefix="tmp-", dir=workdir)).resolve()
archive = subprocess.run(["git", "-C", str(REPO), "archive", tree_ish], capture_output=True, check=True).stdout
subprocess.run(["tar", "-x", "-C", str(root)], input=archive, check=True)
env = _child_env(root, tmp)
rs, ts = str(root), str(tmp)
py = sys.executable
out.mkdir(parents=True, exist_ok=True)
checks: dict[str, Any] = {}
module = _run([py, "-c", "import openultrasast; print(openultrasast.__file__)"], root, env, "import probe").strip()
if not module.startswith(rs):
raise InstrumentError(f"openultrasast imported from {module}, not the export {rs}")
checks["openultrasast_module"] = scrub(module, rs, ts)
inputs = {name: (root / name).stat().st_size for name in INPUTS}
if not all(inputs.values()):
raise InstrumentError(f"empty input manifest: {inputs}")
print(f"export={root} module={module} inputs={inputs}", flush=True)
# 1. Full suite, outcome per test id.
junit = tmp / "suite.xml"
_run([py, "-m", "pytest", "-q", "-p", "no:cacheprovider", f"--junitxml={junit}"], root, env, "pytest")
suite = _suite(junit)
counts = {k: sum(1 for v in suite.values() if v == k) for k in ("passed", "skipped", "failure", "error")}
if counts["passed"] == 0:
raise InstrumentError("suite recorded no passing tests")
(out / "suite.json").write_text(dump(suite))
checks["suite"] = counts
print(f"suite={counts}", flush=True)
# 2. Pair replay (offline, vendored pairs only).
pairs = normalize(json.loads(_run([py, "-m", "openultrasast.cli", "pairs", "--json"], root, env, "pairs")), rs, ts)
checks["pairs"] = _pair_counts(pairs)
(out / "pairs.json").write_text(dump(pairs))
print(f"pairs={checks['pairs']}", flush=True)
# 3. Quick benchmark replay.
empty = tmp / "empty.toml"
empty.write_text("")
cmd = [
py,
"-m",
"openultrasast.cli",
"benchmark",
"benchmarks/manifests/python-vulnerable.toml",
"--mode",
"quick",
"--config",
str(empty),
]
stdout = _run(cmd, root, env, "benchmark")
expected = int(_field(stdout, "benchmark_expected"))
if expected <= 0:
raise InstrumentError("benchmark expected no findings")
checks["benchmark"] = {k: int(_field(stdout, f"benchmark_{k}")) for k in ("expected", "matched", "missed")}
checks["benchmark"]["findings"] = int(_field(stdout, "findings"))
_keep(Path(_field(stdout, "benchmark_run_dir")), BENCH_KEEP, out / "benchmark", rs, ts, checks, "benchmark_files")
(out / "benchmark" / "stdout.txt").write_text(scrub(stdout, rs, ts))
print(f"benchmark={checks['benchmark']}", flush=True)
# 4. One standard-mode scan (verify + fusion, no model configured).
cmd = [py, "-m", "openultrasast.cli", "scan", "benchmarks/fixtures/python-vulnerable", "--mode", "standard", "--config", str(empty)]
stdout = _run(cmd, root, env, "scan")
checks["scan"] = {"findings": int(_field(stdout, "findings")), "file_targets": int(_field(stdout, "file_targets"))}
if checks["scan"]["file_targets"] <= 0:
raise InstrumentError("standard scan saw no file targets")
_keep(Path(_field(stdout, "run_dir")), SCAN_KEEP, out / "scan", rs, ts, checks, "scan_files")
(out / "scan" / "stdout.txt").write_text(scrub(stdout, rs, ts))
print(f"scan={checks['scan']}", flush=True)
# 5. improve --dry-run; the shim keeps the dry-run journal the CLI writes to a throwaway directory.
shim_out = tmp / "improve"
cmd = [py, str(Path(__file__).resolve()), "_improve", str(shim_out), "benchmarks/manifests/java-spring-boot-vulnerable.toml"]
stdout = _run(cmd, root, env, "improve")
(out / "improve").mkdir(exist_ok=True)
(out / "improve" / "stdout.txt").write_text(scrub(stdout, rs, ts))
# A round with no proposals writes no journal (evolve.py returns before append_round); its absence is
# recorded and compared like any file, so a journal appearing after the removal is a difference.
rounds = [line for line in stdout.splitlines() if line.startswith("round ")]
if not rounds or "rounds=" not in stdout:
raise InstrumentError("improve --dry-run printed no round")
journal = shim_out / "improve_journal.json"
checks["improve"] = {"rounds": len(rounds), "journal_bytes": journal.stat().st_size if journal.is_file() else None}
if journal.is_file():
journal_data = normalize(json.loads(journal.read_text()), rs, ts)
(out / "improve" / "improve_journal.json").write_text(dump(journal_data))
print(f"improve={checks['improve']}", flush=True)
head = subprocess.run(["git", "-C", str(REPO), "rev-parse", tree_ish], capture_output=True, text=True, check=True).stdout.strip()
env_record = {
"tree_ish": head,
"freeze_sha256": _freeze_hash(),
"harnessx_importable": False,
"tree_sitter": True,
"python": sys.version.split()[0],
"child_env": {"OPENULTRASAST_SKIP_DOTENV": "1", "PYTHONHASHSEED": "0", "api_keys": "every *_API_KEY unset", "config": "empty"},
"inputs": inputs,
"instrument_checks": checks,
"exclusions": EXCLUSIONS,
}
(out / "env.json").write_text(dump(env_record))
(out / "README").write_text("Excluded from comparison: " + "; ".join(EXCLUSIONS) + "\n")
shutil.rmtree(root, ignore_errors=True)
shutil.rmtree(tmp, ignore_errors=True)
print(f"recorded {out}")
def _pair_counts(pairs: Any) -> dict[str, int]:
overall = pairs.get("overall", {}) if isinstance(pairs, dict) else {}
counts = {k: int(overall.get(k, 0)) for k in ("pairs", "detected_vuln", "pair_correct", "labeled_expected", "labeled_matched")}
counts["outcomes"] = len(pairs.get("outcomes", [])) if isinstance(pairs, dict) else 0
if counts["pairs"] <= 0 or counts["detected_vuln"] <= 0 or counts["outcomes"] <= 0:
raise InstrumentError(f"pair replay saw {counts}")
return counts
def _keep(run_dir: Path, names: tuple[str, ...], dest: Path, rs: str, ts: str, checks: dict[str, Any], key: str) -> None:
if not run_dir.is_dir():
raise InstrumentError(f"run directory missing: {run_dir}")
dest.mkdir(parents=True, exist_ok=True)
present = {}
for name in names:
source = run_dir / name
if source.is_file():
present[name] = source.stat().st_size
(dest / name).write_text(dump(normalize(json.loads(source.read_text()), rs, ts)))
checks[key] = present
def _improve_shim(out: Path, manifest: str) -> int:
"""Run ``ousast improve --dry-run --no-pair-gate`` and copy the throwaway journal out before cleanup."""
from openultrasast import cli
original = tempfile.TemporaryDirectory
class Keeping(original): # type: ignore[misc,valid-type]
def cleanup(self) -> None:
out.mkdir(parents=True, exist_ok=True)
for name in ("improve_journal.json",):
if (Path(self.name) / name).is_file():
shutil.copy2(Path(self.name) / name, out / name)
super().cleanup()
def __exit__(self, *exc: object) -> None:
self.cleanup()
cli.tempfile.TemporaryDirectory = Keeping # type: ignore[misc]
try:
return int(cli.main(["improve", manifest, "--dry-run", "--no-pair-gate"]) or 0)
finally:
cli.tempfile.TemporaryDirectory = original # type: ignore[misc]
def compare(baseline: Path, candidate: Path, suite_map: dict[str, Any]) -> list[str]:
"""Return every difference outside the exclusion list; empty means equal."""
diffs: list[str] = []
diffs += _compare_suite(
json.loads((baseline / "suite.json").read_text()), json.loads((candidate / "suite.json").read_text()), suite_map
)
files = sorted(
{p.relative_to(baseline).as_posix() for p in baseline.rglob("*") if p.is_file()}
| {p.relative_to(candidate).as_posix() for p in candidate.rglob("*") if p.is_file()}
)
for name in files:
if name in {"suite.json", "env.json", "README", "comparison.json", "determinism.json"}:
continue
a, b = baseline / name, candidate / name
if not a.is_file() or not b.is_file():
diffs.append(f"{name}: present only in {'candidate' if b.is_file() else 'baseline'}")
continue
if a.read_bytes() == b.read_bytes():
continue
if name.endswith((".json", ".sarif")):
diffs.append(f"{name}: {_first_json_diff(json.loads(a.read_text()), json.loads(b.read_text()), '$')}")
else:
diffs.append(f"{name}: first differing line {_first_line_diff(a.read_text(), b.read_text())}")
return diffs
def _compare_suite(a: dict[str, str], b: dict[str, str], suite_map: dict[str, Any]) -> list[str]:
deleted = set(suite_map.get("deleted", ()))
renamed: dict[str, str] = dict(suite_map.get("renamed", {}))
added = set(suite_map.get("added", ())) | set(renamed.values())
diffs = []
for test_id, outcome in a.items():
if test_id in deleted:
continue
target = renamed.get(test_id, test_id)
if target not in b:
diffs.append(f"suite.json: {test_id} missing from candidate")
elif b[target] != outcome:
diffs.append(f"suite.json: {test_id} {outcome} -> {b[target]}")
diffs += [f"suite.json: {t} new in candidate" for t in b if t not in a and t not in added]
return diffs
def _first_json_diff(a: Any, b: Any, path: str) -> str:
if type(a) is not type(b):
return f"{path}: type {type(a).__name__} -> {type(b).__name__}"
if isinstance(a, dict):
for key in sorted(set(a) | set(b)):
if key not in a or key not in b:
return f"{path}.{key}: present only in {'candidate' if key in b else 'baseline'}"
if a[key] != b[key]:
return _first_json_diff(a[key], b[key], f"{path}.{key}")
if isinstance(a, list):
if len(a) != len(b):
return f"{path}: length {len(a)} -> {len(b)}"
for i, (x, y) in enumerate(zip(a, b, strict=True)):
if x != y:
return _first_json_diff(x, y, f"{path}[{i}]")
return f"{path}: {json.dumps(a)[:120]} -> {json.dumps(b)[:120]}"
def _first_line_diff(a: str, b: str) -> str:
la, lb = a.splitlines(), b.splitlines()
for i, (x, y) in enumerate(zip(la, lb, strict=False), 1):
if x != y:
return f"{i}: {x[:120]!r} -> {y[:120]!r}"
return f"line count {len(la)} -> {len(lb)}"
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
sub = parser.add_subparsers(dest="command", required=True)
rec = sub.add_parser("record")
rec.add_argument("out", type=Path)
rec.add_argument("--tree-ish", default="HEAD", help="git tree-ish to export (a commit, or `git write-tree` for the index)")
rec.add_argument("--workdir", type=Path, default=Path(tempfile.gettempdir()) / "hxr-equality")
cmp_ = sub.add_parser("compare")
cmp_.add_argument("baseline", type=Path)
cmp_.add_argument("candidate", type=Path)
cmp_.add_argument("--suite-map", type=Path, help='JSON {"deleted": [...], "renamed": {old: new}, "added": [...]}')
cmp_.add_argument("--record", type=Path, help="write the comparison result here as JSON")
shim = sub.add_parser("_improve")
shim.add_argument("out", type=Path)
shim.add_argument("manifest")
args = parser.parse_args(argv)
if args.command == "record":
record(args.out.resolve(), args.tree_ish, args.workdir.resolve())
return 0
if args.command == "_improve":
return _improve_shim(args.out, args.manifest)
suite_map = json.loads(args.suite_map.read_text()) if args.suite_map else {}
diffs = compare(args.baseline, args.candidate, suite_map)
if args.record:
args.record.write_text(
dump(
{
"baseline": str(args.baseline),
"candidate": str(args.candidate),
"equal": not diffs,
"differences": diffs,
"exclusions": EXCLUSIONS,
}
)
)
for line in diffs[:20]:
print(line)
print(f"equal={not diffs} differences={len(diffs)}")
return 1 if diffs else 0
if __name__ == "__main__":
raise SystemExit(main())
+192
View File
@@ -0,0 +1,192 @@
"""Run the UNCHANGED tool hunter over an independent population's pins, scoreable by the evaluation protocol.
The hunter (`tool_hunter.run_tool_hunter`, default prompt, default tools, default step budget) is handed the
regions the ranker selects for the case's family -- the same discovery and ranking the static scan starts from,
top `--max-regions` by rank, in groups of `--per-hunt` -- and nothing else: not the fix diff, not the advisory,
not the declared sites. Each pin's result is written in the scan-result shape `evaluate.py score` reads, into
`~/ousast-results/independent-<vN>-hunter/scans/`, so the same pre-registered matching scores it.
Model calls go to the resolved hunter client (DeepSeek when its key is set); the run stops, and says so, when
the recorded spend reaches `--budget-usd`.
Usage:
python benchmarks/archive/hunt.py --population population-v2.toml [--only ID,...] [--pins a,b]
python benchmarks/archive/hunt.py --population population-v2.toml --score
python benchmarks/archive/hunt.py --pilot-root <checkout> --family injection # a development case
"""
from __future__ import annotations
import argparse
import dataclasses
import json
import shutil
import sys
import time
from pathlib import Path
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
HERE = Path(__file__).resolve().parents[1] / "independent"
sys.path.insert(0, str(HERE))
import evaluate # noqa: E402
from openultrasast.mapping import analyze_entry_points # noqa: E402
from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL # noqa: E402
from openultrasast.model.regions import regions_for # noqa: E402
from openultrasast.model.shipped import declared_sources # noqa: E402
from openultrasast.preprocess import build_file_target, enumerate_source_files # noqa: E402
# The model names a family in its own words; the protocol scores three. Fixed before any v2 pin is hunted.
FAMILY_WORDS = {
"untrusted_destination": ("ssrf", "server-side request", "open redirect", "redirect", "url fetch", "request forgery"),
"config_secrets": ("cors", "debug", "secret", "hardcoded", "hard-coded", "credential", "misconfig", "configuration", "tls", "certificate"),
"injection": ("inject", "sql", "sqli", "nosql", "command", "rce", "code execution", "eval", "template", "ssti", "xss", "deserializ", "ldap", "xpath"),
}
def family_of(raw: str, title: str) -> str:
text = f"{raw} {title}".lower()
for family, words in FAMILY_WORDS.items():
if any(word in text for word in words):
return family
return "other"
def hotspots_for(root: Path, family: str, max_regions: int) -> list:
from openultrasast.complexity.map import Hotspot
files = enumerate_source_files(root)
if not files or not sum(p.stat().st_size for p in files if p.is_file()):
raise SystemExit(f"read nothing under {root}")
targets = [build_file_target(root, p) for p in files]
regions = [
dataclasses.replace(r, families=tuple(f for f in r.families if f == family))
for r in regions_for(analyze_entry_points(root, targets), targets, shipped=declared_sources(root))
]
ranked = sorted((r for r in regions if r.families), key=lambda r: (-r.rank, r.path, r.function or ""))[:max_regions]
return [
Hotspot(
path=r.path,
function_name=r.function or None,
score=float(r.rank),
band="ranker",
signals={},
rationale=f"ranked region offering {family}",
test_hint=None,
inventory_finding_ids=(),
)
for r in ranked
]
def _spent(client: object) -> float:
cost = getattr(client, "cost_usd", None)
return float(cost()) if callable(cost) else 0.0
def hunt(root: Path, family: str, args: argparse.Namespace, spent: list[float]) -> dict:
from openultrasast import tool_hunter
client = tool_hunter.resolve_hunter_client()
if client is None:
raise SystemExit("no hunter client: set DEEPSEEK_API_KEY")
started = time.monotonic()
hotspots = hotspots_for(root, family, args.max_regions)
groups = [hotspots[i : i + args.per_hunt] for i in range(0, len(hotspots), args.per_hunt)]
findings, done, errors = [], 0, []
for group in groups:
if sum(spent) + _spent(client) >= args.budget_usd:
errors.append("budget_exhausted")
break
try:
found = tool_hunter.run_tool_hunter(root, group, client=client, model=args.model, max_steps=tool_hunter.DEFAULT_MAX_STEPS)
done += 1
except Exception as error: # noqa: BLE001 -- recorded per hunt; a pin with no completed hunt is an instrument failure
errors.append(f"{type(error).__name__}: {str(error)[:200]}")
continue
for f in found:
raw = next((t.split(":", 1)[1] for t in f.tags if t.startswith("family:")), "")
findings.append(
{
"site": f"{f.path}:{f.line or 0}:{f.function_name or ''}",
"family": family_of(raw, f.title),
"model_family": raw,
"title": f.title,
"witness": f.rationale[:400],
}
)
cost = _spent(client)
spent.append(cost)
return {
"root": str(root),
"families": [family],
"model": args.model,
"seconds": round(time.monotonic() - started, 1),
"questions": len(groups),
"completed": done,
"completed_regions": sorted({f"{h.path}:{h.function_name or ''}" for g in groups[:done] for h in g}),
"hunt_errors": errors,
"cost_usd": round(cost, 4),
"findings": sorted(findings, key=lambda f: f["site"]),
}
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--population", type=Path, default=evaluate.POPULATION)
parser.add_argument("--only", default="")
parser.add_argument("--pins", default="")
parser.add_argument("--pilot-root", type=Path)
parser.add_argument("--family", default="injection")
parser.add_argument("--model", default=DEFAULT_DETECTOR_MODEL)
parser.add_argument("--max-regions", type=int, default=20)
parser.add_argument("--per-hunt", type=int, default=5)
parser.add_argument("--budget-usd", type=float, default=10.0)
parser.add_argument("--score", action="store_true", help="score the hunter's results with the population's protocol")
args = parser.parse_args()
from openultrasast.config import load_config
load_config() # loads the project's .env: DeepSeek is the chat provider, OpenRouter only embeddings
spent: list[float] = []
if args.pilot_root:
result = hunt(args.pilot_root, args.family, args, spent)
print(json.dumps({k: v for k, v in result.items() if k != "findings"} | {"findings": result["findings"]}, indent=1))
return 0
evaluate.use(args.population)
evaluate.RESULTS = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-hunter")
if args.score:
return evaluate.score_v2()
only = {c.strip() for c in args.only.split(",") if c.strip()}
wanted = {p.strip() for p in args.pins.split(",") if p.strip()}
for case in evaluate.cases():
if only and case["id"] not in only:
continue
for label, commit in evaluate.pins(case).items():
if wanted and label not in wanted:
continue
out = evaluate.RESULTS / "scans" / f"{case['id']}--{label}.json"
if out.is_file():
continue
if sum(spent) >= args.budget_usd:
print(json.dumps({"stopped": "budget", "spent_usd": round(sum(spent), 4)}), flush=True)
return 1
out.parent.mkdir(parents=True, exist_ok=True)
checkout = evaluate.export(case, label, commit)
try:
result = hunt(checkout, case["family"], args, spent)
finally:
shutil.rmtree(checkout, ignore_errors=True)
if not result["completed"]:
result["instrument_error"] = "no hunt completed: " + "; ".join(result["hunt_errors"][:3])
result["root"] = "/case"
out.write_text(json.dumps(result, indent=2) + "\n")
print(
json.dumps({"case": case["id"], "pin": label, "findings": len(result["findings"]), "hunts": f"{result['completed']}/{result['questions']}", "usd": result["cost_usd"], "total_usd": round(sum(spent), 4)}),
flush=True,
)
return 0
if __name__ == "__main__":
raise SystemExit(main())
+175
View File
@@ -0,0 +1,175 @@
"""Diagnostic: does a MODEL, reading source, find the operations the sink vocabulary misses?
Sink-first selection with the shipped vocabulary put the vulnerable function in scope in 2 of 17 v2 cases: the
real sinks (wp_redirect, download_url, spawn, ky, Session.get, parse_expr, Jinja render, Mongo find, JCR
createQuery, ...) are not in any token list, and each population brings new ones. This asks the model instead,
with no vocabulary: per product source file, which functions perform the family's dangerous operation on data
that may not be constant. Candidates are scored exactly as `sink_candidates.py` scores the vocabulary.
Source only. Files: the case's languages, product code (tests, docs, examples, vendored trees, minified bundles
and files over 200 kB excluded), sent in chunks of `--chunk-lines` numbered lines. A chunk whose call fails is
counted as UNCLASSIFIED, never as "no sinks". Hard spend ceiling `--budget-usd`; DeepSeek (`deepseek-flash`).
v2 is spent for tuning; this informs the selection design and qualifies nothing.
Usage: python benchmarks/archive/model_sinks.py [--population population-v2.toml] [--only ID,...]
"""
from __future__ import annotations
import argparse
import concurrent.futures
import json
import re
import shutil
import sys
import threading
import time
from pathlib import Path
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
HERE = Path(__file__).resolve().parents[1] / "independent"
sys.path.insert(0, str(HERE))
import evaluate # noqa: E402
from openultrasast.config import load_config # noqa: E402
from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL # noqa: E402
from openultrasast.preprocess import detect_language, enumerate_source_files # noqa: E402
NOT_PRODUCT = re.compile(r"(^|/)(tests?|__tests__|spec|specs|docs?|examples?|fixtures?|e2e|cypress|storybook)/|\.(test|spec|stories)\.[jt]sx?$|_test\.py$|(^|/)test_[^/]*\.py$|\.min\.js$|(^|/)dist/|(^|/)build/")
OPERATIONS = {
"injection": (
"executes a database query (SQL, NoSQL, JCR/graph query), an operating-system command or shell, code or "
"expression evaluation (eval, exec, dynamic parsing into executable form), server-side template rendering "
"from a template string, or deserialization"
),
"untrusted_destination": "makes an outbound HTTP/network request, downloads a URL, or redirects the user to a URL",
"config_secrets": (
"sets security-relevant configuration: CORS policy, debug mode, TLS/certificate verification, cookie or "
"session security, or embeds a secret, password or API key"
),
}
PROMPT = (
"You are auditing source code. List every function in the code below that {operation}, where the query, "
"command, expression, template, URL or setting is not a compile-time constant. Answer with JSON only: "
'{{"functions": [{{"function": "<name, or <global> for top-level code>", "line": <line number>, '
'"operation": "<the call>"}}]}}. Return {{"functions": []}} if there is none.\n\nFile: {path}\n\n{code}'
)
def product_files(root: Path, languages: set[str]) -> list[Path]:
files = []
for path in enumerate_source_files(root):
relative = path.relative_to(root).as_posix()
if detect_language(path) in languages and not NOT_PRODUCT.search(relative) and path.stat().st_size <= 200_000:
files.append(path)
return files
def chunks(text: str, size: int) -> list[str]:
lines = text.splitlines()
return ["\n".join(f"{n + 1:5d}| {line}" for n, line in enumerate(lines[i : i + size], start=i)) for i in range(0, max(len(lines), 1), size)]
class Meter:
def __init__(self, ceiling: float) -> None:
self.ceiling, self.clients, self.lock = ceiling, [], threading.Lock()
def add(self, client: object) -> None:
with self.lock:
self.clients.append(client)
def spent(self) -> float:
with self.lock:
return sum(float(c.cost_usd()) for c in self.clients)
def classify(root: Path, path: Path, family: str, args: argparse.Namespace, meter: Meter, local: threading.local) -> tuple[list[tuple[str, str, int]], int, int]:
from openultrasast import tool_hunter
if not hasattr(local, "client"):
local.client = tool_hunter.resolve_hunter_client()
meter.add(local.client)
relative = path.relative_to(root).as_posix()
found, failed, asked = [], 0, 0
for code in chunks(path.read_text(errors="ignore"), args.chunk_lines):
if meter.spent() >= args.budget_usd:
return found, failed + 1, asked
asked += 1
prompt = PROMPT.format(operation=OPERATIONS[family], path=relative, code=code)
try:
response = local.client.complete(model=args.model, messages=[{"role": "user", "content": prompt}], tools=[], timeout_seconds=90, json_object=True)
items = json.loads(response.content or "{}").get("functions", [])
except Exception: # noqa: BLE001 -- counted as unclassified, never as "no sinks"
failed += 1
continue
for item in items if isinstance(items, list) else []:
if isinstance(item, dict) and isinstance(item.get("function"), str):
line = item.get("line") if isinstance(item.get("line"), int) else 0
found.append((relative, item["function"].strip() or "<global>", line))
return found, failed, asked
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
parser.add_argument("--only", default="")
parser.add_argument("--model", default=DEFAULT_DETECTOR_MODEL)
parser.add_argument("--chunk-lines", type=int, default=400)
parser.add_argument("--workers", type=int, default=8)
parser.add_argument("--budget-usd", type=float, default=8.0)
args = parser.parse_args()
load_config() # the project's .env: DeepSeek is the chat provider
evaluate.use(args.population)
evaluate.RESULTS = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-modelsinks")
evaluate.RESULTS.mkdir(parents=True, exist_ok=True)
only = {c.strip() for c in args.only.split(",") if c.strip()}
meter, local, rows = Meter(args.budget_usd), threading.local(), []
for case in evaluate.cases():
if only and case["id"] not in only:
continue
if meter.spent() >= args.budget_usd:
print(json.dumps({"stopped": "budget", "spent_usd": round(meter.spent(), 4)}), flush=True)
break
started, before = time.monotonic(), meter.spent()
checkout = evaluate.export(case, "vulnerable", case["vulnerable"])
try:
files = product_files(checkout, {case["language"], *case.get("also", [])})
found, failed, asked = [], 0, 0
with concurrent.futures.ThreadPoolExecutor(max_workers=args.workers) as pool:
for f, fail, ask in pool.map(lambda p: classify(checkout, p, case["family"], args, meter, local), files):
found += f
failed += fail
asked += ask
finally:
shutil.rmtree(checkout, ignore_errors=True)
functions = {(p, fn) for p, fn, _ in found}
changed = set(evaluate.hunks(case, "old"))
sites = {tuple(site.split("::", 1)) for site in case.get("sites", [])}
row = {
"id": case["id"], "family": case["family"], "files": len(files), "chunks": asked, "unclassified_chunks": failed,
"candidate_functions": len(functions), "candidate_files": len({p for p, _ in functions}),
"declared_site_in_candidates": sorted(f"{p}::{fn}" for p, fn in functions if (p, fn) in sites),
"fix_or_site_file_in_candidates": sorted({p for p, _ in functions if p in changed or p in {s[0] for s in sites}})[:5],
"usd": round(meter.spent() - before, 4), "seconds": round(time.monotonic() - started, 1),
} # fmt: skip
(evaluate.RESULTS / f"{case['id']}.json").write_text(json.dumps({**row, "candidates": sorted(found)}, indent=1) + "\n")
rows.append(row)
print(json.dumps(row), flush=True)
print(
json.dumps(
{
"cases": len(rows),
"declared_site_in_candidates": sum(bool(r["declared_site_in_candidates"]) for r in rows),
"fix_or_site_file_in_candidates": sum(bool(r["fix_or_site_file_in_candidates"]) for r in rows),
"spent_usd": round(meter.spent(), 4),
}
),
flush=True,
)
return 0
if __name__ == "__main__":
raise SystemExit(main())
+24
View File
@@ -0,0 +1,24 @@
# Old path to new path (2026-10-02)
Committed records, specs and some docstrings name these scripts by their old path. Those files are
not rewritten. Read the old path as the new one:
| old path | new path |
|---|---|
| `benchmarks/independent/verify_batched.py` | `benchmarks/archive/verify_batched.py` |
| `benchmarks/independent/score_batched.py` | `benchmarks/archive/score_batched.py` |
| `benchmarks/independent/verify_sinks.py` | `benchmarks/archive/verify_sinks.py` |
| `benchmarks/independent/model_sinks.py` | `benchmarks/archive/model_sinks.py` |
| `benchmarks/independent/sink_candidates.py` | `benchmarks/archive/sink_candidates.py` |
| `benchmarks/independent/hunt.py` | `benchmarks/archive/hunt.py` |
| `benchmarks/harnessx_removal_equality.py` | `benchmarks/archive/harnessx_removal_equality.py` |
Files that still use the old path:
- Records: `independent/hunter-protocol-v2.md`, `independent/results-v2-model-pipeline.json`,
`independent/selection-v2-model-sinks.json`, `independent/verifier-batched-check-2026-09-29.json`,
`measurements/2026-09-30-harnessx-removal-equality/README`
- Specs: `.kiro/specs/ai-service-plane/`, `.kiro/specs/harnessx-removal/design.md`,
`.kiro/specs/learned-decision-engine/`
- Docstrings: `src/openultrasast/plane/generate.py`, `plane/tasks/{verify,agree,roles,repo_facts}.py`.
These are left for the owners of those packages.
+95
View File
@@ -0,0 +1,95 @@
"""Reference-only since 2026-09-30: the ai-service-plane tasks `plane/tasks/verify.py` and `agree.py`, run on google/ax
through `ousast plane run plane/runs/validation-46.yaml`, replace this script (benchmarks/independent/
plane-increment-2.json). It stays as the reference behaviour the plane was measured against.
Exploratory score of a batched-verifier run against the population's cases (protocol v2 matching).
The batched verifier writes one vulnerable-pin result per case (and a fixed-pin recheck with --fixed), not the
six pins protocol v2 scores, so this applies the same matching rule (`evaluate.matches_v2`) to what exists:
detected when an AGREED finding matches; unstable when only a DISPUTED one does; fixed-side alert when an agreed
finding at the fixed pin matches the fix's new side. The precision sample is every third agreed vulnerable-pin
finding (all if 15 or fewer), to be adjudicated by hand. v2 is spent for tuning: this qualifies nothing.
Usage: python benchmarks/archive/score_batched.py [--results-suffix batched] [--population population-v2.toml]
"""
from __future__ import annotations
import argparse
import json
import sys
from pathlib import Path
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
HERE = Path(__file__).resolve().parents[1] / "independent"
sys.path.insert(0, str(HERE))
import evaluate # noqa: E402
def anchored(result: dict) -> dict:
"""Sites anchored to the candidate asked about (older records used the model's reported location)."""
for key in ("findings", "disputed"):
for f in result.get(key, []):
path, _, function = f["candidate"].partition("::")
site_path, _, rest = f["site"].partition(":")
line = rest.split(":")[0] if site_path == path else "0"
f["site"] = f"{path}:{line}:{function}"
return result
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
parser.add_argument("--results-suffix", default="batched")
args = parser.parse_args()
evaluate.use(args.population)
results = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-" + args.results_suffix)
scans = results / "scans"
per_case, pool = [], []
for case in evaluate.cases():
path = scans / f"{case['id']}--vulnerable_a.json"
if not path.is_file():
per_case.append({"id": case["id"], "status": "incomplete"})
continue
vulnerable = anchored(json.loads(path.read_text()))
old, new = evaluate.hunks(case, "old"), evaluate.hunks(case, "new")
agreed = [f["site"] for f in vulnerable["findings"] if evaluate.matches_v2(f, case, old)]
disputed = [f["site"] for f in vulnerable.get("disputed", []) if evaluate.matches_v2(f, case, old)]
fixed_path = scans / f"{case['id']}--fixed_a.json"
fixed = anchored(json.loads(fixed_path.read_text())) if fixed_path.is_file() else None
fixed_alerts = [f["site"] for f in fixed["findings"] if evaluate.matches_v2(f, case, new)] if fixed else None
per_case.append(
{
"id": case["id"], "family": case["family"], "detected": bool(agreed), "detected_sites": agreed,
"unstable": bool(disputed) and not agreed, "unstable_sites": disputed,
"agreed_findings": len(vulnerable["findings"]), "disputed_findings": len(vulnerable.get("disputed", [])),
"triaged_out": vulnerable.get("triaged_out"), "candidates": vulnerable.get("questions"),
"fixed_side_alerts": fixed_alerts, "fixed_agreed_total": len(fixed["findings"]) if fixed else None,
"removed_by_fix": fixed.get("removed_by_fix") if fixed else None, "usd": vulnerable.get("usd"), "usage": vulnerable.get("usage"),
}
) # fmt: skip
pool += [{"case": case["id"], **f} for f in vulnerable["findings"]]
pool.sort(key=lambda f: (f["case"], f["site"]))
sample = pool if len(pool) <= 15 else pool[0::3]
scored = [c for c in per_case if c.get("status") != "incomplete"]
detected = sum(c["detected"] for c in scored)
with_fixed = [c for c in scored if c["fixed_side_alerts"] is not None]
result = {
"protocol": "protocol-v2.md matching, exploratory (batched verifier, vulnerable + fixed pins only)",
"cases_scored": len(scored), "cases_incomplete": [c["id"] for c in per_case if c.get("status") == "incomplete"],
"recall": [detected, len(scored)], "recall_wilson95": evaluate.wilson(detected, len(scored)),
"unstable_detections": [c["id"] for c in scored if c["unstable"]],
"fixed_side_alerts": sum(len(c["fixed_side_alerts"]) for c in with_fixed) if with_fixed else None,
"fixed_pins_rechecked": len(with_fixed),
"pooled_agreed_findings": len(pool), "precision_sample_size": len(sample),
"usd": round(sum(float(c.get("usd") or 0) for c in scored), 2),
"cases": per_case, "precision_sample": sample,
} # fmt: skip
(results / "score.json").write_text(json.dumps(result, indent=2) + "\n")
print(json.dumps({k: v for k, v in result.items() if k not in ("cases", "precision_sample")}, indent=1))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+122
View File
@@ -0,0 +1,122 @@
"""Diagnostic: would SINK-FIRST selection, from source alone, put the vulnerable function in scope?
The v2 hunter experiment showed scope selection is the bottleneck: the ranker's top 20 entry-point regions held
the vulnerable file in 1 of 17 cases. This asks the opposite direction without any engine or model: at each
case's vulnerable pin, every call matching the family's shipped sink vocabulary (`taint_specs`) in product
source is a candidate, attributed to its enclosing function by a per-language declaration scan. It reports how
many candidates a repository yields (the budget a hunter would need) and whether a declared site function, or
at least a file the fix changes, is among them.
Source only: the checkout is the pin's tree; git is used to export it and, for the score, to read the fix's
changed files -- never to choose scope.
v2 is spent for tuning; this informs the selection design and qualifies nothing.
Usage: python benchmarks/archive/sink_candidates.py [--population population-v2.toml] [--only ID,...]
"""
from __future__ import annotations
import argparse
import json
import re
import shutil
import sys
from pathlib import Path
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
HERE = Path(__file__).resolve().parents[1] / "independent"
sys.path.insert(0, str(HERE))
import evaluate # noqa: E402
from openultrasast.model.specs import taint_specs # noqa: E402
from openultrasast.preprocess import detect_language, enumerate_source_files # noqa: E402
DECLARATION = {
"python": re.compile(r"^\s*(?:async\s+)?def\s+(\w+)"),
"php": re.compile(r"^\s*(?:(?:public|private|protected|static|final|abstract)\s+)*function\s+&?(\w+)"),
"javascript": re.compile(r"^\s*(?:export\s+)?(?:async\s+)?function\s*\*?\s*(\w+)|^\s*(?:(?:public|private|protected|static|async)\s+)*(\w+)\s*\([^;]*\)\s*(?::[^{=]+)?\{\s*$|^\s*(?:export\s+)?(?:const|let|var)\s+(\w+)\s*=\s*(?:async\s*)?(?:function|\()"),
}
DECLARATION["typescript"] = DECLARATION["javascript"]
KEYWORDS = {"if", "for", "while", "switch", "catch", "return", "function", "else"}
def token(name: str) -> re.Pattern[str]:
before = r"(?<![A-Za-z0-9_$])" if name[:1].isalnum() or name[:1] in "_$" else ""
return re.compile(before + re.escape(name) + r"\s*\(")
def enclosing(lines: list[str], index: int, language: str) -> str:
pattern = DECLARATION.get(language)
if pattern is None:
return ""
for back in range(index, -1, -1):
match = pattern.match(lines[back])
if match:
name = next((g for g in match.groups() if g), "")
if name and name not in KEYWORDS:
return name
return "<global>"
def candidates(root: Path, family: str) -> dict[tuple[str, str], list[int]]:
found: dict[tuple[str, str], list[int]] = {}
patterns: dict[str, list[re.Pattern[str]]] = {}
for path in enumerate_source_files(root):
language = detect_language(path)
if language not in patterns:
spec = taint_specs(language=language).get(family) if language != "unknown" else None
patterns[language] = [token(n) for n in spec.sinks] if spec else []
if not patterns[language]:
continue
try:
lines = path.read_text(errors="ignore").splitlines()
except OSError:
continue
relative = path.relative_to(root).as_posix()
for index, line in enumerate(lines):
stripped = line.strip()
if stripped.startswith(("//", "#", "*", "/*")):
continue
if any(p.search(line) for p in patterns[language]):
found.setdefault((relative, enclosing(lines, index, language)), []).append(index + 1)
return found
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
parser.add_argument("--only", default="")
args = parser.parse_args()
evaluate.use(args.population)
evaluate.RESULTS = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-sinkfirst")
only = {c.strip() for c in args.only.split(",") if c.strip()}
rows = []
for case in evaluate.cases():
if only and case["id"] not in only:
continue
checkout = evaluate.export(case, "vulnerable", case["vulnerable"])
try:
found = candidates(checkout, case["family"])
finally:
shutil.rmtree(checkout, ignore_errors=True)
changed = set(evaluate.hunks(case, "old"))
sites = {tuple(site.split("::", 1)) for site in case.get("sites", [])}
site_hit = sorted(f"{p}::{f}" for p, f in found if (p, f) in sites)
file_hit = sorted({p for p, _ in found if p in changed or p in {s[0] for s in sites}})
row = {
"id": case["id"], "family": case["family"], "candidate_functions": len(found),
"candidate_files": len({p for p, _ in found}), "declared_site_in_candidates": site_hit,
"fix_or_site_file_in_candidates": file_hit[:5],
} # fmt: skip
rows.append(row)
print(json.dumps(row), flush=True)
sites_in = sum(bool(r["declared_site_in_candidates"]) for r in rows)
files_in = sum(bool(r["fix_or_site_file_in_candidates"]) for r in rows)
print(json.dumps({"cases": len(rows), "declared_site_in_candidates": sites_in, "fix_or_site_file_in_candidates": files_in}))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+291
View File
@@ -0,0 +1,291 @@
"""Reference-only since 2026-09-30: the ai-service-plane tasks `plane/tasks/verify.py` and `agree.py`, run on google/ax
through `ousast plane run plane/runs/validation-46.yaml`, replace this script (benchmarks/independent/
plane-increment-2.json). It stays as the reference behaviour the plane was measured against.
Stage 2, second design: triage per file, verify per file, report only what two passes agree on.
The first verifier (`verify_sinks.py`) asked one 6-step tool hunt per candidate function. Measured on pgAdmin
(verifier-check-2026-09-29.json): $0.014-0.020 per candidate, and on 180 candidates verified twice the two runs
flagged 15 and 13 with only 2 in common -- single-pass verdicts on borderline candidates flip. This design
changes three things, each checked against the 15 declared sites and those 26 flickering candidates:
1. TRIAGE, one no-tools call per file: the file's relevant chunks and its candidate functions, asking where the
operation's data ORIGINATES. `request`, `parameter` and `stored` go on to verification; `internal` and
`constant` do not; anything unclassified goes on (a failed call is never a "safe" verdict).
2. BATCHED verification, one 6-step tool hunt per file for up to `--per-hunt` surviving candidates, with one
verdict per listed operation. The file read and the caller searches are shared instead of repeated ~5 times.
3. AGREEMENT: `--passes` independent hunts per file (fresh conversations); a candidate is reported only when
every pass flags it. Findings from single passes are kept in the record as `disputed`.
Source only; the stage-1 candidates come from `model_sinks.py`. Results in `~/ousast-results/independent-<vN>-
<suffix>/scans/` in the shape `evaluate.py` scores; a per-case `.jsonl` makes a stopped run resumable per file.
EXPLORATORY: v2 is spent for tuning; this qualifies nothing.
Usage:
python benchmarks/archive/verify_batched.py [--only ID,...] [--subset sets.json] [--passes 2] [--budget-usd 40]
"""
from __future__ import annotations
import argparse
import concurrent.futures
import json
import shutil
import sys
import time
from pathlib import Path
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
HERE = Path(__file__).resolve().parents[1] / "independent"
sys.path.insert(0, str(HERE))
sys.path.insert(0, str(Path(__file__).resolve().parent)) # archived siblings: model_sinks, verify_sinks
import evaluate # noqa: E402
from model_sinks import OPERATIONS, chunks # noqa: E402
from verify_sinks import ACCOUNT_ERRORS, _Recording # noqa: E402
from openultrasast.config import load_config # noqa: E402
from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL # noqa: E402
VERIFY = {"request", "parameter", "stored"}
TRIAGE = (
"Below is a source file with numbered lines, and a list of functions in it that each {operation}. For EACH "
"listed function, say where the data used by that operation ORIGINATES, tracing within this file only:\n"
"- request: an HTTP request (parameters, body, headers, cookies, uploads), a webhook or API payload;\n"
"- parameter: a parameter of the function whose callers are not in this file;\n"
"- stored: a database row, file or setting written earlier, possibly from external input;\n"
"- internal: values the program computes from constants, the operator's environment, CLI arguments or "
"trusted framework state;\n"
"- constant: a literal.\n"
'Answer with JSON only: {{"functions": [{{"function": "<name exactly as listed>", "data_origin": "<one of the five>"}}]}}.'
"\n\nFunctions: {functions}\n\nFile: {path}\n\n{code}"
)
SYSTEM = (
"You are a security reviewer with repository tools (read_file, grep_repo, find_refs). You are given several "
"operations in ONE file, each named by its function. For EACH operation decide whether data an attacker "
"controls -- HTTP request parameters, body, headers, cookies, uploaded or imported content, webhook or API "
"payloads, values stored earlier from such input -- can reach it without a sufficient guard (validation or "
"allowlisting, escaping that fits the exact context, parameterization, or a fixed destination). Use the tools "
"to follow callers and data. Report an operation only if you can state the concrete path from the source to "
"it. Answer with a JSON array: one object per vulnerable operation with path, line, function_name (exactly as "
"listed), title and rationale (the source, each hop, and why no guard applies); [] if none is vulnerable or "
"you cannot establish a path."
)
_CLIENT = None
def _client():
global _CLIENT
if _CLIENT is None:
from openultrasast import tool_hunter
load_config()
_CLIENT = tool_hunter.resolve_hunter_client()
return _CLIENT
def _relevant_code(text: str, lines: list[int], chunk_lines: int, max_chunks: int = 6) -> str:
parts = chunks(text, chunk_lines)
if not lines or any(line <= 0 for line in lines) or len(parts) <= max_chunks:
return "\n".join(parts[:max_chunks])
wanted = sorted({min((line - 1) // chunk_lines, len(parts) - 1) for line in lines})
return "\n".join(parts[i] for i in wanted[:max_chunks])
def triage(root: Path, path: str, candidates: list[list], family: str, model: str, chunk_lines: int) -> dict[str, str]:
"""Function name -> data_origin for this file's candidates; a name missing from the answer is unclassified."""
client = _client()
try:
text = (root / path).read_text(errors="ignore")
except OSError:
return {}
names = sorted({fn for _, fn, _ in candidates})
prompt = TRIAGE.format(operation=OPERATIONS[family], functions=json.dumps(names), path=path, code=_relevant_code(text, [ln for _, _, ln in candidates], chunk_lines))
response = client.complete(model=model, messages=[{"role": "user", "content": prompt}], tools=[], timeout_seconds=90, json_object=True)
try:
items = json.loads(response.content or "{}").get("functions", [])
except (json.JSONDecodeError, AttributeError):
return {}
return {str(i.get("function")): str(i.get("data_origin", "")).strip().lower() for i in items if isinstance(i, dict) and i.get("function") in names}
def hunt(root: Path, path: str, group: list[list], family: str, model: str, steps: int) -> tuple[list[dict], dict]:
from openultrasast import tool_hunter
from openultrasast.complexity.map import Hotspot
recording = _Recording(_client())
listing = "\n".join(f"- function `{fn}` around line {ln}" for _, fn, ln in group)
prompt = (
f"Operations to judge, all in `{path}`, each of which {OPERATIONS[family]}:\n{listing}\n"
"The file is above. For each, trace where the operation's data comes from and whether it is guarded."
)
spots = [Hotspot(path=path, function_name=fn, score=1.0, band="candidate", signals={}, rationale="model-classified sink", test_hint=None, inventory_finding_ids=()) for _, fn, _ in group]
found = tool_hunter.run_tool_hunter(root, spots, client=recording, model=model, max_steps=steps, system_prompt=SYSTEM, user_prompt=prompt, context_files=[path], tags=("sink-verifier",))
names = {fn for _, fn, _ in group}
rows = []
for f in found:
fn = f.function_name or ""
if fn not in names: # attribute by the nearest listed line when the model renamed the function
fn = min(group, key=lambda c: abs((c[2] or 0) - (f.line or 0)))[1]
# The verdict is about the candidate operation, so the site is the candidate's file and function; the
# model's own location is kept beside it (it often names the file where the input enters instead).
line = f.line if f.path == path else next(c[2] for c in group if c[1] == fn)
rows.append({"candidate": f"{path}::{fn}", "site": f"{path}:{line or 0}:{fn}", "reported_at": f"{f.path}:{f.line or 0}", "family": family, "title": f.title, "witness": f.rationale[:600]})
return rows, recording.usage()
def verify_file(root: str, path: str, candidates: list[list], family: str, args_: dict) -> dict:
"""One file, in a worker process: triage, then `passes` batched hunts over the survivors."""
client = _client()
before = float(client.cost_usd())
record: dict = {"path": path, "candidates": candidates, "triage": {}, "passes": [], "usage": {}}
try:
if not args_["no_triage"]:
record["triage"] = triage(Path(root), path, candidates, family, args_["model"], args_["chunk_lines"])
kept = [c for c in candidates if record["triage"].get(c[1], "unclassified") in VERIFY | {"unclassified"} or record["triage"].get(c[1]) not in {"internal", "constant"}]
record["kept"] = [c[1] for c in kept]
usage: dict[str, int] = {}
for _pass in range(args_["passes"]):
flagged: list[dict] = []
for i in range(0, len(kept), args_["per_hunt"]):
rows, used = hunt(Path(root), path, kept[i : i + args_["per_hunt"]], family, args_["model"], args_["max_steps"])
flagged += rows
usage = {k: usage.get(k, 0) + used.get(k, 0) for k in set(usage) | set(used)}
record["passes"].append(flagged)
record["usage"] = usage
except Exception as error: # noqa: BLE001 -- recorded; an unverified file is not a clean one
record["unverified"] = f"{type(error).__name__}: {str(error)[:160]}"
record["usd"] = round(float(client.cost_usd()) - before, 4)
return record
def agreement(record: dict) -> tuple[list[dict], list[dict]]:
"""(agreed, disputed): flagged in every pass vs in some."""
if not record.get("passes"):
return [], []
by_pass = [{r["candidate"]: r for r in flagged} for flagged in record["passes"]]
every = set.intersection(*(set(b) for b in by_pass))
some = set.union(*(set(b) for b in by_pass))
first = by_pass[0]
pick = lambda c: next(b[c] for b in by_pass if c in b) # noqa: E731
return [dict(first[c], passes=len(by_pass)) for c in sorted(every)], [dict(pick(c), passes=sum(c in b for b in by_pass)) for c in sorted(some - every)]
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
parser.add_argument("--only", default="")
parser.add_argument("--subset", type=Path, help='JSON {case_id: [[path, function], ...]}: verify only these candidates')
parser.add_argument("--model", default=DEFAULT_DETECTOR_MODEL)
parser.add_argument("--max-steps", type=int, default=6)
parser.add_argument("--per-hunt", type=int, default=6)
parser.add_argument("--passes", type=int, default=2)
parser.add_argument("--chunk-lines", type=int, default=400)
parser.add_argument("--no-triage", action="store_true")
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--budget-usd", type=float, default=40.0)
parser.add_argument("--results-suffix", default="batched")
parser.add_argument("--fixed", action="store_true", help="recheck the AGREED candidates of each case at its fixed pin (no triage)")
args = parser.parse_args()
load_config()
evaluate.use(args.population)
candidates_dir = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-modelsinks")
evaluate.RESULTS = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-" + args.results_suffix)
out_dir = evaluate.RESULTS / "scans"
out_dir.mkdir(parents=True, exist_ok=True)
only = {c.strip() for c in args.only.split(",") if c.strip()}
subset = json.loads(args.subset.read_text()) if args.subset else None
settings = {k: getattr(args, k) for k in ("model", "max_steps", "per_hunt", "passes", "chunk_lines", "no_triage")}
spent = 0.0
for case in evaluate.cases():
if only and case["id"] not in only or (subset is not None and case["id"] not in subset):
continue
label = "fixed_a" if args.fixed else "vulnerable_a"
out = out_dir / f"{case['id']}--{label}.json"
if out.is_file():
continue
log = out_dir / f"{case['id']}--{label}.jsonl"
done = {r["path"]: r for r in (json.loads(row) for row in log.read_text().splitlines()) if "unverified" not in r} if log.is_file() else {}
spent += sum(float(r.get("usd", 0.0)) for r in done.values())
record = json.loads((candidates_dir / f"{case['id']}.json").read_text())
unique = list({(p, fn): [p, fn, ln] for p, fn, ln in sorted(record["candidates"])}.values())
if subset is not None:
wanted = {tuple(x) for x in subset[case["id"]]}
unique = [c for c in unique if (c[0], c[1]) in wanted]
if args.fixed:
# The agreed findings at the vulnerable pin, by (path, function), asked again at the fixed pin. Line
# numbers are the vulnerable pin's; the file itself is in the prompt. A function the fix removed
# cannot be asked and counts as removed.
vulnerable_out = out_dir / f"{case['id']}--vulnerable_a.json"
if not vulnerable_out.is_file():
continue
agreed = {tuple(f["candidate"].split("::", 1)) for f in json.loads(vulnerable_out.read_text())["findings"]}
unique = [c for c in unique if (c[0], c[1]) in agreed]
settings = dict(settings, no_triage=True)
by_file: dict[str, list[list]] = {}
for c in unique:
by_file.setdefault(c[0], []).append(c)
pending = {p: cs for p, cs in by_file.items() if p not in done}
started = time.monotonic()
checkout = evaluate.export(case, label, case["fixed" if args.fixed else "vulnerable"])
if args.fixed:
present = [c for c in unique if (checkout / c[0]).is_file()]
removed = [f"{c[0]}::{c[1]}" for c in unique if (checkout / c[0]).is_file() is False]
unique, by_file = present, {}
for c in unique:
by_file.setdefault(c[0], []).append(c)
pending = {p: cs for p, cs in by_file.items() if p not in done}
account_error = ""
try:
with concurrent.futures.ProcessPoolExecutor(max_workers=args.workers) as pool, log.open("a") as sink:
futures = {pool.submit(verify_file, str(checkout), p, cs, case["family"], settings): p for p, cs in pending.items()}
for future in concurrent.futures.as_completed(futures):
if future.cancelled():
continue
rec = future.result()
if any(code in str(rec.get("unverified", "")) for code in ACCOUNT_ERRORS):
account_error = str(rec["unverified"])[:200]
for other in futures:
other.cancel()
continue
spent += float(rec.get("usd", 0.0))
sink.write(json.dumps(rec) + "\n")
sink.flush()
if "unverified" not in rec:
done[rec["path"]] = rec
if spent >= args.budget_usd:
for other in futures:
other.cancel()
finally:
shutil.rmtree(checkout, ignore_errors=True)
agreed, disputed = [], []
for rec in done.values():
a, d = agreement(rec)
agreed += a
disputed += d
triaged_out = sum(len(r["candidates"]) - len(r.get("kept", [])) for r in done.values())
usage = {k: sum(int(r.get("usage", {}).get(k, 0)) for r in done.values()) for k in ("prompt_tokens", "prompt_cache_hit_tokens", "completion_tokens", "calls")}
complete = len(done) >= len(by_file) and not account_error
result = {
"root": "/case", "families": [case["family"]], "questions": len(unique), "completed": sum(len(r["candidates"]) for r in done.values()),
"completed_regions": sorted(f"{c[0]}:{c[1]}" for r in done.values() for c in r["candidates"]),
"triaged_out": triaged_out, "passes": args.passes, "usage": usage, "usd": round(sum(float(r.get("usd", 0.0)) for r in done.values()), 4),
"seconds": round(time.monotonic() - started, 1), "findings": sorted(agreed, key=lambda f: f["site"]), "disputed": sorted(disputed, key=lambda f: f["site"]),
} # fmt: skip
if args.fixed:
result["removed_by_fix"] = removed
if complete:
out.write_text(json.dumps(result, indent=1) + "\n")
print(json.dumps({"case": case["id"], "files": f"{len(done)}/{len(by_file)}", "candidates": len(unique), "triaged_out": triaged_out, "agreed": len(agreed), "disputed": len(disputed), "usd": result["usd"], "total_usd": round(spent, 4), "complete": complete}), flush=True)
if account_error:
print(json.dumps({"stopped": "account", "error": account_error}), flush=True)
return 2
if spent >= args.budget_usd:
print(json.dumps({"stopped": "budget", "spent_usd": round(spent, 4)}), flush=True)
break
return 0
if __name__ == "__main__":
raise SystemExit(main())
+235
View File
@@ -0,0 +1,235 @@
"""Stage 2 of source-only, model-driven detection: verify each model-classified sink candidate.
`model_sinks.py` put the vulnerable function among the candidates in 15 of 17 v2 cases, among ~4,800 candidates.
This asks, per candidate, one bounded question with repository tools (the existing tool hunter with its
per-family overrides): can attacker-controlled data reach THIS operation without a sufficient guard? A candidate
is reported only when the model states a concrete source-to-sink path and names no sufficient guard.
Scope: the candidates `model_sinks.py` recorded for each case's vulnerable pin
(~/ousast-results/independent-v2-modelsinks/<case>.json). One run per pin; the fixed pin is rechecked only for
detected sites (`--fixed`). Findings are written in the scan-result shape `evaluate.py` reads. Hard spend
ceiling. EXPLORATORY: v2 is spent for tuning; this qualifies nothing.
Usage: python benchmarks/archive/verify_sinks.py [--only ID,...] [--budget-usd 40] [--fixed]
"""
from __future__ import annotations
import argparse
import concurrent.futures
import json
import shutil
import sys
import time
from pathlib import Path
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
HERE = Path(__file__).resolve().parents[1] / "independent"
sys.path.insert(0, str(HERE))
sys.path.insert(0, str(Path(__file__).resolve().parent)) # archived siblings: model_sinks, verify_sinks
import evaluate # noqa: E402
from model_sinks import OPERATIONS # noqa: E402
from openultrasast.config import load_config # noqa: E402
from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL # noqa: E402
# The provider refusing the ACCOUNT (no credit, bad key): every later call fails the same way.
ACCOUNT_ERRORS = ("HTTP Error 402", "HTTP Error 401", "Insufficient Balance")
SYSTEM = (
"You are a security reviewer with repository tools (read_file, grep_repo, find_refs). You are given ONE "
"operation in ONE function. Decide whether data an attacker controls -- HTTP request parameters, body, "
"headers, cookies, uploaded or imported content, webhook or API payloads, values stored earlier from such "
"input -- can reach that operation without a sufficient guard (validation or allowlisting, escaping that fits "
"the exact context, parameterization, or a fixed destination). Use the tools to follow callers and data. "
"Report it only if you can state the concrete path from the source to the operation. Answer with a JSON array: "
"one object with path, line, function_name, title, rationale (the source, each hop, and why no guard applies) "
"if it is vulnerable; [] if it is not, or if you cannot establish the path."
)
_CLIENT = None
def _client():
global _CLIENT
if _CLIENT is None:
from openultrasast import tool_hunter
load_config()
_CLIENT = tool_hunter.resolve_hunter_client()
return _CLIENT
class _Recording:
"""Wraps the chat client for one candidate: counts investigation turns and slices token usage."""
def __init__(self, client) -> None:
self.client, self.turns_with_tools, self.first_usage = client, 0, len(getattr(client, "usage", []))
def complete(self, **kw):
response = self.client.complete(**kw)
if response.tool_calls:
self.turns_with_tools += 1
return response
def usage(self) -> dict[str, int]:
rows = getattr(self.client, "usage", [])[self.first_usage :]
keys = ("prompt_tokens", "prompt_cache_hit_tokens", "completion_tokens")
return {k: sum(int(r.get(k) or 0) for r in rows) for k in keys} | {"calls": len(rows)}
def _hunt(root: str, spot, prompt: str, path: str, model: str, steps: int):
from openultrasast import tool_hunter
recording = _Recording(_client())
found = tool_hunter.run_tool_hunter(
Path(root), [spot], client=recording, model=model, max_steps=steps,
system_prompt=SYSTEM, user_prompt=prompt, context_files=[path], tags=("sink-verifier",),
) # fmt: skip
return found, recording
def verify(root: str, candidate: list, family: str, model: str, max_steps: int, escalate_steps: int = 0) -> dict:
"""One candidate, in a worker process: CPU-bound repository tools do not serialise behind one interpreter.
Adaptive budget: the hunt runs at `max_steps`; if it spent EVERY step on tool calls and reported nothing, the
model was still investigating when the budget ended, and the hunt is repeated once at `escalate_steps`. A hunt
that concluded early (fewer tool turns than steps) is final at the small budget. Measured on the seven declared
v2 sites: 3 steps found 5, 8 steps found 6; most candidates conclude in 3.
"""
from openultrasast.complexity.map import Hotspot
client = _client()
before = float(client.cost_usd())
path, function, line = candidate
spot = Hotspot(path=path, function_name=function, score=1.0, band="candidate", signals={}, rationale="model-classified sink", test_hint=None, inventory_finding_ids=())
prompt = (
f"Operation to judge: in `{path}`, function `{function}`, around line {line}, code that {OPERATIONS[family]}. "
"The file is above. Trace where the operation's data comes from and whether it is guarded."
)
usage: dict[str, int] = {}
escalated = False
try:
found, recording = _hunt(root, spot, prompt, path, model, max_steps)
usage = recording.usage()
if not found and escalate_steps > max_steps and recording.turns_with_tools >= max_steps:
escalated = True
found, recording = _hunt(root, spot, prompt, path, model, escalate_steps)
second = recording.usage()
usage = {k: usage.get(k, 0) + second.get(k, 0) for k in set(usage) | set(second)}
except Exception as error: # noqa: BLE001 -- recorded; an unverified candidate is not a clean one
return {"candidate": candidate, "unverified": f"{type(error).__name__}: {str(error)[:120]}", "usd": float(client.cost_usd()) - before}
findings = [
{"site": f"{f.path}:{f.line or line}:{f.function_name or function}", "family": family, "title": f.title, "witness": f.rationale[:600], "candidate": f"{path}::{function}"}
for f in found
]
return {"candidate": candidate, "findings": findings, "usd": float(client.cost_usd()) - before, "escalated": escalated, "usage": usage}
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
parser.add_argument("--only", default="")
parser.add_argument("--model", default=DEFAULT_DETECTOR_MODEL)
parser.add_argument("--max-steps", type=int, default=6)
parser.add_argument("--escalate-steps", type=int, default=0, help="rerun a hunt that used every step without a verdict; 0 disables (measured: 88% escalate, +45% cost)")
parser.add_argument("--workers", type=int, default=4)
parser.add_argument("--budget-usd", type=float, default=40.0)
parser.add_argument("--fixed", action="store_true", help="recheck detected sites at the fixed pin")
parser.add_argument("--results-suffix", default="verified", help="results go to independent-<vN>-<suffix>/")
parser.add_argument("--declared-sites-only", action="store_true", help="verify only the case's declared sites (a known-answer check)")
parser.add_argument("--candidates-from", type=Path, help="verify exactly the candidates recorded in this per-case .jsonl")
args = parser.parse_args()
load_config()
evaluate.use(args.population)
candidates_dir = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-modelsinks")
# Checkouts too, not only results: two runs exporting the same pin into one path deleted each other's tree.
evaluate.RESULTS = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-" + args.results_suffix)
out_dir = evaluate.RESULTS / "scans"
out_dir.mkdir(parents=True, exist_ok=True)
only = {c.strip() for c in args.only.split(",") if c.strip()}
spent = 0.0
for case in evaluate.cases():
if only and case["id"] not in only:
continue
label = "fixed_a" if args.fixed else "vulnerable_a"
out = out_dir / f"{case['id']}--{label}.json"
if out.is_file():
continue
log = out_dir / f"{case['id']}--{label}.jsonl"
done = [json.loads(row) for row in log.read_text().splitlines()] if log.is_file() else []
spent += sum(float(d.get("usd", 0.0)) for d in done)
record = json.loads((candidates_dir / f"{case['id']}.json").read_text())
todo = list({(p, fn): (p, fn, ln) for p, fn, ln in sorted(record["candidates"])}.values()) # one question per function
if args.declared_sites_only:
sites = {tuple(site.split("::", 1)) for site in case.get("sites", [])}
todo = [c for c in todo if (c[0], c[1]) in sites]
if args.candidates_from:
wanted = {tuple(json.loads(row)["candidate"][:2]) for row in args.candidates_from.read_text().splitlines() if row.strip()}
todo = [c for c in todo if (c[0], c[1]) in wanted]
if args.fixed:
vulnerable = json.loads((out_dir / f"{case['id']}--vulnerable_a.json").read_text())
detected = {f["candidate"] for f in vulnerable["findings"]}
todo = [c for c in todo if f"{c[0]}::{c[1]}" in detected]
# Only a VERIFIED candidate is done. An unverified one -- a failed call -- is asked again on resume; counting
# it as done wrote sixteen cases as complete with zero calls made when the account ran out of credit.
done = [d for d in done if "findings" in d]
finished = {tuple(d["candidate"][:2]) for d in done}
pending = [c for c in todo if (c[0], c[1]) not in finished]
started = time.monotonic()
checkout = evaluate.export(case, label, case["fixed" if args.fixed else "vulnerable"])
stopped, account_error = False, ""
try:
with concurrent.futures.ProcessPoolExecutor(max_workers=args.workers) as pool, log.open("a") as sink:
futures = []
for candidate in pending:
futures.append(pool.submit(verify, str(checkout), list(candidate), case["family"], args.model, args.max_steps, args.escalate_steps))
for future in concurrent.futures.as_completed(futures):
if future.cancelled(): # cancelled at the ceiling: never run, nothing to record
continue
result = future.result()
if any(code in str(result.get("unverified", "")) for code in ACCOUNT_ERRORS):
# The account, not the candidate: nothing after this can succeed. Stop, record nothing.
account_error = str(result["unverified"])[:200]
stopped = True
for other in futures:
other.cancel()
continue
spent += float(result.get("usd", 0.0))
sink.write(json.dumps(result) + "\n")
sink.flush()
done.append(result)
if spent >= args.budget_usd and not stopped:
stopped = True
for other in futures:
other.cancel()
finally:
shutil.rmtree(checkout, ignore_errors=True)
verified = {tuple(d["candidate"][:2]) for d in done if "findings" in d}
findings = [f for d in done for f in d.get("findings", [])]
unverified = [d["unverified"] for d in done if "unverified" in d]
complete = len(verified) >= len(todo) and not account_error
result = {
"root": "/case", "families": [case["family"]], "questions": len(todo), "completed": len(verified),
"completed_regions": sorted(f"{p}:{fn}" for p, fn in verified), "unverified": unverified[:20],
"usd": round(sum(float(d.get("usd", 0.0)) for d in done), 4), "seconds": round(time.monotonic() - started, 1),
"findings": sorted(findings, key=lambda f: f["site"]),
} # fmt: skip
if complete:
out.write_text(json.dumps(result, indent=1) + "\n")
tokens = {k: sum(int(d.get("usage", {}).get(k, 0)) for d in done) for k in ("prompt_tokens", "prompt_cache_hit_tokens", "completion_tokens")}
result["usage"] = tokens | {"escalated": sum(1 for d in done if d.get("escalated"))}
print(json.dumps({"case": case["id"], "pin": label, "candidates": len(todo), "verified": len(verified), "findings": len(findings), "unverified": len(unverified), "escalated": result["usage"]["escalated"], "cache_share": round(tokens["prompt_cache_hit_tokens"] / max(tokens["prompt_tokens"], 1), 2), "usd": result["usd"], "total_usd": round(spent, 4), "complete": complete}), flush=True)
if account_error:
print(json.dumps({"stopped": "account", "error": account_error, "spent_usd": round(spent, 4)}), flush=True)
return 2
if stopped or spent >= args.budget_usd:
print(json.dumps({"stopped": "budget", "spent_usd": round(spent, 4)}), flush=True)
break
return 0
if __name__ == "__main__":
raise SystemExit(main())
+62
View File
@@ -0,0 +1,62 @@
"""Bulk reader: answer a question about one or more files with the cheap chat model, citing line numbers.
The Portal pattern (Spotify, 2026-09): reading five files to answer one question is I/O, not reasoning, so a
cheap model reads and the expensive one gets the answer. Files are sent in numbered chunks; the answer must cite
`path:line`, which is what a summary otherwise lacks. Reasoning stays with the caller: this reports what the
files say, it does not decide anything.
Usage: .venv/bin/python benchmarks/dev/bulk_read.py "<question>" FILE [FILE ...] [--max-usd 0.05]
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
CHUNK_LINES = 400
PROMPT = (
"Answer the question using only the files below (numbered lines). Be specific and brief; cite every claim as "
"path:line. If the files do not answer it, say so.\n\nQuestion: {question}\n\n{files}"
)
def numbered(path: Path) -> list[str]:
lines = path.read_text(errors="ignore").splitlines()
chunks = []
for start in range(0, max(len(lines), 1), CHUNK_LINES):
body = "\n".join(f"{n + 1:5d}| {line}" for n, line in enumerate(lines[start : start + CHUNK_LINES], start=start))
chunks.append(f"--- {path} (lines {start + 1}-{min(start + CHUNK_LINES, len(lines))})\n{body}")
return chunks
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("question")
parser.add_argument("files", nargs="+", type=Path)
parser.add_argument("--max-usd", type=float, default=0.05)
args = parser.parse_args()
from openultrasast.config import load_config
from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL
from openultrasast import tool_hunter
load_config()
client = tool_hunter.resolve_hunter_client()
if client is None:
print("no chat client: set DEEPSEEK_API_KEY", file=sys.stderr)
return 2
chunks = [chunk for path in args.files for chunk in numbered(path)]
answers = []
for chunk in chunks:
if float(client.cost_usd()) >= args.max_usd:
answers.append(f"[stopped at the ${args.max_usd} ceiling before: {chunk.splitlines()[0]}]")
break
response = client.complete(model=DEFAULT_DETECTOR_MODEL, messages=[{"role": "user", "content": PROMPT.format(question=args.question, files=chunk)}], tools=[], timeout_seconds=90)
answers.append((response.content or "").strip())
print("\n\n".join(a for a in answers if a))
print(f"\n[bulk_read: {len(chunks)} chunk(s), ${float(client.cost_usd()):.4f}]", file=sys.stderr)
return 0
if __name__ == "__main__":
raise SystemExit(main())
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env python3
"""PreToolUse hook: a whole-file Read of more than LIMIT lines is denied and redirected.
Measured 2026-09-29: 90% of a development session's tokens were tool I/O -- inline scripts and what they
printed -- not reasoning. Whole-file reads are the largest single item that a summary could replace. A Read with
`offset`/`limit` is a deliberate range and passes; a file at or under LIMIT lines passes.
Wired in .claude/settings.local.json (PreToolUse, matcher Read). Stdin: the hook JSON; stdout: a permission
decision only when denying. Never fails the tool: any error here exits 0 silently.
"""
import json
import sys
LIMIT = 350
def main() -> int:
try:
event = json.load(sys.stdin)
except (json.JSONDecodeError, OSError):
return 0
if event.get("tool_name") != "Read":
return 0
tool_input = event.get("tool_input") or {}
path = tool_input.get("file_path")
if not path or tool_input.get("limit") or tool_input.get("offset"):
return 0
try:
with open(path, "rb") as handle:
lines = sum(1 for _ in handle)
except OSError:
return 0
if lines <= LIMIT:
return 0
reason = (
f"{path} has {lines} lines (limit {LIMIT} for a whole-file read). Read a range with offset/limit, or ask "
f"the bulk reader for what you need: .venv/bin/python benchmarks/dev/bulk_read.py '<question>' {path}"
)
print(json.dumps({"hookSpecificOutput": {"hookEventName": "PreToolUse", "permissionDecision": "deny", "permissionDecisionReason": reason}}))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+59
View File
@@ -0,0 +1,59 @@
"""Where a Claude Code session's tokens go: tool input vs tool output vs text, from the transcript.
The 2026-09-29 measurement of this project's main session: tool_use input 47.5% (inline scripts), tool results
42%, assistant text 6%, thinking 2%, user 2%. Re-run after a workflow change to see whether it moved.
Usage: python benchmarks/dev/token_report.py [TRANSCRIPT.jsonl] (default: the newest under ~/.claude/projects)
"""
from __future__ import annotations
import collections
import json
import sys
from pathlib import Path
def newest() -> Path:
candidates = sorted(Path.home().glob(".claude/projects/*/*.jsonl"), key=lambda p: p.stat().st_mtime)
if not candidates:
raise SystemExit("no transcript found")
return candidates[-1]
def main() -> int:
path = Path(sys.argv[1]) if len(sys.argv) > 1 else newest()
chars: collections.Counter[str] = collections.Counter()
tools: collections.Counter[str] = collections.Counter()
for line in path.open():
try:
entry = json.loads(line)
except json.JSONDecodeError:
continue
content = (entry.get("message") or {}).get("content")
kind = entry.get("type", "?")
if isinstance(content, str):
chars[kind] += len(content)
elif isinstance(content, list):
for block in content:
block_type = block.get("type")
if block_type == "text":
chars[f"{kind}:text"] += len(block.get("text", ""))
elif block_type == "tool_use":
tools[str(block.get("name"))] += 1
chars["tool_use_input"] += len(json.dumps(block.get("input")))
elif block_type == "tool_result":
result = block.get("content")
chars["tool_result"] += len(result if isinstance(result, str) else json.dumps(result))
elif block_type == "thinking":
chars["thinking"] += len(block.get("thinking", ""))
total = sum(chars.values()) or 1
print(f"{path.name}: {total / 1e6:.1f}M chars unique (~{total / 4e6:.2f}M tokens once; re-sent every turn)")
for key, value in chars.most_common():
print(f" {key:16s} {value / 1e6:6.2f}M {value / total:5.1%}")
print("tool calls:", sum(tools.values()), dict(tools.most_common(5)))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+89
View File
@@ -0,0 +1,89 @@
<?php
// Benign twin of benchmarks/fixtures/php-vulnerable/app.php: the same handlers, each one repaired the way its
// upstream fix did it. Every quick-mode finding here is a false alert (benchmarks/manifests/php-benign.toml).
$conn = mysqli_connect('localhost', 'app', 'app', 'app');
function order_by_code($code)
{
global $wpdb;
return $wpdb->get_var($wpdb->prepare("SELECT id FROM {$wpdb->prefix}orders WHERE code = %s LIMIT 1", $code));
}
function find_user($conn)
{
$stmt = mysqli_prepare($conn, 'SELECT * FROM users WHERE name = ?');
mysqli_stmt_bind_param($stmt, 's', $_GET['name']);
mysqli_stmt_execute($stmt);
return mysqli_stmt_get_result($stmt);
}
function members_sorted($order)
{
$order = in_array(strtoupper($order), array('ASC', 'DESC'), true) ? strtoupper($order) : 'ASC';
return ' ORDER BY u.user_login ' . ('DESC' === $order ? 'DESC' : 'ASC') . ' ';
}
function create_form($data)
{
return 'INSERT INTO forms (id, label) VALUES (' . intval($data['id']) . ', "x")';
}
function captcha_cleanup($db, $userAgent)
{
$delete = sprintf(
"DELETE FROM captcha WHERE useragent = '%s'",
$db->escape($userAgent)
);
$db->query($delete);
}
function ping()
{
system('ping -c 1 ' . escapeshellarg($_GET['host']));
}
function calculate($formula)
{
return is_numeric($formula) ? (float) $formula : 0.0;
}
function restore_session()
{
return json_decode($_COOKIE['session'], true);
}
function greet()
{
echo 'Hello ' . htmlspecialchars($_GET['name'], ENT_QUOTES);
}
function page()
{
$pages = array('home', 'about');
include in_array($_GET['page'], $pages, true) ? $_GET['page'] . '.php' : 'home.php';
}
function download()
{
readfile('/var/www/files/' . basename($_GET['file']));
}
function fetch_preview()
{
$url = filter_var($_GET['url'], FILTER_VALIDATE_URL);
$ch = curl_init(wp_http_validate_url($url) ? $url : 'about:blank');
return curl_exec($ch);
}
function go_back()
{
wp_safe_redirect(wp_validate_redirect(wp_unslash($_GET['next']), home_url()));
}
function find_order($conn)
{
$id = intval($_POST['id']);
$sql = 'SELECT * FROM orders WHERE id = ' . $id;
return mysqli_query($conn, $sql);
}
@@ -0,0 +1,87 @@
<?php
// Intentionally vulnerable PHP fixture for the quick-mode PHP rules. Do not deploy.
// Each handler has one labeled weakness (benchmarks/manifests/php-vulnerable.toml); the fixed twin is
// benchmarks/fixtures/php-benign/app.php.
$conn = mysqli_connect('localhost', 'app', 'app', 'app');
function order_by_code($code)
{
global $wpdb;
return $wpdb->get_var("SELECT id FROM {$wpdb->prefix}orders WHERE code = '" . $code . "' LIMIT 1");
}
function find_user($conn)
{
return mysqli_query($conn, "SELECT * FROM users WHERE name = '" . $_GET['name'] . "'");
}
function members_sorted($order)
{
$order = esc_sql($order); // escaping does not protect an unquoted ORDER BY
return " ORDER BY u.user_login {$order} ";
}
function create_form($data)
{
return 'INSERT INTO forms (id, label) VALUES (' . $data['id'] . ', "x")';
}
function captcha_cleanup($db, $userAgent)
{
$delete = sprintf(
"DELETE FROM captcha WHERE useragent = '%s'",
$userAgent
);
$db->query($delete);
}
function ping()
{
system('ping -c 1 ' . $_GET['host']);
}
function calculate($formula)
{
eval('$value = ' . $formula . ';');
return $value;
}
function restore_session()
{
return unserialize($_COOKIE['session']);
}
function greet()
{
echo 'Hello ' . $_GET['name'];
}
function page()
{
include $_GET['page'] . '.php';
}
function download()
{
readfile('/var/www/files/' . $_GET['file']);
}
function fetch_preview()
{
$ch = curl_init($_GET['url']);
return curl_exec($ch);
}
function go_back()
{
header('Location: ' . $_GET['next']);
}
// Not visible to a line rule: the value reaches the sink through a variable on another line.
function find_order($conn)
{
$id = $_POST['id'];
$sql = "SELECT * FROM orders WHERE id = '" . $id . "'";
return mysqli_query($conn, $sql);
}
@@ -0,0 +1,65 @@
<?php
// Safe equivalents. None of these lines should produce a finding.
function order_by_code_safe($code)
{
global $wpdb;
return $wpdb->get_var($wpdb->prepare("SELECT id FROM {$wpdb->prefix}orders WHERE code = %s LIMIT 1", $code));
}
function orders_in_safe($ids)
{
global $wpdb;
$ids_format = implode(', ', array_fill(0, count($ids), '%d'));
return $wpdb->prepare('SELECT * FROM orders WHERE id IN (' . $ids_format . ')', $ids);
}
function page_limit_safe($page)
{
return ' LIMIT ' . intval($page) . ', 20';
}
function quoted_escaped_safe($name)
{
return "SELECT * FROM users WHERE name = '" . esc_sql($name) . "'";
}
function ping_safe()
{
system('ping -c 1 ' . escapeshellarg($_GET['host']));
}
function version_safe()
{
exec('git describe --tags', $output);
return $output;
}
function greet_safe()
{
echo 'Hello ' . htmlspecialchars($_GET['name'], ENT_QUOTES);
}
function download_safe()
{
readfile('/var/www/files/' . basename($_GET['file']));
}
function restore_safe($raw)
{
return unserialize($raw, ['allowed_classes' => false]);
}
function query_args_safe()
{
$args = [];
$args['include'] = [(int) $_REQUEST['p']];
return $args;
}
function comment_only()
{
// system('rm -rf ' . $_GET['dir']) would be dangerous but this is a comment
# eval($_POST['code']) is a comment too
return 'ok';
}
File diff suppressed because it is too large Load Diff
+346
View File
@@ -0,0 +1,346 @@
"""Run and score the pre-registered evaluation of the independent population (protocol-v1.md).
`run` exports each case's four pins (vulnerable, fixed, benign base, benign tip) with `git archive` and scans each
for the case's family through the production scan (`benchmarks/push/finding_dump.py`) inside the shipped image,
one container at a time. It is resumable: a scan whose result exists is not repeated. A scan that asked no
question is an instrument failure and is retried once.
`score` applies protocol-v1.md exactly -- matching, detection, fixed-side and benign false alerts, unanswered --
and draws the precision sample (every third pooled vulnerable-pin finding, or all of them if 15 or fewer).
Protocol v2 (protocol-v2.md, population-v2.toml) scans the vulnerable and fixed pins twice each and credits a
finding at a declared site as well; `--population population-v2.toml` selects it for both commands.
Usage:
python benchmarks/independent/evaluate.py run --source <frozen analyzer tree> [--only ID,...] [--population F]
python benchmarks/independent/evaluate.py score [--population F]
"""
from __future__ import annotations
import argparse
import json
import math
import re
import shutil
import subprocess
import tomllib
from pathlib import Path
HERE = Path(__file__).resolve().parent
ROOT = HERE.parents[1]
POPULATION = HERE / "population-v1.toml"
CACHE = Path.home() / ".cache" / "openultrasast" / "independent"
RESULTS = Path.home() / "ousast-results" / "independent-v1"
DEADLINE = 2400
LINE_WINDOW = 10
def cases() -> list[dict]:
return tomllib.loads(POPULATION.read_text())["case"]
def version() -> str:
return POPULATION.stem.removeprefix("population-")
def use(population: Path) -> None:
"""Point the module at a population file and its own results directory."""
global POPULATION, RESULTS
POPULATION = population if population.is_absolute() else HERE / population.name
RESULTS = Path.home() / "ousast-results" / f"independent-{version()}"
def pins(case: dict) -> dict[str, str]:
if version() != "v1":
# Protocol v2: the vulnerable and fixed pins twice each, the benign pair once.
return {
"vulnerable_a": case["vulnerable"],
"vulnerable_b": case["vulnerable"],
"fixed_a": case["fixed"],
"fixed_b": case["fixed"],
"benign_base": case["benign"]["base"],
"benign_tip": case["benign"]["tip"],
}
return {
"vulnerable": case["vulnerable"],
"fixed": case["fixed"],
"benign_base": case["benign"]["base"],
"benign_tip": case["benign"]["tip"],
}
def export(case: dict, label: str, commit: str) -> Path:
target = RESULTS / "checkouts" / case["id"] / label
if target.exists():
shutil.rmtree(target)
target.mkdir(parents=True)
archive = target.parent / f"{label}.tar"
with archive.open("wb") as stream:
subprocess.run(["git", "-C", str(CACHE / case["id"]), "archive", commit], stdout=stream, check=True)
subprocess.run(["tar", "-xf", str(archive), "-C", str(target)], check=True)
archive.unlink()
if not any(target.rglob("*")):
raise RuntimeError(f"{case['id']} {label}: the export is empty")
return target
def scan(case: dict, label: str, checkout: Path, source: Path, out: Path) -> dict:
command = [
"docker", "run", "--rm", "--init", "--network", "none", "--memory", "3g", "--entrypoint", "python",
"-e", "OUSAST_LOG_LEVEL=WARNING", "-e", "PYTHONPATH=/new/src",
"-v", f"{source / 'src'}:/new/src:ro",
"-v", f"{source / 'benchmarks' / 'push' / 'finding_dump.py'}:/dump.py:ro",
"-v", f"{checkout}:/case:ro", "-v", f"{out.parent}:/out",
"openultrasast:dev", "/dump.py", "--root", "/case", "--families", case["family"],
"--out", f"/out/{out.name}", "--deadline", str(DEADLINE), "--max-regions", "5000",
] # fmt: skip
completed = subprocess.run(command, capture_output=True, text=True, timeout=DEADLINE + 900, check=False)
(out.parent / f"{out.stem}.log").write_text((completed.stdout or "") + "\n" + (completed.stderr or "")[-20000:])
if not out.is_file():
return {"instrument_error": f"no result (exit {completed.returncode})"}
result = json.loads(out.read_text())
if not result.get("questions"):
return {"instrument_error": "no question was asked"}
return result
def run(args: argparse.Namespace) -> int:
only = {c.strip() for c in args.only.split(",") if c.strip()}
for case in cases():
if only and case["id"] not in only:
continue
for label, commit in pins(case).items():
out = RESULTS / "scans" / f"{case['id']}--{label}.json"
if out.is_file():
continue
out.parent.mkdir(parents=True, exist_ok=True)
for attempt in (1, 2):
checkout = export(case, label, commit)
result = scan(case, label, checkout, args.source, out)
shutil.rmtree(checkout, ignore_errors=True)
if "instrument_error" not in result:
break
print(json.dumps({"case": case["id"], "pin": label, "attempt": attempt, **result}), flush=True)
out.unlink(missing_ok=True)
else:
out.write_text(json.dumps({"instrument_error": result["instrument_error"], "questions": 0, "findings": []}) + "\n")
summary = json.loads(out.read_text())
keys = ("questions", "completed", "seconds")
progress = {"case": case["id"], "pin": label, "findings": len(summary.get("findings", [])), **{k: summary.get(k) for k in keys}}
print(json.dumps(progress), flush=True)
return 0
def hunks(case: dict, side: str) -> dict[str, list[tuple[int, int]]]:
"""Changed line ranges per file on one side of the fix (`old` = vulnerable, `new` = fixed)."""
diff = subprocess.run(
["git", "-C", str(CACHE / case["id"]), "diff", "-U0", case["vulnerable"], case["fixed"]], capture_output=True, text=True, check=True
).stdout
ranges: dict[str, list[tuple[int, int]]] = {}
current = ""
for line in diff.splitlines():
if (line.startswith("--- a/") and side == "old") or (line.startswith("+++ b/") and side == "new"):
current = line[6:]
elif line.startswith("@@") and current:
match = re.search(r"-(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))?", line)
if match:
start, count = (match.group(1), match.group(2)) if side == "old" else (match.group(3), match.group(4))
first, length = int(start), int(count) if count is not None else 1
ranges.setdefault(current, []).append((first, first + max(length, 1) - 1))
return ranges
def named_functions(case: dict) -> set[str]:
names: set[str] = set()
for field in ("sink", "fix_site"):
_, _, rest = str(case.get(field, "")).partition("::")
names.update(n.split("::")[-1] for n in re.findall(r"[A-Za-z_][A-Za-z0-9_:]*(?=\s*(?:\(|,|;|$))", rest.split("(")[0] + ","))
return {n for n in names if n not in {"used", "by", "source", "callers", "and", "from", "the"}}
def matches(finding: dict, case: dict, ranges: dict[str, list[tuple[int, int]]], functions: set[str]) -> bool:
path, _, rest = finding["site"].partition(":")
line_text, _, function = rest.partition(":")
if finding.get("family") != case["family"] or path not in ranges:
return False
if function in functions:
return True
line = int(line_text) if line_text.isdigit() else -1
return any(first - LINE_WINDOW <= line <= last + LINE_WINDOW for first, last in ranges[path])
def wilson(successes: int, total: int) -> list[float] | None:
if not total:
return None
z, p = 1.96, successes / total
centre, spread = (
(p + z * z / (2 * total)) / (1 + z * z / total),
z * math.sqrt(p * (1 - p) / total + z * z / (4 * total * total)) / (1 + z * z / total),
)
return [round(max(0.0, centre - spread), 3), round(min(1.0, centre + spread), 3)]
def score(args: argparse.Namespace) -> int:
per_case, pool = [], []
for case in cases():
scans = {
label: json.loads((RESULTS / "scans" / f"{case['id']}--{label}.json").read_text())
for label in pins(case)
if (RESULTS / "scans" / f"{case['id']}--{label}.json").is_file()
}
if len(scans) < 4:
per_case.append({"id": case["id"], "status": "incomplete", "scans": sorted(scans)})
continue
old, new, functions = hunks(case, "old"), hunks(case, "new"), named_functions(case)
vulnerable, fixed = scans["vulnerable"].get("findings", []), scans["fixed"].get("findings", [])
detected = [f["site"] for f in vulnerable if matches(f, case, old, functions)]
fixed_alerts = [f["site"] for f in fixed if matches(f, case, new, functions)]
base_regions = {tuple(f["site"].split(":")[0::2]) for f in scans["benign_base"].get("findings", [])}
benign_alerts = [
f["site"] for f in scans["benign_tip"].get("findings", []) if tuple(f["site"].split(":")[0::2]) not in base_regions
]
answered = any(r.split(":")[0] in old for r in scans["vulnerable"].get("completed_regions", []))
errors = {label: s["instrument_error"] for label, s in scans.items() if "instrument_error" in s}
per_case.append(
{
"id": case["id"], "family": case["family"], "language": case["language"],
"detected": bool(detected), "detected_sites": detected,
"unanswered": not detected and not answered, "instrument_errors": errors,
"fixed_side_alerts": fixed_alerts, "benign_alerts": benign_alerts,
"completion": {label: [s.get("completed"), s.get("questions")] for label, s in scans.items()},
"seconds": {label: s.get("seconds") for label, s in scans.items()},
}
) # fmt: skip
pool += [{"case": case["id"], **f} for f in vulnerable]
pool.sort(key=lambda f: (f["case"], f["site"]))
sample = pool if len(pool) <= 15 else pool[0::3]
scored = [c for c in per_case if c.get("status") != "incomplete"]
detected = sum(c["detected"] for c in scored)
result = {
"protocol": "protocol-v1.md",
"cases_scored": len(scored), "cases_incomplete": [c["id"] for c in per_case if c.get("status") == "incomplete"],
"recall": [detected, len(scored)], "recall_wilson95": wilson(detected, len(scored)),
"fixed_side_alerts": sum(len(c["fixed_side_alerts"]) for c in scored),
"benign_alerts": sum(len(c["benign_alerts"]) for c in scored),
"pooled_vulnerable_findings": len(pool), "precision_sample_size": len(sample),
"cases": per_case, "precision_sample": sample,
} # fmt: skip
(RESULTS / "score.json").write_text(json.dumps(result, indent=2) + "\n")
print(json.dumps({k: v for k, v in result.items() if k not in ("cases", "precision_sample")}, indent=1))
return 0
def matches_v2(finding: dict, case: dict, ranges: dict[str, list[tuple[int, int]]]) -> bool:
"""protocol-v2.md: v1's rule with the record's declared site functions, or a declared `path::function`."""
path, _, rest = finding["site"].partition(":")
line_text, _, function = rest.partition(":")
if finding.get("family") != case["family"]:
return False
sites = set(case.get("sites", []))
if f"{path}::{function}" in sites:
return True
if path not in ranges:
return False
if function in {site.partition("::")[2] for site in sites}:
return True
line = int(line_text) if line_text.isdigit() else -1
return any(first - LINE_WINDOW <= line <= last + LINE_WINDOW for first, last in ranges[path])
def stability(one: list[dict], two: list[dict]) -> tuple[int, int]:
"""(findings in exactly one run, findings in either)."""
a, b = {f["site"] for f in one}, {f["site"] for f in two}
return len(a ^ b), len(a | b)
def score_v2() -> int:
per_case, pool, unstable_pool = [], [], 0
changing, total_questions, total_completed = [0, 0], 0, 0
for case in cases():
path = lambda label: RESULTS / "scans" / f"{case['id']}--{label}.json" # noqa: E731
scans = {label: json.loads(path(label).read_text()) for label in pins(case) if path(label).is_file()}
if len(scans) < 6:
per_case.append({"id": case["id"], "status": "incomplete", "scans": sorted(scans)})
continue
old, new = hunks(case, "old"), hunks(case, "new")
va, vb = scans["vulnerable_a"].get("findings", []), scans["vulnerable_b"].get("findings", [])
fa, fb = scans["fixed_a"].get("findings", []), scans["fixed_b"].get("findings", [])
hit_a = {f["site"] for f in va if matches_v2(f, case, old)}
hit_b = {f["site"] for f in vb if matches_v2(f, case, old)}
fixed_alerts = sorted({f["site"] for f in fa + fb if matches_v2(f, case, new)})
base_regions = {tuple(f["site"].split(":")[0::2]) for f in scans["benign_base"].get("findings", [])}
# protocol-v2.md: benign alerts and the precision pool are findings OF THE CASE FAMILY. The engine's dump
# emits only that family, so this was invisible until the hunter, whose findings span every family.
benign_alerts = [
f["site"]
for f in scans["benign_tip"].get("findings", [])
if f.get("family") == case["family"] and tuple(f["site"].split(":")[0::2]) not in base_regions
]
site_files = {site.partition("::")[0] for site in case.get("sites", [])}
covered = set(old) | site_files
answered = any(r.split(":")[0] in covered for label in ("vulnerable_a", "vulnerable_b") for r in scans[label].get("completed_regions", []))
diff_v, union_v = stability(va, vb)
diff_f, union_f = stability(fa, fb)
changing[0] += diff_v + diff_f
changing[1] += union_v + union_f
for scan in scans.values():
total_questions += int(scan.get("questions") or 0)
total_completed += int(scan.get("completed") or 0)
detected = sorted(hit_a & hit_b)
per_case.append(
{
"id": case["id"], "family": case["family"], "language": case["language"],
"detected": bool(detected), "detected_sites": detected,
"unstable": bool(hit_a ^ hit_b) and not detected, "unstable_sites": sorted(hit_a ^ hit_b),
"unanswered": not (hit_a | hit_b) and not answered,
"instrument_errors": {label: s["instrument_error"] for label, s in scans.items() if "instrument_error" in s},
"fixed_side_alerts": fixed_alerts, "benign_alerts": benign_alerts,
"stability": {"vulnerable": [diff_v, union_v], "fixed": [diff_f, union_f]},
"completion": {label: [s.get("completed"), s.get("questions")] for label, s in scans.items()},
"seconds": {label: s.get("seconds") for label, s in scans.items()},
}
) # fmt: skip
stable = {f["site"] for f in vb if f.get("family") == case["family"]}
pool += [{"case": case["id"], **f} for f in va if f.get("family") == case["family"] and f["site"] in stable]
unstable_pool += len({f["site"] for f in va if f.get("family") == case["family"]} ^ stable)
pool.sort(key=lambda f: (f["case"], f["site"]))
sample = pool if len(pool) <= 15 else pool[0::3]
scored = [c for c in per_case if c.get("status") != "incomplete"]
detected = sum(c["detected"] for c in scored)
result = {
"protocol": "protocol-v2.md",
"cases_scored": len(scored), "cases_incomplete": [c["id"] for c in per_case if c.get("status") == "incomplete"],
"recall": [detected, len(scored)], "recall_wilson95": wilson(detected, len(scored)),
"unstable_detections": [c["id"] for c in scored if c["unstable"]],
"fixed_side_alerts": sum(len(c["fixed_side_alerts"]) for c in scored),
"benign_alerts": sum(len(c["benign_alerts"]) for c in scored),
"completion": [total_completed, total_questions],
"stability": changing,
"pooled_stable_vulnerable_findings": len(pool), "unstable_vulnerable_findings": unstable_pool,
"precision_sample_size": len(sample),
"cases": per_case, "precision_sample": sample,
} # fmt: skip
(RESULTS / "score.json").write_text(json.dumps(result, indent=2) + "\n")
print(json.dumps({k: v for k, v in result.items() if k not in ("cases", "precision_sample")}, indent=1))
return 0
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
sub = parser.add_subparsers(dest="command", required=True)
runner = sub.add_parser("run")
runner.add_argument("--source", type=Path, required=True, help="a frozen export of the analyzer (freeze_source.sh)")
runner.add_argument("--only", default="")
runner.add_argument("--population", type=Path, default=POPULATION)
scorer = sub.add_parser("score")
scorer.add_argument("--population", type=Path, default=POPULATION)
args = parser.parse_args()
use(args.population)
if args.command == "run":
return run(args)
return score(args) if version() == "v1" else score_v2()
if __name__ == "__main__":
raise SystemExit(main())
+334
View File
@@ -0,0 +1,334 @@
{
"schema_version": 1,
"population": "population-v1.toml",
"cases": [
{
"id": "wp-ultimate-member-sqli",
"repo": "https://github.com/ultimatemember/ultimatemember",
"vulnerable": "a49b3061ded1630e4f6860b4ebece5ccf0a493d2",
"fixed": "476de92f4c9aaa56c3e2f5527b242ce0476f4655",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "includes/core/class-member-directory-meta.php",
"sink_functions": [
"ajax_get_members"
],
"sink_sha256_vulnerable": "fc6f64e3570f69637a05d747c24bc6fd5e5b350e0a9b1639ef1114d76177c719",
"sink_functions_missing": [],
"sink_sha256_fixed": "2f7a6a913a1ac0c0bed728c8b53d4a32435bafe4f50756f7d8fe30effc789fd7",
"fix_changed_files": 1,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "13151ef558eb6998720cb1ee5f78b5f17535c16c",
"tip": "9130a3226ffec45569a2872ffac338200bd1bfcb",
"subject": "- wpcs;",
"manual": true
},
"benign_review": "replaced: the proposal ea316d04 changed account-status/deactivation logic (authorization-adjacent); 9130a322 is a WPCS reformat of class-dependencies.php"
},
{
"id": "wp-learnpress-sqli",
"repo": "https://github.com/LearnPress/learnpress",
"vulnerable": "73b3cbceef8669056e9da5e2f2c78944eb139af5",
"fixed": "aaf42fe0c6bf05dcf2ff125750b7bd836e76fa3c",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "inc/Databases/class-lp-db.php",
"sink_functions": [
"execute"
],
"sink_sha256_vulnerable": "5a1e22fa525b56771676e6dd7562c796146e52b96e4e0c11471bf2f0d1d329ce",
"sink_functions_missing": [],
"sink_sha256_fixed": "5a1e22fa525b56771676e6dd7562c796146e52b96e4e0c11471bf2f0d1d329ce",
"fix_changed_files": 4,
"fix_touches_sink_file": false,
"fix_touches_fix_site": true,
"benign_proposal": {
"base": "aaf42fe0c6bf05dcf2ff125750b7bd836e76fa3c",
"tip": "179ed7760e6ae5fa46fbe9c5f1324278a42ece04",
"subject": "= 4.2.7.1 = ~ Added: lpOnElementReady function js",
"files": [
"assets/src/js/loadAJAX.js",
"assets/src/js/utils.js",
"inc/class-lp-assets.php"
],
"lines": 50
},
"benign_review": "accepted: JS utility and asset registration"
},
{
"id": "phpmyfaq-captcha-ua-sqli",
"repo": "https://github.com/thorsten/phpMyFAQ",
"vulnerable": "11583abcf4787543d00c25d6c7926af6f509faf0",
"fixed": "545bdffb11244a4741cd29ac909a849c9a6a2e53",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "phpmyfaq/src/phpMyFAQ/Captcha/BuiltinCaptcha.php",
"sink_functions": [
"garbageCollector",
"saveCaptcha"
],
"sink_sha256_vulnerable": "87e468ef69ea87d20d2bafe4ec3a2b5a796fa96faaaac114334e98e72f85a5fb",
"sink_functions_missing": [],
"sink_sha256_fixed": "37c38d1fc7a8ee0dc3a79b69f3fdfb2c9e0f3b90eef078c027e8884b2975dfbc",
"fix_changed_files": 2,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "bf781788ab6b5c2967a09d66216d7580778a2b5a",
"tip": "a4601ae5f01761dc0bb91b8c0c3901be91987ba3",
"subject": "build: updated version strings to 4.2",
"files": [
"CHANGELOG.md",
"docker-compose.yml",
"docs/update.md",
"mkdocs.yml",
"package.json",
"phpmyfaq/src/phpMyFAQ/Setup/AbstractSetup.php",
"phpmyfaq/src/phpMyFAQ/Setup/Installer.php",
"phpmyfaq/src/phpMyFAQ/Setup/Update.php",
"phpmyfaq/src/phpMyFAQ/Setup/UpdateRunner.php",
"phpmyfaq/src/phpMyFAQ/System.php",
"scripts/version.sh",
"tests/phpMyFAQ/SystemTest.php"
],
"lines": 157
},
"benign_review": "accepted: version strings"
},
{
"id": "yeswiki-bazar-form-sqli",
"repo": "https://github.com/YesWiki/yeswiki",
"vulnerable": "6f222915df36fca628fc83325cb0feed8dc2aca0",
"fixed": "fe7244bffcbeffd42777496ebc2a6f6118cc2388",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "tools/bazar/services/FormManager.php",
"sink_functions": [
"create"
],
"sink_sha256_vulnerable": "2051be959575f429ad51367ea6a2e592bc3eb0e86756e93cca3715a942084d3c",
"sink_functions_missing": [],
"sink_sha256_fixed": "f71d844466f0466d8b3e5963670c29b6730f5e675de985b466137a03ad3e1ef9",
"fix_changed_files": 2,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "03f27fdde5f89f90de76cd3f2160cff82e7bffac",
"tip": "70d240c0b43f7e7b0bac55fc0431fb4c2d88b49c",
"subject": "fix(contact): send shared page by mail",
"files": [
"tools/contact/libs/contact.functions.php"
],
"lines": 7
},
"benign_review": "accepted: mail receiver string-to-array normalisation and comments"
},
{
"id": "signalk-remote-conn-ssrf",
"repo": "https://github.com/SignalK/signalk-server",
"vulnerable": "1c479cc0d2d26d14ed74416cff36fa7336c7bf9e",
"fixed": "2816b0dda1bdde893d85c969a9c5950e7825eee7",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "src/serverroutes.ts",
"sink_functions": [
"makeRemoteRequest"
],
"sink_sha256_vulnerable": "10b2224bfdc0308bf1b7f92ee2f37aaabcfdc6578cda59d245e3e97ddea8185f",
"sink_functions_missing": [],
"sink_sha256_fixed": "9aeb864d90df914181683ada2bd49a8e4a86cda4a1a2caa160025225001801bd",
"fix_changed_files": 5,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "8af993d45c898edfe28f820f1726b6fc33b27cad",
"tip": "07416b79601c5d901f671f70a94bd57bed90bad8",
"subject": "fix: allow multiple deltaHandler next calls (#2818)",
"files": [
"src/deltachain.ts",
"test/deltachain.ts"
],
"lines": 61
},
"benign_review": "accepted: delta handler chaining fix"
},
{
"id": "saltcorn-sync-sqli",
"repo": "https://github.com/saltcorn/saltcorn",
"vulnerable": "e7e09254a3c69dc6582eda9e5b5769935db108af",
"fixed": "2b6c8a2eab88955ea6b12b1b4e5c2e613faefef5",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "packages/server/routes/sync.js",
"sink_functions": [
"getSyncRows",
"getDelRows"
],
"sink_sha256_vulnerable": "b69b79171574ba7d8ca4b02d5c0ffcadeb0f885faab016d8066ebf753d839edf",
"sink_functions_missing": [],
"sink_sha256_fixed": "5b5d34758caca6601e327953191ca30475193e389bea542549a91f82af2bf8dc",
"fix_changed_files": 5,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "5c619403c40447bf56ce662570d3a3ffbf781019",
"tip": "df114765fd36d280395b2129f1703fa98c712d06",
"subject": "updated at in trigger.findAllWithTableName",
"files": [
"packages/saltcorn-data/models/trigger.ts"
],
"lines": 2
},
"benign_review": "accepted: adds a constant column to a SQL select -- a query edited without any input, a useful negative control"
},
{
"id": "mongo-express-bson-rce",
"repo": "https://github.com/mongo-express/mongo-express",
"vulnerable": "ea02b364d43f179f191fc91fb9962efdb0843a8d",
"fixed": "d8c9bda46a204ecba1d35558452685cd0674e6f2",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "lib/bson.js",
"sink_functions": [
"toBSON"
],
"sink_sha256_vulnerable": "09032fcec4e4b0782f94bb69bd946afeee9794991daa4b6a9d3c7aa0b1b5531c",
"sink_functions_missing": [],
"sink_sha256_fixed": "a9fa97927fbbb8460902813beb8b99e7d23cda6be7f3761420259aaaf9268640",
"fix_changed_files": 6,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "9550ae2529258b47a2b637d8e64e69a0d4d38ad8",
"tip": "5b9b34f8553d004ceb7b666bbf566b406a2a3153",
"subject": "Merge pull request #538 from damien-malescot/master",
"files": [
"lib/routes/document.js",
"lib/views/document.html"
],
"lines": 7
},
"benign_review": "accepted: UUID document-delete fix"
},
{
"id": "mockoon-admin-api-cors",
"repo": "https://github.com/mockoon/mockoon",
"vulnerable": "b42bdfb7f82e83f0e81bea8e6fe41adf5ec82585",
"fixed": "c420b5a56918475b8663977b51e5f986e45b3299",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "packages/commons-server/src/libs/server/admin-api.ts",
"sink_functions": [
"middleware"
],
"sink_sha256_vulnerable": "1beb42bf35daa8c7291a1f75976a7012cdd29411d909218a27544ec06eae92f9",
"sink_evidence_found": true,
"sink_sha256_fixed": "81866aec8e3a67f3aef78c40980193dab841934102fce53ca4f53c73c2ed6eab",
"fix_changed_files": 34,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "de69f5e30f89e17918855d147f4e8c055d5b2ba9",
"tip": "895961070b55a14d290ad50232ab9c6d800c3cad",
"subject": "Fix serverless lib returning a 404 when no body is present (#2274)",
"files": [
"packages/commons-server/src/libs/server/server.ts",
"packages/commons-server/test/specs/server/server-body-parsing.test.ts"
],
"lines": 40
},
"benign_review": "accepted: serverless 404 on empty body"
},
{
"id": "alerta-query-sqli",
"repo": "https://github.com/alerta/alerta",
"vulnerable": "71c098d8bb8091e76edaefeef4662eb77c753ce7",
"fixed": "fdd52cd1abad8d02d1dfb8ecdcdbb43b6af3b883",
"problems": [
"no benign change found automatically; choose one by hand"
],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "alerta/database/backends/postgres/queryparser.py",
"sink_functions": [
"__repr__"
],
"sink_sha256_vulnerable": "40884d7e83cc38db30a03b11f9686c8cb4b629c2de74191175516cb4c4f1ad3e",
"sink_functions_missing": [],
"sink_sha256_fixed": "d9c13317bb36208bbe83bd305bed1199a9e7a586659ca8256479f6158b9d0ce6",
"fix_changed_files": 3,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "b7111d23ae9aefb5064ef107976f62e3836217b6",
"tip": "71c098d8bb8091e76edaefeef4662eb77c753ce7",
"subject": "fix(deps): replace pkg_resources with importlib.metadata (#2041)",
"manual": true
},
"benign_review": "replaced: no neutral commit after the fix (proposals were login, TLS-verification and filter changes); 71c098d8 replaces pkg_resources with importlib.metadata, both sides still vulnerable"
},
{
"id": "wger-next-open-redirect",
"repo": "https://github.com/wger-project/wger",
"vulnerable": "d30e85b23fb1307ac6579e9e3ae393c50470cea7",
"fixed": "0ba371bbb50cd5668820f240995405e1b2483384",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "wger/core/views/user.py",
"sink_functions": [
"trainer_login",
"use_browser_language"
],
"sink_sha256_vulnerable": "185582190d001b1670738dde2264dac5e6fef2253cc8595dc862c74a270e906f",
"sink_functions_missing": [
"use_browser_language"
],
"sink_sha256_fixed": "c0952e4b1990081906cb90a597a83cabafd1a3f8e2d124c4c9b1e2ab80756534",
"fix_changed_files": 4,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "47d646c24178783eb6787ecf26e79dc8f4a394c6",
"tip": "c1144533bf9bca487ff905d925b567151d89c1aa",
"subject": "Formatting",
"files": [
"wger/core/api/serializers.py"
],
"lines": 5
},
"benign_review": "accepted: formatting"
},
{
"id": "opencve-notification-ssrf",
"repo": "https://github.com/opencve/opencve",
"vulnerable": "928bbadbb1672051bcfc2a6289a9e6a8f029c106",
"fixed": "0af7f894debc95c67fa53e7ba5a6cdecb10c34f0",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "web/projects/notifications.py",
"sink_functions": [
"_http_post_json"
],
"sink_sha256_vulnerable": "392b560fdbfe29f0d56661a5a1e74afcca426ce6658eb96d1c0ff12c8cacb1ad",
"sink_functions_missing": [],
"sink_sha256_fixed": "e8cc8b410e1e43bbf5356066a800c61a32f7d13f03c449512c761dd67553e933",
"fix_changed_files": 8,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "0c8101082ba20628b28fcec9c4611745d32202c2",
"tip": "965dc348e225308edabdfef3ecb76f7d73a0c162",
"subject": "fix: unlink automation executions before cleaning expired reports",
"files": [
"scheduler/dags/includes/constants.py",
"scheduler/tests/tasks/test_reports.py"
],
"lines": 118
},
"benign_review": "accepted: scheduler cleanup of automation executions"
}
],
"reviewed_by": "assistant review of each benign diff; selection of cases by the maintainer",
"frozen_on": "2026-09-24"
}
+475
View File
@@ -0,0 +1,475 @@
{
"schema_version": 1,
"population": "population-v2.toml",
"cases": [
{
"id": "pgadmin-maintenance-sqli",
"repo": "https://github.com/pgadmin-org/pgadmin4",
"vulnerable": "d112dc3b962218efc3613fa727d54f0fc5b1d527",
"fixed": "cf53953d9ad66c182a08e662c1ba29c5831b486b",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "web/pgadmin/tools/maintenance/__init__.py",
"sink_functions": [
"create_maintenance_job"
],
"sink_sha256_vulnerable": "9fa2828e8ff9d3af71b9194a8903eee6301e61f8af33c942f1a447ebbf829ec9",
"sink_functions_missing": [],
"sink_sha256_fixed": "e841eedc2e859240c87899b5de1e5a077e36b6eaa4251c82435a425812bf6be6",
"fix_changed_files": 4,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "e7eaef0dc002c254e4dd15705df800c448e8eb4c",
"tip": "a50a553b0a7e529e5d87f52bdc6d3a5ee3e8b9e8",
"subject": "chore: feature tests use sys.executable; sync yarn.lock to package.json",
"files": [
"web/regression/feature_utils/app_starter.py",
"web/yarn.lock"
],
"lines": 48
}
},
{
"id": "qwed-sympy-eval",
"repo": "https://github.com/QWED-AI/qwed-verification",
"vulnerable": "06801f68cbf9c9b5e54cc17f2a61cb2792da7a26",
"fixed": "6066b68c0c4f4cc2c3771824822aaa864d082ef8",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "src/qwed_new/api/main.py",
"sink_functions": [
"verify_math"
],
"sink_sha256_vulnerable": "bff26eca2c749dbd6bbc20cfd912ef077bd2ab3c62d87b9caea5f6f989a463df",
"sink_functions_missing": [],
"sink_sha256_fixed": "7632a218b3ce20eb50ebf76e3f9d0110a9152f9a38bad9f47b752294fe7a505d",
"fix_changed_files": 8,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "62e74ad0a1760abbb9986292fef3e2e6125b1187",
"tip": "89498ca5eb621a66316b2cf2de1e1e8fdaf6d79d",
"subject": "Merge pull request #211 from QWED-AI/fix/codeql-redos-data-uri",
"files": [
".coverage",
"src/qwed_new/core/output_sanitizer.py",
"tests/test_output_sanitizer.py"
],
"lines": 70
}
},
{
"id": "contextforge-prompt-ssti",
"repo": "https://github.com/IBM/mcp-context-forge",
"vulnerable": "0f8e0ad679f1fff685f486e17f642ac863c8ca9e",
"fixed": "4d31004661858a2e99055b3c0c9f14218b8f7120",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "mcpgateway/services/prompt_service.py",
"sink_functions": [
"_render_template",
"_compile_jinja_template"
],
"sink_sha256_vulnerable": "0a9752890c5324a7ff01bd15890b376b35226cc81d8853525e41f8b31fb66cb6",
"sink_functions_missing": [],
"sink_sha256_fixed": "dd91186b8ee3eed6373761554ad5873e3436b7213305d1d1104f209c6be213af",
"fix_changed_files": 27,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "1f8fd58980f4565d4788ffc3587d6e3bd16b3342",
"tip": "9938c58f7819c526932c4ae4ad646a9c028cb19e",
"subject": "fix: add rust coverage reports (#4458)",
"files": [
"Makefile",
"crates/mcp_runtime/DEVELOPING.md",
"tests/unit/test_makefile_rust_targets.py"
],
"lines": 81
}
},
{
"id": "lmdeploy-media-url-ssrf",
"repo": "https://github.com/InternLM/lmdeploy",
"vulnerable": "71648d239c978452720a6d6692fe914efc8a7b94",
"fixed": "03c313006d17cc3feae86b633c44206a997c44db",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "lmdeploy/vl/media/connection.py",
"sink_functions": [
"_load_http_url"
],
"sink_sha256_vulnerable": "697f169c03cf175c338fee2dddaa81a7649913ed2908e1602bd845806fd19646",
"sink_functions_missing": [],
"sink_sha256_fixed": "ec8efc50b1749326454731e395d92ab5430e6dac3cf1f334c067ab25a2eb566b",
"fix_changed_files": 12,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "b6a29db4d2d218838048975977f125dc9b1ac75d",
"tip": "3d9d23087b59fa44afe83deb03ee7cce04df3b4d",
"subject": "fix(benchmark): seed NumPy RNG so --seed controls random-mode lengths (#4785)",
"files": [
"benchmark/benchmark_guided.py",
"benchmark/profile_pipeline_api.py",
"benchmark/profile_throughput.py"
],
"lines": 3
}
},
{
"id": "winml-cli-serve-cors",
"repo": "https://github.com/microsoft/winml-cli",
"vulnerable": "02179d185f0e3ac23518236b5ca1f61fd37ef680",
"fixed": "f4073e0ef4700a25b623487e7e45c421ca0b9993",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "src/winml/modelkit/serve/cli_api.py",
"sink_functions": [],
"sink_sha256_vulnerable": "efab45cd65ef1cd0a64070fe104dfdcfc2f298c98def37973c0f22cd0140949f",
"sink_functions_missing": [],
"sink_sha256_fixed": "c616799bdaace01883b9e392a3c118eb54295f827eae7f8d7cafcfd40674c012",
"fix_changed_files": 12,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "8d631f6f1e5db26a04e8c13045280408c6f984dc",
"tip": "be3e59dd412d4918c5a852aa4d8f0207c34aaf6f",
"subject": "Remove preload ORT DLL (undo PR 689) (#1380)",
"files": [
"src/winml/modelkit/__init__.py",
"tests/unit/test_modelkit_init.py"
],
"lines": 158
}
},
{
"id": "reactpress-findall-column-sqli",
"repo": "https://github.com/fecommunity/reactpress",
"vulnerable": "c3dd10681b3d2b5e224f7ac647b3769557603975",
"fixed": "78ecb70af1c021455c05fdcbe137212c70e310d6",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "server/src/modules/article/article.service.ts",
"sink_functions": [
"findAll"
],
"sink_sha256_vulnerable": "33fb3c644ab4ee3ea469cb190a58eacdc2f34f8f2379c04437f3df4d7ede7901",
"sink_functions_missing": [],
"sink_sha256_fixed": "8fe3e41ec60351a004455c49d5110fb969acdf2a883b9f22505e66d56c84a3b5",
"fix_changed_files": 9,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "78ecb70af1c021455c05fdcbe137212c70e310d6",
"tip": "70c7dc39e1a885a1ac7d051c57a409432905fe6c",
"subject": "fix(server): correct column whitelist to match entity definitions",
"files": [
"server/src/utils/query-whitelist.util.ts"
],
"lines": 23
}
},
{
"id": "fuxa-tdengine-escape-sqli",
"repo": "https://github.com/frangoteam/FUXA",
"vulnerable": "193a888f8e3f805325072829b1644a2c768c948b",
"fixed": "b78f6159d02a56ce5ff48207feb936afb3534fc8",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "server/runtime/storage/tdengine/index.js",
"sink_functions": [
"getDaqValue"
],
"sink_sha256_vulnerable": "e129a1520f736ef53de1c813ca4295b2b9de09f8041477a4e96bcf242b375d0e",
"sink_functions_missing": [],
"sink_sha256_fixed": "7f89162c781081034a2b6e525c0a2e29f976ee984651b2345b2b5fefdd9ac456",
"fix_changed_files": 2,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "e787e4140b5e71c5b02e23c295fa08b8df0edc53",
"tip": "bf998eeee3158e338724c71979b7cdafcfa74b32",
"subject": "Button control with configurable icon (#2349)",
"files": [
"client/src/app/_models/hmi.ts",
"client/src/app/framework/icon-selector/icon-selector.component.scss",
"client/src/app/gauges/controls/html-button/html-button.component.ts",
"client/src/app/gauges/gauge-property/gauge-property.component.html",
"client/src/app/gauges/gauge-property/gauge-property.component.scss",
"client/src/app/gauges/gauge-property/gauge-property.component.ts",
"client/src/assets/i18n/en.json"
],
"lines": 92
}
},
{
"id": "budibase-mongo-template-nosqli",
"repo": "https://github.com/Budibase/budibase",
"vulnerable": "b7e63df737eede85d951aa9665b0c0778ece7ccb",
"fixed": "dd8c0654b35cd89ce3645f2355f4bf2ff9a5dd80",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "packages/server/src/sdk/workspace/queries/queries.ts",
"sink_functions": [
"enrichContext"
],
"sink_sha256_vulnerable": "27233a212e8227bb0dc5f0045aa64b8afa0ced563b174ddd17f8ff5013a516cb",
"sink_functions_missing": [],
"sink_sha256_fixed": "e127d41a1e33b89173b69e802395d753f3cfc0cda06bffa3ae6952e910102ce8",
"fix_changed_files": 4,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "a4423fc5bf237e6101c2e8348ef5372fff6484cd",
"tip": "272c6a61a2a386688792f9e26c2665f2f431a5c2",
"subject": "Merge pull request #18865 from Budibase/feature/18784-ai-credit",
"files": [
"packages/builder/src/components/usage/Usage.svelte",
"packages/builder/src/settings/pages/usage.svelte",
"packages/shared-core/src/helpers/tests/url.spec.ts",
"packages/shared-core/src/helpers/url.ts"
],
"lines": 92
}
},
{
"id": "unleash-addon-webhook-ssrf",
"repo": "https://github.com/Unleash/unleash",
"vulnerable": "b9f3217d327f24094b4352130baa30387f85cffe",
"fixed": "2100db76af3473f13e6fb40096cf17a9c2b741a1",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "src/lib/addons/addon.ts",
"sink_functions": [
"fetchRetry"
],
"sink_sha256_vulnerable": "a52627ff0742bd8c296a9f63753174b334f74aba50e013cc8bc18d60e6929aa1",
"sink_functions_missing": [],
"sink_sha256_fixed": "7b405bfdf4a981787e70c21f4559e8df94d55f6a56cfc0ab130c7ff8a18549de",
"fix_changed_files": 21,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "d4efb3a31616483c438606edbecce13b868fc587",
"tip": "cb4e6250743fac364bae50b985b2b8348def7a2b",
"subject": "Merge commit '232b5d575c7d455ce7ac767183e61fd389dd8aac' into automation/sync-oss-upstream",
"files": [
"frontend/src/component/providers/UnleashFlagProvider/UnleashFlagProvider.tsx"
],
"lines": 13
}
},
{
"id": "medplum-external-auth-open-redirect",
"repo": "https://github.com/medplum/medplum",
"vulnerable": "5d4a8654b02aa67c444f96ae1eb5663c09854006",
"fixed": "7ae10035ddadde4dba7b18d3156553940465b3a1",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "packages/server/src/auth/external.ts",
"sink_functions": [
"externalCallbackHandler"
],
"sink_sha256_vulnerable": "3ab597c5f268210f56232b07094d80771dce0bc3b78996e9246bb31a0930bd53",
"sink_functions_missing": [],
"sink_sha256_fixed": "74cc58f9a399550945d7a2266c8bd4dac48a852378e4eea56fb5f2a1deb72084",
"fix_changed_files": 7,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "0dd335e8520b859563626d9bf50d602a5f3ad79a",
"tip": "20790b17d76f335e60dd703f65d4eb13bbb4754d",
"subject": "tweak(cdk): make `minHealthyPercent` 100 (#8836)",
"files": [
"packages/cdk/src/backend.ts"
],
"lines": 4
}
},
{
"id": "cct-studio-server-cmdi",
"repo": "https://github.com/davila7/claude-code-templates",
"vulnerable": "acdb5893412a81eaec9862ee24a4613bf296ab45",
"fixed": "bc4618b07232633c1c0aac12a43e436268d31783",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "cli-tool/src/sandbox-server.js",
"sink_functions": [
"executeLocalTask",
"checkAndInstallAgent"
],
"sink_sha256_vulnerable": "e2c24dcf02b90e5056c2b7065ce46027ef02139991117ee70f589a3a971e974d",
"sink_functions_missing": [],
"sink_sha256_fixed": "2414d562c0714689978808349499af8cadf7cf39a28a534194f2ff40954aa25e",
"fix_changed_files": 4,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": null
},
{
"id": "wp-gopay-log-filter-sqli",
"repo": "https://github.com/gopaycommunity/gopay-woocommerce-integration",
"vulnerable": "626bc8b5f8d33de6eefd921e3a05ed52ac320bcd",
"fixed": "7190c985370f9758f226c2f2addd79532a2f93d5",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "admin/views/log.php",
"sink_functions": [],
"sink_sha256_vulnerable": "541b921b7810a7b563a4ffefd2ac296e5b37f81d970c5fd9dd787f107c0593b6",
"sink_functions_missing": [],
"sink_sha256_fixed": "bdc6178b33d87bcc149019555239668bd71a9bd0f9c30c3ec396ee32974810f4",
"fix_changed_files": 1,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "2b5fe58d6dadbfecb01ed34611aed8502d11e612",
"tip": "88b10bda704a37e88121e22dc22f4e5019097272",
"subject": "GPY025-136: fix i18n and remove translation calls on non-literal text",
"files": [
"includes/class-gopay-gateway-options.php",
"includes/class-gopay-wc-blocks-support.php"
],
"lines": 11
}
},
{
"id": "wp-user-registration-login-redirect",
"repo": "https://github.com/themegrill/user-registration",
"vulnerable": "e52cf151cb229852e6209fb3a627aa51485139c1",
"fixed": "75c940dab15fd54d5a3fbb9f2213767b89467b08",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "includes/functions-ur-core.php",
"sink_functions": [
"ur_process_login"
],
"sink_sha256_vulnerable": "7ec30d26e49ed039448d4e2616000987db57d2067533e0b2015fe9621e38e57f",
"sink_functions_missing": [],
"sink_sha256_fixed": "867fccbaa0df3344523aff6516102cdb22e1e7cd8b2eb7b482cdf9e9e3184b06",
"fix_changed_files": 8,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "064dd7ab1585082b9b5cf078f2d22c0234f25e48",
"tip": "3b6acc4695e93d9d48969ed894c21419bd09b33a",
"subject": "Fix - Deactivate Licence button did nothing when the pro-activation transient was set (#1418)",
"files": [
"includes/class-ur-plugin-updater.php"
],
"lines": 4
}
},
{
"id": "wp-directorist-avatar-ssrf",
"repo": "https://github.com/sovware/directorist",
"vulnerable": "77cfc9bc8b453f97ab7e720330706fa4ef056867",
"fixed": "1a67825d5cf6e7c768c3a692da1d18bae732bea9",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "includes/rest-api/functions.php",
"sink_functions": [
"directorist_rest_upload_image_from_url"
],
"sink_sha256_vulnerable": "fb9e13afbf7f1b856d59a75acca4fc8ebbadb06908af5e409d2b2e7eb470e4b4",
"sink_functions_missing": [],
"sink_sha256_fixed": "c451a4853a2ec8a3038741af558f4024cf34edc919a07dd3b16c8f665f834d0f",
"fix_changed_files": 7,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": null
},
{
"id": "wp-groundhogg-confirm-redirect",
"repo": "https://github.com/groundhoggwp/groundhogg",
"vulnerable": "9472a1dd79d4556c40eece501a61930793f170d3",
"fixed": "028247d4b8ec9ba051d3eb48f1466a759bc16bfd",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "templates/preferences.php",
"sink_functions": [],
"sink_sha256_vulnerable": "7887dbbac602984f3c8f754c526dc0656e547c2e66f00e0eb417d56c1af85554",
"sink_functions_missing": [],
"sink_sha256_fixed": "081f9b70f08905b7c2c909306747aaacb4a885d7563f455b54638a3a70c611d8",
"fix_changed_files": 1,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "61c8a03f11b6030c630be6ae0b0c2ad641eb733a",
"tip": "e3a69f1c2814c6bebed940505894008748566b93",
"subject": "output a square image regardless of image dimensions",
"files": [
"includes/utils/html.php"
],
"lines": 13
}
},
{
"id": "froxlor-admin-ip-second-order-sqli",
"repo": "https://github.com/froxlor/froxlor",
"vulnerable": "9bf03de643a2edee4642bfc6c94db1f05253da02",
"fixed": "a1eaca5a1601c8a30e00814a4fc73ad0c185f89e",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "lib/Froxlor/Api/Commands/IpsAndPorts.php",
"sink_functions": [
"listing",
"listingCount"
],
"sink_sha256_vulnerable": "d1494998117a922d6370c9acbc68c9f547207dfae435c9e23a1d4af1786b0854",
"sink_functions_missing": [],
"sink_sha256_fixed": "c4db47c2b4b7d62f6494989c30bd682bd1a39136b6b545d79dad8d33fbff6863",
"fix_changed_files": 3,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "b595f061fbb68874423809fec9f2a42bf6b34175",
"tip": "9d274828bdc4eefec0058a5a3a25bf4eb69d0f70",
"subject": "avoid warnings regarding php8.5 in config/database.php; adjust phpunit for froxlor/framework testing",
"files": [
"composer.json",
"config/database.php",
"phpunit.xml",
"tests/Feature/ExampleTest.php",
"tests/Unit/ExampleTest.php"
],
"lines": 62
}
},
{
"id": "sulu-smart-content-sql2-injection",
"repo": "https://github.com/sulu/sulu",
"vulnerable": "d061094f5b7bb1d5e974544fce30bede9c7adf8e",
"fixed": "d19c01487af8c3de2fb3aa145856707a6367392c",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "src/Sulu/Component/Content/SmartContent/QueryBuilder.php",
"sink_functions": [
"buildCategoriesWhere",
"buildWhere"
],
"sink_sha256_vulnerable": "421e860f75a4953bef494c24b64e951dc0e3748c7d7f592d73ee419af46c73cc",
"sink_functions_missing": [],
"sink_sha256_fixed": "764798a416b14f85c24c9ff7f6342e78fde837dbf1d74207c4284bb162318b99",
"fix_changed_files": 2,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "2c4ee09545cf1c9a400006fcaca3b8499cc9a161",
"tip": "87d30d8f061102ce114556d72d628224e64529e5",
"subject": "Merge branch '2.6' of https://github.com/sulu/sulu into 3.0",
"files": [
"config/packages/security.yaml",
"src/Sulu/Bundle/SecurityBundle/Security/AuthenticationHandler.php",
"src/Sulu/Bundle/SecurityBundle/Tests/Application/config/config.yml",
"src/Sulu/Bundle/SecurityBundle/Tests/Functional/Controller/LoginControllerTest.php",
"src/Sulu/Bundle/SecurityBundle/Tests/Functional/Security/AuthenticationHandlerTest.php",
"src/Sulu/Component/Content/Tests/Unit/Types/LinkTest.php",
"src/Sulu/Component/Content/Types/Link.php"
],
"lines": 183
}
}
]
}
+561
View File
@@ -0,0 +1,561 @@
{
"schema_version": 1,
"population": "population-v3-php.toml",
"cases": [
{
"id": "gitlist-commandline-cmdi",
"repo": "https://github.com/klaussilveira/gitlist",
"vulnerable": "f67609d52c1812fa8a7ed80eae5e795cfd72115f",
"fixed": "88cf2866083d5f7c20d9d565c45f828a7ad1516b",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "src/SCM/System/Git/CommandLine.php",
"sink_functions": [
"getDefaultBranch"
],
"sink_sha256_vulnerable": "a0aa9125dabc9748274b3762b9bd1ad117400b1b8d446593afdc3e6f38951e26",
"sink_functions_missing": [],
"sink_sha256_fixed": "3f1c45d9f0ee68c517bf89fa3fe65c0ead84351ed5d9329ac5d6a4b53d575344",
"fix_changed_files": 2,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "ce5806c849f453d7200c1f488ee6aba19eef9a4c",
"tip": "ecc8e18e03d9b547e82b7c7c148cb00ba01c0cc9",
"subject": "Merge pull request #902 from zoredache/sort_repositories",
"files": [
"src/App/Controller/Repository.php"
],
"lines": 2,
"side": "before the vulnerable pin"
}
},
{
"id": "qloapps-customermessage-bo-query-sqli",
"repo": "https://github.com/Qloapps/QloApps",
"vulnerable": "5b4d65cbb8570a4ba46364ee0f2ded79d294ab84",
"fixed": "123c97c110b7053ea3297d8e58fe95b3c3536560",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sites": [
"classes/CustomerMessage.php::searchByName"
],
"site_basis": {
"classes/CustomerMessage.php::searchByName": "resolved-from-fix-diff"
},
"site_files_unresolved": {},
"sink_file": "classes/CustomerMessage.php",
"sink_functions": [
"searchByName"
],
"sink_sha256_vulnerable": "f4037f17c83d248639dbb6cce36f35e3b7eaae557dd631bc9268a97415c46534",
"sink_functions_missing": [],
"sink_sha256_fixed": "0464ba7047c393a2c873f301717258397bfc50c2b4f423e98f33d0c954678d41",
"fix_changed_files": 5,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "884563ccff83fe42b4689ba32e605276278c3989",
"tip": "e43b45009508eb1465709d4af0f549faacd1d817",
"subject": "Merge pull request #1735 from amitsharma322/gli-3007",
"files": [
"classes/PaymentModule.php",
"controllers/admin/AdminOrdersController.php"
],
"lines": 23,
"side": "after the fix"
}
},
{
"id": "roundcube-markasjunk-cmd-learn-rce",
"repo": "https://github.com/roundcube/roundcubemail",
"vulnerable": "a1afb8fd1f00ed4cb9376c072bb5ca5ded64495e",
"fixed": "b8f90e28a46d42e79a69568cba897f8f4223d9cd",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sites": [
"plugins/markasjunk/drivers/cmd_learn.php::_do_salearn"
],
"site_basis": {
"plugins/markasjunk/drivers/cmd_learn.php::_do_salearn": "resolved-from-fix-diff"
},
"site_files_unresolved": {},
"sink_file": "plugins/markasjunk/drivers/cmd_learn.php",
"sink_functions": [
"_do_salearn"
],
"sink_sha256_vulnerable": "f4f46e54fa8a19d20ab4000e0f363afd544c6f9f73b806c7b8176fd8348dea83",
"sink_functions_missing": [],
"sink_sha256_fixed": "3082db4ed8a79b472ebc2b7d42e34db054737297a81cf595b652a6ba4e34f8eb",
"fix_changed_files": 2,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "1cddafded973c4bf6faa173accaf8f52ca1ac0d0",
"tip": "747f544708ed1d698b677d333346164a9451aeda",
"subject": "Simplify",
"files": [
"tests/Framework/TnefDecoderTest.php"
],
"lines": 6,
"side": "after the fix"
}
},
{
"id": "maccms10-template-blacklist-rce",
"repo": "https://github.com/magicblack/maccms10",
"vulnerable": "2cabecda29947e8f9ea7cd3aa47ab90ed817605b",
"fixed": "71ad3bb29570e110d8e973acff68040a3050ddf0",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sites": [
"application/admin/controller/Template.php::info"
],
"site_basis": {
"application/admin/controller/Template.php::info": "resolved-from-fix-diff"
},
"site_files_unresolved": {},
"sink_file": "application/admin/controller/Template.php",
"sink_functions": [
"info"
],
"sink_sha256_vulnerable": "02e0ff0d172a25fc35b5b386c4280cf65bf3980a612cd833020e43f2dddb5d3c",
"sink_functions_missing": [],
"sink_sha256_fixed": "310d667aa2a99f554f49649b0ff1549b0c7f35bf1a0c167a86569c49167f5cc9",
"fix_changed_files": 1,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "d9f09011a1523d0bfaed5dea9b2139461f2b0ea8",
"tip": "668ba418e50e96ab32ac612f3b6528beb172b3b7",
"subject": "fix #1356 \u4fee\u6b63\u6ce8\u518cAPI\u6821\u9a8c\u5668\u89e3\u6790\u9519\u8bef\u53ca\u7f3a\u53c2\u62a5\u9519",
"files": [
"application/api/controller/User.php",
"application/common/model/User.php",
"tests/regression/user_register_validate.php",
"\"\\350\\257\\264\\346\\230\\216\\346\\226\\207\\346\\241\\243/API\\346\\216\\245\\345\\217\\243\\350\\257\\264\\346\\230\\216V2.txt\""
],
"lines": 166,
"side": "after the fix"
}
},
{
"id": "mantisbt-tokenizer-eval-rce",
"repo": "https://github.com/mantisbt/mantisbt",
"vulnerable": "e7041ecc80c8d35f76a7ea1027a8a231244f7dc5",
"fixed": "78c0af63d1fe0118004744cab21ca3bf2cea0f5c",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sites": [
"core/classes/Tokenizer.class.php::__construct"
],
"site_basis": {
"core/classes/Tokenizer.class.php::__construct": "resolved-from-fix-diff"
},
"site_files_unresolved": {
"adm_config_set.php": "the fix does not change this file"
},
"sink_file": "core/classes/Tokenizer.class.php",
"sink_functions": [
"__construct"
],
"sink_sha256_vulnerable": "cda7f81f7fcb4675c7bf5fc3f8dfd41ccb48ad891e95a85cf5edae90f0cbb1df",
"sink_evidence_found": true,
"sink_sha256_fixed": "f555c63e2dc15dbef7ff2c4db8d7c47848c8bc7abe981364a11f65c5e44b8fb0",
"fix_changed_files": 1,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "088fa767bce246b6d2f083f32d937f22c8de4767",
"tip": "d6ca719b591f70acfa711fc41705388557b6ac04",
"subject": "Improve Workflow Graph display.",
"files": [
"workflow_graph_img.php"
],
"lines": 14,
"side": "after the fix"
}
},
{
"id": "sharp-editor-iframe-srcdoc-xss",
"repo": "https://github.com/code16/sharp",
"vulnerable": "67743149309054493ec8f5505173fc57aaff760f",
"fixed": "ec509a22c808a5bd9dfad6a0a85c92ce6f411e21",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sites": [
"src/Utils/Sanitization/FormatsSanitizedValue.php::sanitizer",
"src/Utils/Sanitization/FormatsSanitizedValue.php::isEncodingNeeded"
],
"site_basis": {
"src/Utils/Sanitization/FormatsSanitizedValue.php::sanitizer": "resolved-from-fix-diff",
"src/Utils/Sanitization/FormatsSanitizedValue.php::isEncodingNeeded": "resolved-from-fix-diff"
},
"site_files_unresolved": {},
"sink_file": "src/Utils/Sanitization/FormatsSanitizedValue.php",
"sink_functions": [
"sanitizer",
"isEncodingNeeded"
],
"sink_sha256_vulnerable": "7e65b0fbcd481c6322f83ae46523b069c4a62383cbf088e7be60be6ace4bcc9a",
"sink_evidence_found": true,
"sink_sha256_fixed": "59a2845f4a8b419f8504b2cb539087eb3dc3fb8bb0ec7c950330ba951c6c75ad",
"fix_changed_files": 2,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "ec509a22c808a5bd9dfad6a0a85c92ce6f411e21",
"tip": "7c601fffe930fc321a5d8bcb25cb072d50a6fc08",
"subject": "Update version to v9.22.5",
"files": [
"src/SharpInternalServiceProvider.php"
],
"lines": 2,
"side": "after the fix"
}
},
{
"id": "privatebin-jsonld-request-uri-injection",
"repo": "https://github.com/PrivateBin/PrivateBin",
"vulnerable": "e0dd4c025c19a182b6a4c6fb77a8bf81ceff6899",
"fixed": "75f056dcda955d94c17ec5a4f8c54a9b7bfcee07",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "lib/Request.php",
"sink_functions": [
"getRequestUri"
],
"sink_sha256_vulnerable": "bd35439be97ac659d94b1d0cde13f70644f99468bfc16ddc2fb214a845bdb7b4",
"sink_functions_missing": [],
"sink_sha256_fixed": "ea17a346372b8259bad1ac7edb1268378168d0de0628cd53b948f52038fd9c1e",
"fix_changed_files": 4,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "75f056dcda955d94c17ec5a4f8c54a9b7bfcee07",
"tip": "3035d6e5d3cceb7f06f7b7bb80968727879c52b0",
"subject": "incrementing version",
"files": [
"CHANGELOG.md",
"Makefile",
"README.md",
"SECURITY.md",
"doc/Installation.md",
"js/package.json",
"lib/Controller.php"
],
"lines": 18,
"side": "after the fix"
}
},
{
"id": "openstamanager-group-name-html-injection",
"repo": "https://github.com/devcode-it/openstamanager",
"vulnerable": "57a2734e2121aa8fcd456eff7c59f03e29e69176",
"fixed": "0b88ad1963871abfa5e134004ca0c74c496d8d2f",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "modules/utenti/edit.php",
"sink_functions": [],
"sink_sha256_vulnerable": "8524dd092b2fdbf68339b80e374c4108fd6925a146223b325370f4ee8414a016",
"sink_evidence_found": true,
"sink_sha256_fixed": "290d5fde2df7713eb592361f3b0abfd3524a3b83bf99ef0fafdf80f0ee29612b",
"fix_changed_files": 4,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "0b88ad1963871abfa5e134004ca0c74c496d8d2f",
"tip": "1832b85d0e570e0b873c7c23dcfcb6b5d1365a28",
"subject": "fix: ricerca riferimenti in importazione fattura per riga con articolo non esistente",
"files": [
"plugins/importFE/actions.php"
],
"lines": 2,
"side": "after the fix"
}
},
{
"id": "wp-geo-my-wp-info-window-lfi",
"repo": "https://github.com/Fitoussi/geo-my-wp",
"vulnerable": "1dbb7b1672a45a8f88fdcd379ceb07b6cf2beceb",
"fixed": "5a768bf1c6e44ded83a65be8789f3587515665ac",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sites": [
"plugins/posts-locator/includes/gmw-posts-locator-functions.php::gmw_posts_locator_ajax_info_window_loader",
"includes/gmw-functions.php::gmw_ajax_info_window_init"
],
"site_basis": {
"plugins/posts-locator/includes/gmw-posts-locator-functions.php::gmw_posts_locator_ajax_info_window_loader": "advisory-named",
"includes/gmw-functions.php::gmw_ajax_info_window_init": "resolved-from-fix-diff"
},
"site_files_unresolved": {},
"sink_file": "plugins/posts-locator/includes/gmw-posts-locator-functions.php",
"sink_functions": [
"gmw_posts_locator_ajax_info_window_loader"
],
"sink_sha256_vulnerable": "605e777dab7653792d5ae2f2ef6c776f76d1fb1a271d0446c65f6ebf1dfb8461",
"sink_functions_missing": [],
"sink_sha256_fixed": "32166bf3941e0a5048b58c49884d513a1be9b11722245f3b7d4d072846400ec2",
"fix_changed_files": 5,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "9c394482d1509ca9222365a91117a10b19619be8",
"tip": "85bcd85541c777b6fd8580cb1915504a2a2b84cf",
"subject": "Release 4.5.5.2",
"files": [
"geo-my-wp.php",
"readme.txt"
],
"lines": 17,
"side": "before the vulnerable pin"
}
},
{
"id": "kirby-realpath-sibling-traversal",
"repo": "https://github.com/getkirby/kirby",
"vulnerable": "e90257fd7504053e13e93a1e9a5a9a5137aed966",
"fixed": "2b6fab950bc89d505ea89576bf71c8de614cc3b2",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "src/Filesystem/Dir.php",
"sink_functions": [
"realpath"
],
"sink_sha256_vulnerable": "d17a09853356ed1d2e08421e3130a4fd0e68dc6c40f3734c796d425ea135f8d3",
"sink_functions_missing": [],
"sink_sha256_fixed": "e0ec9c78eff595b6076ca396646a097a7f37a4f49212c2bc7ef4aa0d68448652",
"fix_changed_files": 4,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "bbc7c251efcbb01a72f5786af15964e596dc6c4d",
"tip": "65dfca053d78bac574c277404adbc8175b2844d8",
"subject": "fix: Improve content salt message",
"files": [
"i18n/translations/en.json",
"tests/Panel/Areas/SystemTest.php"
],
"lines": 4,
"side": "before the vulnerable pin"
}
},
{
"id": "mineadmin-appstore-identifier-traversal",
"repo": "https://github.com/mineadmin/MineAdmin",
"vulnerable": "ac0dcad616c5d304c249d1db4a6675f65cf736dc",
"fixed": "ca41902a2a5422676227e5088f4cc1dec06044f1",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "plugin/mine-admin/app-store/src/Service/Service.php",
"sink_functions": [
"download",
"install",
"unInstall"
],
"sink_sha256_vulnerable": "d06a053c26f7918a179ead518f70d79d60c2211bcbe751185b95cead8a8af22e",
"sink_functions_missing": [],
"sink_sha256_fixed": "b4f2cd97704d2eb76eae07287fd2251d5e072de1a5cba333857a7cc1fd23747c",
"fix_changed_files": 2,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "1f99cfd595f9621f4be572d970821f3509acfe33",
"tip": "15deedde8c106842d9c3584d8680e2f65adaaac1",
"subject": "Upgrade the MineAdmin to 3.2-alpha.1, and update Hyperf to the 3.2 release.",
"files": [
"composer.json",
"config/autoload/cache.php"
],
"lines": 105,
"side": "after the fix"
}
},
{
"id": "pagible-admin-proxy-dns-rebinding-ssrf",
"repo": "https://github.com/aimeos/pagible",
"vulnerable": "03f03b2f81b20b1c1625ff227235c43791ddbb33",
"fixed": "09a8205d513ec89ed22cdd7bdae0f4c181cee082",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"inferred_fix_changes_sites": {
"admin/src/Controllers/AdminController.php::proxy": true,
"core/src/Utils.php::isValidUrl": true
},
"sink_file": "admin/src/Controllers/AdminController.php",
"sink_functions": [
"proxy"
],
"sink_sha256_vulnerable": "cedb2973084f6ba825df0f4cf0d9edc1270c25e236155da0a1f7dddb8fa21340",
"sink_functions_missing": [],
"sink_sha256_fixed": "eb6a56affa39cb0aa88e894227c2dc871f6a7dc508a8121d205cb8bb3c2105a9",
"fix_changed_files": 3,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "981b8d63e52e7ba8875a9a52c1865eb3bc662154",
"tip": "d7577efd68e9c5e36436c8572bfed0df2be2c634",
"subject": "Publish related items (#210)",
"files": [
"core/src/Commands/Publish.php",
"core/src/Models/Element.php",
"core/src/Models/Page.php",
"graphql/src/GraphQL/Mutations/PubElement.php",
"graphql/src/GraphQL/Mutations/PubPage.php",
"graphql/tests/GraphqlPageTest.php",
"mcp/src/Tools/PublishElement.php",
"mcp/src/Tools/PublishPage.php"
],
"lines": 44,
"side": "after the fix"
}
},
{
"id": "wp-graphql-updatepost-publish-bypass",
"repo": "https://github.com/wp-graphql/wp-graphql",
"vulnerable": "d53b0e566f20a8df0664559cb6432d52a7cd285c",
"fixed": "55441663eaa33c3f2e05de038c8286c845916461",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sites": [
"plugins/wp-graphql/src/Mutation/PostObjectUpdate.php::mutate_and_get_payload"
],
"site_basis": {
"plugins/wp-graphql/src/Mutation/PostObjectUpdate.php::mutate_and_get_payload": "resolved-from-fix-diff"
},
"site_files_unresolved": {},
"sink_file": "plugins/wp-graphql/src/Mutation/PostObjectUpdate.php",
"sink_functions": [
"mutate_and_get_payload"
],
"sink_sha256_vulnerable": "e39fe898c11102ea902c0db1a97cfc641c676d88d54bbb12d50c67ac9f9e4447",
"sink_functions_missing": [],
"sink_sha256_fixed": "744928c8eeefdd40b1c7f48ce6ae7fb71f07ff26a95b26321725eaf4bffebdde",
"fix_changed_files": 4,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "b583bb6ec87016965085afc7ea586f114c711c68",
"tip": "cc12aed380414545a005048bfcebe4ba50e9c8f9",
"subject": "fix: point acf.wpgraphql.com links at the wpgraphql.com docs portal (#4251)",
"files": [
"plugins/wp-graphql-acf/README.md",
"plugins/wp-graphql-acf/docs/field-types/advanced-link.md",
"plugins/wp-graphql-acf/docs/field-types/button-group.md",
"plugins/wp-graphql-acf/docs/field-types/checkbox.md",
"plugins/wp-graphql-acf/docs/field-types/code-editor.md",
"plugins/wp-graphql-acf/docs/field-types/color-picker.md",
"plugins/wp-graphql-acf/docs/field-types/countries.md",
"plugins/wp-graphql-acf/docs/field-types/currencies.md",
"plugins/wp-graphql-acf/docs/field-types/date-picker.md",
"plugins/wp-graphql-acf/docs/field-types/date-range-picker.md",
"plugins/wp-graphql-acf/docs/field-types/datetimepicker.md",
"plugins/wp-graphql-acf/docs/field-types/email.md",
"plugins/wp-graphql-acf/docs/field-types/file.md",
"plugins/wp-graphql-acf/docs/field-types/gallery.md",
"plugins/wp-graphql-acf/docs/field-types/google-map.md",
"plugins/wp-graphql-acf/docs/field-types/group.md",
"plugins/wp-graphql-acf/docs/field-types/image-selector.md",
"plugins/wp-graphql-acf/docs/field-types/image-sizes.md",
"plugins/wp-graphql-acf/docs/field-types/image.md",
"plugins/wp-graphql-acf/docs/field-types/languages.md",
"plugins/wp-graphql-acf/docs/field-types/link.md",
"plugins/wp-graphql-acf/docs/field-types/radio-button.md",
"plugins/wp-graphql-acf/docs/field-types/range.md",
"plugins/wp-graphql-acf/docs/field-types/select.md",
"plugins/wp-graphql-acf/docs/field-types/taxonomies.md",
"plugins/wp-graphql-acf/docs/field-types/taxonomy-terms.md",
"plugins/wp-graphql-acf/docs/field-types/taxonomy.md",
"plugins/wp-graphql-acf/docs/field-types/text-area.md",
"plugins/wp-graphql-acf/docs/field-types/text.md",
"plugins/wp-graphql-acf/docs/field-types/time-picker.md",
"plugins/wp-graphql-acf/docs/field-types/true-false.md",
"plugins/wp-graphql-acf/docs/field-types/url.md",
"plugins/wp-graphql-acf/docs/field-types/user-roles.md",
"plugins/wp-graphql-acf/docs/field-types/user.md",
"plugins/wp-graphql-acf/docs/field-types/wysiwyg.md",
"plugins/wp-graphql-acf/readme.txt",
"plugins/wp-graphql/docs/compatibility-guide.md",
"plugins/wp-graphql/src/Admin/AdminNotices.php",
"plugins/wp-graphql/src/Admin/Extensions/Registry.php"
],
"lines": 90,
"side": "before the vulnerable pin"
}
},
{
"id": "sylius-mollie-thankyou-order-idor",
"repo": "https://github.com/Sylius/MolliePlugin",
"vulnerable": "7400f1489a4b699df97ced9499acde4faaa476d5",
"fixed": "01316b3ad3cf82e3c5ad160115d0a2cf89174e49",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sink_file": "src/Controller/Shop/PageRedirectController.php",
"sink_functions": [
"thankYouAction"
],
"sink_sha256_vulnerable": "60578839af598e79b4f3792ad31e8a2c80a360c53fd75609bc25acfb68c4fcd9",
"sink_functions_missing": [],
"sink_sha256_fixed": "950d09980903953d802456cafed8a3efb4f9f0e953cdf49482e9ab4a9ac68de6",
"fix_changed_files": 7,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "7cc02612b4c6b4403a2b0a1bcef1eb7c5382f505",
"tip": "45b01569853d16d04ffd50df281e2fd607650864",
"subject": "[UPMERGE] 3.2 -> 3.3 (#333)",
"files": [
"src/Client/MollieApiClient.php",
"src/SyliusMolliePlugin.php",
"src/Twig/Extension/MolliePluginLatestVersion.php"
],
"lines": 21,
"side": "before the vulnerable pin"
}
},
{
"id": "wp-podlove-generation-guard-poi",
"repo": "https://github.com/podlove/podlove-publisher",
"vulnerable": "4eedbd66796942ab052de2dbd6311a24cc75c4a7",
"fixed": "ea878175f2612d54888d9e2aa49ae4db93b724fd",
"problems": [],
"vulnerable_is_ancestor_of_fixed": true,
"sites": [
"lib/modules/shownotes/rest_api.php::create_link_item",
"lib/image_cache/generation_guard.php::__destruct",
"lib/model/base.php::to_array"
],
"site_basis": {
"lib/modules/shownotes/rest_api.php::create_link_item": "advisory-named",
"lib/image_cache/generation_guard.php::__destruct": "advisory-named",
"lib/model/base.php::to_array": "resolved-from-fix-diff"
},
"site_files_unresolved": {},
"inferred_fix_changes_sites": {
"lib/modules/shownotes/rest_api.php::create_link_item": true,
"lib/image_cache/generation_guard.php::__destruct": false
},
"sink_file": "lib/modules/shownotes/rest_api.php",
"sink_functions": [
"create_link_item"
],
"sink_sha256_vulnerable": "5023f7950ad450c6f0e9ca9bec474f161a509b0b3ddbf8f9f495593738dd8a05",
"sink_functions_missing": [],
"sink_sha256_fixed": "316f85861d95e63a9a7cc1f089e2b6c3f1ac61aa92092e8b279534b226fe042f",
"fix_changed_files": 9,
"fix_touches_sink_file": true,
"fix_touches_fix_site": false,
"benign_proposal": {
"base": "af187ad4bba2739839a0aa1e6cf2cabd6f9b99f5",
"tip": "50da8db0f5a85afaf27cd6199940be0403b368ec",
"subject": "fix PHP warning",
"files": [
"lib/modules/contributors/contributor_list_table.php"
],
"lines": 2,
"side": "after the fix"
}
}
]
}
+366
View File
@@ -0,0 +1,366 @@
"""Verify each reserved case by READING its source, and propose a benign change -- without scanning anything.
Freezing the independent population (pre-push-safety-net 15.1) needs, per case: both pins present and in the
right order, the vulnerable sink confirmed where the advisory says it is, the fix touching that file, and a
benign change from the same history. This does exactly that with git and file reads. It never imports or runs
the analyzer, its queries, or any project tooling: a case this script touched is still unscanned.
The benign change is a PROPOSAL for review, chosen mechanically: the first commit after the fix, on the first
-parent line, that changes source files of the case's language, does not touch the sink file, does not talk
about security in its message, and stays small. A reviewer confirms or replaces it before the status becomes
`frozen`.
Usage: python benchmarks/independent/freeze.py [--population population-vN.toml] [--cache DIR] [--only ID,...]
Writes: benchmarks/independent/freeze-vN.json (v1 by default)
"""
from __future__ import annotations
import argparse
import hashlib
import json
import re
import subprocess
import tomllib
from pathlib import Path
HERE = Path(__file__).resolve().parent
POPULATION = HERE / "population-v1.toml"
def freeze_record(population: Path) -> Path:
"""`population-v2.toml` -> `freeze-v2.json`, next to it."""
return population.with_name("freeze-" + population.stem.removeprefix("population-") + ".json")
EXTENSIONS = {
"php": (".php",),
"javascript": (".js", ".mjs", ".cjs", ".jsx"),
"typescript": (".ts", ".tsx"),
"python": (".py",),
}
# A benign change must not be a security change in disguise. The first pass let through "Restrict access to
# routine logs to owners only", "querybuilder missing the valid_filter_params check" and "add access mode and
# scopes to API tokens" -- access control and input validation, which is exactly what a benign control must not be.
SECURITY_WORDS = re.compile(
r"secur|cve|ghsa|vuln|inject|xss|csrf|ssrf|sanitiz|escap|exploit|attack|auth|permission|cors|access|restrict|harden"
r"|token|scope|valid|filter|privileg|role|owner|password|secret|redirect|sql|query|login|log in|ldap|session|ssl|tls|verif|cert|crypt|hash|cookie",
re.I,
)
MAX_BENIGN_LINES = 200
def git(repo: Path, *args: str, check: bool = True) -> str:
done = subprocess.run(["git", "-C", str(repo), *args], capture_output=True, text=True, check=False)
if check and done.returncode != 0:
raise RuntimeError(f"git {' '.join(args)}: {done.stderr.strip()[:300]}")
return done.stdout
def clone(url: str, target: Path) -> None:
if (target / ".git").is_dir():
git(target, "fetch", "--quiet", "origin")
return
target.parent.mkdir(parents=True, exist_ok=True)
subprocess.run(["git", "clone", "--quiet", "--filter=blob:none", "--no-checkout", url, str(target)], check=True)
def blob(repo: Path, commit: str, path: str) -> bytes | None:
done = subprocess.run(["git", "-C", str(repo), "show", f"{commit}:{path}"], capture_output=True, check=False)
return done.stdout if done.returncode == 0 else None
def sink_parts(sink: str) -> tuple[str, list[str]]:
"""The sink file and the function names the record claims, from `path :: A::b (...), c`."""
path, _, rest = sink.partition("::")
names = re.findall(r"[A-Za-z_][A-Za-z0-9_]*(?=\s*(?:\(|,|$|;))", rest.split("(")[0] + ",")
functions = [n for n in names if n not in {"used", "by", "source", "callers", "and", "from", "the"}]
return path.strip(), functions
PHP_FUNCTION = re.compile(r"\bfunction\s*&?\s*([A-Za-z_][A-Za-z0-9_]*)\s*\(")
def php_code(text: str) -> str:
"""`text` with comments, string contents and inline HTML blanked to spaces, newlines kept (line numbers hold).
Enough to find function bodies by brace matching; heredoc/nowdoc bodies are blanked too. Not a parser: a file
that defeats it yields no function, and the site falls back to `<global>`, which the freeze record shows."""
out, i, n, php = list(text), 0, len(text), False
def blank(start: int, end: int) -> None:
for k in range(start, min(end, n)):
if out[k] != "\n":
out[k] = " "
while i < n:
if not php:
j = text.find("<?", i)
blank(i, n if j < 0 else j)
if j < 0:
break
php, i = True, j + (5 if text.startswith("<?php", j) else 2)
continue
if text.startswith("?>", i):
php, i = False, i + 2
elif text.startswith("//", i) or text[i] == "#" and not text.startswith("#[", i):
j = min(k for k in (text.find("\n", i), text.find("?>", i), n) if k >= 0)
blank(i, j)
i = j
elif text.startswith("/*", i):
j = text.find("*/", i + 2)
j = n if j < 0 else j + 2
blank(i, j)
i = j
elif text.startswith("<<<", i):
label = re.match(r"<<<\s*['\"]?([A-Za-z_][A-Za-z0-9_]*)", text[i:])
end = re.compile(r"^\s*" + re.escape(label.group(1)) + r"\b", re.M) if label else None
found = end.search(text, text.find("\n", i) + 1) if end else None
j = found.end() if found else i + 3
blank(i, j)
i = j
elif text[i] in "'\"":
quote, j = text[i], i + 1
while j < n and text[j] != quote:
j += 2 if text[j] == "\\" else 1
blank(i + 1, j)
i = j + 1
else:
i += 1
return "".join(out)
def php_functions(text: str) -> list[tuple[str, int, int]]:
"""(name, first line, last line) of every named function or method body in a PHP file, 1-based."""
code, spans = php_code(text), []
for match in PHP_FUNCTION.finditer(code):
brace, semi = code.find("{", match.end()), code.find(";", match.end())
if brace < 0 or 0 <= semi < brace: # abstract or interface method: no body
continue
depth, k = 0, brace
while k < len(code):
depth += {"{": 1, "}": -1}.get(code[k], 0)
if depth == 0:
break
k += 1
spans.append((match.group(1), code.count("\n", 0, match.start()) + 1, code.count("\n", 0, k) + 1))
return spans
def enclosing(spans: list[tuple[str, int, int]], line: int, inserted: bool = False) -> str:
"""The innermost named function containing `line`, or `<global>`. For a pure insertion, `line` is the line the
new code follows, so a function whose closing brace is on that line does not contain it."""
inside = [(last - first, name) for name, first, last in spans if first <= line <= last and not (inserted and line == last)]
return min(inside)[1] if inside else "<global>"
def old_hunks(repo: Path, vulnerable: str, fixed: str, path: str) -> list[tuple[int, int, bool]]:
"""Vulnerable-side line ranges the fix changes in `path`, and whether each is a pure insertion (then the range
is the line it follows, at least 1)."""
diff = git(repo, "diff", "-U0", vulnerable, fixed, "--", path)
ranges = []
for start, count in re.findall(r"^@@ -(\d+)(?:,(\d+))? \+", diff, re.M):
first, size = int(start), 1 if count == "" else int(count)
ranges.append((max(first, 1), max(first, 1), True) if size == 0 else (first, first + size - 1, False))
return ranges
def changed_functions(repo: Path, vulnerable: str, fixed: str, path: str) -> list[str]:
"""The functions (or `<global>`) of `path` at the vulnerable pin that the fix's hunks touch, in hunk order."""
text = (blob(repo, vulnerable, path) or b"").decode(errors="replace")
spans, names = php_functions(text), []
for first, last, inserted in old_hunks(repo, vulnerable, fixed, path):
for line in range(first, last + 1):
name = enclosing(spans, line, inserted)
# New code added between functions (a new helper) changes no vulnerable-side code: no site.
if name not in names and not (inserted and name == "<global>"):
names.append(name)
return names
def resolve_sites(repo: Path, case: dict[str, object], changed: list[str]) -> dict[str, object]:
"""Decision 2 of population v3: a file the advisory names without a function gets, as its declared site, the
function(s) the fix's diff of that file changes. Advisory-named sites are kept and marked as such."""
declared = [str(s) for s in case.get("sites", [])] # type: ignore[union-attr]
recorded = case.get("site_basis") or {} # a frozen population keeps each site's basis
basis = {site: recorded.get(site, "advisory-named") for site in declared} # type: ignore[union-attr]
unresolved = {}
named_files = {site.partition("::")[0] for site in declared}
for path in [str(f) for f in case.get("site_files", [])]: # type: ignore[union-attr]
if path in named_files:
continue
if path not in changed:
unresolved[path] = "the fix does not change this file"
continue
for name in changed_functions(repo, str(case["vulnerable"]), str(case["fixed"]), path):
basis.setdefault(f"{path}::{name}", "resolved-from-fix-diff")
return {"sites": list(basis), "site_basis": basis, "site_files_unresolved": unresolved}
def fix_changes_sites(repo: Path, case: dict[str, object], sites: list[str]) -> dict[str, bool]:
"""Per declared site: does the fix's diff change a line inside that function (or the file, for `<global>`)?"""
result = {}
for site in sites:
path, _, name = site.partition("::")
hunks = old_hunks(repo, str(case["vulnerable"]), str(case["fixed"]), path)
if name == "<global>" or not hunks:
result[site] = bool(hunks)
continue
spans = php_functions((blob(repo, str(case["vulnerable"]), path) or b"").decode(errors="replace"))
result[site] = any(enclosing(spans, line, inserted) == name for first, last, inserted in hunks for line in range(first, last + 1))
return result
def benign(
repo: Path, fixed: str, sink_file: str, extensions: tuple[str, ...], strict: bool = False, vulnerable: str = ""
) -> dict[str, object] | None:
"""The first acceptable change after the fix (see the module docstring).
`strict` (a single-language population, v3) tightens the proposal: the change is measured against its first
parent, so a merge counts with everything it brings in, and every message it brings in is checked for security
words (a branch-sync merge can carry the fix itself); it must change a file of the population's language; and
when no commit after the fix qualifies, the history before the vulnerable pin is searched, newest first."""
head = git(repo, "rev-parse", "origin/HEAD", check=False).strip() or git(repo, "rev-parse", "HEAD").strip()
span = [f"{fixed}..{head}"]
if subprocess.run(["git", "-C", str(repo), "merge-base", "--is-ancestor", fixed, head], check=False).returncode:
# A fix on a maintenance branch (population v3: kirby 4.x, roundcube 1.6): the default branch's commits
# after the fix date, not its whole history outside the branch.
span = [f"--since={git(repo, 'log', '-1', '--format=%cI', fixed).strip()}", head]
commits = git(repo, "rev-list", "--first-parent", "--reverse", *span, check=False).split()[:400]
after = set(commits)
if strict and vulnerable:
commits += git(repo, "rev-list", "--first-parent", "--max-count=400", f"{vulnerable}^", check=False).split()
for commit in commits:
parent = git(repo, "rev-parse", f"{commit}^", check=False).strip()
if not parent:
continue
if strict:
message = git(repo, "log", "--format=%s%n%b", f"{parent}..{commit}")
stat = git(repo, "diff", "--numstat", parent, commit).splitlines()
else:
message = git(repo, "log", "-1", "--format=%s%n%b", commit)
stat = git(repo, "show", "--numstat", "--format=", commit).splitlines()
if SECURITY_WORDS.search(message):
continue
files = [line.split("\t") for line in stat if line.count("\t") == 2]
if not files or any(f[2] == sink_file for f in files):
continue
source = [f for f in files if f[2].endswith(extensions)]
lines = sum(int(a) + int(b) for a, b, _ in files if a.isdigit() and b.isdigit())
if source and 0 < lines <= MAX_BENIGN_LINES:
subject = git(repo, "log", "-1", "--format=%s", commit).strip()
side = "after the fix" if commit in after else "before the vulnerable pin"
return {"base": parent, "tip": commit, "subject": subject[:120], "files": [f[2] for f in files], "lines": lines, "side": side}
return None
def verify(case: dict[str, object], cache: Path) -> dict[str, object]:
url, vulnerable, fixed = str(case["repo"]), str(case["vulnerable"]), str(case["fixed"])
repo = cache / str(case["id"])
clone(url, repo)
record: dict[str, object] = {"id": case["id"], "repo": url, "vulnerable": vulnerable, "fixed": fixed, "problems": []}
problems: list[str] = record["problems"] # type: ignore[assignment]
for pin in (vulnerable, fixed):
if git(repo, "cat-file", "-t", pin, check=False).strip() != "commit":
problems.append(f"pin {pin[:12]} is not a commit in the repository")
if problems:
return record
ordered = subprocess.run(["git", "-C", str(repo), "merge-base", "--is-ancestor", vulnerable, fixed], check=False).returncode == 0
record["vulnerable_is_ancestor_of_fixed"] = ordered
if not ordered:
problems.append("the vulnerable pin is not an ancestor of the fixed pin")
changed = git(repo, "diff", "--name-only", vulnerable, fixed).splitlines()
sites = [str(site) for site in case.get("sites", [])] # type: ignore[union-attr]
if case.get("site_files"):
# Population v3: files the advisory names without a function are resolved from the fix's diff.
record.update(resolve_sites(repo, case, changed))
sites = record["sites"] # type: ignore[assignment]
if not sites:
problems.append("no declared site: the advisory names no function and the fix changes no named file")
if case.get("fix_named_by_advisory") is False:
# An inferred fix must change the declared sink file and at least one declared site function.
basis = record.get("site_basis") or {}
named = [s for s in sites if basis.get(s, "advisory-named") == "advisory-named"] # type: ignore[union-attr]
touched = fix_changes_sites(repo, case, named)
record["inferred_fix_changes_sites"] = touched
if not any(touched.values()):
problems.append("the inferred fix changes none of the declared site functions: drop the case")
if sites:
# Declared `path::function` sites (population v2) are what the verification reads; the first names the
# sink file, and every function declared in that file must appear in it.
sink_file = sites[0].partition("::")[0]
functions = [site.partition("::")[2] for site in sites if site.partition("::")[0] == sink_file and not site.endswith("::<global>")]
else:
sink_file, functions = sink_parts(str(case["sink"]))
record["sink_file"], record["sink_functions"] = sink_file, functions
before, after = blob(repo, vulnerable, sink_file), blob(repo, fixed, sink_file)
if before is None:
problems.append(f"sink file {sink_file} is absent at the vulnerable pin")
else:
record["sink_sha256_vulnerable"] = hashlib.sha256(before).hexdigest()
text = before.decode(errors="replace")
evidence = str(case.get("sink_evidence", ""))
if evidence:
record["sink_evidence_found"] = evidence in text
if evidence not in text:
problems.append(f"sink evidence {evidence!r} is absent from {sink_file} at the vulnerable pin")
else:
missing = [f for f in functions if f.split("::")[-1] not in text]
record["sink_functions_missing"] = missing
if functions and len(missing) == len(functions):
problems.append(f"none of {functions} appears in {sink_file} at the vulnerable pin")
if after is not None:
record["sink_sha256_fixed"] = hashlib.sha256(after).hexdigest()
record["fix_changed_files"] = len(changed)
record["fix_touches_sink_file"] = sink_file in changed
fix_site = str(case.get("fix_site", "")).partition("::")[0].strip()
record["fix_touches_fix_site"] = bool(fix_site) and fix_site in changed
if sink_file not in changed and not record["fix_touches_fix_site"]:
problems.append(f"the fix changes neither {sink_file} nor a declared fix_site")
languages = [str(case["language"]), *[str(x) for x in case.get("also", [])]] # type: ignore[union-attr]
extensions = tuple(e for lang in languages for e in EXTENSIONS.get(lang, ()))
strict = bool(case.get("_population_language"))
if strict:
extensions = EXTENSIONS.get(str(case["_population_language"]), extensions)
record["benign_proposal"] = benign(repo, fixed, sink_file, extensions, strict, vulnerable)
if record["benign_proposal"] is None and not case.get("benign"):
problems.append("no benign change found automatically; choose one by hand")
return record
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument("--cache", type=Path, default=Path.home() / ".cache" / "openultrasast" / "independent")
parser.add_argument("--only", default="")
parser.add_argument("--population", type=Path, default=POPULATION, help="a population-vN.toml in this directory")
args = parser.parse_args()
population = args.population if args.population.is_absolute() else HERE / args.population.name
data = tomllib.loads(population.read_text())
cases = data["case"]
for case in cases:
# A single-language population (v3) gets the strict benign proposal (see `benign`).
case["_population_language"] = data.get("language", "")
only = {c.strip() for c in args.only.split(",") if c.strip()}
records = []
for case in cases:
if only and case["id"] not in only:
continue
try:
record = verify(case, args.cache)
except (RuntimeError, subprocess.CalledProcessError) as error:
record = {"id": case["id"], "problems": [f"could not verify: {error}"]}
records.append(record)
print(
json.dumps(
{"id": record["id"], "problems": record["problems"], "benign": (record.get("benign_proposal") or {}).get("subject")}
),
flush=True,
)
freeze_record(population).write_text(
json.dumps({"schema_version": 1, "population": population.name, "cases": records}, indent=2) + "\n"
)
return 0 if all(not r["problems"] for r in records) else 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,55 @@
# Hunter experiment on population v2 (pre-registered)
Written and committed on 2026-09-28, BEFORE the tool hunter is run on any v2 pin. It is the deciding experiment
of the strategy review (`.kiro/specs/pre-push-safety-net/strategy-review-2026-09-28.md`): does a model with
repository tools find what the static engine did not (v2: recall 0/17, precision 0/17)?
## What is under test
- `tool_hunter.run_tool_hunter` as it stands: default system prompt, default tools (`read_file`, `grep_repo`,
`find_refs`), `DEFAULT_MAX_STEPS` (4), the final-answer turn. Its prompt, tools and parsing are unchanged since
the 2026-09-06 measurement; the only changes today are transport (cf57995 and this commit: the client reaches
DeepSeek, and the endpoint is priced so the budget can see spending).
- Model: `deepseek-flash` (DeepSeek, the project's chat provider), through `resolve_hunter_client()`.
- Runner: `benchmarks/independent/hunt.py` at the commit that introduces this file.
## Scope handed to the model, per pin
The regions the static scan's discovery and ranker produce for the case's family (`regions_for` over
`analyze_entry_points`, the same as `finding_dump.py`), sorted by rank, top 20, as hunter hotspots in four groups
of five: one bounded hunt per group. Nothing else: not the fix diff, not the advisory, not the case's declared
sites, not the case id.
## Pins
Protocol v2's six per case: vulnerable a/b, fixed a/b, benign base and tip. Each is a `git archive` of the pin.
## Turning a hunter answer into a scoreable finding
- `site` = `path:line:function`, with the function the model reported, or empty.
- `family` = the model's own family and title mapped by the fixed keyword table in `hunt.py` (`FAMILY_WORDS`,
checked in that order: untrusted_destination, config_secrets, injection; anything else is `other`).
## Scoring
Exactly protocol v2 (`protocol-v2.md`), via `evaluate.score_v2` over the hunter's results: the same matching
(v1's changed-line rule or a declared `path::function` site), detection in BOTH vulnerable runs, a fixed-side
alert in EITHER fixed run, benign alerts, stability, and the precision sample (every third stable
vulnerable-pin finding) adjudicated to v1's standard. A finding with no function can match only by the
changed-line rule. A pin whose four hunts all fail is an instrument failure, reported as such.
"Questions" for the hunter are its hunts (4 per pin), so completion is hunts completed / hunts run; it is not
comparable to the engine's question completion and is reported only as an instrument check.
## Cost
Hard ceiling $10 of recorded model spend, enforced by the runner; a pilot on a v1 development case
(mongo-express, 4 hunts) cost $0.031 and 57 s. If the ceiling stops the run, the unscanned pins are reported
as not run.
## What may not happen
- No change to the hunter's prompt, tools, step budget, parsing, scope rule or family table before this result
is committed.
- Whatever is learned here is a finding about the hunter as it is; tuning it on v2 afterwards is a retune and
needs a new population to qualify.
@@ -0,0 +1,92 @@
{
"measurement": "ai-service-plane first increment (Requirement 6): repo-facts + verify (2 passes) + agree for the 46-candidate validation set, executed by the ax-backed reconciler on google/ax + Agent Substrate (kind, gVisor) on the 7 GB host",
"date": "2026-09-29",
"run": {
"name": "validation-46",
"manifest": "plane/runs/validation-46.yaml",
"tasks": 60,
"workers": 2,
"status": "done",
"results": "~/ousast-results/plane/validation-46/",
"model": "deepseek-flash",
"runner_image": "localhost:5001/ousast-runner@sha256:200e335877b52d68ea0bc53b4c9ba49ef2ee19e8a5ec2c874148b969123ef7ab"
},
"reference": "benchmarks/independent/verifier-batched-check-2026-09-29.json (script pipeline, same cases, candidates, pins and recorded triage)",
"gate": {
"declared_sites_agreed": {
"plane": 14,
"reference": 16,
"of": 20,
"required": ">= 16",
"met": false
},
"cost_per_candidate_with_recorded_triage_usd": {
"plane": 0.0194,
"plane_over_46": 0.0181,
"reference": 0.022,
"required": "< 0.022",
"met": true
},
"result": "NOT MET (recall criterion)"
},
"detail": {
"agreed_candidates": 23,
"disputed_candidates": 10,
"candidates_after_recorded_triage": 43,
"plane_usd": 0.7471,
"recorded_triage_usd": 0.0872,
"usd_with_triage": 0.8343,
"reference_usd": 1.0123,
"calls": 357,
"prompt_tokens": 3820395,
"cache_hit_tokens": 2385152,
"output_tokens": 62257,
"turns_per_hunt": 5.37,
"hunts": 54,
"per_case_sites_agreed_vs_reference": {
"budibase-mongo-template-nosqli": "1/2 vs 2/2 (read: flagged by neither plane pass; the reference agreed on it)",
"froxlor-admin-ip-second-order-sqli": "0/3 vs 2/3 (all three flagged by pass b only)",
"unleash-addon-webhook-ssrf": "1/1 vs 0/1 (reference found it in one pass only)",
"fuxa-tdengine-escape-sqli": "0/1 vs 0/1 (getDaqValue: neither plane pass; the reference found it in one pass)",
"winml-cli-serve-cors": "0/1 vs 0/1 (triage drops the only candidate)",
"all_other_cases": "equal"
},
"declared_sites_flagged_in_either_pass": {
"plane": 17,
"reference": 19,
"of": 20,
"definition": "declared sites whose candidate pass a or pass b flagged; reference = 16 agreed + fuxa, froxlor validateIpAddresses, unleash found in one pass"
},
"declared_sites_covered_by_candidates": {
"plane": 19,
"of": 20,
"definition": "agree's metric declared_sites_matched: candidates whose site falls on a declared site, flagged or not (winml has no candidate)"
}
},
"reading": "Detection is lower, not only agreement: plane passes flagged 17 of 20 declared sites in at least one pass against the reference's 19 (budibase read and fuxa getDaqValue were flagged by neither plane pass), and two-pass agreement moved the three froxlor sites (pass b only) into 'disputed'; unleash moved the other way. Cost fell 17% (62% of prompt tokens were cache hits). By the pre-registered rule the increment is NOT MET and the design is revisited before task 9.",
"dev_token_report": {
"transcript": "the whole development session (c09e30b2), not the increment alone",
"unique_chars": "7.3M (~1.81M tokens)",
"tool_use_input_pct": 45.4,
"tool_result_pct": 40.1,
"tool_io_pct": 85.5,
"previous_tool_io_pct": 90,
"tool_calls": 2701
},
"live_defects_found_by_running_on_ax": [
"no Substrate WorkerPool",
"tag-pinned images rejected",
"ax CLI/server skew",
"unknown-field rejection (atespace, commit)",
"server-set fields in AX_TASK_YAML",
"tasks created Suspended; no Completed phase",
"runner exited after the command",
"golden-snapshot boot ran the command",
"egress gateway not deployed; deny-by-default per-actor policy",
"gateway dials the address the actor dialled",
"workspace clone before the actor was running",
"32 KB env limit on inlined inputs",
"63-byte template name limit"
],
"correction": "2026-09-30: the first version of this record read agree's declared_sites_matched (coverage by candidates, 19) as 'found in either pass' and concluded detection was unchanged; recounted from agreed.json rows it is 17 vs the reference's 19."
}
@@ -0,0 +1,65 @@
{
"measurement": "ai-service-plane increment re-measured after the design revisit (Requirement 6.4): passes a and b as recorded in plane-increment-1.json, plus a tie-break pass c on the 10 candidates a and b disputed; agreement 2-of-3. Executed on google/ax by rerunning the same Run (the 60 done tasks were reused, 30 new tasks ran).",
"date": "2026-09-30",
"run": {
"name": "validation-46",
"tasks": 90,
"status": "done",
"runner_image": "localhost:5001/ousast-runner@sha256:fb9a743027518976659fd11505e58cc3fc84fb42e07627196e13b76ac01d0617"
},
"previous": "benchmarks/independent/plane-increment-1.json",
"gate": {
"declared_sites_agreed": {
"plane": 16,
"reference": 16,
"of": 20,
"required": ">= 16",
"met": true
},
"cost_per_candidate_with_recorded_triage_usd": {
"plane_over_46": 0.0205,
"plane_over_43_after_triage": 0.02195,
"reference_over_46": 0.022,
"required": "< 0.022",
"met": true,
"note": "the reference divides by the 46 candidates before triage; on that basis the plane is 7% cheaper, on the 43 after triage it is 0.2% under"
},
"result": "MET"
},
"detail": {
"tiebreak": {
"asked": 10,
"flipped_to_agreed": 5,
"declared_sites_recovered": [
"froxlor listing",
"froxlor listingCount"
],
"declared_site_still_disputed": "froxlor validateIpAddresses (b only)",
"non_declared_flipped_to_agreed": [
"pgadmin enable_disable",
"pgadmin get_job_args_value",
"pgadmin run_job"
]
},
"agreed_candidates": {
"two_pass": 23,
"two_of_three": 28
},
"declared_sites_flagged_in_any_pass": {
"plane": 17,
"reference": 19,
"of": 20,
"missed_by_every_plane_pass": [
"budibase read",
"fuxa getDaqValue"
]
},
"usd": {
"plane": 0.8567,
"recorded_triage": 0.0872,
"with_triage": 0.9439,
"reference": 1.0123
}
},
"reading": "The plane on ax matches the script pipeline's pre-registered recall (16/20 agreed) at 7% lower cost, so Requirement 6.2 is met after the 6.4 revisit. Two caveats travel with the result: (1) the tie-break is a design change chosen after seeing the first measurement on the same validation set, so this compares plane with script like for like and qualifies nothing on an independent population; (2) the plane's passes flag fewer declared sites than the reference's (17 vs 19 in any pass), and 2-of-3 also promoted three non-declared pgadmin candidates, whose truth is unassessed (an earlier pgadmin spot check found 3 of 5 findings truly privileged). Precision is not part of this gate and remains the open problem."
}
+189
View File
@@ -0,0 +1,189 @@
# Independent evaluation population v1 (pre-push-safety-net task 15.1, milestone M4).
#
# RESERVED, NOT SCANNED. Every case below was chosen from public advisories and fix diffs by web research alone:
# no repository was cloned, no scanner or project tooling was run on any of them, and none appears among the
# 262 repositories referenced anywhere in this project's benchmarks. They exist to answer the question the
# development subjects can no longer answer honestly -- PMPro, WP Statistics, MW WP Form, NodeGoat and VAmPI all
# shaped the rules they would be measured by.
#
# Rules for this file:
# * Do not scan, trace, probe or census any of these repositories until the population is FROZEN (status
# below) and the freeze commit is recorded. A scan before that makes the case a development subject.
# * Freezing requires, per case: the vulnerable and fixed pins verified in the checkout, the sink confirmed at
# the vulnerable pin by reading source, and a BENIGN change chosen from the same history without scanning.
# * No rule, fact or query change may cite anything learned from these cases before their evaluation is
# recorded; after it, a retune needs a new untouched population (release-milestones M4).
#
# Selected by the maintainer on 2026-09-24 from 15 researched candidates (menu selection); rejected alternates
# and the research notes are in the research log referenced below.
schema_version = 1
status = "frozen"
frozen_on = "2026-09-24"
freeze_record = "freeze-v1.json"
reserved_on = "2026-09-24"
selected_by = "maintainer"
research = "web research of GitHub advisories and fix diffs via the GitHub API; no clone, no scan"
multi_language_members = ["wp-ultimate-member-sqli", "wp-learnpress-sqli", "phpmyfaq-captcha-ua-sqli", "yeswiki-bazar-form-sqli"]
also_reserved = ["../push/ghost-reserved.toml"]
[[case]]
id = "wp-ultimate-member-sqli"
repo = "https://github.com/ultimatemember/ultimatemember"
language = "php"
also = ["javascript"]
family = "injection"
advisory = ["CVE-2024-1071", "GHSA-mh7h-gpmx-fggj"]
vulnerable = "a49b3061ded1630e4f6860b4ebece5ccf0a493d2"
fixed = "476de92f4c9aaa56c3e2f5527b242ce0476f4655"
sink = "includes/core/class-member-directory-meta.php :: Member_Directory_Meta::ajax_get_members (ORDER BY built from esc_sql'd sort values)"
privilege = "unauthenticated (only with the custom usermeta table option on)"
license = "GPL"
notes = "Release pair 2.8.2 (e1885a73) -> 2.8.3 (a6a7d13a) spans 94 commits; the commit pair is the clean one. c61008bd is a larger follow-up refactor of the same file."
benign = { base = "13151ef558eb6998720cb1ee5f78b5f17535c16c", tip = "9130a3226ffec45569a2872ffac338200bd1bfcb", subject = "- wpcs;" }
[[case]]
id = "wp-learnpress-sqli"
repo = "https://github.com/LearnPress/learnpress"
language = "php"
also = ["javascript"]
family = "injection"
advisory = ["CVE-2024-8522", "GHSA-3w3r-r6g6-w8x5", "CVE-2024-8529", "GHSA-98wp-w76v-f75p"]
vulnerable = "73b3cbceef8669056e9da5e2f2c78944eb139af5"
fixed = "aaf42fe0c6bf05dcf2ff125750b7bd836e76fa3c"
sink = "inc/Databases/class-lp-db.php :: LP_Database::execute (implode of only_fields into SELECT); source REST get_courses c_only_fields/c_fields"
privilege = "unauthenticated (REST /wp-json/learnpress/v1/courses)"
license = "GPL"
fix_site = "inc/Models/Courses.php :: handle_params_for_query_courses (fields quoted as identifiers); the REST controller stops assigning c_only_fields directly"
notes = "The sink is where the SQL runs (class-lp-db.php:607 implode of only_fields); the fix is on the source side and does not change that file. It also touches assets/src/js/loadAJAX.js: the change itself is multi-language."
benign = { base = "aaf42fe0c6bf05dcf2ff125750b7bd836e76fa3c", tip = "179ed7760e6ae5fa46fbe9c5f1324278a42ece04", subject = "= 4.2.7.1 = ~ Added: lpOnElementReady function js" }
[[case]]
id = "phpmyfaq-captcha-ua-sqli"
repo = "https://github.com/thorsten/phpMyFAQ"
language = "php"
also = ["typescript", "javascript"]
family = "injection"
advisory = ["CVE-2026-46364", "GHSA-289f-fq7w-6q2w"]
vulnerable = "11583abcf4787543d00c25d6c7926af6f509faf0"
fixed = "545bdffb11244a4741cd29ac909a849c9a6a2e53"
sink = "phpmyfaq/src/phpMyFAQ/Captcha/BuiltinCaptcha.php :: garbageCollector, saveCaptcha (User-Agent and client IP in sprintf SQL)"
privilege = "unauthenticated (GET /api/captcha)"
license = "MPL-2.0"
notes = "The advisory's commit b9f25109 is the vulnerable master commit the reporter tested, not the fix. Release pair 4.1.1 (f791858b) -> 4.1.2 (66792f25)."
benign = { base = "bf781788ab6b5c2967a09d66216d7580778a2b5a", tip = "a4601ae5f01761dc0bb91b8c0c3901be91987ba3", subject = "build: updated version strings to 4.2" }
[[case]]
id = "yeswiki-bazar-form-sqli"
repo = "https://github.com/YesWiki/yeswiki"
language = "php"
also = ["javascript"]
family = "injection"
advisory = ["CVE-2026-46670", "GHSA-jwvv-qr7q-cv8j"]
vulnerable = "6f222915df36fca628fc83325cb0feed8dc2aca0"
fixed = "fe7244bffcbeffd42777496ebc2a6f6118cc2388"
sink = "tools/bazar/services/FormManager.php :: FormManager::create (bn_id_nature unquoted in INSERT); source FormController::displayAll imported-form JSON"
privilege = "unauthenticated"
license = "AGPL-3.0"
notes = "Four-line fix: intval plus a Guard::isAllowed check. Default branch doryphore-dev."
benign = { base = "03f27fdde5f89f90de76cd3f2160cff82e7bffac", tip = "70d240c0b43f7e7b0bac55fc0431fb4c2d88b49c", subject = "fix(contact): send shared page by mail" }
[[case]]
id = "signalk-remote-conn-ssrf"
repo = "https://github.com/SignalK/signalk-server"
language = "typescript"
family = "untrusted_destination"
advisory = ["CVE-2026-55591", "GHSA-q59x-jc9f-gfqf"]
vulnerable = "1c479cc0d2d26d14ed74416cff36fa7336c7bf9e"
fixed = "2816b0dda1bdde893d85c969a9c5950e7825eee7"
sink = "src/serverroutes.ts :: makeRemoteRequest (req.body.host/port to http(s).request)"
privilege = "unauthenticated by default (security not configured); admin otherwise"
license = "Apache-2.0"
notes = "Fix adds a schema, an ssrfGuard module and requestId validation (~450 added lines, mostly guard code and tests)."
benign = { base = "8af993d45c898edfe28f820f1726b6fc33b27cad", tip = "07416b79601c5d901f671f70a94bd57bed90bad8", subject = "fix: allow multiple deltaHandler next calls (#2818)" }
[[case]]
id = "saltcorn-sync-sqli"
repo = "https://github.com/saltcorn/saltcorn"
language = "javascript"
also = ["typescript"]
family = "injection"
advisory = ["CVE-2026-41478", "GHSA-jp74-mfrx-3qvh"]
vulnerable = "e7e09254a3c69dc6582eda9e5b5769935db108af"
fixed = "2b6c8a2eab88955ea6b12b1b4e5c2e613faefef5"
sink = "packages/server/routes/sync.js :: getSyncRows, getDelRows (maxLoadedId and timestamps in SQL template literals)"
privilege = "authenticated low-privilege user"
license = "MIT"
notes = "Large multi-package repository; the fix is on the main line (v1.4.6 is a separate backport), so use the commit pair."
benign = { base = "5c619403c40447bf56ce662570d3a3ffbf781019", tip = "df114765fd36d280395b2129f1703fa98c712d06", subject = "updated at in trigger.findAllWithTableName" }
[[case]]
id = "mongo-express-bson-rce"
repo = "https://github.com/mongo-express/mongo-express"
language = "javascript"
family = "injection"
advisory = ["CVE-2019-10758", "GHSA-h47j-hc6x-h3qq"]
vulnerable = "ea02b364d43f179f191fc91fb9962efdb0843a8d"
fixed = "d8c9bda46a204ecba1d35558452685cd0674e6f2"
sink = "lib/bson.js :: toBSON (vm.runInNewContext over request text); callers lib/routes/collection.js, lib/routes/document.js"
privilege = "authenticated (HTTP basic auth, default admin:pass)"
license = "MIT"
notes = "Code injection, not SQL. The advisory's first reference 7d365141 is only the v0.54.0 release commit."
benign = { base = "9550ae2529258b47a2b637d8e64e69a0d4d38ad8", tip = "5b9b34f8553d004ceb7b666bbf566b406a2a3153", subject = "Merge pull request #538 from damien-malescot/master" }
[[case]]
id = "mockoon-admin-api-cors"
repo = "https://github.com/mockoon/mockoon"
language = "typescript"
family = "config_secrets"
advisory = ["CVE-2026-59148", "GHSA-rqx4-3f6q-3x2v"]
vulnerable = "b42bdfb7f82e83f0e81bea8e6fe41adf5ec82585"
fixed = "c420b5a56918475b8663977b51e5f986e45b3299"
sink = "packages/commons-server/src/libs/server/admin-api.ts :: admin-API middleware (Access-Control-Allow-Origin: *)"
sink_evidence = "'Access-Control-Allow-Origin': '*'"
privilege = "unauthenticated"
license = "MIT"
notes = "Weaker fit: the fix commit touches 34 files; only one hunk in admin-api.ts is the CORS fix."
benign = { base = "de69f5e30f89e17918855d147f4e8c055d5b2ba9", tip = "895961070b55a14d290ad50232ab9c6d800c3cad", subject = "Fix serverless lib returning a 404 when no body is present (#2274)" }
[[case]]
id = "alerta-query-sqli"
repo = "https://github.com/alerta/alerta"
language = "python"
family = "injection"
advisory = ["CVE-2026-34400", "GHSA-8prr-286p-4w7j"]
vulnerable = "71c098d8bb8091e76edaefeef4662eb77c753ce7"
fixed = "fdd52cd1abad8d02d1dfb8ecdcdbb43b6af3b883"
sink = "alerta/database/backends/postgres/queryparser.py :: SearchTerm.__repr__ (f-string WHERE) used by utils.py QueryBuilder.from_params"
privilege = "API user; possibly unauthenticated if AUTH_REQUIRED defaults to False (unverified)"
license = "Apache-2.0"
notes = "The advisory's aeba85a3 introduced the parser; the fix is fdd52cd1."
benign = { base = "b7111d23ae9aefb5064ef107976f62e3836217b6", tip = "71c098d8bb8091e76edaefeef4662eb77c753ce7", subject = "fix(deps): replace pkg_resources with importlib.metadata (#2041)" }
[[case]]
id = "wger-next-open-redirect"
repo = "https://github.com/wger-project/wger"
language = "python"
family = "untrusted_destination"
advisory = ["GHSA-vqv8-j3mj-wjxj"]
vulnerable = "d30e85b23fb1307ac6579e9e3ae393c50470cea7"
fixed = "0ba371bbb50cd5668820f240995405e1b2483384"
sink = "wger/core/views/user.py :: trainer_login; wger/core/views/languages.py :: use_browser_language (request.GET next redirected unchecked)"
privilege = "use_browser_language: unauthenticated; trainer_login: authenticated trainer"
license = "AGPL-3.0"
notes = "No CVE. A later commit cfa27a7f fixes the same pattern in core/middleware.py; not part of this fix."
benign = { base = "47d646c24178783eb6787ecf26e79dc8f4a394c6", tip = "c1144533bf9bca487ff905d925b567151d89c1aa", subject = "Formatting" }
[[case]]
id = "opencve-notification-ssrf"
repo = "https://github.com/opencve/opencve"
language = "python"
family = "untrusted_destination"
advisory = ["CVE-2026-62282", "GHSA-ch3g-4xvr-674q"]
vulnerable = "928bbadbb1672051bcfc2a6289a9e6a8f029c106"
fixed = "0af7f894debc95c67fa53e7ba5a6cdecb10c34f0"
sink = "web/projects/notifications.py :: _http_post_json (webhook and Slack test POST to a user-supplied URL)"
privilege = "authenticated (can configure project notifications)"
license = "BUSL-1.1 (verified 2026-10-02: github.com/opencve/opencve LICENSE -- Business Source License 1.1, not OSI; non-production use permitted by the Additional Use Grant; change date 2030-08-14 to Apache-2.0). Published by maintainer decision 2026-10-02: the record holds metadata only (repository, pins, detection), no OpenCVE code."
notes = "Selected by the maintainer despite the licence question; results must not be redistributed until it is checked."
benign = { base = "0c8101082ba20628b28fcec9c4611745d32202c2", tip = "965dc348e225308edabdfef3ecb76f7d73a0c162", subject = "fix: unlink automation executions before cleaning expired reports" }
+338
View File
@@ -0,0 +1,338 @@
# Independent evaluation population v2 (milestone M4). Reserved 2026-09-26.
#
# RESERVED, NOT SCANNED. Chosen by web research alone (research-2026-09-26.md): advisories taken newest-first per
# language under fixed criteria, never by how easy a case looks for this analyzer; 261 advisories examined, 224
# rejected with reasons. No repository was cloned, no scanner or project tooling run on any of them, and none is
# referenced anywhere in this project's development tree (checked against every github.com URL under benchmarks/,
# src/, tests/ and .kiro/, which includes the v1 population).
#
# Why a v2: v1 (2026-09-24) was evaluated and then used to retune the analyzer (file-owned gaps, Node vm sinks,
# Django sources, quoted-only SQL escapes, guards, .gitignore semantics). A retune cannot be qualified on the
# population that informed it.
#
# Rules for this file are v1's: no scan, trace, probe or census of these repositories before the population is
# FROZEN and protocol-v2.md is committed; no rule, fact or query change may cite these cases before their
# evaluation is recorded.
#
# `sites` are the sink or execution locations the ADVISORY names, declared here before any scan. Protocol v2
# credits a finding at one of them as well as one matching the fix's changed lines.
#
# Selected by the maintainer on 2026-09-26 ("balanced" rule-based selection, 17 cases -- the menu called it 16; the
# list it showed held 17). Licences marked "verify" must be checked before any result is published.
schema_version = 1
status = "frozen"
frozen_on = "2026-09-26"
freeze_record = "freeze-v2.json"
reserved_on = "2026-09-26"
selected_by = "maintainer"
research = "research-2026-09-26.md"
protocol = "protocol-v2.md"
multi_language_members = ["wp-gopay-log-filter-sqli", "wp-user-registration-login-redirect", "wp-directorist-avatar-ssrf", "wp-groundhogg-confirm-redirect"]
also_reserved = ["population-v1.toml", "../push/ghost-reserved.toml"]
[[case]]
id = "pgadmin-maintenance-sqli"
repo = "https://github.com/pgadmin-org/pgadmin4"
language = "python"
also = ["javascript"]
family = "injection"
advisory = ["CVE-2026-7815", "GHSA-hp84-p2gq-6fvr"]
vulnerable = "d112dc3b962218efc3613fa727d54f0fc5b1d527"
fixed = "cf53953d9ad66c182a08e662c1ba29c5831b486b"
ancestry = "parent"
sink = "web/pgadmin/tools/maintenance/templates/maintenance/sql/command.sql (string concatenation of vacuum_index_cleanup, vacuum_parallel, buffer_usage_limit, reindex_tablespace) rendered in web/pgadmin/tools/maintenance/__init__.py :: create_maintenance_job() → render_template(...) → BatchProcess(cmd=psql, args=[..., '--command', query])"
source = "JSON body of POST /maintenance/job/<sid>/<did>"
sites = ["web/pgadmin/tools/maintenance/__init__.py::create_maintenance_job"]
privilege = "authenticated (tools_maintenance permission)"
license = "PostgreSQL (verified 2026-10-02: LICENSE, d112dc3b962218efc3613fa727d54f0fc5b1d527)"
confidence = "high"
benign = { base = "e7eaef0dc002c254e4dd15705df800c448e8eb4c", tip = "a50a553b0a7e529e5d87f52bdc6d3a5ee3e8b9e8", subject = "chore: feature tests use sys.executable; sync yarn.lock to package.json", how = "automatic proposal, reviewed" }
[[case]]
id = "qwed-sympy-eval"
repo = "https://github.com/QWED-AI/qwed-verification"
language = "python"
also = []
family = "injection"
advisory = ["CVE-2026-55585", "GHSA-q27q-98j4-9pfv"]
vulnerable = "06801f68cbf9c9b5e54cc17f2a61cb2792da7a26"
fixed = "6066b68c0c4f4cc2c3771824822aaa864d082ef8"
ancestry = "first parent (fixed is the merge of PR #200; the PR commit `dc9d4db72ca4b4ae3f96d0e6a0c27a9e38a06f61` is a direct child of the vulnerable SHA)"
sink = "src/qwed_new/api/main.py :: verify_math() → sympy.parsing.sympy_parser.parse_expr(expression_normalized) (also left_str/right_str); second path src/qwed_new/core/batch.py :: _verify_item() → parse_expr(item.query)"
source = "POST /verify/math JSON expression; POST /verify/batch item query"
sites = ["src/qwed_new/api/main.py::verify_math", "src/qwed_new/core/batch.py::_verify_item"]
privilege = "authenticated tenant (API key; self-signup /auth/signup is on by default)"
license = "Apache-2.0"
confidence = "high"
benign = { base = "5214f2c558641f4d02d3d75abf039f164e393131", tip = "637a880a8333875543090894fc13bf092783d90e", subject = "feat: emit VC v1.0 outputs from Docker action entrypoint (#313)", how = "chosen by hand after review" }
[[case]]
id = "contextforge-prompt-ssti"
repo = "https://github.com/IBM/mcp-context-forge"
language = "python"
also = ["javascript"]
family = "injection"
advisory = ["GHSA-vwf3-4xxj-qg6h"]
vulnerable = "0f8e0ad679f1fff685f486e17f642ac863c8ca9e"
fixed = "4d31004661858a2e99055b3c0c9f14218b8f7120"
ancestry = "parent"
sink = "mcpgateway/services/prompt_service.py :: PromptService._render_template() → _compile_jinja_template() → _get_jinja_env().from_string(template).render() (plain jinja2.Environment)"
source = "prompt template stored through register_prompt (prompts create/update API), rendered on prompt get"
sites = ["mcpgateway/services/prompt_service.py::_render_template", "mcpgateway/services/prompt_service.py::_compile_jinja_template"]
privilege = "authenticated (permission to create or update prompts)"
license = "Apache-2.0"
confidence = "high"
benign = { base = "1f8fd58980f4565d4788ffc3587d6e3bd16b3342", tip = "9938c58f7819c526932c4ae4ad646a9c028cb19e", subject = "fix: add rust coverage reports (#4458)", how = "automatic proposal, reviewed" }
[[case]]
id = "lmdeploy-media-url-ssrf"
repo = "https://github.com/InternLM/lmdeploy"
language = "python"
also = []
family = "untrusted_destination"
advisory = ["GHSA-39wr-7q6h-cf68"]
vulnerable = "71648d239c978452720a6d6692fe914efc8a7b94"
fixed = "03c313006d17cc3feae86b633c44206a997c44db"
ancestry = "parent"
sink = "lmdeploy/vl/media/connection.py :: _load_http_url() → requests.Session().get(url_spec.geturl(), allow_redirects=True), gated by _is_safe_url() (the urlparse host differs from the requests host)"
source = "image_url/video/audio URLs in POST /v1/chat/completions → serve/processors/multimodal.py :: MultimodalProcessor._parse_multimodal_item() → load_from_url()"
sites = ["lmdeploy/vl/media/connection.py::_load_http_url"]
privilege = "unauthenticated by default (api_keys is optional, default None)"
license = "Apache-2.0"
confidence = "medium"
benign = { base = "b6a29db4d2d218838048975977f125dc9b1ac75d", tip = "3d9d23087b59fa44afe83deb03ee7cce04df3b4d", subject = "fix(benchmark): seed NumPy RNG so --seed controls random-mode lengths (#4785)", how = "automatic proposal, reviewed" }
[[case]]
id = "winml-cli-serve-cors"
repo = "https://github.com/microsoft/winml-cli"
language = "python"
also = []
family = "config_secrets"
advisory = ["CVE-2026-84452", "GHSA-96p9-rh4f-92cf"]
vulnerable = "02179d185f0e3ac23518236b5ca1f61fd37ef680"
fixed = "f4073e0ef4700a25b623487e7e45c421ca0b9993"
ancestry = "parent"
sink = "src/winml/modelkit/serve/cli_api.py :: module level app.add_middleware(CORSMiddleware, allow_origins=[\"*\"], ...) (same in serve/app.py :: create_app())"
source = "any web page in the victim's browser calling POST /v1/cli/{command} on 127.0.0.1"
sites = ["src/winml/modelkit/serve/cli_api.py::<global>", "src/winml/modelkit/serve/app.py::create_app"]
privilege = "unauthenticated (cross-origin from any website)"
license = "MIT"
confidence = "high"
benign = { base = "8d631f6f1e5db26a04e8c13045280408c6f984dc", tip = "be3e59dd412d4918c5a852aa4d8f0207c34aaf6f", subject = "Remove preload ORT DLL (undo PR 689) (#1380)", how = "automatic proposal, reviewed" }
[[case]]
id = "reactpress-findall-column-sqli"
repo = "https://github.com/fecommunity/reactpress"
language = "typescript"
also = ["javascript"]
family = "injection"
advisory = ["CVE-2026-61685", "GHSA-wmw4-mw6x-6vfm"]
vulnerable = "c3dd10681b3d2b5e224f7ac647b3769557603975"
fixed = "78ecb70af1c021455c05fdcbe137212c70e310d6"
ancestry = "parent"
sink = "server/src/modules/article/article.service.ts :: ArticleService.findAll() (query-string key interpolated as article.${key} into andWhere); same pattern in comment/file/page/knowledge services"
source = "@Query() queryParams in ArticleController.findAll() (GET /api/article)"
sites = ["server/src/modules/article/article.service.ts::findAll"]
privilege = "unauthenticated"
license = "MIT"
confidence = "high"
benign = { base = "295f5de807ed1ebec7b6b155cce16b24a743f247", tip = "903d4b2e2de1c259843f3a8c67701fa7d50d2205", subject = "docs: update documentation links to new site URL", how = "chosen by hand after review" }
[[case]]
id = "fuxa-tdengine-escape-sqli"
repo = "https://github.com/frangoteam/FUXA"
language = "javascript"
also = ["typescript"]
family = "injection"
advisory = ["CVE-2026-47720", "GHSA-h9fj-c2qr-76g2"]
vulnerable = "193a888f8e3f805325072829b1644a2c768c948b"
fixed = "b78f6159d02a56ce5ff48207feb936afb3534fc8"
ancestry = "parent"
sink = "server/runtime/storage/tdengine/index.js :: TDengine.getDaqValue() (escapeTdString(tagid) doubles quotes but not backslashes, then SQL string for cursor.query)"
source = "sids of GET /api/daq (server/api/daq/index.js) or Socket.IO DAQ_QUERY"
sites = ["server/runtime/storage/tdengine/index.js::getDaqValue"]
privilege = "unauthenticated (guest level; socket handler has no auth check)"
license = "MIT"
confidence = "medium"
benign = { base = "e787e4140b5e71c5b02e23c295fa08b8df0edc53", tip = "bf998eeee3158e338724c71979b7cdafcfa74b32", subject = "Button control with configurable icon (#2349)", how = "automatic proposal, reviewed" }
[[case]]
id = "budibase-mongo-template-nosqli"
repo = "https://github.com/Budibase/budibase"
language = "typescript"
also = ["javascript"]
family = "injection"
advisory = ["GHSA-pmpg-2mxq-6xwr", "GHSA-qw6m-8fw2-2v64"]
vulnerable = "b7e63df737eede85d951aa9665b0c0778ece7ccb"
fixed = "dd8c0654b35cd89ce3645f2355f4bf2ff9a5dd80"
ancestry = "parent"
sink = "packages/server/src/sdk/workspace/queries/queries.ts :: enrichContext() (handlebars processStringSync(..., {noEscaping: true}) into the json template, then JSON.parse) → packages/server/src/integrations/mongodb.ts :: MongoIntegration.read() (collection.find(json))"
source = "query parameters / bindings (e.g. currentUser.email, request parameters) of POST /api/v2/queries/:queryId (packages/server/src/api/controllers/query/index.ts :: executeV2)"
sites = ["packages/server/src/sdk/workspace/queries/queries.ts::enrichContext", "packages/server/src/integrations/mongodb.ts::read"]
privilege = "authenticated app user (BASIC role)"
license = "GPL-3.0 (root LICENSE says \"GPLv3 overall\"; server package.json GPL-3.0; GitHub NOASSERTION)"
confidence = "medium"
benign = { base = "a4423fc5bf237e6101c2e8348ef5372fff6484cd", tip = "272c6a61a2a386688792f9e26c2665f2f431a5c2", subject = "Merge pull request #18865 from Budibase/feature/18784-ai-credit", how = "automatic proposal, reviewed" }
[[case]]
id = "unleash-addon-webhook-ssrf"
repo = "https://github.com/Unleash/unleash"
language = "typescript"
also = ["javascript"]
family = "untrusted_destination"
advisory = ["CVE-2026-63004", "GHSA-5vf6-jrqr-78fj"]
vulnerable = "b9f3217d327f24094b4352130baa30387f85cffe"
fixed = "2100db76af3473f13e6fb40096cf17a9c2b741a1"
ancestry = "parent"
sink = "src/lib/addons/addon.ts :: Addon.fetchRetry() (ky(url, ...)), called from webhook.ts / slack / teams / datadog / new-relic addons"
source = "addon parameters.url saved through the admin addons API (CREATE_ADDON/UPDATE_ADDON), fired on feature events"
sites = ["src/lib/addons/addon.ts::fetchRetry"]
privilege = "authenticated (CREATE_ADDON/UPDATE_ADDON root permission)"
license = "AGPL-3.0"
confidence = "high"
benign = { base = "d4efb3a31616483c438606edbecce13b868fc587", tip = "cb4e6250743fac364bae50b985b2b8348def7a2b", subject = "Merge commit '232b5d575c7d455ce7ac767183e61fd389dd8aac' into automation/sync-oss-upstream", how = "automatic proposal, reviewed" }
[[case]]
id = "medplum-external-auth-open-redirect"
repo = "https://github.com/medplum/medplum"
language = "typescript"
also = ["javascript"]
family = "untrusted_destination"
advisory = ["CVE-2026-53728", "GHSA-m44r-7c5h-m6mj"]
vulnerable = "5d4a8654b02aa67c444f96ae1eb5663c09854006"
fixed = "7ae10035ddadde4dba7b18d3156553940465b3a1"
ancestry = "parent"
sink = "packages/server/src/auth/external.ts :: externalCallbackHandler() (res.redirect(redirectUrl) with URI accepted by getClientRedirectUri(client, body.redirectUri, true) prefix match in packages/server/src/oauth/clients.ts)"
source = "redirectUri inside JSON state query parameter of GET /auth/external"
sites = ["packages/server/src/auth/external.ts::externalCallbackHandler"]
privilege = "unauthenticated (victim completes external IdP login)"
license = "Apache-2.0"
confidence = "high"
benign = { base = "0dd335e8520b859563626d9bf50d602a5f3ad79a", tip = "20790b17d76f335e60dd703f65d4eb13bbb4754d", subject = "tweak(cdk): make `minHealthyPercent` 100 (#8836)", how = "automatic proposal, reviewed" }
[[case]]
id = "cct-studio-server-cmdi"
repo = "https://github.com/davila7/claude-code-templates"
language = "javascript"
also = ["typescript", "python"]
family = "injection"
advisory = ["CVE-2026-73222", "GHSA-79wm-x847-7cvg"]
vulnerable = "acdb5893412a81eaec9862ee24a4613bf296ab45"
fixed = "bc4618b07232633c1c0aac12a43e436268d31783"
ancestry = "parent"
sink = "cli-tool/src/sandbox-server.js :: executeLocalTask() (spawn('claude', [finalPrompt], {shell: true})), route handler POST /api/install-agent and checkAndInstallAgent() (spawn('npx', [..., agentName, ...], {shell: true}))"
source = "req.body.prompt, req.body.agentName, req.body.agent"
sites = ["cli-tool/src/sandbox-server.js::executeLocalTask", "cli-tool/src/sandbox-server.js::checkAndInstallAgent"]
privilege = "unauthenticated (server on 0.0.0.0, wildcard CORS)"
license = "MIT"
confidence = "high"
benign = { base = "c2ed4066bc7ea66806e90d6c0b97988495ff44b2", tip = "025520a8bfc8ed05e9a197bf53fa2f871f666dea", subject = "feat(dashboard): hide Loops from navigation menu (#738)", how = "chosen by hand after review" }
[[case]]
id = "wp-gopay-log-filter-sqli"
repo = "https://github.com/gopaycommunity/gopay-woocommerce-integration"
language = "php"
also = ["javascript"]
family = "injection"
advisory = ["CVE-2026-75959", "GHSA-88c4-2f34-57h5"]
vulnerable = "626bc8b5f8d33de6eefd921e3a05ed52ac320bcd"
fixed = "7190c985370f9758f226c2f2addd79532a2f93d5"
ancestry = "parent"
sink = "admin/views/log.php :: file scope (included by admin/class-gopay-gateway-admin.php), two $wpdb->get_results(sprintf(\"... REGEXP '[\\w\\W]*%s[\\w\\W]*' ...\", ..., strtoupper($log_table_filter)))"
source = "filter_input(INPUT_GET, 'log_table_filter')"
sites = ["admin/views/log.php::<global>"]
privilege = "admin (shop manager, manage_woocommerce)"
license = "GPL-2.0-or-later (verified 2026-10-02: gopay-gateway.php header, 626bc8b5f8d33de6eefd921e3a05ed52ac320bcd)"
confidence = "high"
benign = { base = "2b5fe58d6dadbfecb01ed34611aed8502d11e612", tip = "88b10bda704a37e88121e22dc22f4e5019097272", subject = "GPY025-136: fix i18n and remove translation calls on non-literal text", how = "automatic proposal, reviewed" }
[[case]]
id = "wp-user-registration-login-redirect"
repo = "https://github.com/themegrill/user-registration"
language = "php"
also = ["javascript"]
family = "untrusted_destination"
advisory = ["CVE-2026-80072", "GHSA-w78c-fx7q-9x8m"]
vulnerable = "e52cf151cb229852e6209fb3a627aa51485139c1"
fixed = "75c940dab15fd54d5a3fbb9f2213767b89467b08"
ancestry = "parent"
sink = "includes/functions-ur-core.php :: ur_process_login() (wp_redirect(wp_validate_redirect($redirect, $redirect)), and wp_send_json_success(['message'=>$redirect]) on the AJAX path)"
source = "$post['redirect'] (login form POST) or wp_get_raw_referer()"
sites = ["includes/functions-ur-core.php::ur_process_login"]
privilege = "unauthenticated attacker, victim logs in"
license = "GPL-3.0-or-later (verified 2026-10-02: license.txt, e52cf151cb229852e6209fb3a627aa51485139c1)"
confidence = "high"
benign = { base = "064dd7ab1585082b9b5cf078f2d22c0234f25e48", tip = "3b6acc4695e93d9d48969ed894c21419bd09b33a", subject = "Fix - Deactivate Licence button did nothing when the pro-activation transient was set (#1418)", how = "automatic proposal, reviewed" }
[[case]]
id = "wp-directorist-avatar-ssrf"
repo = "https://github.com/sovware/directorist"
language = "php"
also = ["javascript"]
family = "untrusted_destination"
advisory = ["CVE-2026-84046", "GHSA-6q63-3w56-m722"]
vulnerable = "77cfc9bc8b453f97ab7e720330706fa4ef056867"
fixed = "1a67825d5cf6e7c768c3a692da1d18bae732bea9"
ancestry = "parent"
sink = "includes/rest-api/functions.php :: directorist_rest_upload_image_from_url() (download_url($image_url))"
source = "$request['avater']['src'] in includes/rest-api/Version1/class-users-controller.php :: update_user_meta_fields() (REST user update)"
sites = ["includes/rest-api/functions.php::directorist_rest_upload_image_from_url"]
privilege = "authenticated (subscriber)"
license = "GPL-3.0-or-later (verified 2026-10-02: license.txt + directorist-base.php header, 77cfc9bc8b453f97ab7e720330706fa4ef056867)"
confidence = "medium"
benign = { base = "61ddcdf84f18cbe7749036a912d2931cef8cbd5a", tip = "986becd484f8f2d9b60604737d4fb4bf08fa6d1b", subject = "Replaced Location with Address on the All Listings map view (#2280)", how = "chosen by hand after review" }
[[case]]
id = "wp-groundhogg-confirm-redirect"
repo = "https://github.com/groundhoggwp/groundhogg"
language = "php"
also = ["javascript"]
family = "untrusted_destination"
advisory = ["CVE-2026-81741", "GHSA-m96f-c5vh-h7cm"]
vulnerable = "9472a1dd79d4556c40eece501a61930793f170d3"
fixed = "028247d4b8ec9ba051d3eb48f1466a759bc16bfd"
ancestry = "parent (follow-up is the direct child of the fix)"
sink = "templates/preferences.php :: file scope, case 'confirm': (wp_redirect($redirect_to))"
source = "get_url_var('redirect_to') (query string, only through esc_url_raw(sanitize_text_field(...)))"
sites = ["templates/preferences.php::<global>"]
privilege = "unauthenticated (crafted preferences link)"
license = "GPL-3.0-or-later (verified 2026-10-02: groundhogg.php header, 9472a1dd79d4556c40eece501a61930793f170d3)"
confidence = "high"
benign = { base = "61c8a03f11b6030c630be6ae0b0c2ad641eb733a", tip = "e3a69f1c2814c6bebed940505894008748566b93", subject = "output a square image regardless of image dimensions", how = "automatic proposal, reviewed" }
[[case]]
id = "froxlor-admin-ip-second-order-sqli"
repo = "https://github.com/froxlor/froxlor"
language = "php"
also = ["javascript"]
family = "injection"
advisory = ["CVE-2026-54348", "GHSA-w27m-rmmf-g5w4"]
vulnerable = "9bf03de643a2edee4642bfc6c94db1f05253da02"
fixed = "a1eaca5a1601c8a30e00814a4fc73ad0c185f89e"
ancestry = "parent (ancestor of tag 2.3.8)"
sink = "lib/Froxlor/Api/Commands/IpsAndPorts.php :: IpsAndPorts::listing() / listingCount() (implode(\", \", json_decode($this->getUserDetail('ip'), true)) into WHERE id IN (...)); also lib/Froxlor/Api/Commands/Domains.php :: validateIpAddresses()"
source = "ipaddress API parameter of Admins.add / Admins.update (lib/Froxlor/Api/Commands/Admins.php), stored as JSON in panel_admins.ip, read back via getUserDetail('ip')"
sites = ["lib/Froxlor/Api/Commands/IpsAndPorts.php::listing", "lib/Froxlor/Api/Commands/IpsAndPorts.php::listingCount", "lib/Froxlor/Api/Commands/Domains.php::validateIpAddresses"]
privilege = "admin (API key with change_serversettings)"
license = "LGPL-2.1 (per GitHub)"
confidence = "high"
benign = { base = "b595f061fbb68874423809fec9f2a42bf6b34175", tip = "9d274828bdc4eefec0058a5a3a25bf4eb69d0f70", subject = "avoid warnings regarding php8.5 in config/database.php; adjust phpunit for froxlor/framework testing", how = "automatic proposal, reviewed" }
[[case]]
id = "sulu-smart-content-sql2-injection"
repo = "https://github.com/sulu/sulu"
language = "php"
also = ["javascript"]
family = "injection"
advisory = ["CVE-2026-92692", "GHSA-jg26-q8hg-3pq4"]
vulnerable = "d061094f5b7bb1d5e974544fce30bede9c7adf8e"
fixed = "d19c01487af8c3de2fb3aa145856707a6367392c"
ancestry = "parent (2.6 branch; ancestor of tag 2.6.25)"
sink = "src/Sulu/Component/Content/SmartContent/QueryBuilder.php :: QueryBuilder::buildCategoriesWhere() (also buildTagsWhere(), buildAudienceTargeting()), string-concatenated into the SQL2 WHERE clause built by buildWhere()"
source = "?categories= query parameter, read in src/Sulu/Component/Category/Request/CategoryRequestHandler.php :: getCategories() (explode + trim only), passed by src/Sulu/Component/SmartContent/ContentType.php as websiteCategories"
sites = ["src/Sulu/Component/Content/SmartContent/QueryBuilder.php::buildCategoriesWhere", "src/Sulu/Component/Content/SmartContent/QueryBuilder.php::buildWhere"]
privilege = "unauthenticated (any public page with a category-filtered Smart Content block)"
license = "MIT"
confidence = "high"
benign = { base = "777f4eb47d45dab420c82b43ae6f513638294de0", tip = "6621a429c5284bd05ee1b60f333aff83297fecde", subject = "Removing unused service argument (#7169)", how = "chosen by hand after review" }
@@ -0,0 +1,457 @@
# Independent evaluation population v3 (PHP). Reserved 2026-09-30.
#
# RESERVED, NOT SCANNED. Chosen by web research alone (research-2026-09-30-php.md, candidates-v3-php.json):
# advisories published 2026-06-01 or later, taken newest first per family under criteria fixed before any
# advisory was read, never by how easy a case looks for this analyzer; 470 advisories examined, 26 candidates.
# No repository was cloned, no scanner or project tooling run on any of them, and none is referenced anywhere in
# this project's development tree (checked against every github.com URL under benchmarks/, src/, tests/ and
# .kiro/, which includes populations v1 and v2).
#
# Why a v3: the PHP quick-mode rules were written on 2026-09-30 while looking at PMPro, population-v1's PHP cases,
# Ultimate Member, MediaWiki and WP Statistics, so no PHP number measured so far qualifies them, and population v2
# is reserved for other qualification. v3 is PHP cases nobody in this project has looked at.
#
# HOLD-OUT RULES (v1's and v2's): no scanner, quick rule, engine, plane task or model is run on any of these
# repositories, and no trace, probe or census of them is made, before the population is FROZEN and protocol-v3.md
# is committed. Freezing reads git history, file lists and the fix commit's diff only. No rule, fact, query,
# prompt or scoring change may cite these cases before their evaluation is recorded; any analyzer change made
# after a v3 result is a retune that v3 cannot qualify. tests/test_independent_population.py fails the moment one
# of these repositories is named anywhere under benchmarks/, src/ or tests/ outside this directory.
#
# SELECTION RULE (research-2026-09-30-php.md, "Recommended balanced selection"), applied mechanically: family
# targets injection 5, output_encoding 3, path 3, untrusted_destination 2, access_control 2, deserialization 1;
# within a family, candidates whose fix commit the advisory names come first, then newest first by publication;
# take the first n.
#
# Maintainer's decision 2026-09-30, "Recommended 16, minus v2 mentions": the advisories that
# research-2026-09-26.md (population v2's research) names in its passed-over note are excluded, and each slot
# they held goes to the next candidate in its family by the same rule. Excluded as v2 mentions: Winter, elFinder,
# Easy!Appointments (all three in the recommended 16), and Grav, Backpack CRUD and Koel (alternates).
# - injection: winter-filter-numberrange-sqli -> mantisbt-tokenizer-eval-rce (next named fix, 2026-07-15).
# - untrusted_destination: elfinder-fsock-dns-rebinding-ssrf -> pagible-admin-proxy-dns-rebinding-ssrf (koel,
# next in line, is a v2 mention; pagible's fix is inferred). easyappointments-caldav-ssrf -> EMPTY: the family
# has no remaining candidate among the 26 (the research's further alternates were never verified and hold no
# record), so the slot is left empty rather than bending the rule. The population holds 15 cases.
#
# `sites` are the sink or execution locations the ADVISORY names (`path::function`, `<global>` for file scope),
# declared before any scan. Where an advisory names only a file (`site_files`), the function is resolved at
# freeze from the fix commit's diff of that file (maintainer's decision 2, "Resolve the function at freeze"), and
# the freeze record gives each site's basis. Protocol v3 credits a finding at a declared site as well as one
# matching the fix's changed lines.
#
# Fixes the advisory does not name (`fix_named_by_advisory = false`) are confirmed at freeze: the diff must change
# the declared sink file and function, or the case is dropped and the next candidate taken by the rule.
#
# Licences marked "verify" must be checked before any result is published. Kirby is under a proprietary licence:
# it stays in the population, but its results are unpublishable without the licensor's permission.
#
# FROZEN 2026-09-30 (freeze.py, freeze-v3-php.json; git history, file lists and fix diffs only). No case dropped:
# both inferred fixes change a declared site function (pagible: proxy and isValidUrl; podlove: create_link_item,
# while GenerationGuard::__destruct is not in the diff). Sites resolved from the fix diff are marked in
# `site_basis`; a file the fix does not change stays unresolved (`site_files_unresolved`). Benign controls come
# from freeze.py's strict proposal (single-language population: PHP change, merges judged with everything they
# bring in, history before the vulnerable pin when nothing after the fix qualifies), reviewed on commit messages
# and file names; five proposals were replaced by hand, with the reason in `how`.
schema_version = 1
status = "frozen"
frozen_on = "2026-09-30"
freeze_record = "freeze-v3-php.json"
language = "php"
reserved_on = "2026-09-30"
selected_by = "maintainer"
selection = "Recommended 16, minus v2 mentions (15 cases; one untrusted_destination slot empty)"
research = "research-2026-09-30-php.md"
candidates = "candidates-v3-php.json"
protocol = "protocol-v3.md"
multi_language_members = ["wp-geo-my-wp-info-window-lfi", "wp-graphql-updatepost-publish-bypass", "wp-podlove-generation-guard-poi"]
also_reserved = ["population-v1.toml", "population-v2.toml", "../push/ghost-reserved.toml"]
[[case]]
id = "gitlist-commandline-cmdi"
repo = "https://github.com/klaussilveira/gitlist"
language = "php"
also = []
wordpress = false
family = "injection"
subtype = "OS command / argument injection"
advisory = ["CVE-2026-82668", "GHSA-v64j-pgc3-qgc9"]
published = "2026-08-31"
vulnerable = "f67609d52c1812fa8a7ed80eae5e795cfd72115f"
fixed = "88cf2866083d5f7c20d9d565c45f828a7ad1516b"
ancestry = "parent; ancestor of tag 3.0.0-beta"
fix_named_by_advisory = true
sink = "`getDefaultBranch` of src/SCM/System/Git/CommandLine.php (git command line)"
source = "repository content/arguments reaching the git command line (not stated precisely)"
sites = ["src/SCM/System/Git/CommandLine.php::getDefaultBranch"]
site_files = []
site_derivation = "file and function quoted by the advisory (VulDB-generated text)"
privilege = "not stated ('remotely')"
license = "BSD-3-Clause"
confidence = "medium"
caveats = ["VulDB auto-generated description; the fix message ('Prevent textconv') suggests git config/attribute abuse rather than shell metacharacters, and the patched lines may not be in getDefaultBranch -- verify at freeze"]
benign = { base = "ce5806c849f453d7200c1f488ee6aba19eef9a4c", tip = "ecc8e18e03d9b547e82b7c7c148cb00ba01c0cc9", subject = "Merge pull request #902 from zoredache/sort_repositories", how = "automatic proposal (before the vulnerable pin), reviewed" }
[[case]]
id = "qloapps-customermessage-bo-query-sqli"
repo = "https://github.com/Qloapps/QloApps"
language = "php"
also = ["javascript"]
wordpress = false
family = "injection"
subtype = "SQL injection"
advisory = ["CVE-2026-75497", "GHSA-5w6r-2mxc-fq3v"]
published = "2026-08-25"
vulnerable = "5b4d65cbb8570a4ba46364ee0f2ded79d294ab84"
fixed = "123c97c110b7053ea3297d8e58fe95b3c3536560"
ancestry = "parent"
fix_named_by_advisory = true
sink = "database query in CustomerMessage.php built from the `bo_query` parameter"
source = "`bo_query` request parameter"
sites = ["classes/CustomerMessage.php::searchByName"]
site_files = ["classes/CustomerMessage.php"]
site_basis = { "classes/CustomerMessage.php::searchByName" = "resolved-from-fix-diff" }
site_derivation = "the advisory names 'CustomerMessage.php' (basename) and the parameter; the classes/ path is from the fix commit; no function named"
privilege = "administrator"
license = "OSL-3.0"
confidence = "medium"
caveats = ["one fix covers two advisories (Address.php is GHSA-mj48); repo 129 MB, near the size cap", "function-level site to be resolved at freeze"]
benign = { base = "5c52b864878848a625a5d9cfbb6e3925ab853ad1", tip = "a8f6071627bf97a77d715c861c1157fded5e9e60", subject = "Merge pull request #1813 from Azhar-Webkul/gli-3095", how = "chosen by hand after review: proposal e43b4500 changes payment-status logic; next passing change taken" }
[[case]]
id = "roundcube-markasjunk-cmd-learn-rce"
repo = "https://github.com/roundcube/roundcubemail"
language = "php"
also = ["javascript"]
wordpress = false
family = "injection"
subtype = "OS command injection"
advisory = ["CVE-2026-74997", "GHSA-q5rh-mmp4-ppgq"]
published = "2026-08-17"
vulnerable = "a1afb8fd1f00ed4cb9376c072bb5ca5ded64495e"
fixed = "b8f90e28a46d42e79a69568cba897f8f4223d9cd"
ancestry = "parent; ancestor of tag 1.6.18"
fix_named_by_advisory = true
sink = "the markasjunk plugin's cmd_learn driver runs a shell command with placeholder replacement values"
source = "crafted placeholder replacement values (message/user data) when a message is marked as junk"
sites = ["plugins/markasjunk/drivers/cmd_learn.php::_do_salearn"]
site_files = ["plugins/markasjunk/drivers/cmd_learn.php"]
site_basis = { "plugins/markasjunk/drivers/cmd_learn.php::_do_salearn" = "resolved-from-fix-diff" }
site_derivation = "the advisory names 'the cmd_learn driver of the markasjunk plugin'; the file path follows from that name and is confirmed by the fix commit; no function named"
privilege = "authenticated webmail user"
license = "GPL-3.0"
confidence = "medium"
caveats = ["only instances that enable the markasjunk plugin with the cmd_learn driver"]
benign = { base = "1cddafded973c4bf6faa173accaf8f52ca1ac0d0", tip = "747f544708ed1d698b677d333346164a9451aeda", subject = "Simplify", how = "automatic proposal (after the fix), reviewed" }
[[case]]
id = "maccms10-template-blacklist-rce"
repo = "https://github.com/magicblack/maccms10"
language = "php"
also = ["javascript"]
wordpress = false
family = "injection"
subtype = "code injection (template → PHP, blacklist bypass)"
advisory = ["CVE-2026-71232", "GHSA-cv48-22qx-mpmx"]
published = "2026-08-05"
vulnerable = "2cabecda29947e8f9ea7cd3aa47ab90ed817605b"
fixed = "71ad3bb29570e110d8e973acff68040a3050ddf0"
ancestry = "parent"
fix_named_by_advisory = true
sink = "ThinkPHP `{if condition=...}` compiled into raw PHP; the admin template editor only blocks a function blacklist"
source = "template content saved through the admin template editor"
sites = ["application/admin/controller/Template.php::info"]
site_files = ["application/admin/controller/Template.php"]
site_basis = { "application/admin/controller/Template.php::info" = "resolved-from-fix-diff" }
site_derivation = "file path quoted by the advisory; no function named"
privilege = "administrator"
license = "GitHub: NOASSERTION; no licence declared by the project [verify before publishing]"
confidence = "medium"
caveats = ["guard bypass (incomplete blacklist); the fix is still a blacklist", "licence not declared on GitHub"]
benign = { base = "6ace264920bc50a07944c163b17a8fd8a3f7687f", tip = "c9f8d0e848489a2dc73daf1106748d2cd94d8138", subject = "修复 meilisearch 排序规则语法兼容问题", how = "chosen by hand after review: proposal 668ba418 fixes a registration-API validator (input validation); rejected" }
[[case]]
id = "mantisbt-tokenizer-eval-rce"
repo = "https://github.com/mantisbt/mantisbt"
language = "php"
also = []
wordpress = false
family = "injection"
subtype = "code injection (eval)"
advisory = ["CVE-2026-49273", "GHSA-v84x-qvhg-f36r"]
published = "2026-07-15"
vulnerable = "e7041ecc80c8d35f76a7ea1027a8a231244f7dc5"
fixed = "78c0af63d1fe0118004744cab21ca3bf2cea0f5c"
ancestry = "parent; ancestor of tag release-2.28.4"
fix_named_by_advisory = true
sink = "`Tokenizer` (reached through `ConfigParser`) calls `eval()` with a `return;` prefix; PHP hoists class declarations past the return"
source = "non-string configuration value submitted to adm_config_set.php (Manage Configuration)"
sites = ["core/classes/Tokenizer.class.php::__construct"]
site_files = ["core/classes/Tokenizer.class.php", "adm_config_set.php"]
site_basis = { "core/classes/Tokenizer.class.php::__construct" = "resolved-from-fix-diff" }
site_files_unresolved = { "adm_config_set.php" = "the fix does not change this file" }
sink_evidence = "eval("
site_derivation = "the advisory names adm_config_set.php, ConfigParser, Tokenizer and eval(); the Tokenizer file path and the enclosing function are not named (path from the fix commit)"
privilege = "administrator"
license = "GitHub: none; GPL per project [verify before publishing]"
confidence = "high"
caveats = ["function-level site must be resolved at freeze by reading the file (the advisory names only the class)"]
benign = { base = "088fa767bce246b6d2f083f32d937f22c8de4767", tip = "d6ca719b591f70acfa711fc41705388557b6ac04", subject = "Improve Workflow Graph display.", how = "automatic proposal (after the fix), reviewed" }
[[case]]
id = "sharp-editor-iframe-srcdoc-xss"
repo = "https://github.com/code16/sharp"
language = "php"
also = ["typescript"]
wordpress = false
family = "output_encoding"
subtype = "stored XSS (sanitizer allow-list)"
advisory = ["CVE-2026-61823", "GHSA-qxg3-46rw-79j8"]
published = "2026-09-25"
vulnerable = "67743149309054493ec8f5505173fc57aaff760f"
fixed = "ec509a22c808a5bd9dfad6a0a85c92ce6f411e21"
ancestry = "parent; ancestor of tag v9.22.5"
fix_named_by_advisory = true
sink = "Symfony HtmlSanitizer config allows the iframe `srcdoc` attribute; the value is rendered as live HTML"
source = "Editor field content saved by a user with edit permission"
sites = ["src/Utils/Sanitization/FormatsSanitizedValue.php::sanitizer", "src/Utils/Sanitization/FormatsSanitizedValue.php::isEncodingNeeded"]
site_files = ["src/Utils/Sanitization/FormatsSanitizedValue.php"]
site_basis = { "src/Utils/Sanitization/FormatsSanitizedValue.php::sanitizer" = "resolved-from-fix-diff", "src/Utils/Sanitization/FormatsSanitizedValue.php::isEncodingNeeded" = "resolved-from-fix-diff" }
sink_evidence = "srcdoc"
site_derivation = "file path quoted by the advisory; no function named"
privilege = "authenticated editor"
license = "MIT"
confidence = "medium"
caveats = ["the flaw is a sanitizer allow-list entry, not a missing escape call"]
benign = { base = "ec509a22c808a5bd9dfad6a0a85c92ce6f411e21", tip = "7c601fffe930fc321a5d8bcb25cb072d50a6fc08", subject = "Update version to v9.22.5", how = "automatic proposal (after the fix), reviewed" }
[[case]]
id = "privatebin-jsonld-request-uri-injection"
repo = "https://github.com/PrivateBin/PrivateBin"
language = "php"
also = ["javascript"]
wordpress = false
family = "output_encoding"
subtype = "reflected JSON injection (CWE-116)"
advisory = ["CVE-2026-55891", "GHSA-xrjc-c68j-hp7w"]
published = "2026-08-28"
vulnerable = "e0dd4c025c19a182b6a4c6fb77a8bf81ceff6899"
fixed = "75f056dcda955d94c17ec5a4f8c54a9b7bfcee07"
ancestry = "first parent of a merge; ancestor of tag 2.0.5"
fix_named_by_advisory = true
sink = "`Controller::_jsonld()` splices `$_urlBase` into a .jsonld template with `str_replace()`"
source = "`$_SERVER['REQUEST_URI']` via `Request::getRequestUri()` (FILTER_SANITIZE_URL keeps quotes and angle brackets)"
sites = ["lib/Request.php::getRequestUri", "lib/Controller.php::_jsonld"]
site_files = []
site_derivation = "Class::method names quoted by the advisory; paths lib/*.php from the fix commit"
privilege = "unauthenticated"
license = "GitHub: NOASSERTION; project files state a zlib-style licence [verify before publishing]"
confidence = "high"
caveats = ["the fix changes Request.php only; Controller::_jsonld is a declared site outside the diff", "response type is JSON-LD, not HTML", "licence NOASSERTION on GitHub (zlib-style per project, verify)"]
benign = { base = "75f056dcda955d94c17ec5a4f8c54a9b7bfcee07", tip = "3035d6e5d3cceb7f06f7b7bb80968727879c52b0", subject = "incrementing version", how = "automatic proposal (after the fix), reviewed" }
[[case]]
id = "openstamanager-group-name-html-injection"
repo = "https://github.com/devcode-it/openstamanager"
language = "php"
also = ["javascript"]
wordpress = false
family = "output_encoding"
subtype = "stored HTML injection"
advisory = ["CVE-2026-44701", "GHSA-crx4-7mmq-j74j"]
published = "2026-08-26"
vulnerable = "57a2734e2121aa8fcd456eff7c59f03e29e69176"
fixed = "0b88ad1963871abfa5e134004ca0c74c496d8d2f"
ancestry = "parent; ancestor of tag v2.11-beta"
fix_named_by_advisory = true
sink = "`'<span class=\"text-primary\">'.$group->getTranslation('title').'</span>'` in modules/utenti/edit.php"
source = "group name saved when creating a user group"
sites = ["modules/utenti/edit.php::<global>"]
site_files = []
sink_evidence = "$group->getTranslation('title')"
site_derivation = "file and code quoted by the advisory"
privilege = "authenticated user able to create groups"
license = "GPL-3.0"
confidence = "high"
caveats = ["second advisory commit (7c9eafe9, global GET/POST filtering) is not in the pinned pair"]
benign = { base = "0b88ad1963871abfa5e134004ca0c74c496d8d2f", tip = "1832b85d0e570e0b873c7c23dcfcb6b5d1365a28", subject = "fix: ricerca riferimenti in importazione fattura per riga con articolo non esistente", how = "automatic proposal (after the fix), reviewed" }
[[case]]
id = "wp-geo-my-wp-info-window-lfi"
repo = "https://github.com/Fitoussi/geo-my-wp"
language = "php"
also = ["javascript"]
wordpress = true
family = "path"
subtype = "local file inclusion"
advisory = ["CVE-2026-85200", "GHSA-6hw9-7gx5-mw88"]
published = "2026-09-12"
vulnerable = "1dbb7b1672a45a8f88fdcd379ceb07b6cf2beceb"
fixed = "5a768bf1c6e44ded83a65be8789f3587515665ac"
ancestry = "parent"
fix_named_by_advisory = true
sink = "include of a file chosen through `gmw_posts_locator_ajax_info_window_loader`"
source = "AJAX request parameters (unauthenticated)"
sites = ["plugins/posts-locator/includes/gmw-posts-locator-functions.php::gmw_posts_locator_ajax_info_window_loader", "includes/gmw-functions.php::gmw_ajax_info_window_init"]
site_files = ["includes/gmw-functions.php"]
site_basis = { "plugins/posts-locator/includes/gmw-posts-locator-functions.php::gmw_posts_locator_ajax_info_window_loader" = "advisory-named", "includes/gmw-functions.php::gmw_ajax_info_window_init" = "resolved-from-fix-diff" }
site_derivation = "function named by the advisory text; files from the advisory's own trac reference links (gmw-functions.php#L1945/L1987, gmw-posts-locator-functions.php#L964/L975); the function-to-file pairing is an assumption to confirm at freeze"
privilege = "unauthenticated"
license = "GitHub: none; GPL per project [verify before publishing]"
confidence = "medium"
caveats = ["WordPress plugin; GitHub shows no licence (plugin header GPL presumed, verify)", "release commit (small)"]
benign = { base = "9c394482d1509ca9222365a91117a10b19619be8", tip = "85bcd85541c777b6fd8580cb1915504a2a2b84cf", subject = "Release 4.5.5.2", how = "automatic proposal (before the vulnerable pin), reviewed" }
[[case]]
id = "kirby-realpath-sibling-traversal"
repo = "https://github.com/getkirby/kirby"
language = "php"
also = []
wordpress = false
family = "path"
subtype = "path traversal (prefix containment check)"
advisory = ["CVE-2026-75592", "GHSA-6j4c-mgqr-qv76"]
published = "2026-09-02"
vulnerable = "e90257fd7504053e13e93a1e9a5a9a5137aed966"
fixed = "2b6fab950bc89d505ea89576bf71c8de614cc3b2"
ancestry = "parent; ancestor of tag 4.9.5 (not of 5.5.2)"
fix_named_by_advisory = true
sink = "`Dir::realpath()` / `F::realpath()` containment check accepts a sibling directory sharing the prefix (site vs site2)"
source = "media handler request path for not-yet-generated thumbnails"
sites = ["src/Filesystem/Dir.php::realpath", "src/Filesystem/F.php::realpath"]
site_files = []
site_derivation = "`Dir::realpath()` and `F::realpath()` named by the advisory; paths confirmed by the fix commit"
privilege = "unauthenticated"
license = "Kirby licence (proprietary, GitHub NOASSERTION) -- not OSI; results unpublishable without permission"
confidence = "medium"
caveats = ["only deployments whose index root has a sibling directory with the same name prefix", "Kirby is under a proprietary licence (GitHub NOASSERTION); must be cleared before publishing"]
benign = { base = "7ee5b79dd8eea6aa3d5a7eb0c5411a6631af88f1", tip = "d333470b9d9df7a48ee8d0bc2249db36aa9dd68c", subject = "style: Fix coding style", how = "chosen by hand after review: proposals 65dfca05/0901d0cb concern the content salt, 7bbefc1e updates vendored dependencies; rejected" }
[[case]]
id = "mineadmin-appstore-identifier-traversal"
repo = "https://github.com/mineadmin/MineAdmin"
language = "php"
also = []
wordpress = false
family = "path"
subtype = "path traversal"
advisory = ["CVE-2026-55224", "GHSA-59xm-4m8c-g3xj"]
published = "2026-08-18"
vulnerable = "ac0dcad616c5d304c249d1db4a6675f65cf736dc"
fixed = "ca41902a2a5422676227e5088f4cc1dec06044f1"
ancestry = "parent; the fixed commit is tag v3.2.0-alpha.2"
fix_named_by_advisory = true
sink = "`BASE_PATH . '/plugin/' . $params['identifier']` in download()/install()/unInstall(), then `Plugin::install($params['identifier'])`"
source = "`identifier` request parameter of the app-store plugin endpoints"
sites = ["plugin/mine-admin/app-store/src/Service/Service.php::download", "plugin/mine-admin/app-store/src/Service/Service.php::install", "plugin/mine-admin/app-store/src/Service/Service.php::unInstall"]
site_files = []
site_derivation = "file path and methods quoted by the advisory"
privilege = "authenticated admin with app-store access (verify)"
license = "Apache-2.0"
confidence = "high"
caveats = ["Hyperf framework (not Laravel/Symfony)"]
benign = { base = "c57753e3f547683dc3e14836c3563a51ba4edaee", tip = "ff33969ef5ee378654dacab89e6f4b1928f4edd1", subject = "Fix/service logstash namespace (#663)", how = "chosen by hand after review: proposal 15deedde is a framework/dependency upgrade; rejected" }
[[case]]
id = "pagible-admin-proxy-dns-rebinding-ssrf"
repo = "https://github.com/aimeos/pagible"
language = "php"
also = ["javascript"]
wordpress = false
family = "untrusted_destination"
subtype = "SSRF (DNS rebinding guard bypass)"
advisory = ["CVE-2026-49262", "GHSA-mmj8-wcvw-6789"]
published = "2026-06-26"
vulnerable = "03f03b2f81b20b1c1625ff227235c43791ddbb33"
fixed = "09a8205d513ec89ed22cdd7bdae0f4c181cee082"
ancestry = "parent; ancestor of tag 0.10.4"
fix_named_by_advisory = false
inferred_fix_confirmed = ["admin/src/Controllers/AdminController.php::proxy", "core/src/Utils.php::isValidUrl"]
sink = "`AdminController::proxy` → Guzzle request after `\\Aimeos\\Cms\\Utils::isValidUrl($url)`"
source = "`url` query parameter of `/cmsproxy`"
sites = ["admin/src/Controllers/AdminController.php::proxy", "core/src/Utils.php::isValidUrl"]
site_files = []
site_derivation = "class::method names quoted by the advisory; paths from the fix commit"
privilege = "authenticated (proxy token)"
license = "MIT"
confidence = "medium"
caveats = ["guard bypass (DNS rebinding)"]
benign = { base = "981b8d63e52e7ba8875a9a52c1865eb3bc662154", tip = "d7577efd68e9c5e36436c8572bfed0df2be2c634", subject = "Publish related items (#210)", how = "automatic proposal (after the fix), reviewed" }
[[case]]
id = "wp-graphql-updatepost-publish-bypass"
repo = "https://github.com/wp-graphql/wp-graphql"
language = "php"
also = ["javascript"]
wordpress = true
family = "access_control"
subtype = "missing object-level capability check"
advisory = ["CVE-2026-88974", "GHSA-5mmc-8pc9-wggg"]
published = "2026-09-23"
vulnerable = "d53b0e566f20a8df0664559cb6432d52a7cd285c"
fixed = "55441663eaa33c3f2e05de038c8286c845916461"
ancestry = "parent; ancestor of tag wp-graphql/v2.22.2"
fix_named_by_advisory = true
sink = "`updatePost` mutation checks only `$post_type_object->cap->edit_posts`"
source = "GraphQL `updatePost` mutation input (status PUBLISH, published post id)"
sites = ["plugins/wp-graphql/src/Mutation/PostObjectUpdate.php::mutate_and_get_payload"]
site_files = ["plugins/wp-graphql/src/Mutation/PostObjectUpdate.php"]
site_basis = { "plugins/wp-graphql/src/Mutation/PostObjectUpdate.php::mutate_and_get_payload" = "resolved-from-fix-diff" }
site_derivation = "the advisory names src/Mutation/PostObjectUpdate.php; the repository is a monorepo (plugins/wp-graphql/ prefix from the fix commit); no function named"
privilege = "Contributor"
license = "GPL-3.0"
confidence = "medium"
caveats = ["fix commit also carries an unrelated Model.php change", "function-level site to resolve at freeze"]
benign = { base = "b583bb6ec87016965085afc7ea586f114c711c68", tip = "cc12aed380414545a005048bfcebe4ba50e9c8f9", subject = "fix: point acf.wpgraphql.com links at the wpgraphql.com docs portal (#4251)", how = "automatic proposal (before the vulnerable pin), reviewed" }
[[case]]
id = "sylius-mollie-thankyou-order-idor"
repo = "https://github.com/Sylius/MolliePlugin"
language = "php"
also = []
wordpress = false
family = "access_control"
subtype = "IDOR (missing ownership check)"
advisory = ["CVE-2026-68501", "GHSA-x83g-979r-f5fh"]
published = "2026-07-31"
vulnerable = "7400f1489a4b699df97ced9499acde4faaa476d5"
fixed = "01316b3ad3cf82e3c5ad160115d0a2cf89174e49"
ancestry = "first parent of a merge; ancestor of tag v3.3.1"
fix_named_by_advisory = true
sink = "`findOneBy(['id' => $orderId])` without ownership/session check, returning the order token (302 Location) and QR code"
source = "`orderId` of GET /{_locale}/thank-you and /{_locale}/get-code"
sites = ["src/Controller/Shop/PageRedirectController.php::thankYouAction", "src/Controller/Shop/QrCodeAction.php::fetchQrCodeFromOrder"]
site_files = []
site_derivation = "Class::method names quoted by the advisory; paths from the fix commit"
privilege = "unauthenticated"
license = "MIT"
confidence = "high"
caveats = ["merge commit; one fix covers two advisories (webhook forgery GHSA-rc52 is the other)"]
benign = { base = "19f22cd673fde2b6ac750326fecbfe5fc4c06b9e", tip = "91254cc9c9821acbb3c785918759feedd827bdb9", subject = "Add support for Swish gateway (#324)", how = "chosen by hand after review: proposal 45b01569 is a branch upmerge; rejected" }
[[case]]
id = "wp-podlove-generation-guard-poi"
repo = "https://github.com/podlove/podlove-publisher"
language = "php"
also = ["javascript", "typescript"]
wordpress = true
family = "deserialization"
subtype = "PHP object injection → arbitrary file deletion (in-plugin POP chain)"
advisory = ["CVE-2026-16099", "GHSA-q83h-6954-x259"]
published = "2026-08-16"
vulnerable = "4eedbd66796942ab052de2dbd6311a24cc75c4a7"
fixed = "ea878175f2612d54888d9e2aa49ae4db93b724fd"
ancestry = "linear chain of 3 commits (4eedbd66 → d49c4d7f → b59d7962 → ea878175); ancestor of tag 4.5.4"
fix_named_by_advisory = false
inferred_fix_confirmed = ["lib/modules/shownotes/rest_api.php::create_link_item"]
sink = "unserialize() of model data (lib/model/base.php#L552) instantiating `Podlove\\ImageCache\\GenerationGuard`, whose `__destruct()` calls `wp_delete_file()`"
source = "shownotes REST input handled by `create_link_item` (contributor+)"
sites = ["lib/modules/shownotes/rest_api.php::create_link_item", "lib/image_cache/generation_guard.php::__destruct", "lib/model/base.php::to_array"]
site_files = ["lib/model/base.php"]
site_basis = { "lib/modules/shownotes/rest_api.php::create_link_item" = "advisory-named", "lib/image_cache/generation_guard.php::__destruct" = "advisory-named", "lib/model/base.php::to_array" = "resolved-from-fix-diff" }
site_derivation = "create_link_item and GenerationGuard::__destruct named by the advisory; files from the advisory's own trac reference links"
privilege = "Contributor"
license = "MIT"
confidence = "medium"
caveats = ["fix inferred from file history (three commits)", "the advisory text describes the impact (file deletion) under CWE-502"]
benign = { base = "af187ad4bba2739839a0aa1e6cf2cabd6f9b99f5", tip = "50da8db0f5a85afaf27cd6199940be0403b368ec", subject = "fix PHP warning", how = "automatic proposal (after the fix), reviewed" }
+63
View File
@@ -0,0 +1,63 @@
# Evaluation protocol v1 for the independent population (pre-registered)
Written and committed on 2026-09-24, BEFORE the first scan of any case in `population-v1.toml`. It fixes what is
measured, how a result is scored and what is reported, so that no rule of the scoring can be chosen after seeing
an outcome. Changing it after the first scan makes the run exploratory, and it must then be reported as such.
## What is under test
- The analyzer at the commit that introduces the evaluation runner, exported with
`benchmarks/push/freeze_source.sh`; its hash is recorded in the result.
- Per case, only the case's own family (`injection`, `untrusted_destination` or `config_secrets`), through the
production discovery and scan (`benchmarks/push/finding_dump.py`), unbounded region count (5,000), a 2,400 s
deadline per scan, one engine container at a time.
- Four scans per case: the vulnerable pin, the fixed pin, the benign base and the benign tip. Each checkout is a
`git archive` of the pin, with no project tooling run on it.
## Scoring, per case
A finding MATCHES the case when its file is one the fix changes (`git diff --name-only vulnerable fixed`), its
family is the case family, and either its function is one the record names for the sink or fix site, or its line
lies within 10 lines of a hunk the fix changes on that side.
- **Detected** (recall): at least one matching finding at the vulnerable pin.
- **Fixed-side false alert**: a matching finding at the fixed pin.
- **Benign false alert**: any finding of the case family at the benign tip whose (file, function) has no finding
at the benign base.
- **Unanswered**: the scan completed no question covering the sink file's function. It counts as NOT detected,
and the reason is reported. A scan with no questions at all, or unreadable input, is an instrument failure: it
is rerun once, and if it fails again it is reported as a failure, never as a clean result.
## Precision
All findings of the case family at the VULNERABLE pin, across the population, are pooled and sorted by
(case, site). Every third one, starting from the first, is adjudicated against the source, as are all of them if
the pool holds 15 or fewer. The same standard as the PMPro adjudications applies:
- **True**: attacker-controlled data reaches the sink without a sufficient guard.
- **True, privileged**: the same, reachable only with a privileged role.
- **False, guarded**: a guard makes the flow safe.
- **False, not attacker**: the value does not come from an attacker.
The matched case findings are part of the pool and are adjudicated like any other.
## What is reported
Per case: detected / unanswered, fixed-side and benign false alerts, completion (questions completed / total)
for each of the four scans, and seconds. Overall:
- recall with a Wilson 95% interval;
- sampled precision with a Wilson 95% interval;
- total fixed-side and benign false alerts.
These are reported against the M4 profile: at least 95% precision, at least 90% recall, at least 95% completion,
and zero fixed or benign false alerts. A gate that is not met is reported as not met.
## What may not happen
- No rule, fact, query or scoring change informed by these results before the result is committed.
- Any change made after it is a retune. A retune can only be qualified on a NEW untouched population, as
release-milestones M4 requires.
- The OpenCVE case is not published until its licence is checked. Checked 2026-10-02: BUSL-1.1 (not OSI; non-production
use permitted; converts to Apache-2.0 on 2030-08-14). The maintainer decided to publish the case record, which holds
metadata only (repository, pins, detection) and no OpenCVE code.
+90
View File
@@ -0,0 +1,90 @@
# Evaluation protocol v2 for the independent population v2 (pre-registered)
Written and committed on 2026-09-26, BEFORE the first scan of any case in `population-v2.toml`. It fixes what is
measured, how a result is scored and what is reported, so that no rule of the scoring can be chosen after seeing
an outcome. Changing it after the first scan makes the run exploratory, and it must then be reported as such.
Differences from protocol v1, each decided by the maintainer before any v2 case was scanned, and each a lesson
of the v1 run rather than of any v2 case:
- **Declared sites.** v1 credited a finding only in a file the fix changed. alerta and YesWiki (v1) execute their
SQL in a helper the fix does not touch, so a correct report where the query runs could never count. Each v2
case therefore declares, from its advisory and before any scan, the `sites` (`path::function`) where the
vulnerable operation happens; a finding there is credited too.
- **Two runs.** The engine's path selection is not deterministic: on v1 retunes one verdict flipped in 2 of 4
identical runs, and a PMPro finding came and went between runs. The vulnerable and the fixed pin are scanned
twice each, and the difference is reported.
## What is under test
- The analyzer at the commit that introduces the v2 evaluation runner, exported with
`benchmarks/push/freeze_source.sh`; its hash is recorded in the result.
- Per case, only the case's own family (`injection`, `untrusted_destination` or `config_secrets`), through the
production discovery and scan (`benchmarks/push/finding_dump.py`), unbounded region count (5,000), a 2,400 s
deadline per scan, one engine container at a time.
- Six scans per case: the vulnerable pin twice (runs a, b), the fixed pin twice (a, b), the benign base and the
benign tip once each. Each checkout is a `git archive` of the pin, with no project tooling run on it.
## Scoring, per case
A finding MATCHES the case when its family is the case family and either
1. its file is one the fix changes (`git diff --name-only vulnerable fixed`) and either its function is one the
record names in `sites`, or its line lies within 10 lines of a hunk the fix changes on that side (v1's rule);
or
2. its `path::function` equals one of the record's declared `sites` (`<global>` for file scope).
- **Detected** (recall): a matching finding in BOTH vulnerable runs.
- **Unstable detection**: a matching finding in exactly one vulnerable run. Reported; counts as NOT detected.
- **Fixed-side false alert**: a matching finding in EITHER fixed run.
- **Benign false alert**: any finding of the case family at the benign tip whose (file, function) has no finding
at the benign base.
- **Unanswered**: neither vulnerable run completed a question covering a file the fix changes or a declared site
file. It counts as NOT detected, and the reason is reported. A scan with no questions at all, or unreadable
input, is an instrument failure: it is rerun once, and if it fails again it is reported as a failure, never as
a clean result.
## Stability
Per pin with two runs, the findings present in exactly one run divided by the findings present in either is
reported, per case and pooled. It is not a gate; it says how far a single run can be trusted.
## Precision
All findings of the case family present in BOTH vulnerable runs, across the population, are pooled and sorted by
(case, site). Every third one, starting from the first, is adjudicated against the source, as are all of them if
the pool holds 15 or fewer. Findings present in only one run are counted and reported, not adjudicated. The
standard is v1's:
- **True**: attacker-controlled data reaches the sink without a sufficient guard.
- **True, privileged**: the same, reachable only with a privileged role.
- **False, guarded**: a guard makes the flow safe.
- **False, not attacker**: the value does not come from an attacker.
The matched case findings are part of the pool and are adjudicated like any other.
## What is reported
Per case: detected / unstable / unanswered, fixed-side and benign false alerts, completion (questions completed /
total) for each of the six scans, stability, and seconds. Overall:
- recall with a Wilson 95% interval;
- sampled precision with a Wilson 95% interval;
- total fixed-side and benign false alerts;
- completion over all scans;
- pooled stability.
These are reported against the M4 profile: at least 95% precision, at least 90% recall, at least 95% completion,
and zero fixed or benign false alerts. A gate that is not met is reported as not met.
## What may not happen
- No rule, fact, query or scoring change informed by these results before the result is committed.
- Any change made after it is a retune. A retune can only be qualified on a NEW untouched population.
- No result of a case whose licence is marked "verify" is published until the licence is checked.
- 2026-10-02: the five v2 cases that carried the "verify" marker were checked read-only at their pinned
vulnerable commits (LICENSE / license.txt / plugin header `License:` line via the GitHub API) and the
`license` field now records the SPDX id with the evidence: pgadmin-maintenance-sqli is PostgreSQL;
wp-gopay-log-filter-sqli is GPL-2.0-or-later; wp-user-registration-login-redirect, wp-directorist-avatar-ssrf
and wp-groundhogg-confirm-redirect are GPL-3.0-or-later. All five are OSI-approved. The OpenCVE case in v1
is BSL-1.1, which is not OSI-approved; non-production use is permitted by its Additional Use Grant, and its
change date is 2030-08-14 (to Apache-2.0).
+173
View File
@@ -0,0 +1,173 @@
# Evaluation protocol v3 for the independent PHP population v3 (pre-registered)
Written and committed on 2026-09-30, AFTER `population-v3-php.toml` was frozen and BEFORE any scanner, quick rule,
engine, plane task or model has been run on any v3 repository. It fixes what is measured, how a result is scored
and what is reported, so that no rule of the scoring can be chosen after seeing an outcome. Changing it after the
first scan makes the run exploratory, and it must then be reported as such.
Differences from protocol v2, each decided by the maintainer on 2026-09-30 before any v3 case was scanned:
- **Six families.** v2 scored `injection`, `untrusted_destination` and `config_secrets`. v3 pre-registers matching
for `injection`, `output_encoding`, `path`, `untrusted_destination`, `access_control` and `deserialization`
("Extend protocol v3 to all six"), with one rule for all of them (below).
- **Resolved sites.** Where an advisory names only a file, freeze read that file's diff at the fix commit and
declared the changed function(s) as the site ("Resolve the function at freeze"). `site_basis` in the population
says which sites the advisory named and which were resolved from the fix diff; both are declared sites and are
credited alike. A file the fix does not change was not resolved (`site_files_unresolved`) and is not a site.
- **Three instruments**, each scored on its own (below).
## The population
`population-v3-php.toml`, status `frozen`, record `freeze-v3-php.json`: 15 cases (injection 5, output_encoding 3,
path 3, untrusted_destination 1, access_control 2, deserialization 1). The second untrusted_destination slot is
empty by the selection rule and stays empty; recall is over the 15 cases.
## What is under test
All three instruments at ONE analyzer commit: the commit that introduces the v3 evaluation runner, exported with
`benchmarks/push/freeze_source.sh`; its hash, the runner image digest and the model ids are recorded in the result.
No instrument's configuration may be changed between instruments or between cases.
1. **Quick PHP rules** -- `src/openultrasast/ruleset/php/rules.toml` through the production quick scan
(`ousast scan <checkout> --mode quick`, default configuration: `min_emit_priority` and `min_emit_precision`
0.0). Scored: the ENABLED rules. The shadow rules (`shadow_findings.json`) are scored the same way and reported
separately as exploratory; they do not enter any gate. A quick finding's family is fixed by this table, and a
rule not in it has no family and never matches:
| family | rules |
|---|---|
| injection | `wpdb-sql-composition`, `sql-unquoted-concat`, `sql-unquoted-interpolation`, `sql-sprintf-unescaped`, `command-injection`; shadow `sql-call-composition`, `code-eval` |
| path | `file-inclusion-input`, `path-input` |
| untrusted_destination | `ssrf-input`, `open-redirect` |
| deserialization | shadow `unserialize` |
| output_encoding | shadow `xss-echo-input` |
| access_control | none |
A finding's site is `path:line:function` from the finding's `path`, `line` and `function_name`.
2. **The Joern engine** -- the production discovery and scan (`benchmarks/push/finding_dump.py`) inside the
shipped image, per case only the case's own family (`--families <family>`), 5,000 regions, a 2,400 s deadline
per scan, one engine container at a time, `--network none --memory 3g`, as `evaluate.py run` does.
3. **The plane pipeline** -- the ai-service plane as implemented at that commit: candidates -> triage -> verify
pass a and pass b -> tie-break pass c on the candidates a and b dispute -> agreement. A candidate is a finding
when a and b both flag it, or, for a disputed candidate only, when at least 2 of the 3 passes flag it; pass c
alone never decides. Model: `deepseek-flash` (DeepSeek) for triage and all three verify passes, as in
`plane/models/deepseek-flash.yaml`. Site: the candidate's anchor `path:line:function` (line 0 when the model
names another file); family: the case's family.
Per case and instrument, six pins, protocol v2's: the vulnerable pin twice (runs a, b), the fixed pin twice (a,
b), the benign base and the benign tip once each. Each checkout is a `git archive` of the pin. For the plane,
"run a" and "run b" are two complete, independent executions of the whole pipeline (each with its own a/b/c
passes). The quick rules are deterministic: their two runs must be identical, and a difference is an instrument
failure.
## Instrument checks (before any number is believed)
- Every scan records the files and bytes it read. A scan that read no file of the checkout, asked no question
(engine), or completed no task (plane) is an instrument failure: it is rerun once, and if it fails again it is
reported as a failure, never as a clean result.
- A scan finishing implausibly fast for its instrument (an engine scan under the JVM start time) is treated as a
failed launch and checked before it is scored.
- The plane is smoke-tested once, before the run, on a development case (never a v3 case), to show that model
calls succeed and are priced; the smoke test's cost counts against the budget.
## Scoring, per case (all six families, all three instruments)
A finding MATCHES the case when its family is the case family (exact string) and either
1. its file is one the fix changes (`git diff --name-only vulnerable fixed`) and either its function is one the
record names in `sites`, or its line lies within 10 lines of a hunk the fix changes on that side; or
2. its `path::function` equals one of the record's declared `sites` (`<global>` for file scope), whatever the
site's basis.
This is `evaluate.matches_v2` unchanged; `evaluate.py --population population-v3-php.toml` selects the six-pin
protocol and `score_v2`. There is no cross-family credit: a finding of another family never matches, even at a
declared site.
What a match means, per family (for the adjudicator and the report; the matching rule above is the same):
- **injection** -- the interpreter call (SQL, OS command, code evaluation, template compilation) or the function
that builds its argument.
- **output_encoding** -- the rendering or sanitizing point (an echo/print of the value, a response body, a
sanitizer configuration) at a declared site or in the fix's range. families.toml's limits hold: a clean result
says nothing about template-mediated escaping.
- **path** -- the filesystem operation, the inclusion, or the containment check the fix changes.
- **untrusted_destination** -- the outbound request or redirect, or the destination guard the fix changes.
- **access_control** -- an absence bug: the finding names the protected operation that lacks its guard, at the
declared function where the guard belongs or in the fix's changed lines, where the fix adds it.
- **deserialization** -- the deserializing call, the entry that feeds it, or the gadget method the advisory
names.
Outcomes, as protocol v2:
- **Detected** (recall): a matching finding in BOTH vulnerable runs.
- **Unstable detection**: a matching finding in exactly one vulnerable run. Reported; counts as NOT detected.
- **Fixed-side false alert**: a matching finding in EITHER fixed run.
- **Benign false alert**: any finding of the case family at the benign tip whose (file, function) has no finding
at the benign base. Some benign pairs precede the vulnerable pin (`how` in the population says so); they are
scored the same way.
- **Unanswered**: neither vulnerable run completed a question, rule pass or task covering a file the fix changes
or a declared site file. Counts as NOT detected, with the reason.
- **Not covered**: the instrument has no rule or query for the case family (quick rules: access_control; any
family the engine or plane cannot emit). Counts as NOT detected, reported separately from misses.
- **Not run**: a pin the budget ceiling stopped. Counts as NOT detected and as not completed.
## Stability and precision
Stability: per pin with two runs, findings in exactly one run over findings in either, per case and pooled. Not a
gate.
Precision, per instrument: all findings of the case family present in BOTH vulnerable runs, across the
population, pooled and sorted by (case, site); every third one, starting from the first, is adjudicated against
the source, or all of them if the pool holds 15 or fewer. Findings in only one run are counted, not adjudicated.
The standard is v1's: True; True, privileged; False, guarded; False, not attacker. Matched case findings are part
of the pool.
## Gates
These are reported against the M4 profile: at least 95% precision, at least 90% recall, at least 95% completion,
and zero fixed or benign false alerts. A gate that is not met is reported as not met.
- The gates apply to EACH instrument separately, on its own findings: the quick rules (enabled rules only), the
Joern engine, and the plane pipeline. The plane pipeline is the product's decision path; its result is the
M4 answer for PHP. The quick rules and the engine are reported against the same gates as components.
- Completion is, per instrument: quick rules -- files scanned over PHP files in the checkout; engine -- questions
completed over questions asked (v2); plane -- tasks completed over tasks run, with budget-stopped pins counted
as not completed.
- Recall is detected cases over 15, with a Wilson 95% interval; precision is the sampled fraction True (either
kind), with a Wilson 95% interval. With 15 cases, 90% recall needs 14 detections.
## Budget
Model spend (the plane only; the quick rules and the engine call no model) is metered by the plane's
`MeteredClient` (`OUSAST_BUDGET_USD`, `OUSAST_BUDGET_CALLS`) and recorded per task.
- Hard ceiling for the whole v3 evaluation: **$60** of recorded spend, smoke test and triage included.
- Per case: **$6** across its six pins; per pass, the plane's own per-pass ceiling at the runner commit.
- Reference: $0.02 per candidate for two verify passes with triage (plane-increment-2.json); the largest v2 case
cost $8.91 in the earlier script pipeline.
- When a ceiling stops a case, its remaining pins are "not run"; the run continues with the next case until the
total ceiling. The ceilings are not raised after the first v3 task has started.
## What is reported
Per case and instrument: detected / unstable / unanswered / not covered / not run, fixed-side and benign false
alerts, completion for each of the six scans, stability, seconds and (plane) dollars. Overall, per instrument:
recall and precision with Wilson intervals, total fixed-side and benign false alerts, completion, pooled
stability, spend. Shadow quick rules are reported separately as exploratory.
## What may not happen
- No rule, fact, query, prompt, model, budget or scoring change informed by these results before the result is
committed.
- Any analyzer change made after a v3 result -- to rules, queries, facts, prompts, models, the plane's tasks or
the pipeline's agreement rule -- is a retune. v3 cannot qualify it; a retune can only be qualified on a NEW
untouched population.
- No v3 repository may be named by a test, fixture, rule or measurement outside `benchmarks/independent/`
(`tests/test_independent_population.py`).
- No result of a case whose licence is marked "verify" is published until the licence is checked. Kirby is not
OSI-licensed: its results are unpublishable without the licensor's permission.
## Maintainer confirmation (2026-09-30, before any scan)
The maintainer confirmed the two choices made while drafting: the model-spend ceiling of $60 in total and $6 per
case (a case stopped by its cap counts as not detected), and shadow quick rules reported as exploratory only,
outside every gate. No v3 repository had been scanned when this was confirmed.
@@ -0,0 +1,151 @@
# Independent evaluation population: candidate real-world cases
Research date: 2026-09-24. Selected by the maintainer: 11 of these 15 (see population-v1.toml); not selected: The Events Calendar, Tutor LMS, pyLoad, unfurl. Method: web research only. I read GitHub advisories (GHSA/CVE), commit pages, compare views and diffs through the GitHub REST API (`gh api`) and the web. I did not clone any repository or run any scanner or project tooling.
Exclusion check: I compared every repo URL below against `used-repos.txt` (case-insensitive exact match on `https://github.com/<owner>/<repo>`). None of them is on the list. None is NodeGoat, VAmPI, PMPro, WP Statistics, MW WP Form, libpng, a deliberately vulnerable training app or a benchmark suite. I did not use Ghost.
"Diff seen" means I read the patch hunks of the named fix commit through the GitHub API. "Vuln" is the parent of the fix commit unless a tag is given. Tags come with the commit SHA they resolve to.
## Summary table
| # | id | repo | lang | family | advisory | vulnerable (commit / tag) | fixed (commit / tag) | sink file :: function | privilege | diff seen |
|---|----|------|------|--------|----------|---------------------------|----------------------|-----------------------|-----------|-----------|
| 1 | wp-ultimate-member-sqli | https://github.com/ultimatemember/ultimatemember | PHP+JS (WP plugin) | SQLi | CVE-2024-1071 / GHSA-mh7h-gpmx-fggj | tag 2.8.2 = `e1885a73a8ad542cfd2ed60aef38a6bce4ab8553`; commit-level parent `a49b3061ded1630e4f6860b4ebece5ccf0a493d2` | `476de92f4c9aaa56c3e2f5527b242ce0476f4655` (+ `c61008bdbe1ce8de4597b41c7aad8dfda88f527d`); tag 2.8.3 = `a6a7d13a433b24e468ee10e778e3da5668b6ef59` | includes/core/class-member-directory-meta.php :: `Member_Directory_Meta::ajax_get_members()` (`$this->sql_order` ORDER BY) | unauthenticated | yes |
| 2 | wp-learnpress-sqli | https://github.com/LearnPress/learnpress | PHP+JS (WP plugin) | SQLi | CVE-2024-8522 / GHSA-3w3r-r6g6-w8x5 (+ CVE-2024-8529 / GHSA-98wp-w76v-f75p) | `73b3cbceef8669056e9da5e2f2c78944eb139af5` (4.2.7) | `aaf42fe0c6bf05dcf2ff125750b7bd836e76fa3c` (4.2.7.1) | inc/Databases/class-lp-db.php :: `LP_Database::execute()` (`implode(',', $filter->only_fields)` into SELECT); source inc/jwt/rest-api/version1/class-lp-rest-courses-v1-controller.php :: `get_courses()` | unauthenticated (REST `/wp-json/learnpress/v1/courses`) | yes |
| 3 | wp-events-calendar-sqli | https://github.com/the-events-calendar/the-events-calendar | PHP+JS (WP plugin) | SQLi | CVE-2024-8275 / GHSA-v6x4-m48j-jm68 | tag 6.6.4 = `4e4baa3fee7ef49abdd4cb277d963d555ffab4c7` | tag 6.6.4.1 = `419965a87d99195ad3fca22738e19dd287051f1c` (fix hunk in `e128ec5e2a25fe4fdc4542a9023918bf915b4084`) | src/Events/Custom_Tables/V1/WP_Query/Custom_Tables_Query.php :: `redirect_posts_orderby()` / `parse_orderby()` | unauthenticated per Wordfence (reached through `tribe_has_next_event` `order`) | yes |
| 4 | wp-tutor-lms-coupon-sqli | https://github.com/themeum/tutor | PHP+JS/TS (WP plugin) | SQLi | CVE-2025-13673 / GHSA-f7q9-gg7h-5rff | tag v3.9.6 = `71187a525dbdbdb045137af702307f05dda6a7a6` (= parent of fix) | `e4a25f69f0fb63a46d57eff6ba5b58378a4c088d` (3.9.7, no GitHub tag) | helpers/QueryHelper.php :: `prepare_where_clause()` / `make_clause()`; source models/CouponModel.php :: `get_coupon_details_for_checkout()`, `get_coupon_by_code()` | unauthenticated (checkout `coupon_code`) | yes (mapping partly inferred) |
| 5 | phpmyfaq-captcha-ua-sqli | https://github.com/thorsten/phpMyFAQ | PHP+TS/JS | SQLi | CVE-2026-46364 / GHSA-289f-fq7w-6q2w | `11583abcf4787543d00c25d6c7926af6f509faf0`; tag 4.1.1 = `f791858b2aa4b811f5b19d10c20ba5c4a30786de` | `545bdffb11244a4741cd29ac909a849c9a6a2e53`; tag 4.1.2 = `66792f2575b07d65896b504d47deb5f4f903f9ff` | phpmyfaq/src/phpMyFAQ/Captcha/BuiltinCaptcha.php :: `garbageCollector()`, `saveCaptcha()` | unauthenticated (`GET /api/captcha`, User-Agent header) | yes |
| 6 | yeswiki-bazar-form-sqli | https://github.com/YesWiki/yeswiki | PHP+JS | SQLi | CVE-2026-46670 / GHSA-jwvv-qr7q-cv8j | `6f222915df36fca628fc83325cb0feed8dc2aca0`; tag v4.6.3 = `5e8fc114d01d558dc4799b95219549ccf02ebd11` | `fe7244bffcbeffd42777496ebc2a6f6118cc2388`; tag v4.6.4 = `c1bae9fc260eb3e4346f3b0bc65ab609d1cf783a` | tools/bazar/services/FormManager.php :: `FormManager::create()`; source tools/bazar/controllers/FormController.php :: `displayAll()` | unauthenticated | yes |
| 7 | signalk-remote-conn-ssrf | https://github.com/SignalK/signalk-server | TypeScript (Node/Express) | SSRF | CVE-2026-55591 / GHSA-q59x-jc9f-gfqf | `1c479cc0d2d26d14ed74416cff36fa7336c7bf9e`; tag v2.27.0 = `4592992aaf7c44548842be9390b651cfb816eaf4` | `2816b0dda1bdde893d85c969a9c5950e7825eee7`; tag v2.28.0 = `3645160b235364e1fd3afe1f2f6b2270b8194b7d` | src/serverroutes.ts :: `makeRemoteRequest()` (from `testSignalKConnection`, `requestAccess`, `checkAccessRequest` routes) | unauthenticated by default (security not configured); admin otherwise | yes |
| 8 | saltcorn-sync-sqli | https://github.com/saltcorn/saltcorn | JavaScript/TS (Node/Express) | SQLi | CVE-2026-41478 / GHSA-jp74-mfrx-3qvh | `e7e09254a3c69dc6582eda9e5b5769935db108af`; tag v1.4.5 = `92d24ca5ce8480d3b0cf1067e33113f2a790030c` | `2b6c8a2eab88955ea6b12b1b4e5c2e613faefef5`; tag v1.4.6 = `d49e6a947de777053a203a27ab4d9e1a1754ccd0` (backport) | packages/server/routes/sync.js :: `getSyncRows()`, `getDelRows()` (+ packages/saltcorn-data/models/table.ts sync_info queries) | authenticated low-privilege user | yes |
| 9 | mongo-express-bson-rce | https://github.com/mongo-express/mongo-express | JavaScript (Node/Express) | code injection (vm eval) | CVE-2019-10758 / GHSA-h47j-hc6x-h3qq | `ea02b364d43f179f191fc91fb9962efdb0843a8d`; tag v0.53.0 = `9eb49994f3842260f7c6db75592ea6ba9376fad6` | `d8c9bda46a204ecba1d35558452685cd0674e6f2`; tag v0.54.0 = `7d365141deadbd38fa961cd835ce68eab5731494` | lib/bson.js :: `toBSON()` (`vm.runInNewContext` over request text); callers lib/routes/collection.js, lib/routes/document.js | authenticated (HTTP basic auth, default admin:pass) | yes |
| 10 | mockoon-admin-api-cors | https://github.com/mockoon/mockoon | TypeScript (Node/Express) | permissive CORS | CVE-2026-59148 / GHSA-rqx4-3f6q-3x2v | `b42bdfb7f82e83f0e81bea8e6fe41adf5ec82585`; tag v9.6.1 = `1c874584c058b891f6affbb44d5a5223f3f8f812` | `c420b5a56918475b8663977b51e5f986e45b3299`; tag v9.7.0 = `df02ba3b0d9e8b88651230f3b38d8e6fa3679b9b` | packages/commons-server/src/libs/server/admin-api.ts :: admin-API middleware (`app.use(`${adminApiPrefix}*`)`, `Access-Control-Allow-Origin: *`) | unauthenticated | yes (large diff, one CORS hunk) |
| 11 | pyload-add-package-sqli | https://github.com/pyload/pyload | Python (Flask) | SQLi | CVE-2025-55156 / GHSA-pwh4-6r3m-j2rf | `a123accec0618798db77f9f5bc66838b741d9f58` | `134edcdf6e2a10c393743c254da3d9d90b74258f` (0.5.0b3.dev91) | src/pyload/core/database/file_database.py :: `FileDatabaseMethods.update_link_info()`; source src/pyload/webui/app/blueprints/json_blueprint.py :: `add_package` | authenticated (ADD permission) | yes |
| 12 | alerta-query-sqli | https://github.com/alerta/alerta | Python (Flask) | SQLi | CVE-2026-34400 / GHSA-8prr-286p-4w7j | `71c098d8bb8091e76edaefeef4662eb77c753ce7`; tag v9.0.4 = `b7111d23ae9aefb5064ef107976f62e3836217b6` | `fdd52cd1abad8d02d1dfb8ecdcdbb43b6af3b883`; tag v9.1.0 = `f4312727783dad64198b62ed778bbca98a5144a3` | alerta/database/backends/postgres/queryparser.py :: `SearchTerm.__repr__` (f-string WHERE); used by alerta/database/backends/postgres/utils.py :: `QueryBuilder.from_params()` | API user (API key when `AUTH_REQUIRED`; see notes) | yes |
| 13 | wger-next-open-redirect | https://github.com/wger-project/wger | Python (Django) | open redirect | GHSA-vqv8-j3mj-wjxj (no CVE) | `d30e85b23fb1307ac6579e9e3ae393c50470cea7`; tag 2.5 = `c1fca6bdf77d8ed80048d1b73a8da80cf36ee751` | `0ba371bbb50cd5668820f240995405e1b2483384`; tag 2.6 = `e1d70bcc38cd56ae4a254dca1713c404d069f319` | wger/core/views/user.py :: `trainer_login()`; wger/core/views/languages.py :: `use_browser_language()` | `use_browser_language`: unauthenticated; `trainer_login`: authenticated trainer | yes |
| 14 | unfurl-flask-debug | https://github.com/obsidianforensics/unfurl | Python (Flask) | permissive config (debug on) | GHSA-vg9h-jx4v-cwx2 (no CVE) | `6b8e97853090c909936a18d41228168b7a35fe94` | `4c0a07ab1e9af3a1ddf0e7f47153ec9ba77946dd` | unfurl/app.py :: `web_app()` → `UnfurlApp.__init__` `app.run(debug=...)` | unauthenticated (Werkzeug debugger) | yes |
| 15 | opencve-notification-ssrf | https://github.com/opencve/opencve | Python (Django) | SSRF | CVE-2026-62282 / GHSA-ch3g-4xvr-674q | `928bbadbb1672051bcfc2a6289a9e6a8f029c106` | `0af7f894debc95c67fa53e7ba5a6cdecb10c34f0`; tag v3.0.0 = `928943aa87ceac789061114159d74c2067cc6ae6` | web/projects/notifications.py :: `_http_post_json()` (webhook and Slack test); forms in web/projects/forms.py | authenticated (can configure project notifications) | yes |
Mix: 6 PHP (all six repos ship JS, four are WordPress plugins), 4 JS/TS, 5 Python. By family: 10 injection (9 SQLi: #1–6, #8, #11, #12; 1 code injection: #9), 3 untrusted destination (SSRF #7 and #15, open redirect #13), 2 permissive configuration (CORS #10, debug #14). By privilege: 10 unauthenticated or unauthenticated by default (#1–7, #10, #14, plus #13's `use_browser_language` sink) and 5 authenticated (#8, #9, #11, #15, and #12 depending on `AUTH_REQUIRED`).
## Per-candidate notes
### 1. wp-ultimate-member-sqli (PHP+JS, WordPress)
- Repo: https://github.com/ultimatemember/ultimatemember (GPL, PHP 2.7 MB + JS 0.6 MB).
- Advisory: CVE-2024-1071, GHSA-mh7h-gpmx-fggj (unreviewed GHSA mirror of the Wordfence record). Affects 2.1.3–2.8.2, CVSS 9.8.
- Flaw: `ajax_get_members()` builds `$this->sql_order = " ORDER BY CAST(umm_sort.um_value AS {$custom_sort_type}) {$custom_sort_order} "` and similar ORDER BY strings. Values from the member-directory `sorting` request pass only through `esc_sql()`. That does not protect a context without quotes. Only exploitable when the "custom usermeta table" option is on.
- Fix: `476de92f` checks `$order` / `$custom_sort_order` against `ASC`/`DESC` and `$custom_sort_type` against `$this->sort_data_types`. `c61008bd` ("fixed security issue with mysql query escape") is a larger WPCS refactor of the same file. Use `476de92f` as the minimal fix and the 2.8.2 → 2.8.3 tags for a release-level pair.
- Confidence: high. I read both diffs. The 2.8.2...2.8.3 compare has 94 commits, so a tag pair gives a noisy diff. The commit pair is cleaner.
### 2. wp-learnpress-sqli (PHP+JS, WordPress)
- Repo: https://github.com/LearnPress/learnpress (PHP 5.9 MB, JS 1.4 MB).
- Advisories: CVE-2024-8522 (`c_only_fields`, CVSS 10.0) and CVE-2024-8529 (`c_fields`). Both are fixed by the same commit.
- Flaw: the REST controller `get_courses()` put `explode(',', $params['c_only_fields'])` straight into `$filter->only_fields`. `LP_Database::execute()` then runs `implode(',', array_unique($filter->only_fields))` into the SELECT column list (class-lp-db.php around line 607).
- Fix `aaf42fe0`: `Courses::handle_params_for_query_courses()` now runs every field through `$wpdb->prepare('%i', $field)` (identifier quoting), and the controller's direct assignment is removed. It also touches `assets/src/js/loadAJAX.js`, so the commit is itself multi-language.
- Confidence: high. I read the diff and checked the sink line at the vulnerable commit.
### 3. wp-events-calendar-sqli (PHP+JS, WordPress)
- Repo: https://github.com/the-events-calendar/the-events-calendar (large plugin: PHP 10 MB, JS 0.7 MB; license field NOASSERTION, the plugin is GPL). **It has a git submodule `common` (tribe-common), so a checkout needs `--recursive`.**
- Advisory: CVE-2024-8275, GHSA-v6x4-m48j-jm68. The `order` parameter of `tribe_has_next_event`, ≤ 6.6.4.
- Flaw: `redirect_posts_orderby()` split ORDER BY fragments with a regex and kept any trailing `order` token as is. `parse_orderby()` inserted `$order` into `"ID $order, ..."`.
- Fix (6.6.4.1, commit `e128ec5e`): adds `sanitize_order()` (only ASC/DESC) and applies it at both sites. It also bumps the submodule.
- Privilege: Wordfence rates it unauthenticated. Reachability in practice depends on how the template tag's arguments reach WP_Query. The ground-truth sink is still unambiguous.
- Confidence: high for the diff. Medium for how exploitable it is from a clean install.
### 4. wp-tutor-lms-coupon-sqli (PHP+JS/TS, WordPress)
- Repo: https://github.com/themeum/tutor (GPL-3.0; PHP 3.8 MB, TS 3.3 MB, JS 0.4 MB).
- Advisory: CVE-2025-13673, GHSA-f7q9-gg7h-5rff. ≤ 3.9.6, unauthenticated `coupon_code`. Partial fixes shipped in 3.9.4 and 3.9.6.
- Flaw: `QueryHelper::prepare_where_clause()` built values as `"'" . $val . "'"`. The only escaping was an `esc_sql()` in `get_coupon_details_for_checkout()`. Other coupon paths (`get_coupon_by_code()`, `get_coupon_usage_count()`, `get_coupon_applications()`) pass `$coupon_code` through without escaping.
- Fix `e4a25f69` (parent = tag v3.9.6): `make_clause()` now binds every value through `$wpdb->prepare('%s'|'%d', ...)`, and the manual quoting and `esc_sql` are removed.
- Confidence: medium. I read the diff and it is the only SQL change right after v3.9.6. The trac changeset cited by the CVE (3469242) returned 403, so the exact exploited call chain from the checkout request to QueryHelper is my inference. There is no v3.9.7 tag on GitHub. Use the commit pair.
### 5. phpmyfaq-captcha-ua-sqli (PHP+TS/JS)
- Repo: https://github.com/thorsten/phpMyFAQ (MPL-2.0; PHP 14.7 MB, TS 1.7 MB).
- Advisory: CVE-2026-46364, GHSA-289f-fq7w-6q2w. ≤ 4.1.1. The advisory's commit reference `b9f25109` is the **vulnerable** master commit the reporter tested, not the fix.
- Flaw: the constructor stores `$request->headers->get('user-agent')` and `getClientIp()`. `garbageCollector()` (DELETE) and `saveCaptcha()` (SELECT/INSERT) put them into `sprintf` SQL without escaping. Both run on every `GET /api/captcha`.
- Fix `545bdffb` ("fix: added missing escaping"): adds `escapeQueryValue()` for userAgent, ip, language and code.
- Confidence: high. I read the diff in the 4.1.1...4.1.2 compare and the commit.
### 6. yeswiki-bazar-form-sqli (PHP+JS)
- Repo: https://github.com/YesWiki/yeswiki (AGPL-3.0; PHP 4 MB, JS 0.8 MB). Default branch `doryphore-dev`.
- Advisory: CVE-2026-46670, GHSA-jwvv-qr7q-cv8j. < 4.6.4.
- Flaw: `FormManager::create()` concatenates `$data['bn_id_nature']` unquoted into `INSERT ... VALUES (`. The data comes from `$_POST['imported-form']` JSON in `FormController::displayAll()`, and that path had no permission check.
- Fix `fe7244bf`: `intval($data['bn_id_nature'])` plus a `Guard::isAllowed('saisie_formulaire')` check. The diff is 4 lines.
- Confidence: high. I read the diff. Related YesWiki SQLi advisories (GHSA-qg78-vmvc-fhjw, GHSA-f58v-p6j9-24c2, GHSA-4pf7-cc4r-g63h) could add more cases from the same repo.
### 7. signalk-remote-conn-ssrf (TypeScript, Node/Express)
- Repo: https://github.com/SignalK/signalk-server (Apache-2.0, about 24 MB).
- Advisory: CVE-2026-55591, GHSA-q59x-jc9f-gfqf. ≤ 2.27.0.
- Flaw: `makeRemoteRequest(host, port, useTLS, selfsignedcert, path)` passes `req.body.host`/`port` to `http(s).request` without validation. It is reached from three admin routes.
- Fix `2816b0dd`: TypeBox schema validation (new src/remoteConnectionSchemas.ts), a new src/ssrfGuard.ts (BlockList for loopback, link-local and metadata, plus a DNS lookup guard) wired into `makeRemoteRequest`, and UUID validation of `requestId`.
- Confidence: high. I read the diff. It is medium-sized, about 450 added lines, mostly new guard code and tests.
### 8. saltcorn-sync-sqli (JavaScript/TS, Node/Express)
- Repo: https://github.com/saltcorn/saltcorn (MIT). It is a monorepo of packages and on the large side (about 340 MB of git data), but the relevant code is in `packages/server` and `packages/saltcorn-data`.
- Advisory: CVE-2026-41478, GHSA-jp74-mfrx-3qvh. < 1.4.6 / < 1.5.6 / < 1.6.0-beta.5.
- Flaw: `getSyncRows()` interpolates `${syncInfo.maxLoadedId}` (from `req.body.syncInfos[table].maxLoadedId`) and `${user.id}` into SQL template literals. `getDelRows()` does the same with timestamps.
- Fix `2b6c8a2e`: integer and date validation plus `$1`-style parameterised queries in sync.js and table.ts.
- Confidence: high. I read the diff. The fix commit is on the main line, and v1.4.6 is a separate backport. For a clean pair, use the commit and its parent `e7e09254`.
### 9. mongo-express-bson-rce (JavaScript, Node/Express)
- Repo: https://github.com/mongo-express/mongo-express (MIT, about 16 MB).
- Advisory: CVE-2019-10758, GHSA-h47j-hc6x-h3qq (CWE-94/78). < 0.54.0.
- Flaw: `lib/bson.js` `toBSON()` evaluates user-supplied document/query text with `vm.runInNewContext` in a BSON sandbox. The sandbox can be escaped, which gives RCE through the `/checkValid` and document/collection routes.
- Fix `d8c9bda4`: replaces the vm-based parser with `mongodb-query-parser`. It removes 101 lines from bson.js, deletes lib/json.js and changes collection.js.
- Confidence: high. I read the diff. It is an older case (2019), useful as a known-answer eval sink. Note that the advisory's first reference, `7d365141`, is only the version-bump or release commit (= tag v0.54.0).
### 10. mockoon-admin-api-cors (TypeScript, Node/Express)
- Repo: https://github.com/mockoon/mockoon (MIT; monorepo of app, cli, commons-server and serverless; about 42 MB).
- Advisory: CVE-2026-59148, GHSA-rqx4-3f6q-3x2v (CWE-942/306/352/732). < 9.7.0.
- Flaw: the admin API middleware sets `Access-Control-Allow-Origin: *` with all methods on every `/mockoon-admin/*` endpoint. The admin API is on by default and has no authentication.
- Fix `c420b5a5`: adds a configurable `adminApiCorsOrigins` allowlist (no header by default, 403 on preflight), optional admin API auth, and an env-var prefix check.
- Confidence: high that the CORS hunk is the fix. The commit is **large (34 files)**, but the permissive-CORS sink sits in one clearly identifiable hunk of admin-api.ts. It is a weaker fit for "small diff" than the others.
### 11. pyload-add-package-sqli (Python, Flask)
- Repo: https://github.com/pyload/pyload (AGPL-3.0, GitHub reports NOASSERTION; about 50 MB).
- Advisory: CVE-2025-55156, GHSA-pwh4-6r3m-j2rf. < 0.5.0b3.dev91.
- Flaw: `update_link_info()` builds `"','".join(x[3] for x in data)` into `f"SELECT id FROM links WHERE url IN ('{statuses}')"`. The URLs come from `add_links` on `/json/add_package`. Chain: json_blueprint `add_package` → `Api.add_package` → `FileManager.add_links` → `InfoThread` → `update_file_info` → `update_link_info`.
- Fix `134edcdf`: `?` placeholders. The diff is 5 lines. The source-to-sink path crosses a thread, which makes it a good cross-thread taint test.
- Confidence: high. I read the diff. The same repo also has CVE-2023-0297 (GHSA-pf38-5p22-x6h6, unauthenticated js2py code injection, fix `7d73ba7919e594d783b3411d7ddb87885aea782d`, parent `6aa71cd99a6f510de7137d17ec99f930916704c0`). I kept it as an alternate only, because that fix just calls `js2py.disable_pyimport()`. The `eval_js` sink stays in the fixed tree, so a sink-based analyzer would still correctly flag the fixed version.
### 12. alerta-query-sqli (Python, Flask)
- Repo: https://github.com/alerta/alerta (Apache-2.0, about 11 MB).
- Advisory: CVE-2026-34400, GHSA-8prr-286p-4w7j. < 9.1.0. The advisory's `aeba85a3` reference (2018) is the commit that **introduced** the parser. The fix is `fdd52cd1`.
- Flaw: the Postgres query-string parser (`?q=`) turned parsed search terms into SQL with f-strings in `SearchTerm.__repr__`. `QueryBuilder.from_params()` then used that as a WHERE clause with an empty `qvars`.
- Fix: new `to_sql(params)` methods produce `%(name)s` placeholders, and `from_params()` passes the parsed vars.
- Privilege: alerta requires an API key or login when `AUTH_REQUIRED=True`. I believe the default is `False`, which would make this unauthenticated on a default install, but I did not check settings.py. Record it as "API user, possibly unauthenticated".
- Confidence: high for the diff. The fix is about 100 lines in one file plus tests.
### 13. wger-next-open-redirect (Python, Django)
- Repo: https://github.com/wger-project/wger (AGPL-3.0, about 91 MB with a JS frontend).
- Advisory: GHSA-vqv8-j3mj-wjxj (no CVE assigned). ≤ 2.5, fixed in 2.6.
- Flaw: `trainer_login()` returns `HttpResponseRedirect(request.GET['next'])`. `use_browser_language()` redirects to `remove_language_code(request.GET.get('next','/'))`. Neither checks host or scheme.
- Fix `0ba371bb`: `url_has_allowed_host_and_scheme()` at both sites. The diff is 20 lines of code plus tests.
- Two sinks, two privilege levels: `use_browser_language` is unauthenticated, `trainer_login` needs a trainer. A later follow-up, `cfa27a7fc78c682ad4244020505dce4825fe1e26`, fixes the same pattern in `wger/core/middleware.py` (AuthProxyMiddleware). I found no separate advisory for it. It is optional as a third sink, but it is not in the 2.6 fix.
- Confidence: high. I read the diff.
### 14. unfurl-flask-debug (Python, Flask)
- Repo: https://github.com/obsidianforensics/unfurl (Apache-2.0, about 24 MB).
- Advisory: GHSA-vg9h-jx4v-cwx2 (critical, CWE-489, no CVE). The GHSA lists "patched: none" as a package version, but it references the fix commit.
- Flaw: `web_app()` reads `debug` with `config[...].get('debug')`, which returns a string. Any non-empty string, including `"False"`, is truthy, so `app.run(debug=...)` always enables the Werkzeug debugger.
- Fix `4c0a07ab`: `getboolean('debug')` and `debug = false` in unfurl.ini. The diff is 2 lines.
- Caveat: the function default `debug='True'` is unchanged in the fixed version, so a run without an `[UNFURL_APP]` section still starts in debug mode. The fixed revision may therefore still, correctly, be flagged for debug-on by a strict analyzer.
- Confidence: high. I read the diff.
### 15. opencve-notification-ssrf (Python, Django)
- Repo: https://github.com/opencve/opencve (GitHub license: NOASSERTION. I believe OpenCVE v2+ is under a non-OSI license (BSL), but I did not verify this. Check before use).
- Advisory: CVE-2026-62282, GHSA-ch3g-4xvr-674q. The advisory's version ranges are confusing ("≤ 1.5.0, patched 3.0.0"). Use the commit pair.
- Flaw: webhook and Slack notification "test" requests POST to a user-supplied URL (`_http_post_json` with urllib) and do not restrict the destination.
- Fix `0af7f894`: new web/opencve/utils/ssrf.py (`validate_public_http_url`, `safe_request` with DNS pinning), `clean_url`/`clean_webhook_url` validators in forms.py, and notifications.py switched to `safe_request`.
- Confidence: high for the diff. The license is the concern. It is the weakest Python pick; drop it if an OSS license is mandatory.
## Considered and rejected (for traceability)
- vantage6 CVE-2024-23823 (permissive CORS, fix `70bb4e1d889230a841eb364d6c03accd7dd01a41`): the fix only adds a `cors_allowed_origins` option whose default is still `"*"`, so the fixed version is still permissive by default.
- FlaskBB CVE-2026-46556 (SSRF): the advisory has no patched version. The only related commit (`e87e585f`) just sets `allow_redirects=False`.
- SillyTavern CVE-2026-46372 (SSRF): the fix is a global request filter that is off by default.
- NodeBB CVE-2025-50979 (SQLi): no patched version or fix commit is listed, and I could not confirm the claim.
- Etherpad CVE-2026-55087 (open redirect): the referenced commits introduced the code. The fix is inside a large "assorted hardening" PR (#7784).
- EverShop CVE-2023-46943 (hardcoded HMAC secret): the fix is a large dev→main merge commit.
- claude-code-router CVE-2025-57755 (CORS): no tags, and I found no identifiable fix commit.
- calibre-web CVE-2022-0339 (SSRF): the fix commit is mixed with unrelated Kobo-sync changes. calibre-web SQLi CVE-2022-30765 (GHSA-8ppf-x4gr-2x7g) could be a Python alternate, but I did not check its diff.
- Tutor LMS PRs #3011/#3014/#3018 (SQLi hardening, Sept 2026): no CVE linked yet.
@@ -0,0 +1,658 @@
# Independent evaluation population v2: candidate real-world cases
Research date: 2026-09-26. Method: web research only. I read GitHub advisories (GHSA/CVE), commit pages, compare views and diffs through the GitHub REST API (`gh api`, read-only). I did not clone any candidate repository or run any scanner or project tooling on one. The work ran in four parallel lanes (Python, JS/TS, PHP apps, WordPress plugins) under one shared rule sheet, reproduced below. Every candidate was then re-verified from the merged list.
## Selection method (anti-bias)
- **Enumeration.** Reviewed GitHub advisories per ecosystem (pip, npm, composer): `gh api "/advisories?ecosystem=<eco>&type=reviewed&cwes=<C>&sort=published&direction=desc"` for CWE 89, 78, 77, 94, 95, 1336, 943, 90, 918, 601, 489, 215, 942, 295, 798, 321, 259, 1188, 16, 346, 1004. The results were merged, deduplicated, limited to publications on or after 2023-01-01 and sorted newest first.
- WordPress plugins: the newest 300 *unreviewed* advisories per CWE (89, 78, 94, 918, 601, 798, 942), kept only when the description mentions a WordPress plugin, then worked newest first. Each plugin's slug was searched on GitHub for a real development repository; SVN mirrors do not count.
- The CWE filter is the only prefilter. It removes out-of-family advisories before anyone sees them, so those are not counted.
- **Order.** Every lane took each advisory that met the criteria, strictly newest first, until its quota was filled:
- Python 7, JS/TS 7, PHP 4, WordPress 5 (rows marked `quota`).
- After the quota, each lane continued newest first for `untrusted_destination` and `config_secrets` only, until it had one extra of each or had examined 40 more advisories (rows marked `extra`).
- The quota phase came out heavy on SSRF and redirects (about half the reviewed advisories of 2026 are SSRF in AI and agent servers). So the Python, JS and PHP lanes ran a third pass, `continuation`: newest first, injection only (Python) or CWE-89/943 only (JS, PHP), until 2 more were taken. Python then ran a CWE-89 pass until 1 SQLi was taken.
- At no point was a case chosen or skipped for how easy it would be to detect.
- **Shared rules:**
- In scope means a web application or service. For Python that is Flask, Django or FastAPI (Starlette counts). For JS/TS it is a Node server (Express, Koa, NestJS, Next API routes, plain http). For PHP it is any web application or plugin. Libraries, SDKs, CLIs, MCP stdio tool servers and other frameworks (Zope/Plone, aiohttp, WsgiDAV) were rejected.
- Guard-bypass and incomplete-fix advisories are allowed, with the caveat "vulnerable pin already has a guard".
- A fix spread over up to 3 commits is allowed. A fix inside a large unrelated squash or release commit is rejected.
- One case per repository: the newest qualifying advisory wins.
- **Exclusion check.** I compared every candidate repo against `used-repos-v2.txt` (case-insensitive, `github.com/owner/repo`), which covers the v1 population and all development subjects. There were no hits. No candidate is a training app, CTF, benchmark suite or toy fork.
- **Verification (merged list, re-run independently of the lanes).** For all 37 pairs:
- `gh api repos/O/R/commits/<SHA> --jq .sha` returned both SHAs.
- For 36 pairs, the vulnerable SHA is listed in the fixed commit's `parents`. For qwed, the fixed SHA is a merge and the vulnerable SHA is its first parent.
- For LightRAG, `compare` reports the fixed SHA 3 commits ahead of the vulnerable SHA and 0 behind, so the vulnerable SHA is an ancestor.
"Diff seen" means the patch hunks of the named fix commit were read through the API, and the sink was read at the vulnerable pin (`contents/<path>?ref=<vuln>`).
## Summary table
| # | id | phase | repo | lang (+also) | family | advisory | vulnerable SHA | fixed SHA | ancestry | sink file :: function | source | privilege | license | diff seen | confidence |
|---|----|-------|------|--------------|--------|----------|----------------|-----------|----------|-----------------------|--------|-----------|---------|-----------|------------|
| 1 | lightrag-md-image-ssrf | quota | https://github.com/HKUDS/LightRAG | Python/FastAPI (+TS web UI) | untrusted_destination (SSRF, guard bypass) | CVE-2026-85740 / GHSA-vv3m-f8x4-7377 | `480f4a01741ec1d246842af68e09555a51ebf78f` | `c598545aa0084a427bcb19a84ade5e0ec31fa673` | ancestor (3 commits: `a2586217` → `9207e7fd` → `c598545a`, linear) | lightrag/parser/markdown/parser.py :: `_MarkdownImageResolver._download()` → `_build_guarded_opener().open(req)`, gated by the module function `_validated_addresses()` (image `src` URL from an uploaded markdown/textpack document) | document upload through the FastAPI API (image URLs inside the uploaded markdown) | authenticated (API key through `combined_auth`) | MIT | yes | medium |
| 2 | lmdeploy-media-url-ssrf | quota | https://github.com/InternLM/lmdeploy | Python/FastAPI (+C++/CUDA) | untrusted_destination (SSRF, guard bypass) | GHSA-39wr-7q6h-cf68 (no CVE) | `71648d239c978452720a6d6692fe914efc8a7b94` | `03c313006d17cc3feae86b633c44206a997c44db` | parent | lmdeploy/vl/media/connection.py :: `_load_http_url()` → `requests.Session().get(url_spec.geturl(), allow_redirects=True)`, gated by `_is_safe_url()` (the `urlparse` host differs from the `requests` host) | `image_url`/video/audio URLs in `POST /v1/chat/completions` → serve/processors/multimodal.py :: `MultimodalProcessor._parse_multimodal_item()` → `load_from_url()` | unauthenticated by default (`api_keys` is optional, default None) | Apache-2.0 | yes | medium |
| 3 | litellm-user-config-ssrf | quota | https://github.com/BerriAI/litellm | Python/FastAPI (+TS admin UI) | untrusted_destination (SSRF) | CVE-2026-59823 / GHSA-hx8v-g79f-8w5f | `93faf321df159b1ef65a98cc29a70958a96d184a` | `47214be317f45a2d2ac08258362e0481411eb427` | parent | litellm/proxy/route_llm_request.py :: `_route_user_config_request()` (`litellm.Router(**user_config)` then the provider call to the attacker's `api_base`); the only guard is litellm/proxy/auth/auth_utils.py :: `is_request_body_safe()` (`banned_params`) | JSON body of `POST /chat/completions` etc., key `user_config` | authenticated (any valid virtual key) | NOASSERTION (MIT core + enterprise dir) | yes | medium |
| 4 | open-webui-playwright-ssrf | quota | https://github.com/open-webui/open-webui | Python/FastAPI (+Svelte/TS/JS) | untrusted_destination (SSRF, DNS rebinding) | CVE-2026-87996 / GHSA-4v28-j6q3-5m4r | `73c1f5806aeb6345dad5de8f5aa26d1f3d0bef80` | `27402ff210bfa253445720920dfb86b15a00327b` | parent | backend/open_webui/retrieval/web/utils.py :: `SafePlaywrightURLLoader._intercept_navigation_sync()` / `_intercept_navigation()` (`validate_url(req.url)` then `route.fetch()`, where the browser re-resolves the host) | URL submitted for RAG ingestion or web search | authenticated (any verified user) | Open WebUI License (BSD-3 plus branding clause, not OSI; GitHub: NOASSERTION) | yes | medium |
| 5 | winml-cli-serve-cors | quota | https://github.com/microsoft/winml-cli | Python/FastAPI (+HTML) | config_secrets (wildcard CORS on an unauthenticated local API) | CVE-2026-84452 / GHSA-96p9-rh4f-92cf | `02179d185f0e3ac23518236b5ca1f61fd37ef680` | `f4073e0ef4700a25b623487e7e45c421ca0b9993` | parent | src/winml/modelkit/serve/cli_api.py :: module level `app.add_middleware(CORSMiddleware, allow_origins=["*"], ...)` (same in serve/app.py :: `create_app()`) | any web page in the victim's browser calling `POST /v1/cli/{command}` on 127.0.0.1 | unauthenticated (cross-origin from any website) | MIT | yes | high |
| 6 | chainlit-mcp-ssrf | quota | https://github.com/Chainlit/chainlit | Python/FastAPI (+TS) | untrusted_destination (SSRF) | CVE-2026-45019 / GHSA-hvfh-5mj3-5f3j | `b8506a47fa59d7eaba40ad6d27496c2605a64cb9` | `0565fd0eccb915fce159929598b053ed79f6e0c9` | parent | backend/chainlit/server.py :: `connect_mcp()` → nested `_mcp_session_runner()` → `sse_client(url=...)` / `streamablehttp_client(url=..., headers=...)` (`payload.url`, `payload.headers`) | `POST /mcp` JSON body (`ConnectMCPRequest`) | unauthenticated (needs non-default `features.mcp.enabled = true`) | Apache-2.0 | yes | high |
| 7 | contextforge-prompt-ssti | quota | https://github.com/IBM/mcp-context-forge | Python/FastAPI (+JS, Rust, Go) | injection (SSTI, Jinja2) | GHSA-vwf3-4xxj-qg6h (no CVE) | `0f8e0ad679f1fff685f486e17f642ac863c8ca9e` | `4d31004661858a2e99055b3c0c9f14218b8f7120` | parent | mcpgateway/services/prompt_service.py :: `PromptService._render_template()` → `_compile_jinja_template()` → `_get_jinja_env().from_string(template).render()` (plain `jinja2.Environment`) | prompt template stored through `register_prompt` (prompts create/update API), rendered on prompt get | authenticated (permission to create or update prompts) | Apache-2.0 | yes | high |
| 8 | lemur-acme-update-ssrf | extra | https://github.com/Netflix/lemur | Python/Flask (+JS) | untrusted_destination (SSRF, stored) | CVE-2026-71303 / GHSA-v5rc-cpwc-cfpr | `e9ade7dad5c9bc1369a831554ed6df3277c9cbae` | `edca0390f930344d65ff4ca37a669c2320e3dfad` | parent | lemur/plugins/lemur_acme/acme_handlers.py :: `AcmeHandler.setup_acme_client_no_retry()` → `ClientV2.get_directory(directory_url, net)` (`acme_url` from the stored authority options) | `PUT /api/1/authorities/<id>` `options` → lemur/authorities/service.py :: `update()` stores it without validation | authenticated (user with a role on the ACME authority) | Apache-2.0 | yes | high |
| 9 | glances-restful-cors-credentials | extra | https://github.com/nicolargo/glances | Python/FastAPI (+Vue/JS) | config_secrets (wildcard CORS with credentials) | CVE-2026-68517 / GHSA-fp27-88fp-2phg | `a14fb302a772bf8341b00c3ab8c2b0e95cffc197` | `890858944ab9d03730ec6b1ba42d4015e6d85db5` | parent | glances/outputs/glances_restful_api.py :: `GlancesRestfulApi.__init__()` (`add_middleware(CORSMiddleware, allow_origins=cors_origins, allow_credentials=cors_credentials)` after a guard that only matches `cors_origins == ["*"]`) | configuration file (`[outputs] cors_origins=*,https://x`, `cors_credentials=true`); exploited by any website through the victim's cached Basic auth | unauthenticated cross-origin reader (config-dependent) | LGPL-3.0 per COPYING (GitHub: NOASSERTION) | yes | medium |
| 10 | qwed-sympy-eval | continuation | https://github.com/QWED-AI/qwed-verification | Python/FastAPI (+HTML, small Go/TS/Rust SDKs) | injection (code/eval injection through SymPy `parse_expr`) | CVE-2026-55585 / GHSA-q27q-98j4-9pfv | `06801f68cbf9c9b5e54cc17f2a61cb2792da7a26` | `6066b68c0c4f4cc2c3771824822aaa864d082ef8` | first parent (fixed is the merge of PR #200; the PR commit `dc9d4db72ca4b4ae3f96d0e6a0c27a9e38a06f61` is a direct child of the vulnerable SHA) | src/qwed_new/api/main.py :: `verify_math()` → `sympy.parsing.sympy_parser.parse_expr(expression_normalized)` (also `left_str`/`right_str`); second path src/qwed_new/core/batch.py :: `_verify_item()` → `parse_expr(item.query)` | `POST /verify/math` JSON `expression`; `POST /verify/batch` item `query` | authenticated tenant (API key; self-signup `/auth/signup` is on by default) | Apache-2.0 | yes | high |
| 11 | xinference-llama3-tool-eval | continuation | https://github.com/xorbitsai/inference | Python/FastAPI (+TS/JS web UI) | injection (code/eval injection) | CVE-2026-61539 / GHSA-x2rj-828p-hx9m | `4cefcaa1e63d7e29d26bcd4be96cb267da113871` | `1b3d220f342ce68d34cec4586d9409d457dadc42` | parent | xinference/model/llm/tool_parsers/llama3_tool_parser.py :: `Llama3ToolParser.extract_tool_calls()` → `eval(model_output, {}, {})`; second site xinference/model/llm/utils.py :: `ChatModelMixin._eval_llama3_chat_arguments()` → `eval(text, {}, {})` | model output steered by the prompt in `POST /v1/chat/completions` with `tools` (xinference/api/restful_api.py) | unauthenticated by default (auth not enabled in the default deployment, per the advisory) | Apache-2.0 | yes | medium |
| 12 | pgadmin-maintenance-sqli | continuation | https://github.com/pgadmin-org/pgadmin4 | Python/Flask (+JS/React) | injection (SQLi) | CVE-2026-7815 / GHSA-hp84-p2gq-6fvr | `d112dc3b962218efc3613fa727d54f0fc5b1d527` | `cf53953d9ad66c182a08e662c1ba29c5831b486b` | parent | web/pgadmin/tools/maintenance/templates/maintenance/sql/command.sql (string concatenation of `vacuum_index_cleanup`, `vacuum_parallel`, `buffer_usage_limit`, `reindex_tablespace`) rendered in web/pgadmin/tools/maintenance/__init__.py :: `create_maintenance_job()` → `render_template(...)` → `BatchProcess(cmd=psql, args=[..., '--command', query])` | JSON body of `POST /maintenance/job/<sid>/<did>` | authenticated (`tools_maintenance` permission) | PostgreSQL License (OSI; GitHub: NOASSERTION) | yes | high |
| 13 | reactpress-findall-column-sqli | quota | https://github.com/fecommunity/reactpress | TypeScript, NestJS/TypeORM (+JS, CSS, HTML) | injection (SQLi) | CVE-2026-61685 / GHSA-wmw4-mw6x-6vfm | `c3dd10681b3d2b5e224f7ac647b3769557603975` | `78ecb70af1c021455c05fdcbe137212c70e310d6` | parent | server/src/modules/article/article.service.ts :: `ArticleService.findAll()` (query-string key interpolated as `article.${key}` into `andWhere`); same pattern in comment/file/page/knowledge services | `@Query() queryParams` in `ArticleController.findAll()` (`GET /api/article`) | unauthenticated | MIT | yes | high |
| 14 | 9router-kiro-region-ssrf | quota | https://github.com/decolua/9router | JavaScript, Next.js API routes (+CSS, Shell) | untrusted_destination (SSRF) | CVE-2026-56678 / GHSA-6mwv-4mrm-5p3m | `f2a7ae20309b4af55023eb11d1c02f63be1b80d1` | `126aa244c5b51b74ab8c7594e3418fcf4437bf6f` | parent | src/lib/oauth/services/kiro.js :: `KiroService.listAvailableProfiles()` (`fetch(\`https://codewhisperer.${region}.amazonaws.com\`)` with the API key as Bearer), reached via `validateApiKey()` | `region` in JSON body of `POST /api/oauth/kiro/api-key` (src/app/api/oauth/kiro/api-key/route.js) | authenticated (dashboard session) | MIT | yes | high |
| 15 | omniroute-acp-agent-rce | quota | https://github.com/diegosouzapw/OmniRoute | TypeScript, Next.js API routes (+JS, Python, Shell) | injection (OS command / code) | CVE-2026-88062 / GHSA-hf57-cqmx-p4gr | `c130f2aa1ccc7aaddd7a7685bd6a0e08136dccf1` | `60829241fd64d0317aa6a0dd8cd7a445a5287fed` | parent | src/lib/acp/registry.ts :: `detectAgent()` → `execFileSync(probe.command, probe.args)` with probe from `resolveVersionProbe(binary, versionCommand, true)` | `binary` / `versionCommand` in body of `POST /api/acp/agents` (src/app/api/acp/agents/route.ts → `setCustomAgents` → `refreshAgentCache`) | unauthenticated when `requireLogin=false` or before a management password is set; management session otherwise | MIT | yes | medium |
| 16 | cct-studio-server-cmdi | quota | https://github.com/davila7/claude-code-templates | JavaScript, Express (+Python, TS, HTML, Shell, many template languages) | injection (OS command) | CVE-2026-73222 / GHSA-79wm-x847-7cvg | `acdb5893412a81eaec9862ee24a4613bf296ab45` | `bc4618b07232633c1c0aac12a43e436268d31783` | parent | cli-tool/src/sandbox-server.js :: `executeLocalTask()` (`spawn('claude', [finalPrompt], {shell: true})`), route handler `POST /api/install-agent` and `checkAndInstallAgent()` (`spawn('npx', [..., agentName, ...], {shell: true})`) | `req.body.prompt`, `req.body.agentName`, `req.body.agent` | unauthenticated (server on 0.0.0.0, wildcard CORS) | MIT | yes | high |
| 17 | unleash-addon-webhook-ssrf | quota | https://github.com/Unleash/unleash | TypeScript, Express (+JS, PLpgSQL, SCSS) | untrusted_destination (SSRF) | CVE-2026-63004 / GHSA-5vf6-jrqr-78fj | `b9f3217d327f24094b4352130baa30387f85cffe` | `2100db76af3473f13e6fb40096cf17a9c2b741a1` | parent | src/lib/addons/addon.ts :: `Addon.fetchRetry()` (`ky(url, ...)`), called from webhook.ts / slack / teams / datadog / new-relic addons | addon `parameters.url` saved through the admin addons API (`CREATE_ADDON`/`UPDATE_ADDON`), fired on feature events | authenticated (CREATE_ADDON/UPDATE_ADDON root permission) | AGPL-3.0 | yes | high |
| 18 | nocobase-backup-restore-cmdi | quota | https://github.com/nocobase/nocobase | TypeScript, Koa (+JS, Shell) | injection (OS command) | CVE-2026-55410 / GHSA-p853-83gj-wjj3 | `a89b1ea0218b0d758543734e86f595f98f654314` | `aecc6758810b46dc13790d35ec4a15eea29357cc` | parent | packages/plugins/@nocobase/plugin-backups/src/server/adapters/database.ts :: `PostgresAdapter.restore()` (`-n ${srcSchema}` in pg_restore command string) → module `run()` → `child_process.exec` | `database.schema` in `_metadata.json` of an uploaded backup archive, restore action with `force` (resourcers/backup-cli.ts → managers/restore.ts) | authenticated, backup-management role | repo: NocoBase License (non-OSI, GitHub NOASSERTION); plugin package.json says Apache-2.0 | yes | medium |
| 19 | medplum-external-auth-open-redirect | quota | https://github.com/medplum/medplum | TypeScript, Express (+JS, Python, HCL) | untrusted_destination (open redirect) | CVE-2026-53728 / GHSA-m44r-7c5h-m6mj | `5d4a8654b02aa67c444f96ae1eb5663c09854006` | `7ae10035ddadde4dba7b18d3156553940465b3a1` | parent | packages/server/src/auth/external.ts :: `externalCallbackHandler()` (`res.redirect(redirectUrl)` with URI accepted by `getClientRedirectUri(client, body.redirectUri, true)` prefix match in packages/server/src/oauth/clients.ts) | `redirectUri` inside JSON `state` query parameter of `GET /auth/external` | unauthenticated (victim completes external IdP login) | Apache-2.0 | yes | high |
| 20 | flowise-ipv4-mapped-ssrf | extra | https://github.com/FlowiseAI/Flowise | TypeScript, Express (+JS, HTML, SCSS) | untrusted_destination (SSRF, guard bypass) | CVE-2026-69257 / GHSA-c6xh-wv4j-ppv5 | `b7a2005fd7b394ba7874aded744fe3730d59ccb3` | `0fc769208395641c1411ccdb9c81416e54802155` | parent | packages/components/src/httpSecurity.ts :: `isDeniedIP()` (deny-list check used by `checkDenyList()` / `resolveAndValidate()` before `secureFetch()` / `secureAxiosRequest()`) | URLs in chatflow node inputs that go through `secureFetch`/`secureAxiosRequest`; attacker-controlled DNS returning `::ffff:<internal IPv4>` | authenticated (flow builder), possibly prediction API when a flow takes a URL from input | Apache-2.0 outside `packages/server/src/enterprise` (GitHub NOASSERTION; commercial licence for enterprise dir) | yes | medium |
| 21 | nx-graph-server-wildcard-cors | extra | https://github.com/nrwl/nx | TypeScript, Node `http` server (+JS, Rust, many) | config_secrets (permissive CORS) | CVE-2026-54753 / GHSA-g2r8-wvmj-jf5w | `6ac9f82b04d02ab057b395b0e2b209fb3caf8545` | `e59122c5569cf201561b9995ce4e34a304f4fb03` | parent | packages/nx/src/command-line/graph/graph.ts :: `startServer()` request handler (`res.setHeader('Access-Control-Allow-Origin', '*')` on every response, including `/help` which runs `execSync(target.metadata.help.command)`) | any cross-origin web page the developer visits while `nx graph` runs | unauthenticated (cross-origin, local dev server) | MIT | yes | high |
| 22 | budibase-mongo-template-nosqli | continuation | https://github.com/Budibase/budibase | TypeScript, Koa (+JS, Svelte, Shell) | injection (NoSQL) | GHSA-pmpg-2mxq-6xwr (no CVE); same fix family as GHSA-qw6m-8fw2-2v64 | `b7e63df737eede85d951aa9665b0c0778ece7ccb` | `dd8c0654b35cd89ce3645f2355f4bf2ff9a5dd80` | parent | packages/server/src/sdk/workspace/queries/queries.ts :: `enrichContext()` (handlebars `processStringSync(..., {noEscaping: true})` into the `json` template, then `JSON.parse`) → packages/server/src/integrations/mongodb.ts :: `MongoIntegration.read()` (`collection.find(json)`) | query parameters / bindings (e.g. `currentUser.email`, request parameters) of `POST /api/v2/queries/:queryId` (packages/server/src/api/controllers/query/index.ts :: `executeV2`) | authenticated app user (BASIC role) | GPL-3.0 (root LICENSE says "GPLv3 overall"; server package.json GPL-3.0; GitHub NOASSERTION) | yes | medium |
| 23 | fuxa-tdengine-escape-sqli | continuation | https://github.com/frangoteam/FUXA | JavaScript, Express + Socket.IO (+TS, HTML, SCSS) | injection (SQLi, escape bypass) | CVE-2026-47720 / GHSA-h9fj-c2qr-76g2 | `193a888f8e3f805325072829b1644a2c768c948b` | `b78f6159d02a56ce5ff48207feb936afb3534fc8` | parent | server/runtime/storage/tdengine/index.js :: `TDengine.getDaqValue()` (`escapeTdString(tagid)` doubles quotes but not backslashes, then SQL string for `cursor.query`) | `sids` of `GET /api/daq` (server/api/daq/index.js) or Socket.IO `DAQ_QUERY` | unauthenticated (guest level; socket handler has no auth check) | MIT | yes | medium |
| 24 | contao-feed-reader-ssrf | quota | https://github.com/contao/contao | PHP (+JS, Twig) | untrusted_destination (SSRF) | CVE-2026-57232 / GHSA-87mg-5grr-rhwh | `394d951e7f6a87074ddc1f75f1c0ad9f94d98d68` | `27f6201809553bee767dcef15535bb8f0f4eac5f` | parent (5.7 branch; fixed commit is an ancestor of tag 5.7.9) | core-bundle/src/Controller/FrontendModule/FeedReaderController.php :: `FeedReaderController::getResponse()` (`$this->feedIo->read($url, ...)` over `$model->rss_feed` URLs; HTTP client wired in core-bundle/config/services.yaml `contao.feed.http_adapter`) | `tl_module.rss_feed` field saved through the back end, rendered by any front-end page that holds the module | authenticated back-end user with module-edit rights | LGPL-3.0 | yes | high |
| 25 | formie-form-settings-ssrf | quota | https://github.com/verbb/formie | PHP (+JS, TS) | untrusted_destination (SSRF) | CVE-2026-76086 / GHSA-v3f3-cmj4-cvj9 | `788fc375defb4def857c8737f7c92e487da92ca9` | `6735fe4ae8f6a2a76930716ad7876b236f7c530d` | parent (craft-5 branch; ancestor of tag 3.1.31) | src/controllers/IntegrationsController.php :: `IntegrationsController::actionFormSettings()` (`$integration->setAttributes($settings, false)`), then e.g. src/integrations/emailmarketing/ActiveCampaign.php :: `defineClient()` (`base_uri` from `$this->apiUrl`) → src/base/Integration.php :: `Integration::request()` | POST param `settings[...]` (for example `settings[apiUrl]`) to `formie/integrations/form-settings` | authenticated (any logged-in user, including front-end members) | **Craft License (proprietary, not OSI)**; GitHub says NOASSERTION | yes | medium |
| 26 | sulu-smart-content-sql2-injection | quota | https://github.com/sulu/sulu | PHP (+JS) | injection (JCR-SQL2 query injection) | CVE-2026-92692 / GHSA-jg26-q8hg-3pq4 | `d061094f5b7bb1d5e974544fce30bede9c7adf8e` | `d19c01487af8c3de2fb3aa145856707a6367392c` | parent (2.6 branch; ancestor of tag 2.6.25) | src/Sulu/Component/Content/SmartContent/QueryBuilder.php :: `QueryBuilder::buildCategoriesWhere()` (also `buildTagsWhere()`, `buildAudienceTargeting()`), string-concatenated into the SQL2 WHERE clause built by `buildWhere()` | `?categories=` query parameter, read in src/Sulu/Component/Category/Request/CategoryRequestHandler.php :: `getCategories()` (explode + trim only), passed by src/Sulu/Component/SmartContent/ContentType.php as `websiteCategories` | unauthenticated (any public page with a category-filtered Smart Content block) | MIT | yes | high |
| 27 | freeform-attribute-twig-ssti | quota | https://github.com/solspace/craft-freeform | PHP (+JS, EJS) | injection (SSTI, sandboxed Twig) | CVE-2026-73858 / GHSA-gxrg-x694-283w | `1b1b16a45c289cec7ac0cc2385e9431867427fe2` | `5f7555320635f1cd3b4c478aa0b58e8c3144313b` | parent (ancestor of tag v5.10.14, not of v5.10.13) | packages/plugin/src/Library/Attributes/Attributes.php :: `Attributes::toHtmlTagArray()` (`$twig->render($value, $properties)` through `Library\Helpers\IsolatedTwig`) | submitted form field values that are rendered back as the field's `value` attribute | unauthenticated (public form submission) | **Craft License (proprietary, not OSI)**; GitHub says NOASSERTION | yes | medium |
| 28 | smw-uriresolver-open-redirect | extra | https://github.com/SemanticMediaWiki/SemanticMediaWiki | PHP (+JS) | untrusted_destination (open redirect) | CVE-2026-77609 / GHSA-hw3m-8j5x-94ff | `5821b9b865c58d40e5ecd1aa807b79fa32eb5c2f` | `765a6ac22305e2d5c6fc79415bfee7cbd58514cd` | parent (ancestor of tag 7.2.0, not of 7.1.0) | src/MediaWiki/Specials/SpecialURIResolver.php :: `SpecialURIResolver::execute()` (`$out->redirect($title->getFullURL(), '303')`) | subpage of `Special:URIResolver/<x>` (the `$query` argument), decoded and turned into a Title (interwiki prefixes allowed) | unauthenticated | GPL-2.0-or-later (composer.json; GitHub says NOASSERTION) | yes | high |
| 29 | pheditor-default-admin-password | extra | https://github.com/pheditor/pheditor | PHP only (single file) | config_secrets (hardcoded default credential) | CVE-2026-55579 / GHSA-p4h7-p9rj-2pq2 | `e538f05b6faec99e5b23726bc9c17d6b57774297` | `a64eb935ce699e43f1ea60e46d47f33e2f87ffd6` | parent (= tag 2.0.5 → 2.0.6 compare, one commit) | pheditor.php :: top level `define('PASSWORD', '<sha512 of "admin">')` (line 11), checked by the top-level login block (`hash('sha512', $_POST['pheditor_password']) === PASSWORD`) | `pheditor_password` POST field on the login form | unauthenticated (default credential) | MIT | yes | low |
| 30 | pimcore-studio-datefilter-sqli | continuation | https://github.com/pimcore/studio-backend-bundle | PHP (+Twig, tiny JS) | injection (SQLi, identifier) | CVE-2026-55208 / GHSA-79cw-hfcc-7mw9 | `ea9d329686f5e5aea2eec378d63ac2deb965bb27` | `f532428cfbf4f5d6e299a13cedd5c29541802552` | parent (ancestor of tags v2025.4.6 and v2026.1.6) | src/Listing/Filter/DateFilter.php :: `DateFilter::applyDateFilter()` (`'`' . $key . '` BETWEEN :minTime AND :maxTime'` into `$listing->addConditionParam()`); same pattern in EqualsFilter::apply(), LikeFilter::apply(), src/Note/Service/FilterService.php :: applyFieldFilters() | `columnFilters[].key` in the JSON body of Studio listing endpoints (e.g. `POST /pimcore-studio/api/website-settings`) | authenticated (any Studio back-end user who can reach a listing endpoint) | **Pimcore Open Core License (POCL), not OSI**; GitHub says NOASSERTION | yes | high |
| 31 | froxlor-admin-ip-second-order-sqli | continuation | https://github.com/froxlor/froxlor | PHP (+JS, Twig) | injection (SQLi, second-order) | CVE-2026-54348 / GHSA-w27m-rmmf-g5w4 | `9bf03de643a2edee4642bfc6c94db1f05253da02` | `a1eaca5a1601c8a30e00814a4fc73ad0c185f89e` | parent (ancestor of tag 2.3.8) | lib/Froxlor/Api/Commands/IpsAndPorts.php :: `IpsAndPorts::listing()` / `listingCount()` (`implode(", ", json_decode($this->getUserDetail('ip'), true))` into `WHERE id IN (...)`); also lib/Froxlor/Api/Commands/Domains.php :: `validateIpAddresses()` | `ipaddress` API parameter of `Admins.add` / `Admins.update` (lib/Froxlor/Api/Commands/Admins.php), stored as JSON in `panel_admins.ip`, read back via `getUserDetail('ip')` | admin (API key with `change_serversettings`) | LGPL-2.1 (per GitHub) | yes | high |
| 32 | wp-s2member-tracking-eval | quota | https://github.com/wpsharks/s2member | PHP (+JS, CSS) | injection (code injection / eval) | CVE-2026-19804 / GHSA-fgx2-6hhq-hwhw | `4e1fe70b342f911ae5cb5c66652f2a2aef45b08b` | `6c622839aa0efe14031ff7c8fcc1f7d3107563ba` | parent | src/includes/classes/tracking-codes.inc.php :: `c_ws_plugin__s2member_tracking_codes::display_signup_tracking_codes()` → src/includes/classes/utilities.inc.php :: `c_ws_plugin__s2member_utilities::evl()` (`eval('?>'.$code)` over a transient holding the admin template with `%%first_name%%` replaced by the payer's first name) | `$paypal['first_name']` from the PayPal return/IPN data, substituted in src/includes/classes/paypal-return-in-subscr-or-wa-w-level.inc.php (line ~315) and stored with `set_transient()` | unauthenticated (needs the admin template to contain `%%first_name%%` and the proxy key, see notes) | GitHub: none; plugin header "GNU General Public License" (GPL, version not stated) | yes | medium |
| 33 | wp-directorist-avatar-ssrf | quota | https://github.com/sovware/directorist | PHP (+JS, Vue, TS, CSS) | untrusted_destination (SSRF) | CVE-2026-84046 / GHSA-6q63-3w56-m722 | `77cfc9bc8b453f97ab7e720330706fa4ef056867` | `1a67825d5cf6e7c768c3a692da1d18bae732bea9` | parent | includes/rest-api/functions.php :: `directorist_rest_upload_image_from_url()` (`download_url($image_url)`) | `$request['avater']['src']` in includes/rest-api/Version1/class-users-controller.php :: `update_user_meta_fields()` (REST user update) | authenticated (subscriber) | GitHub: NOASSERTION; plugin header GPLv3 | yes | medium |
| 34 | wp-gopay-log-filter-sqli | quota | https://github.com/gopaycommunity/gopay-woocommerce-integration | PHP (+JS, CSS) | injection (SQLi) | CVE-2026-75959 / GHSA-88c4-2f34-57h5 | `626bc8b5f8d33de6eefd921e3a05ed52ac320bcd` | `7190c985370f9758f226c2f2addd79532a2f93d5` | parent | admin/views/log.php :: file scope (included by admin/class-gopay-gateway-admin.php), two `$wpdb->get_results(sprintf("... REGEXP '[\w\W]*%s[\w\W]*' ...", ..., strtoupper($log_table_filter)))` | `filter_input(INPUT_GET, 'log_table_filter')` | admin (shop manager, `manage_woocommerce`) | GitHub: none; plugin header GPLv2 or later | yes | high |
| 35 | wp-user-registration-login-redirect | quota | https://github.com/themegrill/user-registration | PHP (+JS, TS, SCSS) | untrusted_destination (open redirect) | CVE-2026-80072 / GHSA-w78c-fx7q-9x8m | `e52cf151cb229852e6209fb3a627aa51485139c1` | `75c940dab15fd54d5a3fbb9f2213767b89467b08` | parent | includes/functions-ur-core.php :: `ur_process_login()` (`wp_redirect(wp_validate_redirect($redirect, $redirect))`, and `wp_send_json_success(['message'=>$redirect])` on the AJAX path) | `$post['redirect']` (login form POST) or `wp_get_raw_referer()` | unauthenticated attacker, victim logs in | GitHub: NOASSERTION; repo has license.txt, plugin header GPLv3 | yes | high |
| 36 | wp-groundhogg-confirm-redirect | quota | https://github.com/groundhoggwp/groundhogg | PHP (+JS, SCSS, HTML) | untrusted_destination (open redirect) | CVE-2026-81741 / GHSA-m96f-c5vh-h7cm | `9472a1dd79d4556c40eece501a61930793f170d3` | `028247d4b8ec9ba051d3eb48f1466a759bc16bfd` (+ comment-only follow-up `2f03c8481a2236b0574f67fa8a7415b77556f730`) | parent (follow-up is the direct child of the fix) | templates/preferences.php :: file scope, `case 'confirm':` (`wp_redirect($redirect_to)`) | `get_url_var('redirect_to')` (query string, only through `esc_url_raw(sanitize_text_field(...))`) | unauthenticated (crafted preferences link) | GitHub: none, no LICENSE file; plugin header GPLv3 | yes | high |
| 37 | wp-stripe-payments-thankyou-redirect | extra | https://github.com/Arsenal21/stripe-payments | PHP (+JS, CSS) | untrusted_destination (open redirect) | CVE-2026-81423 / GHSA-rvj6-j43w-hcgw | `71806a51e2c2c2ceb2cdaf00cbbde7a1b9da09f4` | `fb1f647deb6c92a2d2125da7a99fcb2914264f9c` | parent | stripe-payments/includes/class-asp-process-ipn-ng.php :: `ASP_Process_IPN_NG::ipn_completed()` (`wp_redirect($this->asp_redirect_url)`) | POST `asp_thankyou_page_url` (base64-decoded) in `ASP_Process_IPN_NG::process_ipn()`, triggered by POST `asp_process_ipn` on `wp_loaded` | unauthenticated (needs the public payment-form nonce) | GitHub: none; plugin header GPLv2 or later | yes | medium |
**Mix (37 candidates).**
- **By language:** 12 Python, 11 JS/TS, 14 PHP. The 14 PHP split into 8 PHP web apps and 6 WordPress plugins, all of which ship JS.
- **By family:**
- 16 `injection`: SQLi in #12, #13, #23, #30, #31, #34; JCR-SQL2 query injection #26; NoSQL #22; SSTI #7, #27; code/eval #10, #11, #32; OS command #15, #16, #18.
- 17 `untrusted_destination`: SSRF in #1, #2, #3, #4, #6, #8, #14, #17, #20, #24, #25, #33; open redirect in #19, #28, #35, #36, #37.
- 4 `config_secrets`: wildcard CORS in #5, #9, #21; hardcoded default password #29.
- **By phase:** 23 quota, 7 extra, 7 continuation.
- **By privilege:** 19 unauthenticated (some only under a non-default setting, see the notes), 16 authenticated and 2 admin (#31, #34).
- **Quota check (quota phase alone):** PHP 9 (5 WordPress), JS 7, Python 7. The quota rows alone give 5 injection, 15 untrusted_destination and 3 config_secrets. Most injection candidates come from the continuation passes, which is why those passes were run.
## Per-candidate notes
### Python (Flask / Django / FastAPI)
Source list: `gh api /advisories?ecosystem=pip&type=reviewed&cwes=<c>&sort=published&direction=desc` for CWEs 89, 78, 77, 94, 95, 1336, 943, 90, 918, 601, 489, 215, 942, 295, 798, 321, 259, 1188, 16, 346, 1004, merged and sorted newest first (list-pip.tsv). I worked from the top (2026-09-24) down. The quota of 7 was filled at GHSA-vwf3-4xxj-qg6h (2026-08-25, row 37). After that I continued newest first for `untrusted_destination` and `config_secrets` only, and stopped once each had one extra: Lemur (SSRF) and Glances (CORS). The last row examined is row 72 of list-pip.tsv (GHSA-fp27-88fp-2phg, 2026-08-17).
#### lightrag-md-image-ssrf (Python, FastAPI)
- Repo: https://github.com/HKUDS/LightRAG (MIT, about 117 MB; Python plus a TypeScript web UI).
- Advisory: CVE-2026-85740, GHSA-vv3m-f8x4-7377. ≤ 1.5.4, fixed in 1.5.5.
- Flaw: the native markdown parser downloads images referenced by an uploaded document (`NATIVE_MD_IMAGE_DOWNLOAD_ENABLED` defaults to True). `_download()` calls `_host_is_public()`, then opens the URL through a guarded opener. Both rely on `_validated_addresses()`, which accepts an address when `ip.is_global` is true. Python classifies NAT64 (`64:ff9b::/96`, `64:ff9b:1::/48`), IPv4-compatible (`::a.b.c.d`) and 6to4 (`2002::/16`) wrappers around internal IPv4 addresses as global, so those addresses pass.
- Fix: three consecutive commits on the same day. `a2586217` adds `_unwrap_embedded_ipv4()` and judges `is_global` on the embedded IPv4. `9207e7fd` corrects the RFC 8215 /48 decoding to deny by default. `c598545a` stops the 6to4 branch from loosening the stdlib verdict. The commits touch only parser.py and its tests.
- Caveats: the vulnerable pin already has a guard (this is a guard bypass). The fix spans 3 commits, so use `480f4a01` → `c598545a`. Exploitation needs NAT64/DNS64 routing on the host. The sink is in the same file as the fix.
- Confidence: medium. I read all three diffs and the sink at the vulnerable pin. The flaw is a subtle address-classification bug rather than a missing check.
#### lmdeploy-media-url-ssrf (Python, FastAPI)
- Repo: https://github.com/InternLM/lmdeploy (Apache-2.0, about 17 MB; Python plus C++/CUDA kernels). This is an inference engine. The in-scope surface is its FastAPI OpenAI-compatible server (`lmdeploy serve api_server`).
- Advisory: GHSA-39wr-7q6h-cf68 (no CVE), ≥ 0.12.3, < 0.15.0. **The advisory names no fix commit.** I found `03c31300` ("fix: restrict remote media domains", #4734) through the history of `lmdeploy/vl/media/connection.py`. It is an ancestor of tag v0.15.0 and the only commit in that range that changes the URL parser.
- Flaw: `_is_safe_url()` validates `urlparse(url).hostname`, but `requests` connects to a different host for inputs such as `http://127.0.0.1:6666\@1.1.1.1`. `_load_http_url()` then fetches with `allow_redirects=True`, and redirect targets are not re-validated.
- Fix: `_is_safe_url()` uses urllib3 `parse_url` (the parser `requests` uses). `_load_http_url()` follows redirects manually and re-validates each hop. The commit also adds an optional `allowed_media_domains` allowlist, plumbed through api.py, cli/serve.py, pipeline.py, the async engines and api_server.py (12 files).
- Privilege: `api_keys` defaults to None in `serve()`, so the server is unauthenticated by default.
- Confidence: medium. I read the diff and the sink. The fix commit is my inference because the advisory does not reference it. The vulnerable pin already has a guard (parser differential).
#### litellm-user-config-ssrf (Python, FastAPI)
- Repo: https://github.com/BerriAI/litellm. **It is very large (about 1.7 GB of git data).** GitHub reports the license as NOASSERTION: the core is MIT and the `enterprise/` directory has its own license. Verify before publication.
- Advisory: CVE-2026-59823, GHSA-hx8v-g79f-8w5f. ≤ 1.83.8, fixed in 1.83.9.
- Flaw: `is_request_body_safe()` blocks `api_base`/`base_url` in the request body but not `user_config`. `route_request()` sends any request containing `user_config` to `_route_user_config_request()`, which builds `litellm.Router(**user_config)` and makes the call. An `api_base` nested in that config therefore redirects the proxy's outbound request, together with the provider credentials in the config.
- Fix `47214be3`: adds `"user_config"` to `banned_params`. The same commit makes unrelated hardening changes: a ContextVar for internal calls, stripping `proxy_server_request`/`user_api_key_*` metadata keys, formatting churn, and file_search emulation (6 files).
- Caveats: the sink (route_llm_request.py) is outside the changed files. The fix is a one-word denylist change.
- Confidence: medium. The diff is clear. The outbound HTTP call itself sits deep in provider code, so I name the Router construction as the sink.
#### open-webui-playwright-ssrf (Python, FastAPI)
- Repo: https://github.com/open-webui/open-webui (about 440 MB; Python backend plus a Svelte/TS frontend). **License: "Open WebUI License", which is BSD-3 plus a branding-preservation clause, not OSI-approved. GitHub reports NOASSERTION.**
- Advisory: CVE-2026-87996, GHSA-4v28-j6q3-5m4r. ≥ 0.9.6, < 0.11.1.
- Flaw: with `WEB_LOADER_ENGINE=playwright` (not the default), the route interceptor checks `validate_url(req.url)` and then lets the browser perform the fetch through `route.fetch()`. The browser resolves the hostname again, so DNS rebinding reaches internal addresses. Redirect hops are checked the same way.
- Fix `27402ff2`: the interceptor now performs each request with the project's SSRF-safe requests/aiohttp sessions, which pin the validated address at connect time, and fulfils the page from that response. The change is 2 files.
- Caveats: the vulnerable pin already has a guard (a TOCTOU bypass). The sink is a Playwright browser call, not an HTTP client. The feature is not on by default.
- Confidence: medium. I read the diff and the vulnerable interceptor.
- Two other Open WebUI advisories in the window (GHSA-34r3-9m95-vq73 and GHSA-5x7x-4c3c-qf5w, both fixed by `e3e4bd87`) were not taken, because the repo already has a candidate.
#### winml-cli-serve-cors (Python, FastAPI)
- Repo: https://github.com/microsoft/winml-cli (MIT, about 26 MB). The package is a CLI whose `serve` subcommand runs a FastAPI server that exposes every CLI command over HTTP on 127.0.0.1.
- Advisory: CVE-2026-84452, GHSA-96p9-rh4f-92cf (CWE-306/942). < 0.4.0.
- Flaw: cli_api.py (module level) and app.py `create_app()` register `CORSMiddleware(allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])`. The API has no authentication, so any website can drive `POST /v1/cli/{command}`. With `--trust-remote-code` this leads to RCE through `AutoConfig.from_pretrained(..., trust_remote_code=True)`.
- Fix `f4073e0e`: replaces CORSMiddleware with a new `SameOriginMiddleware` (serve/_security.py), makes the CLI and model-mutation routes loopback-only, and refuses remote-code execution requested over HTTP. It touches 12 files, all related.
- Caveat: `allow_credentials=False` at the vulnerable pin. This is wildcard CORS on an API with no authentication, not wildcard CORS combined with credentials.
- Confidence: high. I read the diff and the vulnerable module.
#### chainlit-mcp-ssrf (Python, FastAPI)
- Repo: https://github.com/Chainlit/chainlit (Apache-2.0, about 12 MB; Python plus TypeScript).
- Advisory: CVE-2026-45019, GHSA-hvfh-5mj3-5f3j. ≥ 2.4.0rc0, ≤ 2.11.1, fixed in 2.12.0.
- Flaw: `connect_mcp()` (`POST /mcp`) builds `SseMcpConnection`/`HttpMcpConnection` from `payload.url` and `payload.headers` without any check, and its background runner opens `sse_client()` / `streamablehttp_client()` to that URL. The attacker also controls the forwarded headers.
- Fix `0565fd0e` ("Merge commit from fork"): adds `validate_mcp_url()` against a new `features.mcp.user_servers.allowed_urls` allowlist, header filtering and connect-time destination checks, plus the fix for the sibling stdio command-injection advisory. It touches 24 files, all MCP-related (Python backend, React client and tests), about 4k added lines. That makes it large but coherent.
- Caveats: the same commit fixes GHSA-w3fx-mc44-mf6j (command injection through the stdio transport, same endpoint). MCP is disabled by default since 2.7.0.
- Confidence: high. I read the server.py hunks and the vulnerable handler.
#### contextforge-prompt-ssti (Python, FastAPI)
- Repo: https://github.com/IBM/mcp-context-forge (Apache-2.0, about 450 MB; Python plus JS, Rust, Go).
- Advisory: GHSA-vwf3-4xxj-qg6h (no CVE). < 1.0.0.
- Flaw: prompt templates supplied by users are compiled with a plain `jinja2.Environment` (`_get_jinja_env()`) and rendered in `PromptService._render_template()`. That allows `__class__`/`__subclasses__` gadget chains, which give code execution on the gateway. The advisory quotes the 0.9.0 layout (`self._jinja_env` built in `__init__`). At the vulnerable pin the environment comes from the module-level `_get_jinja_env()`, and `__init__` stores that.
- Fix `4d310046`: `Environment` becomes `SandboxedEnvironment`, and a regex template validator is added. The hunk is small and clear, but it sits inside a squashed feature PR (#4072, "content security US-3 and US-4", 27 files) that also adds malicious-pattern detection for resources and tools.
- Caveats: the fix is mixed with related feature work. The template is stored first and rendered later (second order).
- Confidence: high for the sink and the fix hunk.
#### lemur-acme-update-ssrf (Python, Flask). Extra `untrusted_destination`
- Repo: https://github.com/Netflix/lemur (Apache-2.0, about 13 MB). **The repository is archived (read-only)** since after this fix. It is a real Netflix product, not a toy.
- Advisory: CVE-2026-71303, GHSA-v5rc-cpwc-cfpr (incomplete fix of GHSA-v2wp-frmc-5q3v).
- Flaw: `acme_url` is checked against `ACME_DIRECTORY_HOST_ALLOWLIST` only when an authority is created. `PUT /api/1/authorities/<id>` → `authorities.service.update()` stores new `options` as they are. At the next issuance, `AcmeHandler.setup_acme_client_no_retry()` reads `acme_url` and fetches it with `ClientV2.get_directory(directory_url, net)`.
- Fix `edca0390`: `update()` re-validates `acme_url` with the now-public `validate_acme_url()`. The code change is 10 lines.
- Caveats: stored (second-order) SSRF. The sink is outside the changed files. The outbound call runs inside the third-party `acme` library, called from project code.
- Confidence: high. I read the diff, the handler and the PUT view.
- The same repo has GHSA-f3qq-49m6-rw8f and GHSA-xpmj-wjcp-6pww (fixes `ed504a83`, `6dcb19b6`), which I did not take (one case per repo).
#### glances-restful-cors-credentials (Python, FastAPI). Extra `config_secrets`
- Repo: https://github.com/nicolargo/glances (about 60 MB; Python plus a Vue web UI). **COPYING is LGPL-3.0; GitHub reports NOASSERTION.**
- Advisory: CVE-2026-68517, GHSA-fp27-88fp-2phg (CWE-942). < 4.5.6.
- Flaw: `GlancesRestfulApi.__init__()` disables credentials only when `cors_origins == ["*"]`. Starlette treats any list that contains `"*"` as allow-all and reflects the request Origin with `Access-Control-Allow-Credentials: true`. A configuration such as `cors_origins=*,https://trusted` with `cors_credentials=true` therefore exposes authenticated API data to any website.
- Fix `89085894`: a membership test (`"*" in cors_origins`) at two sites. The code change is 4 lines, plus a test config and tests.
- Caveats: the vulnerable behaviour depends on a non-default operator configuration. The defaults are `cors_origins=["*"]` with `cors_credentials=False`. The vulnerable pin already has a guard. The same repo has newer command-injection advisories (GHSA-59fj-m2j6-hcxh, GHSA-73wf-9vmv-5pv9, GHSA-qcpp-8x79-hhp3). I passed over them because the injection quota was full when I reached them.
- Confidence: medium. The diff is clear, but whether the vulnerability exists depends on configuration.
#### qwed-sympy-eval (Python, FastAPI). Injection continuation
- Repo: https://github.com/QWED-AI/qwed-verification (Apache-2.0, about 30 MB, 59 stars, created 2025-12). It is small but a real product (PyPI `qwed`), not a training app.
- Advisory: CVE-2026-55585, GHSA-q27q-98j4-9pfv. < 5.1.2.
- Flaw: `verify_math()` reads `request.get("expression")`, applies a cosmetic regex, and calls SymPy `parse_expr()`, which uses `eval` internally, with no restricted `global_dict`. `_verify_item()` in batch.py does the same with `item.query` for MATH items.
- Fix: PR #200. The fix commit `dc9d4db7` (child of the vulnerable SHA) and its merge `6066b68c` add src/qwed_new/core/safe_parser.py (`safe_parse_expr`, validation of the input's syntax tree plus restricted namespaces) and switch every `parse_expr` call in main.py, batch.py and verifier.py to it. The merge also changes a binary `.coverage` file.
- Caveat: the fixed SHA is a merge commit. Its first parent is the vulnerable SHA. The direct parent/child alternative is `06801f68` → `dc9d4db7`.
- Confidence: high. I read the diffs and the vulnerable route.
#### xinference-llama3-tool-eval (Python, FastAPI). Injection continuation
- Repo: https://github.com/xorbitsai/inference (Apache-2.0, about 94 MB; Python plus a TS/JS web UI). This is a model-serving platform with a FastAPI OpenAI-compatible REST API.
- Advisory: CVE-2026-61539, GHSA-x2rj-828p-hx9m. ≤ 2.5.0, fixed in 2.7.0.
- Flaw: when a chat request carries `tools`, the Llama3 tool-call post-processing calls `eval()` on the raw model output, with empty globals, which is not a sandbox. The attacker controls that output through the prompt.
- Fix `1b3d220f`: `json.loads`, with a fall back to `ast.literal_eval`, at both eval sites. It touches 4 files and the change is small.
- Caveats: the tainted value reaches `eval` only through the model. There is no direct data-flow edge from the HTTP request to the eval argument, so the source-to-sink link is model-mediated. Exploitation needs a Llama3-family model served with the Transformers backend.
- Confidence: medium. The diff and sinks are unambiguous. The medium rating reflects the model-mediated source.
#### pgadmin-maintenance-sqli (Python, Flask). Injection continuation (SQLi)
- Repo: https://github.com/pgadmin-org/pgadmin4 (about 370 MB; Python/Flask plus a JS/React UI). **The LICENSE file says PostgreSQL Licence, which is OSI-approved. GitHub reports NOASSERTION.**
- Advisory: CVE-2026-7815, GHSA-hp84-p2gq-6fvr. < 9.15. The advisory references the short SHA `cf53953d9`, which I resolved to the full SHA.
- Flaw: the Maintenance tool's Jinja SQL template concatenates `vacuum_index_cleanup`, `vacuum_parallel`, `buffer_usage_limit` (inside double quotes) and `reindex_tablespace` (inside double quotes) into a VACUUM/ANALYZE/REINDEX statement. `create_maintenance_job()` renders it and runs `psql --command <query>` through `BatchProcess`. Injected SQL can reach `COPY ... TO PROGRAM`.
- Fix `cf53953d`: `validate_maintenance_data()` allow-lists the four fields before rendering, and the template switches `reindex_tablespace` to `conn|qtIdent`. 4 files, 2 of them tests.
- Caveats: the sink is a Jinja SQL template executed by an external `psql` process, not a DB-API call. The same repo has GHSA-j74f-g7vx-fh4x (OS command injection in Import/Export, published at the same time). I passed over it because this phase took SQLi only.
- Confidence: high. I read the diff and the vulnerable route.
### JavaScript / TypeScript (Node servers)
Source list: `list-npm.tsv` (reviewed npm advisories, CWE-prefiltered at fetch time to 89/78/77/94/95/1336/943/90/918/601/489/215/942/295/798/321/259/1188/16/346/1004, published >= 2023-01-01, sorted by publication date descending). Lines 1-59 (2026-09-24 → 2026-08-17) filled the quota of 7. Lines 60-94 (2026-08-14 → 2026-07-31) were the extra phase (RULES.md): only config_secrets and untrusted_destination considered, until +1 of each was taken.
"Diff seen" = I read the patch hunks of the named fix commit through `gh api repos/O/R/commits/SHA`. Every SHA below was confirmed with `gh api repos/O/R/commits/SHA --jq .sha`. Every fixed commit was confirmed to be an ancestor of the default branch (`compare/<default>...<sha>` = `behind`) unless stated.
#### reactpress-findall-column-sqli (TypeScript, NestJS)
- Repo: https://github.com/fecommunity/reactpress (MIT, about 50 MB, 757 stars). Server in `server/` (NestJS + TypeORM), Next.js client alongside.
- Advisory: CVE-2026-61685, GHSA-wmw4-mw6x-6vfm. ≤ 3.6.0, fixed in 3.7.0.
- Flaw: `findAll(queryParams)` destructures `page`, `pageSize`, `status` and loops over every remaining query key: `query.andWhere(\`article.${key} LIKE :${key}\`)`. TypeORM binds the value but the key goes into the SQL text as an identifier. The controller passes `@Query() queryParams` straight in. The same loop exists in comment, file, page and knowledge services.
- Fix `78ecb70a`: new `server/src/utils/query-whitelist.util.ts` (`filterByWhitelist(entity, params)`), applied in all five services. The same commit also adds HTML sanitisation after markdown rendering, a `JwtAuthGuard` on one comment route and `starter.ts` changes (9 files total).
- Confidence: high. I read the diff and the vulnerable `findAll()` and controller at the parent.
- Caveat: five parallel sinks; the fix commit also carries unrelated XSS and auth hardening.
#### 9router-kiro-region-ssrf (JavaScript, Next.js API routes)
- Repo: https://github.com/decolua/9router (MIT, about 16 MB, ~30k stars). Next.js app with API routes.
- Advisory: CVE-2026-56678, GHSA-6mwv-4mrm-5p3m. ≤ 0.5.2 per advisory, fixed in 0.5.6.
- Flaw: the route reads `{ apiKey, region }` from the JSON body and calls `kiroService.validateApiKey(apiKey, region || "us-east-1")`, which calls `listAvailableProfiles()`: `fetch(\`https://codewhisperer.${region}.amazonaws.com\`, { headers: { Authorization: \`Bearer ${accessToken}\` } })`. A region like `evil.host:8443#` moves the request to another host and forwards the key. The route also reflected the upstream error body.
- Fix `126aa244`: `assertValidAwsRegion()` (regex `^[a-z]{2}-[a-z]+-\d{1,2}$`) in `constants/oauth.js`, called before every region-interpolated URL in `kiro.js` and `providers.js`; the route stops reflecting upstream errors. 4 files, ~30 lines.
- Confidence: high. I read the diff and the vulnerable `listAvailableProfiles()` / `validateApiKey()`.
- Caveat: the same function family builds `https://oidc.${region}.amazonaws.com/...` in four more places, all fixed by the same commit; the advisory's exploited path is the codewhisperer one. The repo has three more SSRF advisories and a hardcoded-JWT-secret advisory (GHSA-jphh-m39h-6gwx, CWE-798) that I did not take (one case per repo).
#### omniroute-acp-agent-rce (TypeScript, Next.js API routes)
- Repo: https://github.com/diegosouzapw/OmniRoute (MIT; **very large: GitHub reports about 1 GB, 75 MB of TypeScript**). Default branch `release/v3.8.51`.
- Advisory: CVE-2026-88062, GHSA-hf57-cqmx-p4gr. ≤ 3.8.50; the GHSA lists no patched version but references the fix commit.
- Flaw: `POST /api/acp/agents` accepts `binary` and `versionCommand`, stores the custom agent and calls `refreshAgentCache()`. `detectAgent()` runs `execFileSync(probe.command, probe.args)`. `resolveVersionProbe()` only checks that the first token equals the (also attacker-supplied) `binary` and that there are no shell metacharacters, so `binary: "node"`, `versionCommand: "node -e ..."` runs arbitrary code.
- Fix `60829241`: in `resolveVersionProbe()`, untrusted probes may only carry a single recognised version flag (`SAFE_VERSION_PROBE_ARG`). The same commit closes three other advisories (db-backups auth path, uppercase path authz bypass, a spawn-capable prefix), 10 files.
- Privilege: unauthenticated when `requireLogin=false` or during first-run bootstrap; a management session otherwise.
- Confidence: medium. The diff and the vulnerable sink are clear; the vulnerable pin already has a guard (metacharacter filter and binary match), the fix bundles four advisories, and the repo size is a practical concern.
#### cct-studio-server-cmdi (JavaScript, Express)
- Repo: https://github.com/davila7/claude-code-templates (MIT, about 250 MB, mostly template content in many languages). The Express server is `cli-tool/src/sandbox-server.js`, started by `npx claude-code-templates --studio`.
- Advisory: CVE-2026-73222, GHSA-79wm-x847-7cvg. ≤ 1.29.2, fixed in 1.29.4.
- Flaw: two POST routes pass body fields to `child_process.spawn(cmd, args, { shell: true })`: `prompt` (via `finalPrompt` in `executeLocalTask()`) and `agentName` (`/api/install-agent` and `checkAndInstallAgent()`). With `shell: true` the args are joined and parsed by `sh -c`. The server listens on all interfaces and sends `Access-Control-Allow-Origin: *`.
- Fix `bc4618b0` ("Merge commit from fork", single parent): drops `shell: true`, adds a slug regex for `agentName`, restricts CORS to the localhost UI origin with a 403 for other origins, binds to 127.0.0.1. One source file (+57/-17) plus version bumps and CHANGELOG.
- Confidence: high. I read the diff; the vulnerable code is visible in the removed lines.
- Caveat: this is a developer-local server shipped in a CLI package, not a deployed web app. The same fix also removes a wildcard CORS header, so the case carries a secondary config_secrets aspect.
#### unleash-addon-webhook-ssrf (TypeScript, Express)
- Repo: https://github.com/Unleash/unleash (AGPL-3.0, about 313 MB).
- Advisory: CVE-2026-63004, GHSA-5vf6-jrqr-78fj. < 7.5.2, 7.6.0–7.6.4, 8.0.0–8.0.1.
- Flaw: every addon (webhook, Slack, Teams, Datadog, New Relic) sends the configured `parameters.url` to `Addon.fetchRetry()`, which calls `ky(url, ...)` with no host checks. The webhook addon also forwards the configured `Authorization` header and custom headers. The request is blind, but success and status land in the integration-event log.
- Fix `2100db76` (main, 8.x): new `src/lib/addons/validate-url.ts` (private/loopback/link-local/metadata deny by default, allow-list, http/https only) and `fetchPinned()` with a pinned DNS lookup and `redirect: 'manual'` in `addon.ts`. The advisory also lists two backports, `d45f99df` and `d862562a` (7.x lines), which are not ancestors of main.
- Confidence: high. I read the addon.ts diff and the vulnerable `fetchRetry()` / webhook call site at the parent.
- Caveat: stored input; the request fires later on a feature event, not in the request that sets the URL.
#### nocobase-backup-restore-cmdi (TypeScript, Koa)
- Repo: https://github.com/nocobase/nocobase (**repo licence: "NocoBase License Agreement", non-OSI, GitHub NOASSERTION; the `plugin-backups` package.json declares Apache-2.0**; about 358 MB).
- Advisory: CVE-2026-55410, GHSA-p853-83gj-wjj3. `@nocobase/plugin-backups` < 2.1.19.
- Flaw: `PostgresAdapter.restore()` builds `${restoreCmd} -U ... -n ${srcSchema} -d ${database} ...` where `srcSchema` is the `database.schema` value read from `_metadata.json` in the uploaded backup, and runs it through the module-level `run()` → `promisify(exec)`. `#restoreSchema()` also puts `srcSchema` into SQL. Reached when the restore action is called with `force`.
- Fix: the advisory references `0e1aba1b7b112ffc841588963f7343c00edd9806` (switch from exec strings to spawn argument arrays). **At that commit's parent the schema value is already validated**: `aecc6758` ("reject unsafe PostgreSQL schema names", 2026-06-12, 11 lines) added `assertPostgresSchemaIdentifier()` for `schema` and `schemaOption` in `restore()`. I use `aecc6758` and its parent as the pair because it is the commit that closes the reported path. `0e1aba1b` (parent `3f4ec12fc60157ea15c41ea3f696f0ddb7366e7c`) is the follow-up hardening.
- Confidence: medium. I read both diffs and the vulnerable `restore()` / `run()` at `a89b1ea0`. Medium because the advisory's own fix reference points at the later commit, and because of the licence.
- Caveat: sink input comes from a file inside an uploaded archive, not directly from a request field. The same repo has another advisory (GHSA-p849-8hwh-84j9, SQLi, 2026-07-31) that I did not take (one case per repo, and it came up in the extra phase).
#### medplum-external-auth-open-redirect (TypeScript, Express)
- Repo: https://github.com/medplum/medplum (Apache-2.0, **large: about 440 MB**, monorepo; the server is `packages/server`).
- Advisory: CVE-2026-53728, GHSA-m44r-7c5h-m6mj. `@medplum/core` ≤ 5.1.5 (the flaw is in `packages/server`), fixed in 5.1.6.
- Flaw: `externalCallbackHandler()` parses `req.query.state` as JSON and calls `getClientRedirectUri(client, body.redirectUri, true)`. With `allowPartial` the function returns the requested URI whenever it `startsWith` a registered one (`http://cb.example` accepts `http://cb.example.evil.com/`). The exact-match check afterwards only logs. The handler then appends `login` and `code` and calls `res.redirect()`.
- Fix `7ae10035`: exact match first; partial match only when the project sets `allow-dangerous-redirect`, and then only same origin with a path prefix (`isAllowedPartialRedirectUri`). Server change is ~40 lines in two files; the rest of the 7 files are FHIR definitions, docs and tests.
- Confidence: high. I read the server diff and the vulnerable handler at the parent.
- Caveat: the vulnerable pin already has a (prefix-match) guard.
#### flowise-ipv4-mapped-ssrf (TypeScript, Express) — extra, untrusted_destination
- Repo: https://github.com/FlowiseAI/Flowise (Apache-2.0 for everything outside `packages/server/src/enterprise`, commercial licence inside it; GitHub NOASSERTION; about 83 MB).
- Advisory: CVE-2026-69257, GHSA-c6xh-wv4j-ppv5. ≤ 3.1.2, fixed in 3.1.3. Bypass of the earlier fix for CVE-2026-31829.
- Flaw: `isDeniedIP()` compares the resolved address to CIDR deny entries only when `parsedIp.kind() === parsedRange.kind()`. An IPv4-mapped IPv6 address (`::ffff:169.254.169.254`) has kind `ipv6`, so every IPv4 deny rule is skipped and `secureFetch()` / `secureAxiosRequest()` go ahead.
- Fix `0fc76920`: normalise IPv4-mapped addresses (and deny entries) to IPv4 before matching, strip IPv6 brackets in `checkDenyList()` and `resolveAndValidate()`. One source file (+54/-9) plus a new test file.
- Confidence: medium. I read the diff and the vulnerable `isDeniedIP()`. The flaw is in the guard itself; the request call sites are in many node implementations and I did not trace a specific one.
- Caveat: vulnerable pin already has a guard; exploitation needs attacker-controlled DNS (AAAA record).
#### nx-graph-server-wildcard-cors (TypeScript, Node http) — extra, config_secrets
- Repo: https://github.com/nrwl/nx (MIT; **very large: about 1 GB**).
- Advisory: CVE-2026-54753, GHSA-g2r8-wvmj-jf5w. 17.0.4 – < 22.7.2 and 23.0.0-beta.0 – beta.1.
- Flaw: the `nx graph` local server (`http.createServer` in `startServer()`) set `Access-Control-Allow-Origin: *` on every response, so any page could read the project graph and the output of `/help`, which runs a target's configured help command with `execSync`.
- Fix: the advisory lists two identical 2-line commits. `e59122c5` is on master (parent `6ac9f82b`); `7620c1ff` (parent `0591aa5f`) is the 22.x backport and is not an ancestor of master. I use the master pair.
- Confidence: high for the diff (2 lines removed).
- Caveat: developer-local server inside a CLI, not a deployed web app. Huge repo.
#### budibase-mongo-template-nosqli (TypeScript, Koa) — SQLi/NoSQLi continuation phase
- Repo: https://github.com/Budibase/budibase (GPL-3.0 per root LICENSE and `packages/server/package.json`; GitHub NOASSERTION; **very large, about 790 MB**; also contains a `pro` package).
- Advisory: GHSA-pmpg-2mxq-6xwr (no CVE, published 2026-07-24). `@budibase/server` ≤ 3.38.1; release 3.39.9 referenced. GHSA-qw6m-8fw2-2v64 describes the same flaw through `POST /api/v2/queries/:queryId` and references the PR #18907 merge commit `2d6c1d17`.
- Flaw: `enrichContext()` runs every string field of a saved query through handlebars with `noEscaping: true`, then `JSON.parse`s the `json`/`customData`/`requestBody` result. A binding value containing `"` and `{` breaks out of its JSON string and adds MongoDB operators (e.g. `$ne`), so a BASIC user reads documents the builder's filter was meant to hide. `MongoIntegration.read()` passes the parsed object to `collection.find(json)`. `validateQueryInputs()` in the query controller blocks only `{{ }}` in parameter values.
- Fix `dd8c0654` ("Fix JSON template binding handling", on the PR branch, ancestor of master): new `processJsonStringSync()` in packages/string-templates that substitutes values in JSON-aware form, used by `enrichContext()` for the three JSON fields.
- Confidence: medium. I read the diff and the vulnerable `enrichContext()` and `read()` at the parent. Medium because master already had an interim escaping helper (`jsonEscapeForContext`) when PR #18907 was merged; the merge `2d6c1d17` replaced it. The pair `b7e63df7` → `dd8c0654` is on the feature branch, where the parent has no escaping at all. Use that pair, not the merge.
- Caveat: the sink is a datasource integration; the injected structure goes into a MongoDB query, not SQL.
#### fuxa-tdengine-escape-sqli (JavaScript, Express/Socket.IO) — SQLi/NoSQLi continuation phase
- Repo: https://github.com/frangoteam/FUXA (MIT; SCADA/HMI web server; **large: about 600 MB**, mostly the Angular client and assets).
- Advisory: CVE-2026-47720, GHSA-h9fj-c2qr-76g2. `fuxa-server` ≤ 1.1.14-1243 per GHSA (no patched package version); the text says fixed in 1.3.2. **The advisory references no commit.** I found the fix from the path history of the connector: `b78f6159` ("fix: tdengine string escaping (#2343)", 2026-05-03). It is in tag v1.3.2 and not in v1.3.1.
- Flaw: `escapeTdString()` only doubles single quotes. TDengine treats `\'` as an escaped quote, so a tag id like `x\' OR 1=1--` closes the string and appends SQL in `getDaqValue()`'s SELECT. The ids come from `query.sids` in `GET /api/daq` (guest-level `secureFnc`) or the Socket.IO `DAQ_QUERY` handler, which has no authorization check.
- Fix `b78f6159`: also escape backslashes (one line), plus a unit test.
- Confidence: medium. I read the diff and the vulnerable connector and DAQ route at the parent. Medium because I inferred the fix commit from history rather than an advisory reference, and because the path only exists when TDengine is the configured DAQ backend.
- Caveat: the vulnerable pin already has a guard (incomplete escaping). An earlier commit `5b59296c` (2026-03-08, "parameterize queries") introduced `escapeTdString`; before it the connector concatenated raw values.
### PHP web applications (not WordPress)
Source list: `list-composer.tsv` (GitHub Advisory Database, ecosystem composer, reviewed, CWE-prefiltered to the three families, published >= 2023-01-01, newest first). The quota of 4 was filled at the 5th advisory examined (2026-09-23). After that I continued newest first only for the extra `untrusted_destination` and `config_secrets` slots (RULES.md), which gave one extra of each.
#### contao-feed-reader-ssrf (PHP, Symfony-based CMS)
- Repo: https://github.com/contao/contao (LGPL-3.0; monorepo, about 150 MB, PHP 10 MB, JS 0.2 MB, Twig 0.5 MB).
- Advisory: CVE-2026-57232, GHSA-87mg-5grr-rhwh. Affects 5.3.35–5.3.47 and 5.4.0–5.7.8.
- Flaw: the Feed Reader front-end module splits the module's `rss_feed` text into URLs and passes each to `$this->feedIo->read($url, new Feed())`. The FeedIo adapter used the plain `@psr18.http_client`, with no scheme check and no private-network block, so loopback, RFC1918 and cloud metadata hosts are fetched.
- Fix `27f62018` (5.7 line; the 5.3 backport is `53b939ff2c4718e3a1d7c54ddd8886e9370618e4`, parent `96b216faa19cee9a3a21a103300b7c85d8f759d5`): new `core-bundle/src/HttpClient/NoPrivateNetworkExceptRootPagesHttpClient.php` (Symfony `NoPrivateNetworkHttpClient` variant that allows the site's own root pages), and `services.yaml` wires `contao.feed.http_adapter` and the crawler to it.
- Caveat: **the sink file is not in the diff.** The fix is a dependency-injection rewiring in YAML plus a new decorator class. `FeedReaderController.php` is byte-identical before and after. The fixed revision is only distinguishable by the service configuration.
- Confidence: high. I read the diff and the sink at the vulnerable pin. The source is stored configuration written by a back-end user, not a request parameter at the sink.
#### formie-form-settings-ssrf (PHP, Craft CMS plugin)
- Repo: https://github.com/verbb/formie (about 196 MB; PHP 4.8 MB, JS 0.6 MB, TS small). **License: Craft License (proprietary, commercial); not OSI.** Must be cleared before publishing any result.
- Advisory: CVE-2026-76086, GHSA-v3f3-cmj4-cvj9. Incomplete remediation of GHSA-cvpc-hccg-wmw4 (the `form-settings` action was left out of the 3.1.28 permission gate).
- Flaw: `actionFormSettings()` had no permission check and called `$integration->setAttributes($settings, false)` with the request's `settings` array (unsafe mass assignment). An attacker can overwrite `apiUrl` (and similar host properties) of a configured integration. `getFormSettings()` then makes the integration call its API through `defineClient()`/`request()`, sending stored credentials to the attacker-chosen host and returning the response body in JSON.
- Fix `6735fe4a` (Craft 5 line; Craft 4 line is `dde7799dfa7e4d0a11e28754ad8544dba62d5def`): `requireCpRequest()`, a per-form integration permission check, and an allowlist filter that drops keys containing `url`, `host`, `key`, `token`, etc.
- Caveat: the outbound request is several dynamic-dispatch hops away from the changed file (controller → integration subclass `defineClient()` → base `Integration::request()` → Guzzle). The sink is not in the diff. The case is equally a missing-authorization / mass-assignment bug.
- Confidence: medium. Diff read and the concrete ActiveCampaign sink checked at the vulnerable pin. The weakness is the licence and the indirect sink.
#### sulu-smart-content-sql2-injection (PHP, Symfony CMS)
- Repo: https://github.com/sulu/sulu (MIT; about 165 MB, PHP 11 MB, JS 5 MB admin UI). Default branch is 3.x; this fix is on the 2.6 line.
- Advisory: CVE-2026-92692, GHSA-jg26-q8hg-3pq4. < 2.6.25 and 3.0.0–3.0.7.
- Flaw: `CategoryRequestHandler::getCategories()` reads `?categories=`, explodes on commas and trims. `ContentType` passes the list as `websiteCategories`. `QueryBuilder::buildWhere()` → `buildCategoriesWhere()` appends `'page.[' . $name . '] = ' . $category` to a JCR-SQL2 WHERE clause. The tag filter already validated numbers; the category and target-group paths did not.
- Fix `d19c0148`: three `(int)` casts in QueryBuilder.php plus a new unit test. The diff is 3 changed lines of code.
- Caveat: the query language is JCR-SQL2 (PHPCR content repository, executed through Jackalope), not SQL sent to the RDBMS. The impact is read/enumeration of content nodes and DoS. Record it as query-language injection, not classic SQLi. 3.0.8 has a separate fix commit (not examined).
- Confidence: high. I read the diff, the sink, and the source-to-sink path at the vulnerable pin.
#### freeform-attribute-twig-ssti (PHP, Craft CMS plugin)
- Repo: https://github.com/solspace/craft-freeform (about 100 MB, PHP 6 MB). **License: Craft License (proprietary); not OSI.**
- Advisory: CVE-2026-73858, GHSA-gxrg-x694-283w. 5.0.0–5.10.13. "Limited" SSTI: the isolated Twig environment only exposed constants such as `PHP_VERSION` and `CRAFT_BASE_PATH`; RCE was not shown.
- Flaw: `Attributes::toHtmlTagArray()` renders every attribute key and value through `IsolatedTwig::render()` with the form properties. When a submitted field value comes back as the `value` attribute, attacker text is evaluated as a Twig template.
- Fix `5f755532` (PR #1986): skips Twig rendering for the `value`, `name` and `id` attributes. The diff is 5 lines. The commit's author date is 2025-05-13, but it only lands after v5.10.13 (compare: ahead of v5.10.13, behind v5.10.14).
- Caveat: attribute keys and other attribute values are still rendered through Twig in the fixed version. It is a deny-list fix. I inferred the exact path from the submission to the `value` attribute from the advisory and did not trace it in the field classes.
- Confidence: medium (licence, limited impact, source path partly inferred).
#### smw-uriresolver-open-redirect (PHP, MediaWiki extension), extra untrusted_destination slot
- Repo: https://github.com/SemanticMediaWiki/SemanticMediaWiki (GPL-2.0-or-later per composer.json, GitHub says NOASSERTION; about 66 MB, JS 0.9 MB).
- Advisory: CVE-2026-77609, GHSA-hw3m-8j5x-94ff. ≤ 7.1.0, fixed in 7.2.0. The advisory lists no commit. I found the fix through the history of `SpecialURIResolver.php` and confirmed it is between the 7.1.0 and 7.2.0 tags.
- Flaw: `execute($query)` decodes the subpage, builds a Title with `TitleFactory::newFromText()` (interwiki prefixes such as `mw:` resolve off-site) and issues `$out->redirect($title->getFullURL(), '303')` with no host check.
- Fix `765a6ac2`: new `isLocalRedirectTarget()` compares the target host with the server host at the sink and rejects userinfo. It shows a bad-title page otherwise.
- Caveat: the redirect primitive (`OutputPage::redirect`) and URL building live in MediaWiki core, which is not in this repository. A checkout is an extension, not a runnable app.
- Confidence: high. I read the diff and the vulnerable function.
#### pheditor-default-admin-password (PHP, single-file web file editor), extra config_secrets slot
- Repo: https://github.com/pheditor/pheditor (MIT, 182 stars, one 97 KB PHP file, not archived).
- Advisory: CVE-2026-55579, GHSA-p4h7-p9rj-2pq2. 2.0.1–2.0.5. The advisory lists no commit. The 2.0.5...2.0.6 compare has exactly one commit, `a64eb935`.
- Flaw: `define('PASSWORD', ...)` holds the unsalted SHA-512 of `admin`. The login block accepts it and nothing forces a change, so a default install gives unauthenticated access to the editor, upload and terminal.
- Fix `a64eb935`: when `PASSWORD` still equals `hash('sha512','admin')`, login shows a forced "change password" form that rewrites the `define` line in the file. The same commit also tightens the terminal command filter (`|`, backtick, newlines), which is the fix for sibling advisory GHSA-wg4w-wr5q-6vjc (command injection).
- Caveat: **the hardcoded default hash is still present, unchanged, on line 11 of the fixed file.** The fix is a behavioural gate, not removal of the constant, so the fixed revision still contains the same literal. The commit also mixes in an unrelated command-injection fix. Ground truth for "fixed = clean" is therefore doubtful for this family.
- Confidence: low (see caveat). Kept because it is the first config_secrets advisory newest-first. The next config_secrets candidate in the list, not examined because the slot was filled, is Kimai GHSA-jr9p-4h4j-6c58 (default `APP_SECRET` in the Docker image, CWE-1188).
#### pimcore-studio-datefilter-sqli (PHP, Symfony bundle, SQLi continuation phase)
- Repo: https://github.com/pimcore/studio-backend-bundle (about 10 MB, PHP 7.9 MB). It is the REST backend of Pimcore Studio, a Symfony bundle installed into a Pimcore application, so a checkout is not a runnable app by itself. **License: Pimcore Open Core License (POCL, June 2025), not OSI.**
- Advisory: CVE-2026-55208, GHSA-79cw-hfcc-7mw9. < 2025.4.6 and 2026.1.0–2026.1.5. The advisory is filed under pimcore/pimcore, but the code and fix are in studio-backend-bundle.
- Flaw: listing endpoints map a `columnFilters` array into `ColumnFilter` objects. `DateFilter::applyDateFilter()` wraps the user-supplied `key` in literal backticks and concatenates it into the condition string. Because the bound parameters are the fixed names `:minTime`/`:maxTime`, a backtick in the key breaks out and allows time-based blind SQLi. EqualsFilter, LikeFilter and the Note FilterService share the pattern (there the key also names the bound parameter, which limits exploitation).
- Fix `f532428c` (PR #1883): all four sites use `$this->dbResolver->get()->quoteIdentifier(...)`. About 30 changed code lines plus tests.
- Caveat: the SQL is executed in Pimcore core (`AbstractListing::addConditionParam()` → Doctrine DBAL), which is not in this repository. The sink in-repo is the string build handed to the listing object.
- Confidence: high. Diff read and vulnerable pin checked.
#### froxlor-admin-ip-second-order-sqli (PHP, server-management panel, SQLi continuation phase)
- Repo: https://github.com/froxlor/froxlor (LGPL-2.1 per GitHub; about 30 MB).
- Advisory: CVE-2026-54348, GHSA-w27m-rmmf-g5w4. < 2.3.8.
- Flaw: `Admins::add()` / `update()` store any `ipaddress` array as `json_encode($ipaddress)` in `panel_admins.ip` (the write itself is a prepared statement). Later, `IpsAndPorts::listing()` and `listingCount()` run `implode(", ", json_decode($this->getUserDetail('ip'), true))` into `WHERE id IN (...)`, and that string reaches `Database::prepare()`. `Domains::validateIpAddresses()` does the same.
- Fix `a1eaca5a` ("verify numeric values for ipaddress-id's"): `array_filter(..., 'is_numeric')` and `array_map('intval', ...)` at the store and at both read sites. 3 files, about 20 lines.
- Caveat: second-order. The source (API parameter) and sink (a later API call by the poisoned account) are in different requests, joined through the database. The attacker must already be an admin with an API key.
- Confidence: high. Diff read, and the sink checked at the vulnerable pin.
### WordPress plugins (PHP+JS)
#### wp-s2member-tracking-eval (PHP+JS, WordPress)
- Repo: https://github.com/wpsharks/s2member (PHP 4.0 MB, JS 0.15 MB; about 28 MB of git data). Default branch `dev`. It has a git submodule `phings` (build tooling only).
- Advisory: CVE-2026-19804, GHSA-fgx2-6hhq-hwhw (Wordfence, unreviewed GHSA). "Up to and including 260814". Unauthenticated RCE.
- Flaw: the PayPal return handler replaces `%%first_name%%` (and last/full name) in the admin's Signup Tracking Codes template with the payer's name. The only escaping is `esc_refs()`, which strips regex back-references, not PHP tags. The result is saved in a transient. `display_signup_tracking_codes()` (on `wp_footer`/`login_footer`) reads the transient and runs it through `c_ws_plugin__s2member_utilities::evl()`, which does `eval('?>'.trim($code))`. The same pattern is in `display_modification_tracking_codes()`, `display_ccap_tracking_codes()` and `display_sp_tracking_codes()` (the last one calls `eval("?>".$code)` directly).
- Fix `6c622839` ("Update notification template handling", 9 files, small): removes the `evl()`/`eval` calls in all four display functions and echoes the code through `do_shortcode()` behind a new filter. Related template-handling changes in the admin menu pages and PayPal notify handlers are part of the same commit.
- Caveats: the RCE chain also needs the site-global proxy verification key, which the advisory says leaked in PayPal Checkout AJAX responses. That leak was addressed separately (probably `37c29337d9256bf17a2bd115ac1d16bf07923061` "Fix PayPal proxy request handling", which I did not read). The advisory's version bound (≤ 260814) does not match the GitHub history, where the eval removal landed on 2026-09-14 and shipped in v260917. The source-to-sink path crosses a WordPress transient (second-order). Confidence medium because of the version mismatch and the two-part chain. The eval sink and its removal are unambiguous.
#### wp-directorist-avatar-ssrf (PHP+JS, WordPress)
- Repo: https://github.com/sovware/directorist (JS 21 MB of built assets, PHP 4.8 MB). Default branch `trunk`. Tags v8.9.4 = `d0fbb4f8c63582d43c0860e9add24d95b04e4a43`, v8.9.5 = `a3aa54a9a58e3c6d7784302dbe48a16ddcc14712` (101 commits apart, so use the commit pair).
- Advisory: CVE-2026-84046, GHSA-6q63-3w56-m722 (WPScan). < 8.9.5, subscriber and above.
- Flaw: the REST users controller passes `$request['avater']['src']` to `directorist_rest_upload_image_from_url()`, which calls WordPress `download_url($image_url)` after only `wp_parse_url`/`esc_url_raw`. The same helper is also called from the categories, locations and tools code paths.
- Fix `1a67825d` ("readme update & security fix", 7 files): adds `directorist_rest_validate_remote_image_url()` (http/https only, rejects localhost, resolves the host and rejects private or reserved ranges) and forces `reject_unsafe_urls` on the download. The same commit also tightens permission checks in the orders, users and favorites controllers.
- Caveat: WordPress core's `download_url()` already uses `wp_safe_remote_get()`, which blocks 127/8, 10/8, 172.16/12 and 192.168/16 but not 169.254/16 (cloud metadata). The practical exposure is therefore narrower than the advisory text suggests. Confidence medium for that reason. The sink and the fix are clear.
#### wp-gopay-log-filter-sqli (PHP+JS, WordPress/WooCommerce)
- Repo: https://github.com/gopaycommunity/gopay-woocommerce-integration (small: PHP 0.14 MB; the plugin ships a committed `vendor/` with the GoPay SDK, but the sink is in the plugin's own code). Plugin slug `gopay-gateway`.
- Advisory: CVE-2026-75959, GHSA-88c4-2f34-57h5 (Wordfence). ≤ 1.0.36 per advisory, shop manager and above.
- Flaw: admin/views/log.php reads `filter_input(INPUT_GET, 'log_table_filter')` and puts `strtoupper()` of it into `sprintf("... REGEXP '[\w\W]*%s[\w\W]*' ...")` for both the COUNT and the page query, without escaping. The view is included from the admin class's log submenu page (capability `manage_woocommerce`).
- Fix `7190c985` ("fix authenticated SQL injection in log admin view (CVE-2026-75959)"): one file. Both queries become `$wpdb->prepare(... LIKE %s ...)` with `$wpdb->esc_like()`.
- Caveats: the sink is at file scope in a view template, not inside a function. The advisory says ≤ 1.0.36, but the fix commit precedes the 1.0.35 bump (`329113ac`). The commit message names the CVE, so I trust the commit. Confidence high.
#### wp-user-registration-login-redirect (PHP+JS/TS, WordPress)
- Repo: https://github.com/themegrill/user-registration (PHP 4.9 MB, JS 2.5 MB, TS 0.2 MB; about 75 MB). Tags 5.2.7 = `c532e6aee2efc73ab711bdb6f8b36a06674e5d7d`, 5.2.8 = `28e2e931c3c0b13a8e1b577d56d6224c494fdae0`. That pair is 360 commits and 117 files apart, so use the commit pair.
- Advisory: CVE-2026-80072, GHSA-w78c-fx7q-9x8m (WPScan). < 5.2.8, unauthenticated.
- Flaw: `ur_process_login()` takes `$redirect = esc_url_raw(wp_unslash($post['redirect']))` (or the raw referer), then calls `wp_redirect(wp_validate_redirect($redirect, $redirect))`. The fallback equals the input, so validation is a no-op. On the AJAX-login path it returns `$redirect` in JSON, and the front-end JS navigates to it. The failed-login branch also redirects to `wp_get_raw_referer()` with `wp_redirect`.
- Fix `75c940da` ("#1408 Fix - Prevent membership privilege escalation and open redirect after login", 8 files): `wp_validate_redirect($redirect, get_home_url())` before both branches, `wp_safe_redirect` in both success and failure, and an `allowed_redirect_hosts` filter for the admin-configured external URL. The same commit also fixes a membership privilege escalation in modules/membership. Those hunks are separate. A later commit, `e537e78738aff363b5f250253d2b64bf9dc79c80` "Fix - Open redirect after login via unvalidated redirect parameter (#1417)", has an empty diff against its parent (already applied).
- Confidence high: the diff is readable and the no-op validation is visible at the vulnerable pin.
#### wp-groundhogg-confirm-redirect (PHP+JS, WordPress)
- Repo: https://github.com/groundhoggwp/groundhogg (PHP 4.8 MB, JS 2.0 MB). Tags 4.7.1 = `3cc86bfa09e8bb85cbf15f7c8affa727c6b65961`, 4.7.2 = `0f19a08505d0bd0ec27dd4668ac39f00ad3d0d5d` (17 commits).
- Advisory: CVE-2026-81741, GHSA-m96f-c5vh-h7cm (WPScan). < 4.7.2, unauthenticated.
- Flaw: in the email-preferences template, `case 'confirm':` sets `$redirect_to = esc_url_raw(sanitize_text_field(get_url_var('redirect_to')))`, passes it through a filter and calls `wp_redirect($redirect_to)` (with a phpcs ignore comment for SafeRedirect).
- Fix `028247d4` ("use safe redirect"): one line, `wp_redirect` → `wp_safe_redirect`. Its child `2f03c848` only removes the phpcs comment.
- Caveats: the sink is at file scope in a template. The preferences page needs a contact context (the tracking/permissions link from an email), which the advisory treats as a crafted link. Confidence high. There is no LICENSE file in the repo, but the plugin header says GPLv3.
#### wp-stripe-payments-thankyou-redirect (PHP+JS, WordPress), extra untrusted_destination pick
- Repo: https://github.com/Arsenal21/stripe-payments (PHP 1.5 MB, JS 0.13 MB). The plugin lives in the `stripe-payments/` subdirectory.
- Advisory: CVE-2026-81423, GHSA-rvj6-j43w-hcgw (WPScan). < 2.1.4, unauthenticated.
- Flaw: `process_ipn()` (hooked on `wp_loaded` when POST `asp_process_ipn` is set) takes POST `asp_thankyou_page_url`, base64-decodes it into `$this->asp_redirect_url`, and `ipn_completed()` ends with `wp_redirect($this->asp_redirect_url)`.
- Fix `fb1f647d` (2 files, 5 lines): `wp_validate_redirect(..., home_url('/'))` plus `wp_safe_redirect`.
- Caveats: `process_ipn()` first checks the `asp_payment_form_nonce` (a front-end nonce on any page with a payment form), and `ipn_completed()` runs after payment processing (or on the "already processed" branch). The attacker needs a victim to submit such a request. Confidence medium, because of those preconditions. The sink and the fix are unambiguous. Taken as the "+1 untrusted_destination" extra after the 5-case quota, per RULES.md.
## Open decisions for the maintainer
- **Licences that are not OSI or not verified.** These must be resolved before any result is published:
- Not OSI: Open WebUI (#4, Open WebUI License), NocoBase (#18, NocoBase License), Formie and Freeform (#25, #27, proprietary Craft License), Pimcore Studio (#30, Pimcore Open Core License). LiteLLM (#3) and Flowise (#20) have enterprise directories under a commercial licence; the core is MIT or Apache.
- GitHub reports NOASSERTION, but the repo's own files state an OSI licence: Glances (#9), pgAdmin (#12), Budibase (#22), Semantic MediaWiki (#28).
- All six WordPress plugins (#32–#37) have no GitHub licence metadata and rely on the plugin-header GPL declaration. s2Member's header does not state a GPL version.
- **Fix commits the advisory does not name (pair inferred from file history):** LMDeploy (#2), FUXA (#23), Semantic MediaWiki (#28). NocoBase (#18) uses an earlier commit than the one the advisory cites, because at the cited commit's parent the reported path was already closed.
- **Fixed SHA is a merge commit:** qwed (#10). The PR commit `dc9d4db72ca4b4ae3f96d0e6a0c27a9e38a06f61` is a plain child of the vulnerable SHA if a non-merge pair is preferred.
- **Weak ground truth:**
- Pheditor (#29, low): the default hash is still in the fixed file.
- xinference (#11): the input reaches `eval` only through model output.
- s2Member (#32): the advisory's version bound does not match the history, and exploitation needs a second key leak.
- Glances (#9) and Chainlit (#6): vulnerable only with a non-default configuration.
- Directorist (#33): WordPress core already blocks most private ranges.
- **Sink outside the repository or outside the changed files:**
- The sink's final call is outside the repo: Semantic MediaWiki (#28, MediaWiki core), Pimcore Studio (#30, Pimcore core), Lemur (#8, the `acme` library).
- The sink is outside the fix's changed files: LiteLLM (#3), Contao (#24).
- **Scope edge cases:** winml-cli (#5), cct-studio (#16) and nx (#21) are servers started from a developer CLI, not deployed web apps. Lemur (#8) is archived.
- **Guard bypasses (the vulnerable pin already has a check):** #1, #2, #4, #9, #15, #19, #20, #23.
- **Very large repositories:** LiteLLM (about 1.7 GB), OmniRoute and nx (about 1 GB each), Budibase (about 790 MB), FUXA (about 600 MB), Open WebUI, ContextForge and medplum (about 440–450 MB).
- **Family balance:** in strict newest-first order, recent advisories are dominated by SSRF. A selection of 12–15 with "the rest injection" will need most of the 16 injection rows. Taking fewer untrusted_destination rows is a selection choice; no case was excluded for being hard.
## Rejection log
Total: 261 advisories examined (Python 72, JS/TS 112, PHP 20, WordPress 57), 37 taken, 224 rejected. Each lane's log follows, one line per advisory. Advisories that were passed over after a lane's family slot filled are listed separately where the lane recorded them, and are not counted as examined.
### Python (Flask / Django / FastAPI)
- GHSA-mr95-65j8-9mxp / CVE-2026-57170 (oscal-compass/compliance-trestle, SSTI): library/CLI, not a web app.
- GHSA-rr49-f9g6-c9r5 / CVE-2026-57149 (plone/plone.app.portlets, TALES injection): framework out of scope (Zope/Plone).
- GHSA-4ghv-53cq-7wp3 / CVE-2026-91129 (home-assistant/core, SSRF): framework out of scope (aiohttp), and a huge repository.
- GHSA-v9m3-wfh8-5646 / CVE-2026-77249 (sooperset/mcp-atlassian, SSRF): fix mixed with unrelated changes. The single squash `b0417334` (#1448, 34 files) fixes about ten advisories across the attachment, transport, SSRF, authorization, filter and OAuth layers. The named sink (jira/users.py `_lookup_user_by_permissions`) is not in the diff. The MCP transport is not Flask/Django/FastAPI.
- GHSA-hgcf-4mq8-5266 / CVE-2026-77274 (mcp-atlassian, SSRF): same squash as above.
- GHSA-6vmq-24h2-pj7j / CVE-2026-77271 (mcp-atlassian, path traversal to RCE): family out of scope (path traversal), same squash.
- GHSA-5wf4-jqxh-8gm3 / CVE-2026-77267 (mcp-atlassian, SSRF): same squash as above.
- GHSA-6529-c226-h328 / CVE-2026-77261 (mcp-atlassian, SSRF): same squash as above.
- GHSA-49xv-9743-pw8w / CVE-2026-77265 (mcp-atlassian, SSRF): same squash as above.
- GHSA-ch3g-4xvr-674q / CVE-2026-62282 (opencve/opencve, SSRF): repo in used list.
- GHSA-82r6-8w77-94w6 / CVE-2026-63374 (agronholm/anyio, TLS verification): library, not a web app.
- GHSA-rqfv-2mw9-78g2 / CVE-2026-59971 (designcomputer/mysql_mcp_server, CWE-306/346): family out of scope (missing Origin/Host validation, DNS rebinding against an MCP SSE transport, not a permissive-configuration setting). The MCP transport is not Flask/Django/FastAPI.
- GHSA-34r3-9m95-vq73 / CVE-2026-87999 (open-webui, SSRF): repo already has a candidate.
- GHSA-5x7x-4c3c-qf5w / CVE-2026-88001 (open-webui, SSRF through redirect): repo already has a candidate.
- GHSA-jf6q-chmf-3h3v / CVE-2026-55073 (kozea/weasyprint, SSRF): library, not a web app.
- GHSA-4hhp-h66f-j5j7 / CVE-2026-73560 (vllm-project/vllm, SSRF): the sink (`mimo_v2_omni.py` `_fetch_image`) is reached through the offline `LLM.generate(multi_modal_data=...)` API in the advisory. The HTTP server fetches media through `MediaConnector`, so this is a library path, not a web app.
- GHSA-284h-m62q-gf8w / CVE-2026-78676 (gitpython-developers/gitpython, config injection): library, not a web app.
- GHSA-6ww7-3frv-cqxh / CVE-2026-78682 (nltk/nltk, SSRF): library, not a web app.
- GHSA-4mvj-m6j5-pmf7 / CVE-2026-71428 (Unstructured-IO/unstructured, SSRF): library, not a web app.
- GHSA-jrrm-9hc7-2v3h / CVE-2026-62674 (omnigent-ai/omnigent, CWE-94): family out of scope. This is missing authorization on a shared-agent overwrite. Command execution comes from the intended stdio-MCP feature.
- GHSA-756x-9hf6-q4h4 / CVE-2026-62675 (omnigent, CWE-94): family out of scope (uploaded agent bundle runs Python tools by design; authorization issue), and the repo was already examined.
- GHSA-p3m8-78j2-g5p3 / CVE-2026-62388 (nltk, CWE-1188): library, not a web app.
- GHSA-3gqm-fcw5-w839 / CVE-2026-63311 (nltk, SSRF fail-open): library, not a web app.
- GHSA-p6gw-4frg-j7jw / CVE-2026-55509 (mar10/wsgidav, SQLi): framework out of scope (its own WSGI server, not Flask/Django/FastAPI).
- GHSA-jw39-3688-r4rx / CVE-2026-54757 (compliance-trestle, SSTI): library/CLI, not a web app.
- GHSA-6hx8-3wjj-gr8g / CVE-2026-54770 (Pylons/webob, open redirect): library, not a web app.
- GHSA-6wvf-77m9-58rm / CVE-2026-37004 (BerriAI/litellm, SSTI in /prompts/test): repo already has a candidate.
- GHSA-jrw6-7x4q-w25j / CVE-2026-54569 (senaite/senaite.core, eval injection): framework out of scope (Plone/Zope).
- GHSA-mv8m-v9v6-5f94 / CVE-2026-54548 (siemens/kas, SSH host key checking disabled): library/CLI, not a web app.
- GHSA-w3fx-mc44-mf6j / CVE-2026-45018 (Chainlit/chainlit, command injection): repo already has a candidate (published 2 minutes before GHSA-hvfh and fixed by the same commit).
- GHSA-8cp3-qxj6-px34 (universal-tool-calling-protocol/python-utcp, SSRF): library, not a web app.
- GHSA-ppx3-28rw-8fpf / CVE-2026-12210 (python-utcp, SSRF): library, not a web app.
- GHSA-9qhg-99ww-9mqc (python-utcp, SSRF through redirect): library, not a web app.
- GHSA-pvph-5j39-v8qc / CVE-2026-55532 (MervinPraison/PraisonAI, CWE-346/352): family out of scope (a prefix-match Origin check leading to CSRF on an MCP HTTP transport, not a permissive CORS setting).
- GHSA-hmfx-4v44-9qw9 / CVE-2026-55535 (PraisonAI, SSRF by DNS rebinding in `webhook_url`): the fix is mixed with unrelated changes. The single commit `2f9677ab` ("harden input validation and query parameterization", 16 files) fixes about seven advisories across memory, tools, workflows, the jobs, browser and MCP servers. It only turns `gaierror` into a rejection. `executor.py` still re-resolves at send time, so the fixed tree does not remove the rebinding the advisory describes.
- GHSA-rg5q-pp8p-f7jm / CVE-2026-55537 (PraisonAI, SSRF by DNS fail-open): same multi-advisory commit as above.
- GHSA-vg6p-v9vm-6fgj / CVE-2026-55524 (PraisonAI `praisonaiagents` web_crawl, SSRF): library agent tool, not a web app; same commit.
- GHSA-x44h-65qv-cw74 / CVE-2026-55526 (PraisonAI `praisonaiagents` spider_tools, SSRF): library agent tool, not a web app.
- GHSA-wj6g-v78p-6fx3 / CVE-2026-55529 (PraisonAI MCP HTTP stream, origin validation): family out of scope (CSRF/origin).
- GHSA-8hjw-25cg-g52h / CVE-2026-55523 (PraisonAI web_crawl, SSRF): library agent tool, not a web app.
- GHSA-5r34-2g38-6569 / CVE-2026-55525 (PraisonAI web_crawl, SSRF): library agent tool, not a web app.
- GHSA-crp9-r7rq-c8cg (nltk): duplicate advisory.
- GHSA-qg9p-xrhj-435m (nltk): duplicate advisory.
- GHSA-8vh5-mgjj-w6hg (nltk): duplicate advisory.
- GHSA-5p3m-vhh6-9236 (eidetic-labs/stigmem, SSRF): negligible real-world use (4 stars, repo created 2026-05, alpha release 0.9.0a11). I treated it as a toy project.
- GHSA-p77j-g7h5-r2vw (geolens-io/geolens, 9 CWEs incl. 918): family out of scope (the advisory is mainly authorization and cache-scope data disclosure).
- GHSA-f3qq-49m6-rw8f / CVE-2026-70667 (Netflix/lemur, SSRF): repo already has a candidate.
- GHSA-xpmj-wjcp-6pww / CVE-2026-70666 (Netflix/lemur, SSRF): repo already has a candidate.
Injection-only continuation (examined):
- GHSA-mw6r-2hvm-4rp2 / CVE-2026-55546 (QWED-AI/qwed-mcp, SymPy eval): MCP stdio server, not a Flask/Django/FastAPI web app.
- GHSA-hxmv-c4g6-5fqc / CVE-2026-55522 (MervinPraison/PraisonAI, tools.py autoload in workflow include): library workflow feature, not a web app. The fix is the same multi-advisory commit `2f9677ab`.
- GHSA-xm98-3vcf-fph7 / CVE-2026-53710 (IBM/mcp-context-forge, RestrictedPython sandbox bypass): repo already has a candidate.
- GHSA-2cp2-2r3c-7p7r / CVE-2026-68508 (hydra-ecosystem/hydra, hydra.utils.instantiate): library, not a web app.
SQLi-only continuation (CWE-89, pip, newest first from 2026-08-21):
- GHSA-7xw9-549r-8jrc (DIRACGrid/DIRAC, SQLi in PilotManager): framework out of scope (DIRAC's own DISET/Tornado RPC services).
- GHSA-m4m7-4cw8-62j6 / CVE-2026-61667 (DIRAC, SQLi + eval in FileCatalog): framework out of scope (same repo, DISET services).
- GHSA-6xc5-4r68-67fc / CVE-2026-54760 (langroid/langroid, SQLChatAgent blocklist bypass): library, not a web app.
- GHSA-pmch-g965-grmr / CVE-2026-50180 (langroid, SQLChatAgent blocklist): library, not a web app.
- GHSA-pqrj-4gwg-h5f7 / CVE-2025-66336 (apache/doris-mcp-server, SQLi): MCP server, not a Flask/Django/FastAPI web app.
- GHSA-9pc9-4crj-mhpj (eidetic-labs/stigmem, identifier quoting): negligible real-world use (toy project, see above). The advisory also describes defensive hardening rather than a demonstrated injection.
- GHSA-82m5-3pcp-hccq / CVE-2026-10105 (agno-agi/agno, ClickHouse `delete_by_metadata`): library vector-DB backend, not a web app. There is no patched version, and the fix sits in PR #7883 with no merged commit reference.
- GHSA-mxfr-6hcw-j9rq / CVE-2026-25879 (langroid, prompt-to-SQL): library, not a web app.
Passed over in the continuation phase and not examined, because their family quota was already met. Listed for completeness, not counted as examined:
- Injection, older than the last row the injection continuation reached (GHSA-x2rj, 2026-08-21): GHSA-533j-2v4q-mw5h (langchain-mongodb), GHSA-w47q-945m-q9pc (document-merge-service), GHSA-9gmc-jqmh-3rvm (copier), GHSA-rghg-q7wp-9767 (MONAI), GHSA-59fj-m2j6-hcxh, GHSA-73wf-9vmv-5pv9 (glances), GHSA-3496-9g83-7v6x (sqlparse).
- Non-SQLi injection passed over in the SQLi-only phase: GHSA-j74f-g7vx-fh4x (pgadmin4 OS command injection, same repo as the SQLi taken).
- Untrusted destination, passed after Lemur filled the extra slot: GHSA-95px-34x5-p37h (MobSF), GHSA-7gwp-5pfp-969j (mlflow).
Counts: Examined 72, taken 12 (7 quota, 2 extra-family, 3 injection continuation), rejected 60. The first two phases examined 57 advisories (rows 1–72 of list-pip.tsv down to 2026-08-17, with 15 rows passed over). The injection continuation examined 6 of those 15 passed-over rows, which leaves 9 passed over. The CWE-89 continuation examined 9 more entries from pip-89.json (2026-07-13 down to 2026-05-11) and passed over 1 non-SQLi entry (pgadmin GHSA-j74f). In total, 10 rows were passed over without being examined. Advisories outside the CWE filter never entered the list. Their number is not known, because the list was built from per-CWE queries.
### JavaScript / TypeScript (Node servers)
Main phase (lines 1-59 of list-npm.tsv):
- GHSA-3cj3-hqcr-g934 / CVE-2026-59723 (cline/cline, CWE-346 cross-origin WebSocket hijacking): VS Code extension, not a Node web app; family out of scope.
- GHSA-fm8p-53ww-hf6w / CVE-2026-61742 (bytebase/dbhub, 306/346 DNS rebinding): MCP tool server; missing Host/Origin check is out of the three families.
- GHSA-cmhj-wh2f-9cgx / CVE-2026-56676 (decolua/9router, SSRF DNS rebinding): repo already has a candidate.
- GHSA-798p-78g2-v556 / CVE-2026-61612 (ondata/ckan-mcp-server, SSRF): MCP tool server (input is LLM tool arguments), not a web app.
- GHSA-q69g-4hcv-6jg4 / CVE-2026-71538 (cyclonedx/cyclonedx-node-npm, cmdi): CLI, not a web app.
- GHSA-q8hw-4fvp-9rwv / CVE-2026-61793 (nuxt-modules/og-image, SSRF): library (Nuxt module), not an app.
- GHSA-2h44-8472-frjj / CVE-2026-61559 (zereight/gitlab-mcp, SSRF): MCP tool server.
- GHSA-rqx4-3f6q-3x2v / CVE-2026-59148 (mockoon/mockoon, CORS): repo in used list.
- GHSA-65h7-9wrw-629c / CVE-2026-59973 (agentfront/frontmcp, SSRF): library / MCP framework.
- GHSA-4x45-gxvp-6283 / CVE-2026-59960 (argos-ci/argos-javascript, cmdi): CI client library/CLI.
- GHSA-q5wm-mgqx-fv2f / CVE-2026-86074 (n8n-io/n8n, SSRF): no fix commit referenced; 2.38.1...2.38.2 is a 213-file "Bundle 2.x" commit `8c8b059e` (fix mixed with unrelated changes); licence is the non-OSI Sustainable Use License.
- GHSA-6xcw-7xm6-48c6 / CVE-2026-86083 (n8n, code injection): same, fix inside 213-file bundle commit.
- GHSA-34ff-336r-5q23 / CVE-2026-86082 (n8n, SSRF): same.
- GHSA-hw8v-xxg5-vvvx / CVE-2026-86076 (n8n, code injection): same.
- GHSA-f6mr-pjwc-34m4 / CVE-2026-88056 (angular/angular platform-server, SSRF): framework library, not an app.
- GHSA-8m3c-c648-2xjj (nodemailer, SSRF/file): library.
- GHSA-gx45-xrj5-g6c4 / CVE-2026-75911 (hmbown/codewhale, code injection): TUI/CLI.
- GHSA-wrj3-vj8c-784f / CVE-2026-75858 (codewhale): TUI/CLI.
- GHSA-6v2g-fpxh-pmmh / CVE-2026-75856 (codewhale, SSRF): TUI/CLI.
- GHSA-6hxq-p678-4hr2 (masterkale/simplewebauthn, 295): library; attestation-chain validation is not a permissive-config case.
- GHSA-fg9p-mrxr-hvq7, GHSA-88f2-fpv8-89q2, GHSA-w727-8j6c-2rj4, GHSA-2h9g-j24r-h63g, GHSA-8j6p-r8jg-mxqh, GHSA-2w86-xfrc-g85r, GHSA-3575-w9fc-c2j6, GHSA-653q-5476-x79g, GHSA-6437-gxhq-pqv8, GHSA-cxq5-97v7-87j8, GHSA-p4cg-3328-rvfg, GHSA-6mr6-jvcr-2f25 (orval-labs/orval, 12 advisories, code injection / SSRF in generated code): code-generator CLI, not a web app.
- GHSA-4q39-2jhr-7qx8 / CVE-2026-65842 (udecode/plate, SSRF): library.
- GHSA-cpjf-6666-r8fx / CVE-2026-61704 (op-engineering/link-preview-js, SSRF): library.
- GHSA-f65p-4m7j-42xc / CVE-2026-75975 (fastify/fast-uri, SSRF): library.
- GHSA-fph4-wmhf-6fwf / CVE-2026-75899 (fastify/fast-uri, SSRF): library.
- GHSA-83x6-42hr-jc76 / CVE-2026-73845 (ondata/ckan-mcp-server, SSRF): MCP tool server.
- GHSA-vq4v-j7r6-jq4m / CVE-2026-82393 (pnpm/pnpm): CLI; path traversal/file write, family out of scope.
- GHSA-qxc2-j82w-r537 / CVE-2026-73231 (faker-js/faker, eval): library.
- GHSA-3wgp-x9p5-c7cc / CVE-2026-58191 (appium/appium base-driver, XSS/489): library; the flaw is reflected XSS in test routes, family out of scope.
- GHSA-g5xc-5w98-jfvm / CVE-2026-55855 (mariadb-connector-nodejs, SQLi in escaping): database driver library.
- GHSA-cqhc-2h57-wpxf / CVE-2026-55215 (mariadb-connector-nodejs, TLS): database driver library.
- GHSA-86m2-fcxq-5q7c / CVE-2026-55641 (decolua/9router, Host spoofing / SSRF): repo already has a candidate.
- GHSA-66mm-25pp-rfff / CVE-2026-77415 (jsonata-js/jsonata, code execution): library.
- GHSA-2943-5xfg-gq5f / CVE-2026-77414 (jsonata): library.
- GHSA-8gq3-vp5j-2grp / CVE-2026-77413 (jsonata): library.
- GHSA-wppf-h75h-6pm6 / CVE-2026-54689 (ihor-sokoliuk/mcp-searxng, SSRF): MCP tool server.
- GHSA-q87f-qc2r-2gw4 / CVE-2026-54688 (mcp-searxng, SSRF): MCP tool server.
- GHSA-2xhg-73j7-rrgx / CVE-2026-53957 (contentful/contentful-mcp-server, SSRF): MCP tool server.
- GHSA-w6x9-28jw-hq7j / CVE-2026-63643 (MagicMirrorOrg/MagicMirror, SSRF calendar): the referenced fix `58c2a5e6` only enforces the IP whitelist on Socket.IO handshakes; the `fetch(url)` sink is unchanged in the fixed version (fix is access control, not destination validation).
- GHSA-998g-7v5w-cr7g / CVE-2026-63642 (MagicMirror, SSRF newsfeed): same fix commit, same reason.
- GHSA-8g4w-4ffg-8vgx / CVE-2026-56677 (decolua/9router, SSRF OIDC test): repo already has a candidate.
Extra phase (lines 60-94; only config_secrets and untrusted_destination sought, +1 each):
- GHSA-49mq-fc6q-3h46 / CVE-2026-55157 (ooples/token-optimizer-mcp, cmdi): quota filled, family not sought in extra phase (also an MCP tool server).
- GHSA-5fpj-28rv-84r7 / CVE-2026-35219 (Budibase/budibase, SSRF): no fix commit referenced; the 3.41.2...3.41.3 compare (47 commits) does not touch the automation steps (fix not identifiable, possibly in the private pro submodule).
- GHSA-fjgc-3mj7-8rg8 / CVE-2026-55087 (ether/etherpad, open redirect): fix inside a large "assorted hardening" PR (#7784), as found in the v1 research.
- GHSA-vhh6-v828-x62f / CVE-2026-58230 (@sap/approuter, 601): no public source repository.
- GHSA-wcx4-wpfv-mc5c (aws/jsii jsii-diff), GHSA-6x64-9x62-f2gx (mermaid), GHSA-9473-5f9j-94wq and GHSA-279x-mwfv-vcqv (nuxt/nuxt), GHSA-f2r8-jv7c-xqmp and GHSA-4f78-qhmw-8j8m (electron): injection family, not sought in extra phase.
- GHSA-v64r-4m7r-3mvq / CVE-2026-70605 (electron/electron, SSRF): framework out of scope (Electron).
- GHSA-v3j7-r9gq-3gjw / CVE-2026-70604 (electron, 942 CORS): framework out of scope (Electron).
- GHSA-9pf5-hg6p-4pwp / CVE-2026-70599 (electron, 346): framework out of scope.
- GHSA-x5mm-wm4g-j5xv, GHSA-gcvv-72q8-9v76, GHSA-g366-23fw-ggp6, GHSA-ch52-px8q-f22j, GHSA-wvp2-4qqp-4h3r (tryghost/ghost, 5x SSRF): repo in used list.
- GHSA-5xvg-pmgg-3mxr, GHSA-4j8x-x6v7-w9rq, GHSA-x3hf-7cj6-3r4m (Flowise, code injection, listed before the SSRF advisory on the same day): injection family, not sought in extra phase.
- GHSA-x6vm-w76m-8j7g, GHSA-vmv7-4m6c-3cg5, GHSA-3769-jgqc-cxm7, GHSA-wg86-r78f-74mp, GHSA-g32j-mmxr-gfq5 (Flowise, code injection): repo already has a candidate; injection not sought.
- GHSA-v8fg-2rw7-q452 / CVE-2026-69240 (sequelize, SQLi Oracle): ORM library; injection not sought.
- GHSA-mwp4-54f8-5fhr, GHSA-4xrf-jv44-h6hh, GHSA-22jq-vg5j-6vgg (beaugunderson/ip-address, SSRF classification): library; untrusted extra already filled.
- GHSA-34pj-2622-jvxq / CVE-2026-53607 (apostrophecms/apostrophe, SSRF via Host header): untrusted_destination extra already filled by Flowise (newer); not examined further.
- GHSA-p849-8hwh-84j9 / CVE-2026-52887 (nocobase/nocobase, SQLi): repo already has a candidate; injection not sought.
- GHSA-5846-7qm3-r52j / CVE-2026-54729 (hackingrepo/dssrf-js, SSRF): library; untrusted extra already filled.
SQLi/NoSQLi continuation phase (CWE-89 / CWE-943 entries of list-npm.tsv after GHSA-p849-8hwh-84j9, newest first; requested by the coordinating session because SQLi was thin):
- GHSA-pqh8-p93p-2rx7 (dynatrace-oss/dynatrace-mcp, DQL injection): MCP tool server.
- GHSA-6wcc-39rp-hh9p / CVE-2026-54658 (hypequery/hypequery, ClickHouse escaping): library (query builder).
- GHSA-pmpg-2mxq-6xwr (Budibase, MongoDB NoSQLi): TAKEN (budibase-mongo-template-nosqli).
- GHSA-2xgg-r2wc-c5r2 (Budibase, MySQL DESCRIBE backtick): repo already has a candidate.
- GHSA-qw6m-8fw2-2v64 (Budibase, MongoDB NoSQLi via query endpoint): repo already has a candidate (same flaw family as the taken case).
- GHSA-q6x4-v3qx-85qw / CVE-2026-73300 (Budibase, MySQL multipleStatements): repo already has a candidate.
- GHSA-652q-gvq3-74qv (n8n, Snowflake node SQLi): no fix commit referenced; 2.32.0...2.32.1 has only a "Bundle/2.x" backport commit `f69dfc6d` plus two unrelated commits (fix mixed into bundle); non-OSI licence.
- GHSA-jqwr-vx3p-r266 (n8n, PostgresTrigger SQLi): same release bundle as above.
- GHSA-hwmj-qg4v-cvg9 / CVE-2026-59257 (n8n, MySQL v1 SQLi): 2.28.0...2.28.1 fix is again inside a "Bundle/2.x" commit `08ffbcc1`.
- GHSA-2qp2-6frj-p9pq (n8n MySQL v1): duplicate advisory.
- GHSA-5c7w-4wm3-85vw (asymmetric-effort/specifyjs, GraphQL tag): library.
- GHSA-8qv3-p479-cj62 / CVE-2026-54350 (Budibase, NoSQL operator injection): repo already has a candidate.
- GHSA-9ggv-8w38-r7pm (typeorm/typeorm, orderBy SQLi): ORM library.
- GHSA-qqf5-x7mj-v43p (Budibase, connector SQLi): repo already has a candidate.
- GHSA-jpq7-226w-6cxx / CVE-2026-54313 (n8n, MongoDB node): n8n release-bundle pattern (not re-checked per release); non-OSI licence.
- GHSA-c37g-w77q-m4vp / CVE-2026-54310 (n8n, Postgres v1 node): same.
- GHSA-98xf-r82g-9mhx / CVE-2026-48121 (langchain-ai/langgraphjs, MongoDBSaver NoSQLi): library.
- GHSA-h9fj-c2qr-76g2 / CVE-2026-47720 (frangoteam/FUXA, TDengine SQLi): TAKEN (fuxa-tdengine-escape-sqli). Quota of 2 reached; stopped.
Counts: Examined 112 advisories (59 in the main phase, 35 in the extra phase, 18 in the SQLi/NoSQLi continuation phase), taken 11 (7 main + 2 extra + 2 SQLi phase), rejected 101. CWE-prefiltered skips: 0 inside the list (the list was CWE-filtered at fetch time, so reviewed npm advisories with other CWEs never entered it and were not counted).
Families in this lane: injection 6 (2 SQLi, 1 NoSQLi, 3 OS command/code), untrusted_destination 4 (2 SSRF, 1 SSRF guard bypass, 1 open redirect), config_secrets 1 (wildcard CORS). Privilege: unauthenticated 5 (reactpress, cct, medplum, nx, fuxa) + 1 conditional (omniroute); authenticated 5.
### PHP web applications (not WordPress)
- GHSA-rw77-vq4g-x3hp / CVE-2026-56738 (thorsten/phpMyFAQ, SQLi): repo in used list.
- GHSA-9ccq-2jfg-qw33 / CVE-2026-72702 (getgrav/grav, CWE-346): extra phase; Referer prefix-match origin check, neither a configuration weakness nor a redirect sink as described; untrusted slot already filled by SMW.
- GHSA-68jx-f42c-7599 / CVE-2026-19418 (typo3/typo3, CWE-346/352): extra phase; broken access control, family out of scope.
- GHSA-qh7h-6c7g-x8m6 (getgrav/grav, CWE-346): duplicate advisory.
- GHSA-4f2f-jr2m-j7p4 (typo3, CWE-346): duplicate advisory.
- GHSA-f7vp-7xgx-4w4r / CVE-2026-69245 (guzzle/guzzle, CWE-346): library, not a web app.
- GHSA-g446-98w2-8p5w / CVE-2026-59883 (guzzle/guzzle, CWE-346): library, not a web app.
SQLi-only continuation phase (requested after the cross-lane family balance; CWE-89 entries in list-composer.tsv, newest first, resuming after Sulu on 2026-09-23):
- GHSA-vjqc-q4mp-2rvf / CVE-2026-79752 (cakephp/cakephp, SQLi): framework library (`FunctionsBuilder`), not a web app.
- GHSA-23rh-xw42-fq82 / CVE-2026-55416 (pimcore/pimcore, SQLi): Custom Reports lets a `reports_config` admin write SQL fragments by design (only a keyword blacklist). No fix commit is linked, and `Sql.php` history shows several successive hardening commits (`3fd77334`, `1999fee2`, `62ad7c5a`). No single identifiable fix.
- GHSA-fxf7-vhh8-7vpq / CVE-2026-77635 (cakephp/cakephp, SQLi): framework library, not a web app.
- GHSA-9x44-4gxf-8c25 / CVE-2026-55634 (pimcore/pimcore, CWE-89+94): primary sink is PHP code generation (field name injected into a generated class file, RCE). The ALTER TABLE SQLi is a secondary sink of the same fix (`a4f8c3cf`). Rejected in the SQLi-only phase as mixed-family ground truth. It would be a valid `injection` (code) case, licence POCL.
- (taken) GHSA-79cw-hfcc-7mw9: pimcore-studio-datefilter-sqli.
- (taken) GHSA-w27m-rmmf-g5w4: froxlor-admin-ip-second-order-sqli.
config_secrets alternate, examined on request:
- GHSA-jr9p-4h4j-6c58 / CVE-2026-52824 (kimai/kimai, CWE-1188, default `APP_SECRET`): rejected. The advisory links no commit. The fix is in release commit `31a8f887a5cda517db7b4320a7ad997c87d08601` ("Release 2.58", 85 files, fixes four GHSAs plus unrelated changes; parent `8d245ae223fd8fcbd508362dfe7fd04254a066c4`). That is a fix mixed with unrelated changes. The default secret also lives in `Dockerfile` (`ENV APP_SECRET=...`) and `.docker/entrypoint.sh`, not in PHP code. The fix adds a shell `ensureAppSecret()` function, and the bare-metal `.env.dist` default is, per the advisory, unchanged. pheditor stays the only (low-confidence) config_secrets case from this lane.
Passed over without examination because the injection quota was filled and only the extra families were being sought (2026-09-17 → 2026-07-16, 39 advisories), including: Grav CWE-94 ×5 (GHSA-r94f-hx44-8jqf, -f8wv-xp27-6gq7, -7pgq-cr25-xvc8, -2c4f-86xc-cr74, -fj2p-qj2f-74v5), CakePHP SQLi ×2 (GHSA-vjqc-q4mp-2rvf, -fxf7-vhh8-7vpq; framework library), Chamilo GHSA-g4c3-4g96-6g4m, Pimcore SQLi ×4, Smarty SSRF GHSA-cq55-c7wv-pxmq, elFinder SSRF GHSA-8x3q-jpjh-qh5c, Snipe-IT open redirect GHSA-wg2f-x2c2-c4rp, SilverStripe ×2, Phalcon, Backpack CRUD, LibreNMS ×3, Froxlor, jmespath.php, Winter, CodeIgniter4, PHP_CodeSniffer, Easy!Appointments SSRF, Poweradmin, WPCS, Pheditor ×2 other, PHPSpreadsheet, Koel SSRF ×6 (partly beyond 07-16). Note for the maintainer: the untrusted_destination ones among them (Smarty, elFinder, Snipe-IT, Easy!Appointments, Poweradmin, Koel) are the next-in-line alternates for that family.
No WordPress plugin appeared in the composer list up to 2026-07-16 (WPCS GHSA-3pwp-g2mj-5p3v is a coding-standards tool, not a plugin).
Counts: Examined 20, taken 8 (4 quota + 1 extra untrusted_destination + 1 extra config_secrets + 2 SQLi continuation), rejected 12. SQLi continuation phase: 6 examined, 2 taken. Kimai alternate: 1 examined, rejected. Passed over after the quota was filled (other families): 39. CWE-prefiltered skips: the composer list was prefiltered by CWE at fetch time, so out-of-family advisories never entered it (not counted individually).
### WordPress plugins (PHP+JS)
- GHSA-7jgm-vpwg-fj48 / CVE-2026-93899 (Better Messages – Chat Rooms, Group Chat, Private Messages & AI Chat Bots, SQLi, published 2026-09-25): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-g4cp-fj5c-7599 / CVE-2026-12037 (Asset CleanUp: Page Speed Booster, SSRF, published 2026-09-25): only GitHub repo is gabelivan/wp-asset-clean-up, last push 2019; fix only on wp.org SVN
- GHSA-6rpr-h6cv-7xm2 / CVE-2026-78397 (Link Library, SSRF, published 2026-09-25): the author repo ylefebvre/link-library stops at 7.9.2 (2026-04-26); the fix is in 7.9.6, which is only on wp.org SVN
- GHSA-8q4x-8xp2-6ww3 / CVE-2026-18335 (Kirki – Freeform Page Builder, Website Builder & Customizer, SSRF, published 2026-09-24): themeum/kirki exists but last push 2026-04-10 and has no 6.x code; fix (trac changeset 3636487) only on wp.org SVN
- GHSA-4rj9-f57h-53hg / CVE-2026-93368 (Rename wp-login.php to anything you want, SQLi, published 2026-09-23): only unrelated/stale repo (ellatrix/rename-wp-login, 2020); fix only on wp.org SVN
- GHSA-8743-f7m5-qxwx / CVE-2026-91024 (Booking Manager, SQLi, published 2026-09-23): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-ppcx-fj6x-46xf / CVE-2026-75799 (YAHMAN Add-ons, code injection, published 2026-09-23): file-type validation (upload), family out of scope; also no GitHub repo
- GHSA-9rxg-m349-fgqj / CVE-2022-4997 (jet-form-builder-stripe-gateway, SQLi, published 2026-09-23): CVE-2022 id (disclosed 2022, GHSA published 2026); no GitHub repo
- GHSA-gg2m-8p7j-v3mx / CVE-2026-92235 (The WP Ultimate Review, code injection, published 2026-09-22): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-h9fx-ggrg-2982 / CVE-2026-16542 (Import and export users and customers, SSRF, published 2026-09-20): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-h6cq-4ghp-3r32 / CVE-2026-87067 (Forminator Forms, code injection, published 2026-09-20): Forminator: no public source repo; also object-injection style (class restriction), family doubtful
- GHSA-fv9m-mw9j-43cg / CVE-2026-9855 (Custom Field Template, SQLi, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-wc9h-566w-4rw2 / CVE-2026-7527 (WP Ghost (Hide My WP Ghost) – Security & Firewall, open redirect, published 2026-09-19): johndarrel/hide-my-wp is a release-dump mirror; the fix (<=7.0.02 -> 7.0.08) sits in one 7.0.08 release commit of 60+ files (fix mixed with unrelated changes)
- GHSA-jvc3-5jgr-chw6 / CVE-2026-85658 (Paid Membership Plugin, Ecommerce, User Registration Form, Login Form, User Profile &..., code injection, published 2026-09-19): ProfilePress: only profilepress/profilepress-readmes on GitHub; fix only on wp.org SVN
- GHSA-cv43-j772-67wp / CVE-2026-6295 (WP Optimizer, SQLi, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-c79v-6v55-pvvh / CVE-2026-13191 (Create, SQLi, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-mw9j-687w-wh97 / CVE-2026-1641 (Wow Elements Addons for Elementor, SSRF, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-fmwg-rf74-rjqj / CVE-2026-4327 (The Welcomizer, code injection, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-5j33-g9fw-4r4f / CVE-2026-88926 (VikRentItems Flexible Rental Management System, SQLi, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-jhj7-pcmr-9q9p / CVE-2026-13200 (Create, SQLi, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-2vrv-wr6r-v79v / CVE-2026-92229 (The Forminator Forms – Contact Form, Payment Form & Custom Form Builder, code injection, published 2026-09-19): Forminator: no public source repo (wpmudev has only forminator-ui); fix only on wp.org SVN
- GHSA-3fg9-62hh-37mh / CVE-2026-92807 (Save as PDF Plugin by PDFCrowd, code injection, published 2026-09-19): pdfcrowd/save-as-pdf-addon is the browser add-on, not the WP plugin; fix only on wp.org SVN
- GHSA-x57m-9xmj-gh9p / CVE-2026-89274 (WP Recipe Maker, code injection, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-34mw-p6wx-m38q / CVE-2026-85652 (Photo Gallery by 10Web – Mobile-Friendly Image Gallery, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-7wpp-2mgh-3vfp / CVE-2026-75961 (NEX-Forms – Ultimate Forms Plugin for, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-jvcr-7q46-jqm3 / CVE-2026-85705 (Location Manager, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-35gj-4m99-3jxg / CVE-2026-18442 (WCFM Marketplace – Multivendor Marketplace for WooCommerce, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-q2g2-xwwx-rp3m / CVE-2026-17607 (WP Inventory Manager, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-prxh-6hcq-7gvg / CVE-2026-15275 (WP Multi Store Locator Pro, SQLi, published 2026-09-18): wpexpertsio/WP-Multi-Store-Locator last push 2023-12 (Pro advisory); fix not on GitHub
- GHSA-xcfp-7r4f-6367 / CVE-2026-17576 (InfiniteWP Client, SQLi, published 2026-09-18): Revmakx/iwp-client last push 2025-09-23, before the fix; fix only on wp.org SVN
- GHSA-7v2c-5xw3-f2hc / CVE-2026-12106 (Auto Upload Images, SSRF, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-93x5-88q4-f8c3 / CVE-2026-87775 (Tz Weekly Radio Schedule, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-hm83-7h6c-m5rj / CVE-2026-87774 (Tz Weekly Radio Schedule, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-c3g6-9rgm-rg4m / CVE-2026-90984 (Generate PDF using Contact Form 7, SSRF, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-4778-xmgc-pc88 / CVE-2026-87770 (Price Drop Alert for Woo Commerce, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-rjgv-8pq9-p3c3 / CVE-2026-87771 (Product Question and Answer, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-52cc-m98v-q64v / CVE-2026-87767 (wp shortcut link and advertisement baner, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-7mpj-9p6h-7gr6 / CVE-2026-84738 (AF Companion, code injection, published 2026-09-18): file upload validation, family out of scope; also no GitHub repo
- GHSA-vfmj-6hfh-5m4v / CVE-2026-87963 (Yo, SQLi, published 2026-09-17): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-xg7f-297r-82hv / CVE-2026-88795 (wpShopGermany IT-RECHT KANZLEI, code injection, published 2026-09-17): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-xhvp-v8j9-2mrf / CVE-2026-86823 (Newsletter, open redirect, published 2026-09-16): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-49j8-gcx6-2f92 / CVE-2026-76559 (WP Import Export Lite, SSRF, published 2026-09-16): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-g78q-54w3-px42 / CVE-2026-76550 (WP Import Export Lite, code injection, published 2026-09-16): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-v7v7-79q5-j8cf / CVE-2026-76551 (WP Import Export Lite, code injection, published 2026-09-16): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-9c57-9fxg-8x9j / CVE-2026-78159 (The Events Calendar, code injection, published 2026-09-12): repo in used list (the-events-calendar)
- GHSA-3q5p-wcf4-wvm5 / CVE-2026-14560 (teddy-bear-customize-addon, code injection, published 2026-09-11): file upload validation, family out of scope; also no GitHub repo
- GHSA-p236-jmm4-v336 / CVE-2026-18021 (The Beaver Builder Page Builder – Drag and Drop Website Builder, code injection, published 2026-09-08): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-c2fp-562c-5p2q / CVE-2026-12757 (The Email Subscribers & Newsletters – Email Marketing, Post Notifications & Newsletter Plugin..., code injection, published 2026-09-07): icegram/email-subscribers last push 2019; fix only on wp.org SVN
- GHSA-368m-gh85-cq79 / CVE-2026-81342 (MasterStudy LMS, open redirect, published 2026-08-29): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-qc2w-xhvw-6xmh / CVE-2026-16296 (Clearfy Cache, open redirect, published 2026-08-04): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
- GHSA-66xh-mw5m-g493 / CVE-2026-14236 (Contact Form 7, open redirect, published 2026-07-27): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
Counts: - Enumeration: `gh api "/advisories?type=unreviewed&cwes=<C>&per_page=100&page=<1..3>&sort=published&direction=desc"` for C in 89, 78, 94, 918, 601, 798, 942. That gives the 300 newest unreviewed advisories per CWE, covering roughly 2026-09-14 to 2026-09-25 for 89/918, and back to 2026-07-27 for 601. I kept those whose description mentions "WordPress" and "plugin" (57 advisories, 2026-07-27 → 2026-09-25) and worked through them newest first. For each, I took the plugin slug from the trac/wordpress.org references and searched GitHub (`search/repositories` by slug, name and vendor). I rejected repos that are only SVN mirrors (common-repository, WordPressBugBounty, WPPlugins, wp-plugins, WordpressPluginDirectory) or that stop before the fix. No WordPress advisory with CWE 798 or 942 appeared in that window.
- Examined 57, taken 6 (5 quota + 1 extra untrusted_destination), rejected 51.
- Family outcome: 1 code injection, 1 SQLi, 1 SSRF, 3 open redirect, 0 config_secrets. Most WordPress SQLi advisories in the window (Better Messages, Custom Field Template, WP Optimizer, Mediavine Create, Photo Gallery, NEX-Forms, WCFM, InfiniteWP, etc.) have no public development repository, so SQLi is thin in this lane for that reason only.
@@ -0,0 +1,842 @@
# Independent evaluation population v3 (PHP): candidate real-world cases
Research date: 2026-09-30. Method: web research only. I read GitHub advisory records (the global advisory API, reviewed and unreviewed), commit pages, compare views, repository metadata and file histories through `gh api` (read-only) and GitHub search. I did not clone, download or check out any candidate repository, did not run any scanner, rule, engine or project tool on one, and did not read candidate source beyond what an advisory itself quotes. Commit pages were read for their metadata (parents, message, changed-file names and line counts), not their patch text.
Output: this file and `candidates-v3-php.json` (26 candidates, machine-readable, one record per case in the v2 schema plus research fields). Nothing is reserved yet: the maintainer chooses, then a `population-v3.toml` and a protocol are written and frozen as for v2.
## Why a v3, and what it must not contain
PHP quick-mode rules were written on 2026-09-30 while looking at PMPro, population-v1's PHP cases, Ultimate Member, MediaWiki and WP Statistics. None of today's PHP numbers qualifies, and population v2 is reserved for other qualification. v3 must be PHP cases nobody in this project has looked at.
## Selection criteria (fixed before looking at any advisory)
From the maintainer's brief, unchanged:
- **Language** PHP. WordPress plugins allowed but capped at half the population; non-WordPress PHP applications and frameworks included.
- **Recency**: advisory published 2026-06-01 or later, taken newest first; a public fix commit or tag diff in a public GitHub repository; identifiable vulnerable and fixed versions.
- **Family spread**: at most about 40% injection; include output_encoding, path, untrusted_destination, deserialization, access_control and OS command injection where advisories exist. Families are the ids of `src/openultrasast/ruleset/families.toml`, and an advisory's family is found from its CWE ids through that file's `cwes` lists (so CWE-434 upload and CWE-74 generic injection map to no family).
- **Declarable site**: the advisory names the vulnerable function or file, so a `sites` entry can be declared from its text alone.
- **Exclusions**: any repository whose `github.com/owner/repo` appears under `benchmarks/`, `src/`, `tests/` or `.kiro/` (341 repositories, extracted by grep and compared case-insensitively); PMPro, Ultimate Member, LearnPress, phpMyFAQ, YesWiki, MediaWiki (including MediaWiki extensions), WP Statistics, froxlor, GoPay, Directorist and Groundhogg by name; repositories over about 150 MB by the GitHub API `size` (153,600 KB); advisories without a fix diff.
Operational rules, carried over from the v2 research (`research-2026-09-26.md`) and fixed before the first advisory was read:
- **In scope** means a web application, CMS, web framework/bundle or plugin whose vulnerable code is reached from HTTP request handling inside the repository. General-purpose libraries and framework APIs whose attacker input only exists in a calling application, developer tools/CLIs, and client-side JavaScript sinks are rejected (they give the analyzer no in-repo source).
- **Fix**: at most 3 commits, or one commit that is plainly about this weakness; a fix buried in a large unrelated squash or release commit is rejected, as is a fix that changes no PHP line (the fixed pin must differ in PHP). A fix commit the advisory does not name may be taken from the release compare or the named file's history, and is then marked `fix_named_by_advisory: false`.
- **One case per repository.** Old vulnerabilities republished in 2026 under an older CVE id are rejected (v2 did the same).
- **Guard-bypass and incomplete-fix advisories are allowed**, with the caveat recorded.
- **At no point was a case chosen or skipped for how easy it would be for this analyzer.** No rule or query of the project was consulted.
### Order and quotas
- **Family quotas** (targeting about 27 candidates): injection 8, output_encoding 4, path 4, untrusted_destination 4, access_control 4, deserialization 3.
- **Passes by family, rarest first**, so that one-case-per-repository does not hand a rare family's only repository to a common family. Rarity was counted from metadata alone (distinct eligible repositories per family after the automatic exclusions, before any advisory text was read): deserialization 1, injection 9, path 10, untrusted_destination 11, access_control 16, output_encoding 21. Within a pass, advisories are taken newest first (by `published_at`); a repository taken in an earlier pass is skipped.
- **Sources (lanes)**, consulted in this order within a pass, and each later lane only for a family still short after the earlier ones:
- **Lane A** — reviewed GitHub advisories, ecosystem composer, published >= 2026-06-01 (582 records; 386 in the six families). `gh api --paginate "/advisories?ecosystem=composer&type=reviewed&published=>=2026-06-01&sort=published&direction=desc"`.
- **Lane B** — unreviewed advisories for the family CWEs of injection, path, deserialization and untrusted_destination (5,246 records), kept when a reference is a `github.com/<o>/<r>/(commit|pull|compare)/` link and the GitHub API reports the repository's language as PHP (92 records on 56 PHP repositories). Output_encoding and access_control were not fetched, because lane A filled them.
- **Lane C** — deserialization only: unreviewed CWE-502 advisories whose text names WordPress (42), each plugin searched on GitHub for a development repository (SVN mirrors do not count, as in v2), plus the non-WordPress CWE-502 records naming a PHP product (9). The Patchstack one-line records ('Unauthenticated PHP Object Injection in X <= v') never name a function and were not examined individually.
- Lanes B and C were added after lane A left injection 4, path 3 and deserialization 0 against their quotas; the lane rule was fixed then, before any lane-B/C advisory was read.
- **Verification** of every candidate: `gh api repos/O/R/commits/<sha>` for the fixed commit (parents, message, changed files); the vulnerable pin is the fixed commit's (first) parent, except where a short linear chain is noted; `compare/<fixed>...<release tag>` confirms the fixed commit is an ancestor of the first fixed release wherever a tag exists; repository size, licence and default branch from `repos/O/R`.
## Counts
- **Examined: 470** advisories (lane A 379, lane B 40, lane C 51). **Taken: 26. Rejected: 444.**
- Automatic exclusions by metadata count as examined, as in v2; advisories passed over after their family's quota filled were not examined and are listed separately (lane A 7, lane B 34; lane B's 18 untrusted_destination rows were never needed).
| Rejection reason | Count |
|---|---|
| repository over ~150 MB | 161 |
| one case per repository | 64 |
| repository already referenced in the project tree (used list) | 46 |
| advisory names no vulnerable file or function | 46 |
| library/framework API: no attacker source inside the repository | 42 |
| duplicate advisory | 26 |
| no usable fix diff (none, unmerged, reverted, release squash, or non-PHP) | 25 |
| no public development repository | 10 |
| not a web application (developer tool, CLI, game server) | 9 |
| excluded by name | 6 |
| old vulnerability republished (CVE-2022..2025) | 4 |
| sink is client-side JavaScript | 3 |
| other: corrupt advisory record: the description is about draytek vi | 1 |
| other: personal/student project (`harsh21patel/inventory-management | 1 |
## Family mix of the 26 candidates
| Family | Candidates | Of which WordPress | Subtypes |
|---|---|---|---|
| injection | 8 | 0 | OS command / argument injection; OS command injection; SQL injection; code injection |
| output_encoding | 4 | 0 | XSS; reflected JSON injection; stored HTML injection; stored XSS |
| path | 4 | 1 | local file inclusion; path traversal |
| untrusted_destination | 4 | 0 | SSRF |
| access_control | 4 | 2 | IDOR; IDOR in REST permission callback; missing object-level capability check; unvalidated URL-writable properties override provisioning parameters |
| deserialization | 2 | 2 | PHP object injection; PHP object injection → arbitrary file deletion |
| **total** | **26** | **5** (19%) | |
Injection is 8 of 26 (31%): SQL 2 (Winter, QloApps), OS command 3 (Backpack, GitList, Roundcube), code/eval 3 (Grav, MantisBT, MacCMS). Untrusted_destination is all SSRF: no open-redirect advisory reached the top of its pass. WordPress is 5 of 26 (19%), under the cap.
## Candidates
Full records (sink, source, sites, site derivation, fix notes, caveats, benign proposal) are in `candidates-v3-php.json`. `R` marks the recommended selection below.
| # | R | id | repo | family | advisory | published | vulnerable → fixed | fixed release | sites (advisory) | licence | MB | WP |
|---|---|----|------|--------|----------|-----------|--------------------|---------------|------------------|---------|----|----|
| 1 | | grav-blueprint-bare-function-rce | https://github.com/getgrav/grav | injection | CVE-2026-75827 / GHSA-f8wv-xp27-6gq7 | 2026-09-17 | `24e8717ae317` → `af31066e9b6a` | 2.0.15 | `system/src/Grav/Common/Data/Blueprint.php::isSafeDynamicCall`, `system/src/Grav/Common/Data/Blueprint.php::init` | MIT | 34 | |
| 2 | | backpack-stats-host-header-cmdi | https://github.com/Laravel-Backpack/CRUD | injection | CVE-2026-54182 / GHSA-mrc5-3mm3-45c5 | 2026-08-20 | `baae0420b6eb` → `282ba2a0b887` | 7.0.36 | `src/Stats.php::makeCurlRequest` | MIT | 76 | |
| 3 | R | winter-filter-numberrange-sqli | https://github.com/wintercms/winter | injection | CVE-2026-32593 / GHSA-m7jc-g4rc-jmvh | 2026-08-12 | `555e67c84b50` → `50713de95adf` | 1.2.13 | `modules/backend/widgets/Filter.php::numbersFromAjax` | MIT | 68 | |
| 4 | | mantisbt-tokenizer-eval-rce | https://github.com/mantisbt/mantisbt | injection | CVE-2026-49273 / GHSA-v84x-qvhg-f36r | 2026-07-15 | `e7041ecc80c8` → `78c0af63d1fe` | 2.28.4 | file only: `core/classes/Tokenizer.class.php`, `adm_config_set.php` | GitHub: none [verify] | 89 | |
| 5 | R | gitlist-commandline-cmdi | https://github.com/klaussilveira/gitlist | injection | CVE-2026-82668 / GHSA-v64j-pgc3-qgc9 | 2026-08-31 | `f67609d52c18` → `88cf2866083d` | 3.0.0-beta | `src/SCM/System/Git/CommandLine.php::getDefaultBranch` | BSD-3-Clause | 6 | |
| 6 | R | qloapps-customermessage-bo-query-sqli | https://github.com/Qloapps/QloApps | injection | CVE-2026-75497 / GHSA-5w6r-2mxc-fq3v | 2026-08-25 | `5b4d65cbb857` → `123c97c110b7` | commit 123c97c | file only: `classes/CustomerMessage.php` | OSL-3.0 | 129 | |
| 7 | R | roundcube-markasjunk-cmd-learn-rce | https://github.com/roundcube/roundcubemail | injection | CVE-2026-74997 / GHSA-q5rh-mmp4-ppgq | 2026-08-17 | `a1afb8fd1f00` → `b8f90e28a46d` | 1.6.18 / 1.7.3 | file only: `plugins/markasjunk/drivers/cmd_learn.php` | GPL-3.0 | 67 | |
| 8 | R | maccms10-template-blacklist-rce | https://github.com/magicblack/maccms10 | injection | CVE-2026-71232 / GHSA-cv48-22qx-mpmx | 2026-08-05 | `2cabecda2994` → `71ad3bb29570` | commit 71ad3bb | file only: `application/admin/controller/Template.php` | GitHub: NOASSERTION [verify] | 29 | |
| 9 | R | kirby-realpath-sibling-traversal | https://github.com/getkirby/kirby | path | CVE-2026-75592 / GHSA-6j4c-mgqr-qv76 | 2026-09-02 | `e90257fd7504` → `2b6fab950bc8` | 4.9.5 / 5.5.2 | `src/Filesystem/Dir.php::realpath`, `src/Filesystem/F.php::realpath` | GitHub: NOASSERTION [verify] | 66 | |
| 10 | R | mineadmin-appstore-identifier-traversal | https://github.com/mineadmin/MineAdmin | path | CVE-2026-55224 / GHSA-59xm-4m8c-g3xj | 2026-08-18 | `ac0dcad616c5` → `ca41902a2a54` | 3.2.0-alpha.2 | `plugin/mine-admin/app-store/src/Service/Service.php::download`, `plugin/mine-admin/app-store/src/Service/Service.php::install`, `plugin/mine-admin/app-store/src/Service/Service.php::unInstall` | Apache-2.0 | 53 | |
| 11 | | facturascripts-uploadedfile-move-traversal | https://github.com/NeoRazorX/facturascripts | path | GHSA-hgjx-r89m-m7v4 | 2026-07-14 | `6baefcdc00c2` → `d3770be64a3e` | not stated | `Core/UploadedFile.php::move` | LGPL-3.0 | 60 | |
| 12 | R | wp-geo-my-wp-info-window-lfi | https://github.com/Fitoussi/geo-my-wp | path | CVE-2026-85200 / GHSA-6hw9-7gx5-mw88 | 2026-09-12 | `1dbb7b1672a4` → `5a768bf1c6e4` | 4.5.5.4 | `plugins/posts-locator/includes/gmw-posts-locator-functions.php::gmw_posts_locator_ajax_info_window_loader` | GitHub: none [verify] | 9 | yes |
| 13 | R | elfinder-fsock-dns-rebinding-ssrf | https://github.com/Studio-42/elFinder | untrusted_destination | CVE-2026-81889 / GHSA-8x3q-jpjh-qh5c | 2026-08-31 | `dd73e702820c` → `6d997386cd0f` | 2.1.70 | `php/elFinder.class.php::fsock_get_contents` | GitHub: NOASSERTION [verify] | 43 | |
| 14 | R | easyappointments-caldav-ssrf | https://github.com/alextselegidis/easyappointments | untrusted_destination | CVE-2026-52840 / GHSA-pm5p-7w5h-jm5q | 2026-07-29 | `725eafa64730` → `4abb10545d83` | 1.6.0 | `application/controllers/Caldav.php::connect_to_server`, `application/libraries/Caldav_sync.php::test_connection` | GPL-3.0 | 49 | |
| 15 | | koel-podcast-enclosure-ipv6-ssrf | https://github.com/koel/koel | untrusted_destination | CVE-2026-54494 / GHSA-rjg7-r26h-cfp2 | 2026-07-15 | `8ee1de0f5589` → `5f6ce2cefd08` | 9.7.1 | `app/Helpers/Network.php::isPublicHost`, `app/Values/Podcast/EpisodePlayable.php::createForEpisode` | MIT | 29 | |
| 16 | | pagible-admin-proxy-dns-rebinding-ssrf | https://github.com/aimeos/pagible | untrusted_destination | CVE-2026-49262 / GHSA-mmj8-wcvw-6789 | 2026-06-26 | `03f03b2f81b2` → `09a8205d513e` | 0.10.4 | `admin/src/Controllers/AdminController.php::proxy`, `core/src/Utils.php::isValidUrl` | MIT | 73 | |
| 17 | R | wp-graphql-updatepost-publish-bypass | https://github.com/wp-graphql/wp-graphql | access_control | CVE-2026-88974 / GHSA-5mmc-8pc9-wggg | 2026-09-23 | `d53b0e566f20` → `55441663eaa3` | 2.22.2 | file only: `plugins/wp-graphql/src/Mutation/PostObjectUpdate.php` | GPL-3.0 | 120 | yes |
| 18 | R | sylius-mollie-thankyou-order-idor | https://github.com/Sylius/MolliePlugin | access_control | CVE-2026-68501 / GHSA-x83g-979r-f5fh | 2026-07-31 | `7400f1489a4b` → `01316b3ad3cf` | 2.2.8 / 3.2.4 / 3.3.1 | `src/Controller/Shop/PageRedirectController.php::thankYouAction`, `src/Controller/Shop/QrCodeAction.php::fetchQrCodeFromOrder` | MIT | 9 | |
| 19 | | paymenter-checkout-config-override | https://github.com/Paymenter/Paymenter | access_control | CVE-2026-47198 / GHSA-5q4q-834j-g8g4 | 2026-06-30 | `9d305ea444f7` → `75719ae46b40` | 1.5.1 | file only: `app/Livewire/Products/Checkout.php`, `app/Livewire/Cart.php` | MIT | 15 | |
| 20 | | wp-buddypress-messages-rest-idor | https://github.com/buddypress/buddypress | access_control | CVE-2026-53673 / GHSA-j3j5-5m8v-7gvc | 2026-06-10 | `56b1fd3b4054` → `27d41e9749a8` | 14.5.0 | `src/bp-messages/classes/class-bp-messages-rest-controller.php::get_item_permissions_check` | GitHub: none [verify] | 79 | yes |
| 21 | R | sharp-editor-iframe-srcdoc-xss | https://github.com/code16/sharp | output_encoding | CVE-2026-61823 / GHSA-qxg3-46rw-79j8 | 2026-09-25 | `677431493090` → `ec509a22c808` | 9.22.5 | file only: `src/Utils/Sanitization/FormatsSanitizedValue.php` | MIT | 109 | |
| 22 | R | privatebin-jsonld-request-uri-injection | https://github.com/PrivateBin/PrivateBin | output_encoding | CVE-2026-55891 / GHSA-xrjc-c68j-hp7w | 2026-08-28 | `e0dd4c025c19` → `75f056dcda95` | 2.0.5 | `lib/Request.php::getRequestUri`, `lib/Controller.php::_jsonld` | GitHub: NOASSERTION [verify] | 16 | |
| 23 | R | openstamanager-group-name-html-injection | https://github.com/devcode-it/openstamanager | output_encoding | CVE-2026-44701 / GHSA-crx4-7mmq-j74j | 2026-08-26 | `57a2734e2121` → `0b88ad196387` | 2.11-beta | `modules/utenti/edit.php::<global>` | GPL-3.0 | 43 | |
| 24 | | symfony-ux-live-child-tag-xss | https://github.com/symfony/ux | output_encoding | CVE-2026-49210 / GHSA-38x5-rcv4-xf7x | 2026-06-19 | `a6e9bf808476` → `fbc5e9a1bda7` | 2.36.0 / 3.1.0 | `src/LiveComponent/src/Util/ChildComponentPartialRenderer.php::createHtml` | MIT | 59 | |
| 25 | R | wp-podlove-generation-guard-poi | https://github.com/podlove/podlove-publisher | deserialization | CVE-2026-16099 / GHSA-q83h-6954-x259 | 2026-08-16 | `4eedbd667969` → `ea878175f261` | 4.5.4 (not stated in the advisory; first release after the fix) | `lib/modules/shownotes/rest_api.php::create_link_item`, `lib/image_cache/generation_guard.php::__destruct` | MIT | 33 | yes |
| 26 | | wp-admin-columns-ids-to-collection-poi | https://github.com/codepress/admin-columns | deserialization | CVE-2026-7654 / GHSA-h6x4-229g-5f43 | 2026-06-06 | `ea6db9a02e01` → `c64e9aaecd20` | 7.0.19 (not stated; first release after the fix) | `classes/Formatter/IdsToCollection.php::get_ids_from_string` | GPL-3.0 | 37 | yes |
Full SHAs are in the JSON. Fix commits the advisory does not name (inferred from a release compare or file history): grav, backpack, facturascripts, pagible, paymenter, podlove, admin-columns.
### Per-candidate notes
#### grav-blueprint-bare-function-rce (injection; code injection (callable allow-list bypass → arbitrary PHP function))
- Repo https://github.com/getgrav/grav (MIT, 34 MB, languages PHP 100%). Lane A. Advisory CVE-2026-75827 / GHSA-f8wv-xp27-6gq7, published 2026-09-17; affected <= 2.0.14; fixed 2.0.15.
- Sink: system/src/Grav/Common/Data/Blueprint.php:455-458 `call_user_func_array($o, $params)` (bare-function dynamic-data provider), gated by `Blueprint::isSafeDynamicCall()` (Blueprint.php:514-536), whose bare-function branch uses only the denylist `Utils::isDangerousFunction()`; twin sink system/src/Grav/Framework/Flex/FlexDirectory.php:936-938 `call_user_func_array($function, $params)`. Source: a blueprint `data-*@` directive naming a bare PHP function (e.g. `error_log`) with attacker parameters, resolved by `Blueprint::init()`. Privilege: authenticated account that can edit pages/blueprint data (per the advisory's root-cause section; verify).
- Fix: The advisory links no commit. Found in the release compare 2.0.14...2.0.15 (14 commits, each security fix labelled): `af31066e` '[security] Gate the blueprint bare-function data directive on a positive allowlist', 1 file (Blueprint.php +11-1). A later commit in the same release, `c51987f5` 'Trust blueprint dynamic-data providers by provenance, not by allowlist', reworks the same gate. Ancestry: parent; ancestor of tag 2.0.15.
- Sites: file paths and function names quoted by the advisory. sink_evidence `call_user_func_array($o, $params)`.
- Benign: fixed pin as fixed-side control; proposed benign change `8661b3b56448` "Deduplicate repeated debugger deprecation traces" (172 lines, CHANGELOG.md, system/src/Grav/Common/Debugger.php, tests/unit/Grav/Common/DebuggerDeprecationsTest.php).
- Caveats: guard bypass: the vulnerable pin already has a denylist check; the enclosing function of the call_user_func_array line is not named by this advisory (sink_evidence is used instead). Confidence: high.
#### backpack-stats-host-header-cmdi (injection; OS command injection)
- Repo https://github.com/Laravel-Backpack/CRUD (MIT, 76 MB, languages PHP 80%, Blade 17%). Lane A. Advisory CVE-2026-54182 / GHSA-mrc5-3mm3-45c5, published 2026-08-20; affected 7.x < 7.0.36 (also 4.1.x < 4.1.72, 5.x < 5.6.2, 6.x < 6.8.13); fixed 7.0.36.
- Sink: `Backpack\CRUD\Stats::makeCurlRequest` builds a shell command and passes it to `exec()`. Source: HTTP `Host` header, on every production request through `BackpackServiceProvider::boot()` (1-in-100 random gate). Privilege: unauthenticated.
- Fix: The advisory links only release tags. The compare 7.0.35...7.0.36 contains exactly one commit, `282ba2a0` 'refactor stats to use guzzle only' (src/Stats.php +6-31), matching the advisory's 'makeCurlRequest was replaced with the Guzzle-based path'. Ancestry: parent; the fixed commit is tag 7.0.36.
- Sites: class and method named by the advisory; the path src/Stats.php follows from the PSR-4 class name and is confirmed only by the fix commit's file list.
- Benign: fixed pin as fixed-side control; proposed benign change `608652b1020b` "fix color display (#5992)" (8 lines, src/resources/views/crud/columns/color.blade.php).
- Caveats: needs exec() and curl available, and a Host header that the web server passes through (the advisory rates AC:H). Confidence: high.
#### winter-filter-numberrange-sqli (injection; SQL injection)
- Repo https://github.com/wintercms/winter (MIT, 68 MB, languages PHP 70%, JavaScript 19%, Less 8%). Lane A. Advisory CVE-2026-32593 / GHSA-m7jc-g4rc-jmvh, published 2026-08-12; affected <= 1.2.12; fixed 1.2.13.
- Sink: `Backend\Widgets\Filter` builds SQL for a `numberrange` scope configured with a `conditions` key. Source: the filter widget's AJAX handler (`numbersFromAjax`, per the advisory title). Privilege: authenticated backend user with access to an affected list view.
- Fix: Advisory workaround names commit `50713de9` 'Use parameter binding for filter scope values in conditions statements' (modules/backend/widgets/Filter.php +64-24, plus a test). Ancestry: parent; ancestor of tag v1.2.13, not of v1.2.12.
- Sites: class `Backend\Widgets\Filter` and `numbersFromAjax` named by the advisory; path confirmed by the fix commit.
- Benign: fixed pin as fixed-side control; proposed benign change `2a342f94ae7c` "Improved Ukrainian translation (#1467)" (46 lines, modules/backend/lang/uk/lang.php).
- Caveats: no built-in view uses the vulnerable scope/config combination: exploitable only with a plugin that registers a numberrange scope with `conditions` (the sink is in core, the configuration is not). Confidence: medium.
#### mantisbt-tokenizer-eval-rce (injection; code injection (eval))
- Repo https://github.com/mantisbt/mantisbt (GitHub: none [verify], 89 MB, languages PHP 82%, CSS 10%). Lane A. Advisory CVE-2026-49273 / GHSA-v84x-qvhg-f36r, published 2026-07-15; affected >= 1.3.0, <= 2.28.3; fixed 2.28.4.
- Sink: `Tokenizer` (reached through `ConfigParser`) calls `eval()` with a `return;` prefix; PHP hoists class declarations past the return. Source: non-string configuration value submitted to adm_config_set.php (Manage Configuration). Privilege: administrator.
- Fix: Advisory patch link `78c0af63` 'Remove use of eval() in Tokenizer class' (core/classes/Tokenizer.class.php +2-6). Ancestry: parent; ancestor of tag release-2.28.4.
- Sites: the advisory names adm_config_set.php, ConfigParser, Tokenizer and eval(); the Tokenizer file path and the enclosing function are not named (path from the fix commit). sink_evidence `eval(`.
- Benign: fixed pin as fixed-side control; proposed benign change `c867fcc0c119` "Web UI: use project category commands" (58 lines, manage_proj_cat_add.php, manage_proj_cat_copy.php, manage_proj_cat_delete.php…).
- Caveats: function-level site must be resolved at freeze by reading the file (the advisory names only the class). Confidence: high.
#### gitlist-commandline-cmdi (injection; OS command / argument injection)
- Repo https://github.com/klaussilveira/gitlist (BSD-3-Clause, 6 MB, languages PHP 87%, Twig 9%). Lane B. Advisory CVE-2026-82668 / GHSA-v64j-pgc3-qgc9, published 2026-08-31; affected 2.0.0; fixed 3.0.0-beta.
- Sink: `getDefaultBranch` of src/SCM/System/Git/CommandLine.php (git command line). Source: repository content/arguments reaching the git command line (not stated precisely). Privilege: not stated ('remotely').
- Fix: Advisory names patch `88cf2866` 'Prevent textconv. Fixes #947.' (src/SCM/System/Git/CommandLine.php +2-2, plus a test). Ancestry: parent; ancestor of tag 3.0.0-beta.
- Sites: file and function quoted by the advisory (VulDB-generated text).
- Benign: fixed pin as fixed-side control; no API proposal passed the freeze.py filter within 60 commits after the fix; let freeze.py propose one from the clone, or pick by hand (a small non-security PHP change on the default branch that does not touch the sink file).
- Caveats: VulDB auto-generated description; the fix message ('Prevent textconv') suggests git config/attribute abuse rather than shell metacharacters, and the patched lines may not be in getDefaultBranch -- verify at freeze. Confidence: medium.
#### qloapps-customermessage-bo-query-sqli (injection; SQL injection)
- Repo https://github.com/Qloapps/QloApps (OSL-3.0, 129 MB, languages PHP 80%, Smarty 9%, JavaScript 7%). Lane B. Advisory CVE-2026-75497 / GHSA-5w6r-2mxc-fq3v, published 2026-08-25; affected not stated ('Fixed in 123c97c'); fixed commit 123c97c.
- Sink: database query in CustomerMessage.php built from the `bo_query` parameter. Source: `bo_query` request parameter. Privilege: administrator.
- Fix: Advisory names `123c97c1` 'Fixed: validate request parameters before database query' (PR #1783): classes/CustomerMessage.php +1-1, classes/Address.php +8-8 (twin advisory GHSA-mj48-jjc7-6wqr) and three hotel-module classes. Ancestry: parent.
- Sites: the advisory names 'CustomerMessage.php' (basename) and the parameter; the classes/ path is from the fix commit; no function named.
- Benign: fixed pin as fixed-side control; proposed benign change `54c12b4efe41` "fixed issues" (106 lines, classes/order/Order.php, classes/order/OrderPayment.php, modules/hotelreservationsystem/classes/HotelBookingDetail.php…).
- Caveats: one fix covers two advisories (Address.php is GHSA-mj48); repo 129 MB, near the size cap; function-level site to be resolved at freeze. Confidence: medium.
#### roundcube-markasjunk-cmd-learn-rce (injection; OS command injection)
- Repo https://github.com/roundcube/roundcubemail (GPL-3.0, 67 MB, languages PHP 90%, JavaScript 7%). Lane B. Advisory CVE-2026-74997 / GHSA-q5rh-mmp4-ppgq, published 2026-08-17; affected < 1.6.18; 1.7.x < 1.7.3; fixed 1.6.18 / 1.7.3.
- Sink: the markasjunk plugin's cmd_learn driver runs a shell command with placeholder replacement values. Source: crafted placeholder replacement values (message/user data) when a message is marked as junk. Privilege: authenticated webmail user.
- Fix: Advisory links `14044f84` (master) and `b8f90e28` (1.6 line), both 'Fix RCE via cmd_learn driver of markasjunk plugin' (plugins/markasjunk/drivers/cmd_learn.php +14-10), and a follow-up warning fix `495d2116`. The 1.6-line pin is used because it is an ancestor of tag 1.6.18. Ancestry: parent; ancestor of tag 1.6.18.
- Sites: the advisory names 'the cmd_learn driver of the markasjunk plugin'; the file path follows from that name and is confirmed by the fix commit; no function named.
- Benign: fixed pin as fixed-side control; proposed benign change `747f544708ed` "Simplify" (6 lines, tests/Framework/TnefDecoderTest.php); tests-only change, on master (the fixed pin is on the 1.6 line).
- Caveats: only instances that enable the markasjunk plugin with the cmd_learn driver. Confidence: medium.
#### maccms10-template-blacklist-rce (injection; code injection (template → PHP, blacklist bypass))
- Repo https://github.com/magicblack/maccms10 (GitHub: NOASSERTION [verify], 29 MB, languages PHP 66%, HTML 23%, JavaScript 9%). Lane B. Advisory CVE-2026-71232 / GHSA-cv48-22qx-mpmx, published 2026-08-05; affected before commit 71ad3bb (2026-06-22); fixed commit 71ad3bb.
- Sink: ThinkPHP `{if condition=...}` compiled into raw PHP; the admin template editor only blocks a function blacklist. Source: template content saved through the admin template editor. Privilege: administrator.
- Fix: Advisory names `71ad3bb2` (application/admin/controller/Template.php +1-1: adds the missing functions to the blacklist regex). Ancestry: parent.
- Sites: file path quoted by the advisory; no function named.
- Benign: fixed pin as fixed-side control; proposed benign change `668ba418e50e` "fix #1356 修正注册API校验器解析错误及缺参报错" (166 lines, application/api/controller/User.php, application/common/model/User.php, tests/regression/user_register_validate.php…); review: subject says it fixes a register-API validator.
- Caveats: guard bypass (incomplete blacklist); the fix is still a blacklist; licence not declared on GitHub. Confidence: medium.
#### kirby-realpath-sibling-traversal (path; path traversal (prefix containment check))
- Repo https://github.com/getkirby/kirby (GitHub: NOASSERTION [verify], 66 MB, languages PHP 77%, Vue 15%). Lane A. Advisory CVE-2026-75592 / GHSA-6j4c-mgqr-qv76, published 2026-09-02; affected < 4.9.5; 5.0.0 - 5.5.1; fixed 4.9.5 / 5.5.2.
- Sink: `Dir::realpath()` / `F::realpath()` containment check accepts a sibling directory sharing the prefix (site vs site2). Source: media handler request path for not-yet-generated thumbnails. Privilege: unauthenticated.
- Fix: Advisory links `2b6fab95` 'fix: Ensure containment in ::realpath()' (4.x line: src/Filesystem/Dir.php, src/Filesystem/F.php) and `e0dca5f7` (5.x line, which also carries the Media::thumb fix of GHSA-9vx2-j98c-p72w). The 4.x pin keeps the case to this advisory alone. Ancestry: parent; ancestor of tag 4.9.5 (not of 5.5.2).
- Sites: `Dir::realpath()` and `F::realpath()` named by the advisory; paths confirmed by the fix commit.
- Benign: fixed pin as fixed-side control; proposed benign change `3989964ac1cb` "fix: Stuck unsaved changes for untouched fields (#8355)" (80 lines, config/fields/object.php, config/fields/toggle.php, src/Form/Field.php…); on the default branch (5.x); the fixed pin is on the 4.x line.
- Caveats: only deployments whose index root has a sibling directory with the same name prefix; Kirby is under a proprietary licence (GitHub NOASSERTION); must be cleared before publishing. Confidence: medium.
#### mineadmin-appstore-identifier-traversal (path; path traversal)
- Repo https://github.com/mineadmin/MineAdmin (Apache-2.0, 53 MB, languages PHP 99%). Lane A. Advisory CVE-2026-55224 / GHSA-59xm-4m8c-g3xj, published 2026-08-18; affected < 3.2.0-alpha.2; fixed 3.2.0-alpha.2.
- Sink: `BASE_PATH . '/plugin/' . $params['identifier']` in download()/install()/unInstall(), then `Plugin::install($params['identifier'])`. Source: `identifier` request parameter of the app-store plugin endpoints. Privilege: authenticated admin with app-store access (verify).
- Fix: Advisory links `ca41902a` 'fix(app-store): 修复app-store几处安全风险。 (#728)' (Service.php +81-9, IndexController.php +4-0). Ancestry: parent; the fixed commit is tag v3.2.0-alpha.2.
- Sites: file path and methods quoted by the advisory.
- Benign: fixed pin as fixed-side control; proposed benign change `c2f97b772c10` "feat(install): 添加前端项目下载与依赖安装功能 (#735)" (193 lines, app/Command/InstallCommand.php).
- Caveats: Hyperf framework (not Laravel/Symfony). Confidence: high.
#### facturascripts-uploadedfile-move-traversal (path; path traversal (arbitrary file write))
- Repo https://github.com/NeoRazorX/facturascripts (LGPL-3.0, 60 MB, languages PHP 90%, Twig 9%). Lane A. Advisory GHSA-hgjx-r89m-m7v4, published 2026-07-14; affected >= 2025, <= 2026.2; fixed not stated.
- Sink: `UploadedFile::move($destiny, $destinyName)` concatenates the client filename into `move_uploaded_file()`. Source: `getClientOriginalName()` of an uploaded file. Privilege: authenticated user.
- Fix: The advisory links no commit and states no patched version. File history of Core/UploadedFile.php has `d3770be6` 'Fix uploaded file path traversal' (2026-05-19; UploadedFile.php +38-4 plus tests). Inferred, not confirmed by the advisory. Ancestry: parent.
- Sites: file and method quoted by the advisory.
- Benign: fixed pin as fixed-side control; proposed benign change `add96d2bbee5` "test: add API smoke test" (192 lines, Test/Core/Controller/ApiSmokeTest.php); tests-only change.
- Caveats: fix commit inferred from file history; the advisory's affected range ends at 2026.2 while the inferred fix is dated 2026-05-19 -- confirm at freeze that the fixed pin is really unaffected. Confidence: medium.
#### wp-geo-my-wp-info-window-lfi (path; local file inclusion)
- Repo https://github.com/Fitoussi/geo-my-wp (GitHub: none [verify], 9 MB, languages PHP 47%, JavaScript 44%, CSS 9%). Lane B. Advisory CVE-2026-85200 / GHSA-6hw9-7gx5-mw88, published 2026-09-12; affected <= 4.5.5.3; fixed 4.5.5.4.
- Sink: include of a file chosen through `gmw_posts_locator_ajax_info_window_loader`. Source: AJAX request parameters (unauthenticated). Privilege: unauthenticated.
- Fix: Advisory links `5a768bf1` 'Release 4.5.5.4' (5 files: includes/gmw-functions.php +93-6, the posts/members locator functions files, version bumps). A release commit, but small and on-topic. Ancestry: parent.
- Sites: function named by the advisory text; files from the advisory's own trac reference links (gmw-functions.php#L1945/L1987, gmw-posts-locator-functions.php#L964/L975); the function-to-file pairing is an assumption to confirm at freeze.
- Benign: fixed pin as fixed-side control; no API proposal passed the freeze.py filter within 60 commits after the fix; let freeze.py propose one from the clone, or pick by hand (a small non-security PHP change on the default branch that does not touch the sink file).
- Caveats: WordPress plugin; GitHub shows no licence (plugin header GPL presumed, verify); release commit (small). Confidence: medium.
#### elfinder-fsock-dns-rebinding-ssrf (untrusted_destination; SSRF (DNS rebinding guard bypass))
- Repo https://github.com/Studio-42/elFinder (GitHub: NOASSERTION [verify], 43 MB, languages JavaScript 65%, PHP 29%). Lane A. Advisory CVE-2026-81889 / GHSA-8x3q-jpjh-qh5c, published 2026-08-31; affected <= 2.1.69; fixed 2.1.70.
- Sink: `fsock_get_contents()` fallback reconnects by hostname after the URL validation resolved it (TOCTOU). Source: URL submitted for server-side upload. Privilege: depends on the integrating connector.
- Fix: Advisory links `6d997386` 'Merge commit from fork' (php/elFinder.class.php +113-12, i18n) and the build commit `191372c1`. `6d997386` has a single parent `dd73e702` (the fix of the sibling ZIP-extraction advisory). Ancestry: parent; ancestor of tag 2.1.70-src (tag 2.1.70 is a build branch with no common history).
- Sites: function named by the advisory, file linked by the advisory.
- Benign: fixed pin as fixed-side control; no API proposal passed the freeze.py filter within 60 commits after the fix; let freeze.py propose one from the clone, or pick by hand (a small non-security PHP change on the default branch that does not touch the sink file).
- Caveats: only when PHP lacks cURL; guard bypass (DNS rebinding); repository is 65% JavaScript; licence NOASSERTION on GitHub (verify). Confidence: medium.
#### easyappointments-caldav-ssrf (untrusted_destination; SSRF)
- Repo https://github.com/alextselegidis/easyappointments (GPL-3.0, 49 MB, languages PHP 89%, JavaScript 9%). Lane A. Advisory CVE-2026-52840 / GHSA-pm5p-7w5h-jm5q, published 2026-07-29; affected <= 1.5.2; fixed 1.6.0.
- Sink: `$this->caldav_sync->test_connection($caldav_url, ...)` → Guzzle REPORT request. Source: `caldav_url` request parameter of Caldav::connect_to_server. Privilege: authenticated backend user (admin, provider or secretary).
- Fix: Advisory links four linear commits of 2026-05-26: `4abb1054` 'fix: harden caldav url validation against ssrf' (the validation), then `6b34b78c`, `2da2baed`, `6eb9336a` (opt-in private-host toggle and its relocation). The first carries the fix and is used as the fixed pin; `6eb9336a` is the end of the chain if the maintainer prefers it. Ancestry: parent of 4abb1054; ancestor of tag 1.6.0.
- Sites: controller path and method quoted by the advisory; the library path follows from `$this->caldav_sync` and the fix commit.
- Benign: fixed pin as fixed-side control; proposed benign change `1b51132026c7` "Minor translation change." (2 lines, application/language/german/translations_lang.php).
- Caveats: fix spread over four linear commits (v2 allowed up to three; the first commit alone is the fix). Confidence: high.
#### koel-podcast-enclosure-ipv6-ssrf (untrusted_destination; SSRF (guard bypass via IPv6 transition addresses))
- Repo https://github.com/koel/koel (MIT, 29 MB, languages PHP 55%, TypeScript 27%, Vue 18%). Lane A. Advisory CVE-2026-54494 / GHSA-rjg7-r26h-cfp2, published 2026-07-15; affected <= 9.7.0; fixed 9.7.1.
- Sink: `EpisodePlayable::createForEpisode()` → `Http::sink($file)->get($url)`, guarded only by `Network::isPublicHost()`. Source: `<enclosure url>` of an attacker-hosted RSS feed subscribed by any user. Privilege: authenticated user.
- Fix: Advisory links `5f6ce2ce` 'fix: close remaining SSRF advisories (DNS rebinding, IPv6 transition, bail) (#2549)', 28 files; one commit that closes several SSRF advisories (same family). Ancestry: parent; ancestor of tag v9.7.1.
- Sites: app/Helpers/Network.php and isPublicHost quoted by the advisory; EpisodePlayable path from the class name `App\Values\Podcast\EpisodePlayable`.
- Benign: fixed pin as fixed-side control; proposed benign change `eafb61deaad0` "test: serve a real response to the radio stream connector (#2671)" (178 lines, tests/Integration/Services/Radio/RadioStreamConnectorTest.php, tests/Unit/Services/Radio/RadioStreamConnectorTest.php); tests-only change.
- Caveats: large multi-advisory fix commit (28 files), same family; exploitation needs NAT64/6to4 routing. Confidence: medium.
#### pagible-admin-proxy-dns-rebinding-ssrf (untrusted_destination; SSRF (DNS rebinding guard bypass))
- Repo https://github.com/aimeos/pagible (MIT, 73 MB, languages PHP 57%, JavaScript 16%, CSS 12%, Vue 11%). Lane A. Advisory CVE-2026-49262 / GHSA-mmj8-wcvw-6789, published 2026-06-26; affected < 0.10.4; fixed 0.10.4.
- Sink: `AdminController::proxy` → Guzzle request after `\Aimeos\Cms\Utils::isValidUrl($url)`. Source: `url` query parameter of `/cmsproxy`. Privilege: authenticated (proxy token).
- Fix: The advisory links no commit. The compare 0.10.3...0.10.4 has three commits; `09a8205d` 'Fixed SSRF via DNS rebinding in admin proxy' (admin/src/Controllers/AdminController.php +21-7, core/src/Utils.php +22-4). Ancestry: parent; ancestor of tag 0.10.4.
- Sites: class::method names quoted by the advisory; paths from the fix commit.
- Benign: fixed pin as fixed-side control; proposed benign change `bc5a10a515bd` "Fixed admin test" (2 lines, admin/tests/AdminControllerTest.php); tests-only change.
- Caveats: guard bypass (DNS rebinding). Confidence: medium.
#### wp-graphql-updatepost-publish-bypass (access_control; missing object-level capability check)
- Repo https://github.com/wp-graphql/wp-graphql (GPL-3.0, 120 MB, languages PHP 68%, JavaScript 26%). Lane A. Advisory CVE-2026-88974 / GHSA-5mmc-8pc9-wggg, published 2026-09-23; affected < 2.22.2 (confirmed 2.19.0); fixed 2.22.2.
- Sink: `updatePost` mutation checks only `$post_type_object->cap->edit_posts`. Source: GraphQL `updatePost` mutation input (status PUBLISH, published post id). Privilege: Contributor.
- Fix: Advisory links `55441663` (PR #4270) whose subject is 'do not invoke bare callable-string field definitions in Model'; it also changes plugins/wp-graphql/src/Mutation/PostObjectUpdate.php (+27-0) -- two fixes in one commit. Ancestry: parent; ancestor of tag wp-graphql/v2.22.2.
- Sites: the advisory names src/Mutation/PostObjectUpdate.php; the repository is a monorepo (plugins/wp-graphql/ prefix from the fix commit); no function named.
- Benign: fixed pin as fixed-side control; no API proposal passed the freeze.py filter within 60 commits after the fix; let freeze.py propose one from the clone, or pick by hand (a small non-security PHP change on the default branch that does not touch the sink file).
- Caveats: fix commit also carries an unrelated Model.php change; function-level site to resolve at freeze. Confidence: medium.
#### sylius-mollie-thankyou-order-idor (access_control; IDOR (missing ownership check))
- Repo https://github.com/Sylius/MolliePlugin (MIT, 9 MB, languages PHP 90%). Lane A. Advisory CVE-2026-68501 / GHSA-x83g-979r-f5fh, published 2026-07-31; affected < 2.2.8; 3.0.0-3.2.3; 3.3.0; fixed 2.2.8 / 3.2.4 / 3.3.1.
- Sink: `findOneBy(['id' => $orderId])` without ownership/session check, returning the order token (302 Location) and QR code. Source: `orderId` of GET /{_locale}/thank-you and /{_locale}/get-code. Privilege: unauthenticated.
- Fix: Advisory links three merge commits (one per branch). `01316b3a` (PR #354, 3.3 line) is used: PageRedirectController.php +17-6, QrCodeAction.php +9-11, PaymentWebhookController.php (the sibling webhook advisory GHSA-rc52-c4hv-w89p). Ancestry: first parent of a merge; ancestor of tag v3.3.1.
- Sites: Class::method names quoted by the advisory; paths from the fix commit.
- Benign: fixed pin as fixed-side control; proposed benign change `ad215c9c0fc3` "Refactor `PaymentMethodLogoUploader` to use `FilesystemAdapterInterface` instead of `Gaufrette\Filesystem`." (15 lines, config/services/uploader.xml, src/Uploader/PaymentMethodLogoUploader.php).
- Caveats: merge commit; one fix covers two advisories (webhook forgery GHSA-rc52 is the other). Confidence: high.
#### paymenter-checkout-config-override (access_control; unvalidated URL-writable properties override provisioning parameters)
- Repo https://github.com/Paymenter/Paymenter (MIT, 15 MB, languages PHP 87%, Blade 12%). Lane A. Advisory CVE-2026-47198 / GHSA-5q4q-834j-g8g4, published 2026-06-30; affected < 1.5.1; fixed 1.5.1.
- Sink: `$checkoutConfig` (`#[Url]`, alias `config`) keys stored by Cart.php into service properties, later passed to createServer(). Source: `config[...]` query parameters of the checkout page. Privilege: authenticated customer.
- Fix: The advisory links no commit. The compare v1.5.0...v1.5.1 has 17 commits; `75719ae4` 'Merge commit from fork' changes only app/Livewire/Products/Checkout.php (+7-1). Ancestry: parent; ancestor of tag v1.5.1.
- Sites: file paths quoted by the advisory; no function named.
- Benign: fixed pin as fixed-side control; proposed benign change `e60567981acf` "fix: reCAPTCHA v3 not working after Livewire navigation" (24 lines, themes/default/views/components/captcha/recaptcha-v3.blade.php); review: subject mentions reCAPTCHA (anti-abuse).
- Caveats: mass-assignment shape (CWE-20/639); classed access_control by its CWE; fix commit inferred from the release compare. Confidence: medium.
#### wp-buddypress-messages-rest-idor (access_control; IDOR in REST permission callback)
- Repo https://github.com/buddypress/buddypress (GitHub: none [verify], 79 MB, languages PHP 76%, CSS 14%, SCSS 6%). Lane A. Advisory CVE-2026-53673 / GHSA-j3j5-5m8v-7gvc, published 2026-06-10; affected < 14.5.0 (confirmed 14.4.0); fixed 14.5.0.
- Sink: `get_item_permissions_check` validates the supplied user_id instead of the logged-in user; reused by update/delete. Source: `user_id` parameter of the messages REST API. Privilege: authenticated (any user).
- Fix: Advisory links `27d41e97` 'Improve security of messages endpoint.' (class-bp-messages-rest-controller.php +113-42, tests). The commit is dated 2026-07-07, after the advisory; confirm at freeze that the release carrying it is 14.5.0. Ancestry: parent.
- Sites: method named by the advisory; file path from the fix commit.
- Benign: fixed pin as fixed-side control; proposed benign change `ea5cb85f4d8a` "PHPDoc: Improve parameter descriptions." (132 lines, src/bp-activity/bp-activity-embeds.php, src/bp-activity/bp-activity-filters.php, src/bp-activity/bp-activity-functions.php…).
- Caveats: the GitHub repository is the project's own mirror of its development SVN; no GitHub licence metadata (GPL-2.0+ per project, verify). Confidence: medium.
#### sharp-editor-iframe-srcdoc-xss (output_encoding; stored XSS (sanitizer allow-list))
- Repo https://github.com/code16/sharp (MIT, 109 MB, languages PHP 66%, Vue 23%, TypeScript 9%). Lane A. Advisory CVE-2026-61823 / GHSA-qxg3-46rw-79j8, published 2026-09-25; affected < 9.22.5; fixed 9.22.5.
- Sink: Symfony HtmlSanitizer config allows the iframe `srcdoc` attribute; the value is rendered as live HTML. Source: Editor field content saved by a user with edit permission. Privilege: authenticated editor.
- Fix: Advisory links `ec509a22` 'Improve sanitization' (src/Utils/Sanitization/FormatsSanitizedValue.php +4-2, test). The same commit fixes the sibling GHSA-vj3q-vp3g-j9c8. Ancestry: parent; ancestor of tag v9.22.5.
- Sites: file path quoted by the advisory; no function named. sink_evidence `srcdoc`.
- Benign: fixed pin as fixed-side control; proposed benign change `7c601fffe930` "Update version to v9.22.5" (2 lines, src/SharpInternalServiceProvider.php).
- Caveats: the flaw is a sanitizer allow-list entry, not a missing escape call. Confidence: medium.
#### privatebin-jsonld-request-uri-injection (output_encoding; reflected JSON injection (CWE-116))
- Repo https://github.com/PrivateBin/PrivateBin (GitHub: NOASSERTION [verify], 16 MB, languages PHP 46%, JavaScript 38%, CSS 16%). Lane A. Advisory CVE-2026-55891 / GHSA-xrjc-c68j-hp7w, published 2026-08-28; affected <= 2.0.4; fixed 2.0.5.
- Sink: `Controller::_jsonld()` splices `$_urlBase` into a .jsonld template with `str_replace()`. Source: `$_SERVER['REQUEST_URI']` via `Request::getRequestUri()` (FILTER_SANITIZE_URL keeps quotes and angle brackets). Privilege: unauthenticated.
- Fix: Advisory links `75f056dc` 'Merge commit from fork' (first-parent diff: lib/Request.php +2-2, lib/Configuration.php, CHANGELOG, test). Its first parent `e0dd4c02` is the fix of the sibling attachment-XSS advisory GHSA-f2xf-7x3g-4272. Ancestry: first parent of a merge; ancestor of tag 2.0.5.
- Sites: Class::method names quoted by the advisory; paths lib/*.php from the fix commit.
- Benign: fixed pin as fixed-side control; proposed benign change `48ae4ed5d3d4` "fix: quote administration storage path" (54 lines, bin/administration, tst/AdministrationEmptyDirsTest.php); review: 'quote administration storage path' may be a shell-quoting fix.
- Caveats: the fix changes Request.php only; Controller::_jsonld is a declared site outside the diff; response type is JSON-LD, not HTML; licence NOASSERTION on GitHub (zlib-style per project, verify). Confidence: high.
#### openstamanager-group-name-html-injection (output_encoding; stored HTML injection)
- Repo https://github.com/devcode-it/openstamanager (GPL-3.0, 43 MB, languages PHP 80%, JavaScript 17%). Lane A. Advisory CVE-2026-44701 / GHSA-crx4-7mmq-j74j, published 2026-08-26; affected <= 2.10.1; fixed 2.11-beta.
- Sink: `'<span class="text-primary">'.$group->getTranslation('title').'</span>'` in modules/utenti/edit.php. Source: group name saved when creating a user group. Privilege: authenticated user able to create groups.
- Fix: Advisory links `0b88ad19` 'fix: HTML Injection in modules/utenti/edit.php' (edit.php +2-2, ajax/select.php, src/AJAX.php, src/Modules.php) and a broader input-filter refactor `7c9eafe9`. The first is used. Ancestry: parent; ancestor of tag v2.11-beta.
- Sites: file and code quoted by the advisory. sink_evidence `$group->getTranslation('title')`.
- Benign: fixed pin as fixed-side control; proposed benign change `570eff68c504` "fix: definizione variabile" (2 lines, modules/fatture/src/Gestori/Scadenze.php).
- Caveats: second advisory commit (7c9eafe9, global GET/POST filtering) is not in the pinned pair. Confidence: high.
#### symfony-ux-live-child-tag-xss (output_encoding; XSS (unescaped tag name))
- Repo https://github.com/symfony/ux (MIT, 59 MB, languages PHP 61%, TypeScript 22%, JavaScript 10%, Twig 7%). Lane A. Advisory CVE-2026-49210 / GHSA-38x5-rcv4-xf7x, published 2026-06-19; affected ux-live-component >= 2.8.0, < 2.36.0; >= 3.0.0, < 3.1.0; fixed 2.36.0 / 3.1.0.
- Sink: `ChildComponentPartialRenderer::createHtml()` interpolates `$childTag` into HTML. Source: client JSON `children[id].tag` parsed by `LiveComponentSubscriber` and passed through `InterceptChildComponentRenderSubscriber`. Privilege: unauthenticated (Accept header gate; CORS or same-origin needed).
- Fix: Advisory links `fbc5e9a1` '[LiveComponent] Reject malicious child component tags' (ChildComponentPartialRenderer.php +7-0, test). Ancestry: parent; ancestor of tag v2.36.0.
- Sites: FQCN::method quoted by the advisory; monorepo path from the fix commit.
- Benign: fixed pin as fixed-side control; no API proposal passed the freeze.py filter within 60 commits after the fix; let freeze.py propose one from the clone, or pick by hand (a small non-security PHP change on the default branch that does not touch the sink file).
- Caveats: the advisory calls it primarily defense in depth (custom Accept header cannot be sent cross-origin without preflight); component framework in a monorepo (UX packages). Confidence: medium.
#### wp-podlove-generation-guard-poi (deserialization; PHP object injection → arbitrary file deletion (in-plugin POP chain))
- Repo https://github.com/podlove/podlove-publisher (MIT, 33 MB, languages PHP 63%, JavaScript 18%, Vue 10%, TypeScript 6%). Lane C. Advisory CVE-2026-16099 / GHSA-q83h-6954-x259, published 2026-08-16; affected <= 4.5.3; fixed 4.5.4 (not stated in the advisory; first release after the fix).
- Sink: unserialize() of model data (lib/model/base.php#L552) instantiating `Podlove\ImageCache\GenerationGuard`, whose `__destruct()` calls `wp_delete_file()`. Source: shownotes REST input handled by `create_link_item` (contributor+). Privilege: Contributor.
- Fix: The advisory links wp.org trac only. The GitHub history of the referenced files has three consecutive commits of 2026-07-26: `d49c4d7f` 'fix: constrain image lock deletion' (generation_guard.php), `b59d7962` 'fix: harden shownotes REST input' (rest_api.php +173-20), `ea878175` 'fix: limit model unserialization' (lib/model/base.php). The pair spans all three (vulnerable = parent of d49c4d7f). Ancestry: linear chain of 3 commits (4eedbd66 → d49c4d7f → b59d7962 → ea878175); ancestor of tag 4.5.4.
- Sites: create_link_item and GenerationGuard::__destruct named by the advisory; files from the advisory's own trac reference links.
- Benign: fixed pin as fixed-side control; proposed benign change `50da8db0f5a8` "fix PHP warning" (2 lines, lib/modules/contributors/contributor_list_table.php).
- Caveats: fix inferred from file history (three commits); the advisory text describes the impact (file deletion) under CWE-502. Confidence: medium.
#### wp-admin-columns-ids-to-collection-poi (deserialization; PHP object injection (bundled POP chain))
- Repo https://github.com/codepress/admin-columns (GPL-3.0, 37 MB, languages PHP 69%, Svelte 10%, SCSS 7%, JavaScript 6%, TypeScript 6%). Lane C. Advisory CVE-2026-7654 / GHSA-h6x4-229g-5f43, published 2026-06-06; affected <= 7.0.18; fixed 7.0.19 (not stated; first release after the fix).
- Sink: `unserialize()` without `allowed_classes` in `IdsToCollection::get_ids_from_string()`. Source: post meta values set by a Contributor. Privilege: Contributor.
- Fix: The advisory links wp.org trac only. The compare 7.0.18...7.0.19 has one commit touching the named file: `c64e9aae` 'Update IdsToCollection.php' (+1-1). An identical change `25e276e0` sits on the 7.1 development line. Inferred; the commit message does not say what changed. Ancestry: parent; ancestor of tag 7.0.19.
- Sites: Class::method named by the advisory; files from its trac reference links. sink_evidence `unserialize(`.
- Benign: fixed pin as fixed-side control; proposed benign change `e998321863c4` "Todo ans PHP code fixer" (40 lines, classes/ColumnCollection.php, classes/DI/Container.php, classes/Exception/IteratorException.php…).
- Caveats: fix inferred (one-line change, uninformative message) -- confirm at freeze that it adds allowed_classes. Confidence: medium.
## Recommended balanced selection (16)
**Rule** (applied mechanically to the 26, never by how easy a case looks): per-family targets injection 5, output_encoding 3, path 3, untrusted_destination 2, access_control 2, deserialization 1. Within a family, candidates whose fix commit the advisory names come first, then newest first by publication; take the first *n*. The WordPress cap is checked afterwards, not used to choose.
- **injection** (5): **gitlist-commandline-cmdi** (2026-08-31), **qloapps-customermessage-bo-query-sqli** (2026-08-25), **roundcube-markasjunk-cmd-learn-rce** (2026-08-17), **winter-filter-numberrange-sqli** (2026-08-12), **maccms10-template-blacklist-rce** (2026-08-05), mantisbt-tokenizer-eval-rce (2026-07-15), grav-blueprint-bare-function-rce (2026-09-17, fix inferred), backpack-stats-host-header-cmdi (2026-08-20, fix inferred)
- **output_encoding** (3): **sharp-editor-iframe-srcdoc-xss** (2026-09-25), **privatebin-jsonld-request-uri-injection** (2026-08-28), **openstamanager-group-name-html-injection** (2026-08-26), symfony-ux-live-child-tag-xss (2026-06-19)
- **path** (3): **wp-geo-my-wp-info-window-lfi** (2026-09-12), **kirby-realpath-sibling-traversal** (2026-09-02), **mineadmin-appstore-identifier-traversal** (2026-08-18), facturascripts-uploadedfile-move-traversal (2026-07-14, fix inferred)
- **untrusted_destination** (2): **elfinder-fsock-dns-rebinding-ssrf** (2026-08-31), **easyappointments-caldav-ssrf** (2026-07-29), koel-podcast-enclosure-ipv6-ssrf (2026-07-15), pagible-admin-proxy-dns-rebinding-ssrf (2026-06-26, fix inferred)
- **access_control** (2): **wp-graphql-updatepost-publish-bypass** (2026-09-23), **sylius-mollie-thankyou-order-idor** (2026-07-31), wp-buddypress-messages-rest-idor (2026-06-10), paymenter-checkout-config-override (2026-06-30, fix inferred)
- **deserialization** (1): **wp-podlove-generation-guard-poi** (2026-08-16, fix inferred), wp-admin-columns-ids-to-collection-poi (2026-06-06, fix inferred)
Selected (bold above): winter-filter-numberrange-sqli, gitlist-commandline-cmdi, qloapps-customermessage-bo-query-sqli, roundcube-markasjunk-cmd-learn-rce, maccms10-template-blacklist-rce, kirby-realpath-sibling-traversal, mineadmin-appstore-identifier-traversal, wp-geo-my-wp-info-window-lfi, elfinder-fsock-dns-rebinding-ssrf, easyappointments-caldav-ssrf, wp-graphql-updatepost-publish-bypass, sylius-mollie-thankyou-order-idor, sharp-editor-iframe-srcdoc-xss, privatebin-jsonld-request-uri-injection, openstamanager-group-name-html-injection, wp-podlove-generation-guard-poi.
Mix: injection 5 (31%; SQL 2, OS command 2, code 1), output_encoding 3, path 3, untrusted_destination 2, access_control 2, deserialization 1; WordPress 3 of 16. Benign controls: the fixed pin for every case; a proposed benign change exists for 12 of 16 (gitlist, geo-my-wp, elfinder and wp-graphql need one chosen at freeze).
The deserialization pick has an inferred fix (both deserialization candidates do); it is the only family where the named-fix rule could not be met.
## Open decisions for the maintainer
- **Protocol coverage.** Protocol v2 scores only `injection`, `untrusted_destination` and `config_secrets`. v3 as proposed also needs scoring rules for `path`, `output_encoding`, `access_control` and `deserialization` (including what counts as a match for an absence bug such as a missing check, and the output_encoding limits stated in families.toml). Decide the family set before the population is reserved; dropping families changes the selection rule's targets, not the candidates.
- **File-only sites.** Seven candidates have no advisory-named function (mantisbt, qloapps, roundcube, maccms10, wp-graphql, paymenter, sharp); several others name a second file without a function (`site_files`). Protocol v2 credits `path::function`; for these the function has to be resolved at freeze by reading the file, or the protocol must credit a file-level site. Where the advisory quotes code, `sink_evidence` carries it for freeze.py.
- **Sites derived from class names.** Several advisories name `Class::method` but not the file; the path follows from PSR-4 or the plugin layout and is confirmed only by the fix commit's file list (marked in `site_derivation`).
- **Licences to clear before publishing:** Kirby (proprietary Kirby licence, not OSI), MacCMS10 (no licence metadata), elFinder and PrivateBin (NOASSERTION; project files state BSD-3 and zlib-style respectively, verify), MantisBT, BuddyPress and GEO my WP (no GitHub licence metadata; GPL per project, verify).
- **Inferred fixes:** grav, backpack, pagible and paymenter come from release compares with a single on-topic commit; facturascripts, podlove and admin-columns from file history (admin-columns' commit message is only 'Update IdsToCollection.php'). Freeze must confirm each diff changes the sink as the advisory describes.
- **Multi-advisory fixes:** koel (28 files, several SSRF advisories), sylius mollie (merge also fixes the webhook advisory), sharp, privatebin, qloapps and wp-graphql (commit also changes Model.php).
- **Non-default configuration required:** winter (plugin-registered scope), roundcube (markasjunk cmd_learn), elfinder (no cURL), kirby (sibling directory), backpack (exec/curl and Host passthrough), symfony-ux (CORS or same-origin).
- **Branch pins:** kirby uses the 4.x-line fix (the 5.x fix also carries another advisory); roundcube uses the 1.6-line fix; sylius mollie uses the 3.3-line merge. Benign proposals come from the default branch, which may not contain these pins.
- **Prior mention.** `research-2026-09-26.md` lists some of these advisory ids by name in its 'passed over without examination' note (Grav, elFinder, Backpack CRUD, Winter, Easy!Appointments, Koel). They were never examined and their URLs are not in the tree, so they pass the exclusion rule; the maintainer may still prefer to drop them.
- **Alternates** if a candidate fails at freeze (first in line for its family, newest first): injection — Krayin CRM leads DataGrid SQLi (GHSA-8879-fh4f-q3qm), FrontAccounting SQLi ×3 (GHSA-w5j4-x499-pfwp …), LMS SQLi/command injection (GHSA-qrw9-xj5g-qxw8, GHSA-765c-7qf8-cw9w), Dockwatch command injection (GHSA-cgfv-42v7-7f53), MantisBT history_order SQLi (GHSA-mw6p-33vw-46cc, same repository as the MantisBT candidate); path — GEOFlow ImageLibraryController (GHSA-gf7h-mv4h-gg7c), rConfig download_export (GHSA-qrrp-6pvj-xfvf); untrusted_destination — CakePHP Authentication open redirect (GHSA-hhpq-7wg4-36jm), Koillection SSRF (GHSA-gmxh-hjfv-qc2w), Cotonti open redirects (GHSA-g8jm-3grw-8wh2 …); deserialization — Cypht logout back_query (GHSA-46j6-w3hw-4r9x; fix `e4aa2f34` known, but the advisory names no file or function). None of the alternates was fully verified.
## What constrained the search
- **Deserialization is nearly absent with a usable fix.** Lane A had one eligible record (October CMS, whose public repository only receives bulk 'Sync upstream' commits). Lanes B and C yielded two, both WordPress, both with fixes inferred from file history; most WordPress object-injection records name no function or have no development repository.
- **SQL injection with a fix diff is scarce outside excluded or oversized repositories.** Most 2026 PHP SQLi advisories are on repositories already used (phpMyFAQ, YesWiki, froxlor, pimcore studio), over the size cap (Pimcore, TYPO3, Mautic, LimeSurvey, CodeIgniter4, Dolibarr, LibreNMS), in framework query builders (CakePHP), or in WordPress plugins with no GitHub repository. Only two SQLi candidates qualified in quota order; more exist as alternates.
- **The size cap is the largest single filter** (161 rejections): Craft CMS, Snipe-IT (32 advisories), TYPO3, Pimcore, Shopware, Sylius core, Mautic, Filament, AVideo, LibreNMS, MISP, Dolibarr, OpenEMR, ILIAS, Joomla.
- **Naming the site.** 46 advisories name no file or function (Statamic, Silverstripe and most WordPress records).
- **Kimai** shipped the fixes for nine of this window's advisories inside one 'Release 2.58' squash, which removes a whole application from the pool.
## Rejection log
One line per examined advisory, newest first within each lane. `taken` rows name the candidate. Automatic exclusions by metadata are listed first within lane A.
### Lane A: reviewed composer advisories
- GHSA-r6hr-vr92-vv28 / CVE-2026-65954 (phpcsstandards/phpcsutils, injection, 2026-09-29): developer tool / CLI, not a web application. _PHPCSUtils: Remote code execution via eval() in AbstractArrayDeclarationSniff::getActualAr_
- GHSA-jh5r-qr3c-85q8 / CVE-2026-102279 (laravel/framework, output_encoding, 2026-09-29): DOM XSS in the debug page's JavaScript tooltip (Tippy `allowHTML`); the sink is client-side JavaScript, and the advisory names no PHP file or function. _Laravel: XSS in Debug Page Information_
- GHSA-v65j-hff3-753c / CVE-2026-57440 (starcitizenwiki/mediawiki-extensions-embedvideo, output_encoding, 2026-09-25): excluded by name. _Mediawiki EmbedVideo Extension has stored XSS via malformed src url with $wgEmbedVideoRequ_
- GHSA-qxg3-46rw-79j8 / CVE-2026-61823 (code16/sharp, output_encoding, 2026-09-25): **taken** as sharp-editor-iframe-srcdoc-xss. _code16 Sharp vulnerable to stored XSS via iframe srcdoc Attribute_
- GHSA-vj3q-vp3g-j9c8 / CVE-2026-61825 (code16/sharp, output_encoding, 2026-09-25): one case per repository (repository taken as sharp-editor-iframe-srcdoc-xss). _code16/sharp has a stored XSS via data-html-content Sanitizer Bypass_
- GHSA-87mg-5grr-rhwh / CVE-2026-57232 (contao/contao, untrusted_destination, 2026-09-24): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Contao: Server-Side Request Forgery (SSRF) via Unvalidated RSS Feed URL in Feed Reader Mod_
- GHSA-rw77-vq4g-x3hp / CVE-2026-56738 (thorsten/phpmyfaq, injection, 2026-09-24): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ has SQL Injection in `StopWords::add()` — Unescaped Stop Word Insertion_
- GHSA-pgwp-vc7q-cvj3 / CVE-2026-56736 (thorsten/phpmyfaq, output_encoding, 2026-09-24): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ has Stored XSS in Admin FAQ Editor via HTML Entity Bypass in Frontend FAQ Submiss_
- GHSA-396x-xmvh-p563 / CVE-2026-63498 (grokability/snipe-it, output_encoding, 2026-09-24): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Stored XSS via Inline XML Rendering in the Uploaded Files API_
- GHSA-p9h3-gvpq-5539 / CVE-2026-62368 (grokability/snipe-it, output_encoding, 2026-09-24): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Stored XSS via Custom Field name in asset-list column headers_
- GHSA-9rg8-2wvr-fgjh / CVE-2026-76089 (verbb/formie, access_control, 2026-09-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Formie: Missing authorization on sent notification resend modal exposes submission PII_
- GHSA-584p-f93j-wpgc / CVE-2026-76087 (verbb/formie, access_control, 2026-09-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Formie: Unauthenticated users can overwrite incomplete submissions via submit action_
- GHSA-v3f3-cmj4-cvj9 / CVE-2026-76086 (verbb/formie, access_control/untrusted_destination, 2026-09-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Formie: Integration form-settings action allows SSRF and exfiltration of stored integratio_
- GHSA-jg26-q8hg-3pq4 / CVE-2026-92692 (sulu/sulu, injection, 2026-09-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Sulu: JCR-SQL2 injection via `categories` query parameter (unauthenticated)_
- GHSA-gxrg-x694-283w / CVE-2026-73858 (solspace/craft-freeform, injection, 2026-09-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Solspace Freeform: Limited Twig template injection via submitted field values_
- GHSA-w998-qmw9-mf4m / CVE-2026-63002 (redaxo/core, output_encoding, 2026-09-23): repo over ~150 MB (500 MB per GitHub API). _REDAXO: Stored XSS in Mediapool Sync Page via Unescaped Filesystem Filenames_
- GHSA-mf2p-wjp4-99pq / CVE-2026-63001 (redaxo/core, output_encoding, 2026-09-23): repo over ~150 MB (500 MB per GitHub API). _REDAXO: Stored XSS via Unescaped Media Manager Type Name in `mediaIsInUse()`_
- GHSA-5mmc-8pc9-wggg / CVE-2026-88974 (wp-graphql/wp-graphql, access_control, 2026-09-23): **taken** as wp-graphql-updatepost-publish-bypass. _WPGraphQL: Contributor can publish and modify posts without the required capabilities via _
- GHSA-jr78-w6w5-m8f8 (semanticmediawiki/semanticmediawiki, access_control, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki'a missing authorization in the smwtask API module allows unauthenticate_
- GHSA-9rcc-pmj8-ffhr (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki's Special:FacetedSearch cstate hidden inputs enable reflected XSS (resi_
- GHSA-cx86-7xwp-w9wf / CVE-2026-77616 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki affected by reflected XSS in `Special:Ask` via a forged cursor paginati_
- GHSA-q5fm-9mx6-44f4 / CVE-2026-77610 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki has a query debug output XSS (`DebugFormatter`)_
- GHSA-hw3m-8j5x-94ff / CVE-2026-77609 (semanticmediawiki/semanticmediawiki, untrusted_destination, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki has an open redirect in Special:URIResolver_
- GHSA-7xv3-gf2g-498h / CVE-2026-77607 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki affected by Special:Ask table `sep` parameter reflected XSS_
- GHSA-59xw-qv23-j3rc / CVE-2026-77608 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki has reflected XSS in `Special:SearchByProperty` (`property` and `value`_
- GHSA-3jp5-3h47-28qf / CVE-2026-77606 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki has reflected XSS in Special:Ask plain table headers_
- GHSA-hg8h-557g-q8pp / CVE-2025-61682 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki vulnerable to stored XSS through wikitext via improper use of non-reser_
- GHSA-g4c3-4g96-6g4m / CVE-2026-45140 (chamilo/chamilo-lms, injection/path, 2026-09-17): repo over ~150 MB (1625 MB per GitHub API). _Chamilo LMS CStudio upload flow allows unauthenticated remote code execution_
- GHSA-r94f-hx44-8jqf / CVE-2026-72819 (getgrav/grav, injection, 2026-09-17): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) (only the attacker's `shell.php`). _Grav CMS vulnerable to remote code execution via .zip file upload_
- GHSA-xhfv-7758-r9hx / CVE-2026-75837 (getgrav/grav, access_control, 2026-09-17): the fix changes only a YAML blueprint (`system/blueprints/user/group.yaml`); no PHP line changes, so the fixed pin cannot be told apart in PHP. _Grav: Missing admin.super guard on core group blueprint access field allows admin.users op_
- GHSA-q2j8-x8hf-63ch / CVE-2026-75834 (getgrav/grav, output_encoding, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Single invalid UTF-8 byte disables every rule in Security::detectXss(), bypassing th_
- GHSA-f8wv-xp27-6gq7 / CVE-2026-75827 (getgrav/grav, injection, 2026-09-17): **taken** as grav-blueprint-bare-function-rce. _Grav: Blueprint dynamic-data bare-function branch is denylist-gated and omits error_log, g_
- GHSA-vfmf-q6x9-cw96 / CVE-2026-75828 (getgrav/grav, output_encoding, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: detectXss() misses an event-handler attribute after an unpaired quote in an unquoted_
- GHSA-4v9q-p283-qc2m / CVE-2026-74907 (getgrav/grav, path, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Unauthenticated Path Traversal via Missing Directory-Boundary Check in `plugin-asset_
- GHSA-jq29-c7v8-rg55 / CVE-2026-72695 (getgrav/grav, path, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Path Traversal in MediaUploadTrait::deleteFile() Allows Arbitrary File Deletion_
- GHSA-vjqc-q4mp-2rvf / CVE-2026-79752 (cakephp/cakephp, injection, 2026-09-17): framework query-builder API (`FunctionsBuilder`); vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository. _CakePHP: Multiple methods in FunctionsBuilder vulnerable to SQL injection_
- GHSA-47ch-6w46-6xm7 / CVE-2026-72697 (getgrav/grav, path, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: media_directory() Twig function allows filesystem path traversal and file content di_
- GHSA-6qw9-4vv5-jr97 / CVE-2026-75831 (getgrav/grav, output_encoding, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Stored XSS via Markdown audio/video media <source> URL_
- GHSA-269c-h76q-8cxw / CVE-2026-72832 (getgrav/grav, output_encoding, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Stored XSS via quoted-attribute bypass in detectXss_
- GHSA-w3f4-8pj2-599w / CVE-2026-69089 (getgrav/grav, path, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Path Traversal in ImageMedium::watermark() — arbitrary file disclosure via publicly-_
- GHSA-7pgq-cr25-xvc8 / CVE-2026-69088 (getgrav/grav, injection, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Incomplete callable validation in blueprint dynamic fields allows arbitrary static m_
- GHSA-ffmg-hfvg-jhg9 / CVE-2026-58657 (getgrav/grav, output_encoding, 2026-09-16): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Stored CSS injection via Markdown image resize() bypasses prior media style sanitize_
- GHSA-2c4f-86xc-cr74 / CVE-2026-61453 (getgrav/grav, injection/output_encoding, 2026-09-16): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: XSS Blueprint Validation Bypass via Twig String Concatenation_
- GHSA-2ff2-mx52-q8wp / CVE-2026-49400 (octobercms/october, deserialization, 2026-09-14): no fix diff in the public repository: `modules/backend/traits/SessionMaker.php` last changed in a bulk 'Sync upstream (4.2)' commit (2026-05-20), and there is no v4.2.23 tag. _October CMS: PHP Object Injection via Backend Widget Session Storage_
- GHSA-xv9m-fm3w-8w5x / CVE-2026-46696 (octobercms/october, access_control, 2026-09-14): no fix diff in the public repository (same bulk 'Sync upstream' history, no v4.2.23 tag); advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text). _October CMS: Safe Mode Sandbox Bypass via Session Store and Forwarded Builder Calls_
- GHSA-2cg9-97gq-9mqp / CVE-2026-56825 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopper: Missing authorization on product removal actions in CollectionProducts component_
- GHSA-99h5-jhh7-v3r3 / CVE-2026-56830 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopper: Media sub-form store() still lacks authorization (Incomplete fix for GHSA-h4mp-g9_
- GHSA-g3f9-g5vj-p62f / CVE-2026-56829 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopper: Unauthorized inventory stock manipulation via unlocked variant property in Varian_
- GHSA-j328-xmgp-j4q3 / CVE-2026-56828 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopper: privilege escalation via improper Livewire admin component authorization_
- GHSA-f7h9-qv4x-9x57 / CVE-2026-56826 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopping privilege escalation through missing authorization in Settings components_
- GHSA-243p-f3cv-c5wh / CVE-2026-56827 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopper: Authorization bypass in Filament bulk actions allows browse-only staff to mass-de_
- GHSA-23rh-xw42-fq82 / CVE-2026-55416 (pimcore/pimcore, injection, 2026-09-10): repo over ~150 MB (534 MB per GitHub API). _Pimcore: SQL Injection in Custom Reports via Malicious Report Configuration_
- GHSA-65fr-j4p9-vc33 / CVE-2026-81525 (mongodb/mongo-php-library, injection, 2026-09-08): general-purpose library: the attacker source lies in the calling application, not in the repo. _mongodb: Reject "." and NUL bytes in database and collection names_
- GHSA-rvx4-ffvw-m9q3 / CVE-2026-84361 (composer/composer, injection, 2026-09-08): developer tool / CLI, not a web application. _Composer arbitrary command execution via a malicious package's Perforce source URL_
- GHSA-c7r6-vx3h-w5g2 / CVE-2026-84374 (spartnernl/laravel-excel, path, 2026-09-08): general-purpose library: the attacker source lies in the calling application, not in the repo. _Laravel Excel writes exports outside the configured filesystem disk when given a caller-co_
- GHSA-fxf7-vhh8-7vpq / CVE-2026-77635 (cakephp/cakephp, injection, 2026-09-08): framework query-builder API (`FunctionsBuilder::jsonValue()`); vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository. _CakePHP: FunctionsBuilder::jsonValue() vulerable to SQL injection with PostgresDriver_
- GHSA-pp4x-ccxq-6r33 / CVE-2026-82396 (sulu/sulu, output_encoding, 2026-09-02): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Sulu: Stored XSS via media download inline-disposition override_
- GHSA-65cv-w493-7vhq / CVE-2026-82394 (sulu/sulu, access_control, 2026-09-02): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Sulu: Fix authorization bypass when creating preview links_
- GHSA-h6cx-gjxx-v25c / CVE-2026-82395 (sulu/sulu, access_control, 2026-09-02): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Sulu: Media move/update authorization bypass (IDOR)_
- GHSA-6j4c-mgqr-qv76 / CVE-2026-75592 (getkirby/kirby, path, 2026-09-02): **taken** as kirby-realpath-sibling-traversal. _Kirby: Access to image files outside of the site root via path traversal in the media hand_
- GHSA-fj2p-qj2f-74v5 / CVE-2026-64850 (getgrav/grav, injection, 2026-09-02): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Remote code execution via unrestricted callable in Blueprint::dynamicData()_
- GHSA-g2fm-8hr4-j82h / CVE-2026-81892 (easycorp/easyadminbundle, access_control, 2026-09-02): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) (only the public API `Action::linkToRoute()`; the dispatcher that swaps controllers is described, not named). _EasyAdmin custom-action dispatcher bypasses access_control on other routes_
- GHSA-g3hc-697w-wm82 / CVE-2026-81887 (livewire/livewire, output_encoding, 2026-09-02): DOM-based XSS in Livewire's client-side JavaScript; not a PHP sink. _Livewire DOM-based cross-site scripting during client-side state handling_
- GHSA-f8fg-pg57-v4j8 (thephpleague/commonmark, output_encoding, 2026-09-01): general-purpose library: the attacker source lies in the calling application, not in the repo. _league/commonmark XSS: `on*` event-handler filter in `AttributesExtension` bypassed with a_
- GHSA-cq55-c7wv-pxmq / CVE-2026-62993 (smarty-php/smarty, untrusted_destination, 2026-09-01): general-purpose library: the attacker source lies in the calling application, not in the repo. _Smarty: SSRF via redirect bypass of trusted_uri using {fetch}_
- GHSA-67mx-6wf2-92xp / CVE-2026-71415 (getkirby/kirby, access_control, 2026-08-31): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: File upload permissions are not checked during processing of chunk data_
- GHSA-9vx2-j98c-p72w / CVE-2026-75594 (getkirby/kirby, path, 2026-08-31): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: Access to image files and limited access to JSON files outside of the site root via_
- GHSA-8x3q-jpjh-qh5c / CVE-2026-81889 (studio-42/elfinder, untrusted_destination, 2026-08-31): **taken** as elfinder-fsock-dns-rebinding-ssrf. _elFinder: SSRF protection bypass via DNS rebinding in the `fsock_get_contents()` fallback_
- GHSA-9x44-4gxf-8c25 / CVE-2026-55634 (pimcore/pimcore, injection, 2026-08-28): repo over ~150 MB (534 MB per GitHub API). _Pimcore Vulnerable to Remote Code Execution via DataObject Class-Definition Field Name_
- GHSA-w23p-wrp7-ch38 / CVE-2026-55220 (pimcore/pimcore, deserialization, 2026-08-28): repo over ~150 MB (534 MB per GitHub API). _Pimcore Hotspotimage getDataFromResource() unrestricted Serialize::unserialize over object_
- GHSA-f97c-ph8j-8vff / CVE-2026-55212 (pimcore/pimcore, access_control, 2026-08-28): repo over ~150 MB (534 MB per GitHub API). _Pimcore: Insufficient Permission Check on Class Definition Creation Endpoint Allows Privil_
- GHSA-79cw-hfcc-7mw9 / CVE-2026-55208 (pimcore/pimcore, injection, 2026-08-28): repo over ~150 MB (534 MB per GitHub API). _Pimcore: SQL Injection via Column Name in DateFilter allows authenticated user to extract _
- GHSA-575r-357h-fhch / CVE-2026-55516 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT vulnerable to cross-company asset maintenance re-parenting via API update_
- GHSA-35cr-9hqq-p2mg / CVE-2026-55515 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Cross-company deletion of pending checkout acceptances via unscoped report endpo_
- GHSA-w7qw-5wfv-gwx9 / CVE-2026-55481 (grokability/snipe-it, output_encoding, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has CSS Injection via `header_color` Setting_
- GHSA-8frh-vhgh-64cf / CVE-2026-55479 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has incorrect permission for legacy license checkin API _
- GHSA-crv3-j83j-f3r6 / CVE-2026-55478 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has missing object-level authorization in Kits API_
- GHSA-53jc-27pc-x8r8 / CVE-2026-55476 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT Vulnerable to Unauthorized Asset Request Cancellation via Unguarded cancel_by_adm_
- GHSA-5wx7-xq8j-v4qm / CVE-2026-55475 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT's import created_by can be overwritten_
- GHSA-c6f4-wj38-m3g3 / CVE-2026-55474 (grokability/snipe-it, path, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT vulnerable to directory traversal in displaySig_
- GHSA-8w8c-8mx9-52cw / CVE-2026-55472 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT's API Location Creation Bypasses FMCS Parent-Child Company Boundary Validation_
- GHSA-xr9m-gphc-9p63 / CVE-2026-55469 (grokability/snipe-it, path, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has a path traversal vulnerability via CSV import `image` field_
- GHSA-jhph-5q74-pmfx / CVE-2026-55466 (grokability/snipe-it, output_encoding, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT vulnerable to stored XSS via inline-served attachment_
- GHSA-r52f-r9v5-66xr / CVE-2026-55464 (grokability/snipe-it, output_encoding, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT vulnerable to stored XSS via Markdown custom field _
- GHSA-fc33-6w3q-538h / CVE-2026-55462 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has an authorization bypass on print inventory page_
- GHSA-wg2f-x2c2-c4rp / CVE-2026-55461 (grokability/snipe-it, untrusted_destination, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has an Open Redirect After User Edit_
- GHSA-vgx7-c78r-69w9 / CVE-2026-55460 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has an authorization bypass on bulk editing users_
- GHSA-m4g4-86qc-v8w7 / CVE-2026-55779 (silverstripe/silverstripe-versioned, output_encoding, 2026-08-28): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) (only the `ArchiveAdmin` screen; the fix is in `src/RestoreAction.php`). _silverstripe/versioned has XSS in archive admin restore_
- GHSA-xrjc-c68j-hp7w / CVE-2026-55891 (privatebin/privatebin, output_encoding, 2026-08-28): **taken** as privatebin-jsonld-request-uri-injection. _PrivateBin has reflected JSON injection in backend responses via unescaped REQUEST_URI_
- GHSA-f2xf-7x3g-4272 / CVE-2026-55696 (privatebin/privatebin, output_encoding, 2026-08-28): one case per repository (repository taken as privatebin-jsonld-request-uri-injection). _PrivateBin has stored Cross-Side-Scripting (XSS) vulnerability in attachment download link_
- GHSA-39mm-rwm3-29jp / CVE-2026-54718 (silverstripe/silverstripe-advancedworkflow, injection, 2026-08-27): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text); the fix only bumps a dependency in composer.json (the repair lives in another package). _silverstripe-advancedworkflow vulnerable to remote code execution via advanced workflow em_
- GHSA-g8wr-r2v2-vqc6 / CVE-2026-54721 (silverstripe/silverstripe-userforms, injection, 2026-08-27): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text); the fix only bumps a dependency in composer.json (the repair lives in another package). _silverstripe/userforms vulnerable to remote code execution via userforms email subject_
- GHSA-gvrw-qqp5-jgc5 / CVE-2026-54720 (silverstripe/silverstripe-framework, output_encoding, 2026-08-27): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) ('Insert media from web'). _Silverstripe Framework: Possible XSS attack through media embed_
- GHSA-7w8c-qgxg-m7jx (librenms/librenms, output_encoding, 2026-08-26): repo over ~150 MB (369 MB per GitHub API). _LibreNMS — Stored XSS via SNMP/Syslog Data in Legacy Templates_
- GHSA-crx4-7mmq-j74j / CVE-2026-44701 (devcode-it/openstamanager, output_encoding, 2026-08-26): **taken** as openstamanager-group-name-html-injection. _OpenSTAManager has HTML Injection in modules/utenti/edit.php_
- GHSA-mf8r-wm2w-f8c5 (thorsten/phpmyfaq, access_control, 2026-08-25): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ public FAQ APIs expose inactive FAQ content_
- GHSA-88g4-74f3-63x9 (thorsten/phpmyfaq, path, 2026-08-25): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ has Potential Authenticated Path Traversal in PDF Export_
- GHSA-896w-cw95-xq7w (, path, 2026-08-25): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Path Traversal in MediaUploadTrait::deleteFile() Allows Arbitrar_
- GHSA-rj4c-4q9x-543x (, path, 2026-08-25): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: media_directory() Twig function allows filesystem path traversal_
- GHSA-5h77-88j3-r659 / CVE-2026-63135 (yourls/yourls, output_encoding, 2026-08-21): repo over ~150 MB (489 MB per GitHub API). _YOURLS has stored XSS in referrer statistics chart via crafted Referer header_
- GHSA-hrwp-4hh9-c8r8 / CVE-2026-59989 (phalcon/cphalcon, injection, 2026-08-21): repo over ~150 MB (273 MB per GitHub API). _Phalcon Volt compiler `join` filter compile-time PHP code injection (SSTI leads to RCE)_
- GHSA-8hgv-xc77-jmcr (getgrav/grav, output_encoding, 2026-08-21): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Page editors can inject arbitrary script into rendered pages via the Twig sandbox's _
- GHSA-hq84-x37p-j6q5 (wintercms/winter, output_encoding, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Reflected XSS through the search query parameter in the backend Table widget_
- GHSA-5cwr-5jxg-pcf6 (wintercms/winter, output_encoding, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Stored XSS through cached Brand Settings and Editor Settings custom styles_
- GHSA-fm29-4mq3-phg6 (wintercms/winter, access_control, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: ImportExportController AJAX handlers bypass granular import/export permission gate_
- GHSA-mpmw-f6h6-3g26 (wintercms/winter, access_control, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: My Account preview exposes another backend user's profile by record ID_
- GHSA-7mpf-4465-7fc2 (wintercms/winter, output_encoding, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Stored XSS through Backend List widget image columns_
- GHSA-2223-f22x-24cq (wintercms/winter, path, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Local File Inclusion through =include directives in JavaScript asset compilation_
- GHSA-58fp-mcx6-7qf9 / CVE-2026-63179 (wintercms/winter, path, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Local File Inclusion through @import directives in LESS compilation of backend cus_
- GHSA-42vx-43vc-x6pr / CVE-2026-57570 (laravel-backpack/crud, access_control, 2026-08-20): one case per repository (repository taken as backpack-stats-host-header-cmdi). _Laravel Backpack CRUD: HasMany/MorphMany relation fields allow cross-tenant record re-pare_
- GHSA-3277-h8g9-qj5f / CVE-2026-54256 (wintercms/winter, access_control, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Authenticated IDOR in backend FileUpload widget allows cross-user access to attach_
- GHSA-mrc5-3mm3-45c5 / CVE-2026-54182 (laravel-backpack/crud, injection/output_encoding, 2026-08-20): **taken** as backpack-stats-host-header-cmdi. _Laravel Backpack CRUD: OS command injection in Stats::makeCurlRequest via attacker-control_
- GHSA-mmg4-322v-6jvc / CVE-2026-54181 (laravel-backpack/crud, output_encoding, 2026-08-20): one case per repository (repository taken as backpack-stats-host-header-cmdi). _Laravel Backpack CRUD: Stored XSS in the color column — the `@if($column['escaped'])` bran_
- GHSA-vgmv-8xjc-6rch / CVE-2026-54180 (laravel-backpack/crud, access_control, 2026-08-20): one case per repository (repository taken as backpack-stats-host-header-cmdi). _Laravel Backpack CRUD: CRUD panel query scopes are not enforced on Update, Delete, and Reo_
- GHSA-8hw4-7qjr-3wxg / CVE-2026-54179 (laravel-backpack/crud, output_encoding, 2026-08-20): one case per repository (repository taken as backpack-stats-host-header-cmdi). _Laravel Backpack CRUD: SingleBase64Image accepts any base64 payload behind a `data:image` _
- GHSA-8xjm-wqrp-2f25 / CVE-2026-54178 (laravel-backpack/crud, access_control/path, 2026-08-20): one case per repository (repository taken as backpack-stats-host-header-cmdi). _Laravel Backpack CRUD: Arbitrary file deletion via attacker-controlled clear_<attr>[] in H_
- GHSA-c8qc-wf67-342w / CVE-2026-61807 (grokability/snipe-it, output_encoding, 2026-08-19): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Stored DOM XSS via table selected-count IDs_
- GHSA-r9r3-g9fp-3q4q / CVE-2026-55703 (grokability/snipe-it, access_control, 2026-08-19): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Maintenance Record Disclosure via Missing Authorization on GET_
- GHSA-3hgv-jr5j-cg9x / CVE-2026-55694 (grokability/snipe-it, access_control, 2026-08-19): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Chained Information Disclosure and IDOR Leads to Full EULA File Takeover_
- GHSA-c6w2-j4wq-mvwg / CVE-2026-55643 (grokability/snipe-it, access_control, 2026-08-19): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Tenant Isolation Bypass in FMCS Floater Mode_
- GHSA-7gww-x7fh-jf9j (librenms/librenms, output_encoding/untrusted_destination, 2026-08-18): repo over ~150 MB (369 MB per GitHub API). _LibreNMS: SSRF-driven stored XSS via Oxidized API response fields in device showconfig pag_
- GHSA-7cj5-v4pp-v632 (librenms/librenms, output_encoding, 2026-08-18): repo over ~150 MB (369 MB per GitHub API). _LibreNMS: Stored XSS via graph_descr admin config settings echoed without escaping to all _
- GHSA-jf24-8g2h-2wg7 (librenms/librenms, injection, 2026-08-18): repo over ~150 MB (369 MB per GitHub API). _LibreNMS Vulnerable to Remote Code Execution via AboutController_
- GHSA-w27m-rmmf-g5w4 / CVE-2026-54348 (froxlor/froxlor, injection, 2026-08-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Froxlor: Second-Order SQL Injection via `Admins.add` `ipaddress` Parameter Allows Full Dat_
- GHSA-43gm-9rr3-cx7g / CVE-2026-54347 (froxlor/froxlor, output_encoding, 2026-08-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Froxlor: Stored XSS in DNS TXT Record Content Allows Customer-to-Admin Account Takeover_
- GHSA-pcw8-m77r-2528 / CVE-2026-54133 (jmespath/jmespath.php, injection/output_encoding, 2026-08-18): general-purpose library: the attacker source lies in the calling application, not in the repo. _jmespath.php has CompilerRuntime code injection via unescaped function names_
- GHSA-c9fv-cgmm-2wg7 / CVE-2026-55182 (librenms/librenms, injection, 2026-08-18): repo over ~150 MB (369 MB per GitHub API). _LibreNMS Vulnerable to Remote Code Execution by Signal Alert Transportation module_
- GHSA-q8cg-5m48-5c25 (, output_encoding, 2026-08-18): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Stored XSS via Markdown audio/video media <source> URL_
- GHSA-9pr6-8r9w-wvmj (, output_encoding, 2026-08-18): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: detectXss() misses an event-handler attribute after an unpaired _
- GHSA-993v-76jg-67xr (, output_encoding, 2026-08-18): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Single invalid UTF-8 byte disables every rule in Security::detec_
- GHSA-fwwx-3362-3947 (, injection, 2026-08-18): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Blueprint dynamic-data bare-function branch is denylist-gated an_
- GHSA-2rhw-8953-48q3 (, path, 2026-08-18): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Unauthenticated Path Traversal via Missing Directory-Boundary Ch_
- GHSA-59xm-4m8c-g3xj / CVE-2026-55224 (mineadmin/mineadmin, path, 2026-08-18): **taken** as mineadmin-appstore-identifier-traversal. _MineAdmin Vulnerable to Path Traversal via Unsanitized identifier in Plugin Install/Uninst_
- GHSA-cgvr-f65r-pjv3 (, output_encoding, 2026-08-14): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Stored XSS via quoted-attribute bypass in detectXss_
- GHSA-wvxr-6v52-gfmh (, injection, 2026-08-14): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Remote code execution via .zip file upload in Grav CMS_
- GHSA-2mhj-fhvg-v428 / CVE-2026-55072 (pimcore/pimcore, injection, 2026-08-13): repo over ~150 MB (534 MB per GitHub API). _Pimcore: ClassDefinition UID regex missing end anchor allows SQL injection via Block.php u_
- GHSA-6pvm-2vjj-rx4w / CVE-2026-47132 (thorsten/phpmyfaq, injection, 2026-08-12): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ: SQL LIKE Wildcard Injection in Chat User Search Allows Authenticated User Enumer_
- GHSA-jmqm-f8q4-v7wx / CVE-2026-45694 (librenms/librenms, output_encoding, 2026-08-12): repo over ~150 MB (369 MB per GitHub API). _LibreNMS: Reflected XSS via Proxmox instance/vmid GET parameters injected into document.ti_
- GHSA-j5jq-cr68-v2xx / CVE-2026-35445 (wintercms/winter, access_control, 2026-08-12): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Authenticated backend users can bypass Users controller permission checks_
- GHSA-m7jc-g4rc-jmvh / CVE-2026-32593 (wintercms/winter, injection, 2026-08-12): **taken** as winter-filter-numberrange-sqli. _Winter: SQL Injection in Backend Filter Widget numberrange Scope via numbersFromAjax_
- GHSA-vgp4-2fc4-qff2 / CVE-2026-32258 (wintercms/winter, output_encoding, 2026-08-12): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Stored XSS through Editor Settings custom styles_
- GHSA-v7cf-8gh9-gxmj / CVE-2026-32257 (wintercms/winter, output_encoding, 2026-08-12): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Stored XSS through Brand Settings custom styles_
- GHSA-4hc4-qjfx-wjf3 (, access_control, 2026-08-11): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: Authorization bypass: view-only Categories user can modify _
- GHSA-9w6w-8x3c-hfqp (, path, 2026-08-11): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: Incorrect path validation could potentially lead to path tr_
- GHSA-hhmc-q9hp-r662 / CVE-2026-63222 (codeigniter4/codeigniter4, path, 2026-08-07): repo over ~150 MB (387 MB per GitHub API). _CodeIgniter: Path traversal in UploadedFile::move() when using client-provided filenames_
- GHSA-c9w5-rwh3-7pm9 / CVE-2026-63221 (codeigniter4/codeigniter4, injection, 2026-08-07): repo over ~150 MB (387 MB per GitHub API). _CodeIgniter: SQL injection in Query Builder deleteBatch() when used with where() condition_
- GHSA-rjhh-76wf-8xmw / CVE-2026-62996 (smarty-php/smarty, path, 2026-08-07): general-purpose library: the attacker source lies in the calling application, not in the repo. _Smarty Security stream restriction bypass through stream: resource_
- GHSA-f6wf-28g6-769x / CVE-2026-62992 (smarty-php/smarty, path, 2026-08-07): general-purpose library: the attacker source lies in the calling application, not in the repo. _Smarty: Symlink path traversal out of trusted directories_
- GHSA-xxpx-f366-4xpq / CVE-2026-72785 (craftcms/cms, access_control, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Authorization bypass: view-only Categories user can modify category structure v_
- GHSA-rvmm-v933-jgxq / CVE-2026-14794 (craftcms/cms, access_control, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Missing authorization check allows non-admin control panel users access to user_
- GHSA-7hxc-f267-h5q7 / CVE-2026-72783 (craftcms/cms, path, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Incorrect path validation could potentially lead to path traversal_
- GHSA-2rp4-x2j7-qmcc (craftcms/cms, output_encoding, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Stored XSS in the control panel via unescaped draft name_
- GHSA-hmqg-cxww-wqhq / CVE-2026-67434 (phpcsstandards/php_codesniffer, injection, 2026-08-06): repo over ~150 MB (210 MB per GitHub API). _PHP_CodeSniffer gitblame report command injection via crafted filename_
- GHSA-p8x7-9vfw-p7vc (craftcms/cms, access_control, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Arbitrary user password reset leading to administrator account takeover_
- GHSA-9p7c-v5x3-rfx8 / CVE-2026-14793 (craftcms/cms, access_control, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Missing authorization check allows non-admin control panel users to reorder Glo_
- GHSA-29pj-957v-52mc / CVE-2026-71478 (thephpleague/commonmark, output_encoding, 2026-08-06): general-purpose library: the attacker source lies in the calling application, not in the repo. _league/commonmark: AttributesExtension href/src unsafe-link filter bypass via embedded con_
- GHSA-grm4-wm43-9jh5 / CVE-2026-55825 (contao/contao, path, 2026-08-06): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Contao: Possible path traversal in job download URIs_
- GHSA-w3cp-g2pf-65wh / CVE-2026-54717 (silverstripe/silverstripe-cms, output_encoding, 2026-08-06): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) (breadcrumbs in page list view). _Silverstripe: XSS in breadcrumbs in page list view_
- GHSA-vx89-p3j7-8xqc / CVE-2026-71435 (statamic/cms, output_encoding, 2026-08-06): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) ('automagic' form notification email). _Statamic: Stored Cross-Site Scripting in Automagic Form Notification Email Template_
- GHSA-qh8c-7588-qfrv / CVE-2026-64662 (statamic/cms, access_control, 2026-08-06): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text). _Statamic: Missing authorization on navigation endpoint allows disclosure of restricted ent_
- GHSA-225x-3jhx-wh4q / CVE-2026-64664 (statamic/cms, access_control, 2026-08-06): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text). _Statamic: Missing authorization on Control Panel endpoint allows disclosure of user existe_
- GHSA-v5mv-p594-2x33 / CVE-2026-69246 (guzzle/guzzle, untrusted_destination, 2026-08-03): general-purpose library: the attacker source lies in the calling application, not in the repo. _Guzzle: Noncanonical host can bypass host-based checks_
- GHSA-vj8j-973f-r65j (, injection, 2026-08-03): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Incomplete callable validation in blueprint dynamic fields allow_
- GHSA-mmwh-j75q-gxp8 (, path, 2026-08-03): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Path Traversal in ImageMedium::watermark() — arbitrary file disc_
- GHSA-x83g-979r-f5fh / CVE-2026-68501 (sylius/mollieplugin, access_control, 2026-07-31): **taken** as sylius-mollie-thankyou-order-idor. _Sylius Mollie Plugin has unauthenticated IDOR that leaks order token and customer PII_
- GHSA-rc52-c4hv-w89p / CVE-2026-68500 (sylius/mollieplugin, access_control, 2026-07-31): one case per repository (repository taken as sylius-mollie-thankyou-order-idor). _Sylius Mollie Plugin vulnerable to payment status forgery via the payment webhook_
- GHSA-996f-334j-67g7 / CVE-2026-52838 (alextselegidis/easyappointments, output_encoding, 2026-07-29): one case per repository (repository taken as easyappointments-caldav-ssrf). _Easy!Appointments disable_booking_message rendered as raw HTML on public booking page — St_
- GHSA-8hm4-r66f-29wr / CVE-2026-52841 (alextselegidis/easyappointments, access_control, 2026-07-29): one case per repository (repository taken as easyappointments-caldav-ssrf). _Easy!Appointments: Authorization bypass in Google OAuth provider binding lets any backend _
- GHSA-xgr6-pqjv-3pf8 / CVE-2026-52837 (alextselegidis/easyappointments, access_control, 2026-07-29): one case per repository (repository taken as easyappointments-caldav-ssrf). _Easy!Appointments has unauthenticated customer PII disclosure on booking reschedule page_
- GHSA-w8xc-8g92-v77h / CVE-2026-52839 (alextselegidis/easyappointments, access_control, 2026-07-29): one case per repository (repository taken as easyappointments-caldav-ssrf). _Easy!Appointments appointments/store and appointments/update allow cross-provider appointm_
- GHSA-pm5p-7w5h-jm5q / CVE-2026-52840 (alextselegidis/easyappointments, untrusted_destination, 2026-07-29): **taken** as easyappointments-caldav-ssrf. _Easy!Appointments has server-side request forgery in CalDAV connection test that exposes t_
- GHSA-3735-5339-xfwx / CVE-2026-54588 (poweradmin/poweradmin, untrusted_destination, 2026-07-28): repo over ~150 MB (508 MB per GitHub API). _Poweradmin has Host Header Injection in OIDC redirect_uri, SAML ACS/SLO URL, and Logout Re_
- GHSA-3pwp-g2mj-5p3v / CVE-2026-45293 (wordpress/wordpress-coding-standards, injection, 2026-07-28): developer tool / CLI, not a web application. _WordPress Coding Standards (WordPressCS) contains an arbitrary code execution vulnerabilit_
- GHSA-rm67-g9ch-vxff (poweradmin/poweradmin, access_control, 2026-07-24): repo over ~150 MB (508 MB per GitHub API). _Poweradmin: Broken access control (IDOR): any zone owner can modify DNS records in zones t_
- GHSA-h4hf-v6w5-897x (poweradmin/poweradmin, access_control, 2026-07-24): repo over ~150 MB (508 MB per GitHub API). _Poweradmin: API user-update endpoint leads to a non-admin reset any user's password and ta_
- GHSA-g3hq-hphg-8fhh (pheditor/pheditor, injection, 2026-07-24): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Pheditor: Terminal command-allowlist bypass via argument injection leads to RCE — survivin_
- GHSA-6hq5-7373-42rg / CVE-2026-59931 (phpoffice/phpspreadsheet, untrusted_destination, 2026-07-23): general-purpose library: the attacker source lies in the calling application, not in the repo. _PHPSpreadsheet: SSRF bypass via HTTP redirect in WEBSERVICE() domain whitelist_
- GHSA-cx96-42px-69fm / CVE-2026-56722 (dompdf/dompdf, path, 2026-07-22): general-purpose library: the attacker source lies in the calling application, not in the repo. _Dompdf: Local file read due to improper file path validation in SVG images encoded as data_
- GHSA-wvh6-f5jh-8gw4 / CVE-2026-55554 (dompdf/dompdf, path, 2026-07-22): general-purpose library: the attacker source lies in the calling application, not in the repo. _Dompdf: Chroot Validation Bypass_
- GHSA-gjfg-22fp-rrxx / CVE-2026-59946 (composer/composer, path, 2026-07-20): developer tool / CLI, not a web application. _Composer: Path traversal in package bin field lets dependencies chmod arbitrary host files_
- GHSA-499r-g7pc-vmp9 / CVE-2026-59948 (composer/composer, path, 2026-07-20): developer tool / CLI, not a web application. _Composer: Arbitrary file write outside vendor via malicious transitive package name_
- GHSA-cvpc-hccg-wmw4 (verbb/formie, access_control, 2026-07-17): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Formie: Missing authorization in administrative settings allows low-privileged CP users to_
- GHSA-wg4w-wr5q-6vjc / CVE-2026-55578 (pheditor/pheditor, injection, 2026-07-16): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Pheditor: Incomplete command sanitization in terminal feature allows RCE via pipe operator_
- GHSA-9643-6xjp-vx57 / CVE-2026-54540 (pheditor/pheditor, injection, 2026-07-16): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Pheditor has an authenticated terminal command whitelist bypass_
- GHSA-373m-p57p-8665 (, output_encoding, 2026-07-15): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: XSS Blueprint Validation Bypass via Twig String Concatenation_
- GHSA-h2wf-967x-gxvw / CVE-2026-62944 (mantisbt/mantisbt, output_encoding, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: Stored XSS in print_all_bug_page_word.php_
- GHSA-3v2j-6fw9-f57c / CVE-2026-52882 (mantisbt/mantisbt, access_control, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: REST and SOAP API Issue Update Accepts Unreleased Product Versions From Updaters_
- GHSA-vcrw-4xvv-jh49 / CVE-2026-52881 (mantisbt/mantisbt, output_encoding, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: Reflected XSS in admin/install.php via unescaped printf _
- GHSA-77x8-3v3h-hrhv / CVE-2026-52847 (mantisbt/mantisbt, output_encoding, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: Reflected XSS in admin/install.php_
- GHSA-rjg7-r26h-cfp2 / CVE-2026-54494 (koel/koel, untrusted_destination, 2026-07-15): **taken** as koel-podcast-enclosure-ipv6-ssrf. _Koel: Full-read SSRF via podcast enclosure URL: isPublicHost() filter_var guard does not r_
- GHSA-jr4p-4xjh-fwvw / CVE-2026-50552 (koel/koel, untrusted_destination, 2026-07-15): one case per repository (repository taken as koel-podcast-enclosure-ipv6-ssrf). _Koel: Server-Side Request Forgery (SSRF) in radio station creation due to missing validati_
- GHSA-6qvr-wjmv-v8mm / CVE-2026-54491 (koel/koel, untrusted_destination, 2026-07-15): one case per repository (repository taken as koel-podcast-enclosure-ipv6-ssrf). _Koel: Incomplete fix for CVE-2026-47260 — systemic SSRF in podcast & radio fetch paths_
- GHSA-8q6q-m837-fv64 (koel/koel, untrusted_destination, 2026-07-15): one case per repository (repository taken as koel-podcast-enclosure-ipv6-ssrf). _ Koel has SSRF through Authenticated Subsonic podcast feed URLs_
- GHSA-6p96-cfg5-4vhp / CVE-2026-54493 (koel/koel, untrusted_destination, 2026-07-15): one case per repository (repository taken as koel-podcast-enclosure-ipv6-ssrf). _Koel: Authenticated Full-Read SSRF via Subsonic Internet Radio Stations_
- GHSA-w79m-f3jx-779v / CVE-2026-54492 (koel/koel, untrusted_destination, 2026-07-15): one case per repository (repository taken as koel-podcast-enclosure-ipv6-ssrf). _Koel: Authenticated Blind SSRF via Subsonic Podcast Channel Creation_
- GHSA-m7ph-9558-mrx3 / CVE-2026-49280 (mantisbt/mantisbt, access_control, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: REST API unauthorized Issue status change_
- GHSA-v84x-qvhg-f36r / CVE-2026-49273 (mantisbt/mantisbt, injection, 2026-07-15): **taken** as mantisbt-tokenizer-eval-rce. _MantisBT: Remote Code Execution via eval() Class Hoisting in adm_config_set.php_
- GHSA-c2xg-qjqw-2v98 / CVE-2026-47156 (mantisbt/mantisbt, access_control, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: SOAP API Authentication Bypass with Privilege Escalation to Administrator_
- GHSA-mw6p-33vw-46cc / CVE-2026-47142 (mantisbt/mantisbt, injection, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: SQL Injection via history_order Configuration Value_
- GHSA-hgjx-r89m-m7v4 (neorazorx/facturascripts, path, 2026-07-14): **taken** as facturascripts-uploadedfile-move-traversal. _FacturaScripts: Path traversal in UploadedFile::move() via getClientOriginalName() — arbit_
- GHSA-8559-gwj3-q37r / CVE-2026-54087 (easycorp/easyadminbundle, output_encoding, 2026-07-14): vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository (uploads must be stored under the public web root); the flaw is a missing default upload restriction, with no rendering sink in the repository. _EasyAdmin: Stored Cross-Site Scripting (XSS) via uploaded files served inline in FileField_
- GHSA-3x7p-v8hj-xh5m / CVE-2026-45710 (neorazorx/facturascripts, output_encoding, 2026-07-14): one case per repository (repository taken as facturascripts-uploadedfile-move-traversal). _FacturaScripts: Stored XSS in WidgetVariante and WidgetSubcuenta modal lists via HTML-attr_
- GHSA-cv65-7cg8-r623 / CVE-2026-45693 (neorazorx/facturascripts, path, 2026-07-14): one case per repository (repository taken as facturascripts-uploadedfile-move-traversal). _FacturaScripts: Unauthenticated Path Traversal in Static File Controllers Reads Private My_
- GHSA-5qmh-x653-g8qj / CVE-2026-45262 (neorazorx/facturascripts, injection/untrusted_destination, 2026-07-14): no fixed version stated (`<= 2026.1`, patched: none) and no identifiable fix: the only candidate (`14ee211c`, 'centralizar parseFieldFilter con validación anti-SQLi') was reverted the same day (`7392b489`). _FacturaScripts: Authenticated SQL injection in the FacturaScripts REST API filter paramete_
- GHSA-rw46-qg69-vg6h / CVE-2026-52828 (kimai/kimai, access_control, 2026-07-14): no fix commit referenced; per the v2 research, Kimai's fixes for this release shipped in the single 'Release 2.58' commit (`31a8f887`, 120 files across 2.57.0...2.58.0), mixed with unrelated changes. _Kimai: ExportTemplate CRUD Missing Authorization Check Allows Unauthorized TEAMLEAD Access_
- GHSA-2xgg-2x8h-8xw4 / CVE-2026-52826 (kimai/kimai, access_control, 2026-07-14): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai: Improper Authorization in Project, Customer, and Activity Rate Edit Endpoints Allow_
- GHSA-xv4r-4885-gwpg / CVE-2026-52825 (kimai/kimai, access_control, 2026-07-14): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai has Improper Authorization in Team Member and Team Activity Assignment APIs Which Al_
- GHSA-c6w6-57jj-62vh / CVE-2026-52822 (kimai/kimai, access_control, 2026-07-14): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Improper Authorization in Kimai Timesheet Restart and Duplicate Allows New Timesheets Afte_
- GHSA-3q6q-26vg-v97x / CVE-2026-52821 (kimai/kimai, access_control, 2026-07-14): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai: Improper Authorization Through Activity Creation with Preset Project Allows Creatio_
- GHSA-xv8g-76mx-2rxc / CVE-2026-49970 (plank/laravel-mediable, path, 2026-07-13): general-purpose library: the attacker source lies in the calling application, not in the repo. _Laravel-Mediable: path traversal vulnerability in the File::sanitizePath()_
- GHSA-4chg-4752-w88r / CVE-2026-55372 (nukeviet/nukeviet, untrusted_destination, 2026-07-13): repo over ~150 MB (190 MB per GitHub API). _NukeViet: Pre-authentication SSRF via X-Forwarded-Host_
- GHSA-c9xg-64p9-f2jj / CVE-2026-54065 (nukeviet/nukeviet, path, 2026-07-13): repo over ~150 MB (190 MB per GitHub API). _NukeViet: Path Traversal to Arbitrary File Deletion in Edit Comment Function_
- GHSA-465g-4q99-5x86 / CVE-2026-54064 (nukeviet/nukeviet, output_encoding, 2026-07-13): repo over ~150 MB (190 MB per GitHub API). _NukeViet: Multiple Anti-XSS Filter Bypasses Leading to Stored XSS in News Module_
- GHSA-w2w5-w2pw-r929 / CVE-2026-49259 (nukeviet/nukeviet, output_encoding, 2026-07-13): repo over ~150 MB (190 MB per GitHub API). _NukeViet: Improper Neutralization of Input During Web Page Generation ('Cross-site Scripti_
- GHSA-mxpf-qgg6-v3ff / CVE-2026-48118 (nukeviet/nukeviet, output_encoding, 2026-07-13): repo over ~150 MB (190 MB per GitHub API). _NukeViet: Unauthenticated Reflected XSS in Comment Module_
- GHSA-vrr2-g9gh-c3jc / CVE-2026-52820 (kimai/kimai, access_control, 2026-07-13): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai: Timesheet PATCH/POST allows assigning to project outside user's team via query_buil_
- GHSA-4m8q-55qv-9pwp / CVE-2026-52819 (kimai/kimai, access_control, 2026-07-13): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai: Teamlead authorization bypass in GET /api/timesheets allows reading other users' ti_
- GHSA-qv4m-m73m-8hj7 (notrinos/notrinoserp, path, 2026-07-10): no fixed version stated and no identifiable fix commit (the history of `hrm/manage/employees.php` shows no matching change); the weakness is also mainly unrestricted upload (CWE-434). _NotrinosERP: Authenticated arbitrary file upload leads to remote code execution via HRM em_
- GHSA-pj8j-p4g4-4vw8 / CVE-2026-49865 (kimai/kimai, untrusted_destination, 2026-07-10): no fix commit referenced; the fix is in Kimai's 'Release 2.58' squash (see above). _Kimai has Server-Side Request Forgery in Invoice PDF Rendering via Markdown Image URLs_
- GHSA-pjhx-3c3w-9v23 / CVE-2026-49858 (api-platform/core, access_control, 2026-07-10): a per-process cache-key bug in the serializer; exposure depends on application-declared `#[ApiProperty(security: ...)]` predicates, and no attacker-controlled value reaches the flaw inside the repository. _API Platform Core vulnerable to cross-user attribute leak in JSON:API and HAL item normali_
- GHSA-mr9r-h354-966r / CVE-2026-53639 (sylius/sylius, access_control, 2026-07-09): repo over ~150 MB (355 MB per GitHub API). _Sylius: IDOR on Shop Payment Request API endpoints_
- GHSA-6955-hrm5-c4qp / CVE-2026-53638 (sylius/sylius, access_control, 2026-07-09): repo over ~150 MB (355 MB per GitHub API). _Sylius: Channel-based payment method restriction bypass on shop account orders API endpoin_
- GHSA-px5m-h76g-p7p8 / CVE-2026-52778 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki has Unsafe eval() in its Formula Calculato, Leading to Remote Code Execution & Den_
- GHSA-9369-69wj-7m2f / CVE-2026-52777 (yeswiki/yeswiki, deserialization, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki Vulnerable to Authenticated PHP Object Injection in BazarImportAction via unserial_
- GHSA-4pf7-cc4r-g63h / CVE-2026-52775 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki has Authenticated SQL Injection via ReactionManager _
- GHSA-r5xw-gcgw-hwp5 / CVE-2026-52774 (yeswiki/yeswiki, output_encoding, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki Vulnerable to Reflected XSS via Unescaped `id` Parameter in Bazar Widget HTML Attr_
- GHSA-35f3-pg38-486f / CVE-2026-52773 (yeswiki/yeswiki, output_encoding, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki Vulnerable to Reflected XSS via Unescaped Archived-Revision `time` Parameter in `h_
- GHSA-xc7j-3g8q-9vh4 / CVE-2026-52772 (yeswiki/yeswiki, output_encoding, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki has stored XSS in Bazar form-field templates via unescaped field.label / field.hin_
- GHSA-8f2v-2qhj-gfwg / CVE-2026-52771 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki: Second-Order SQL Injection in Page Delete API via Unescaped Page Tag (`ApiControl_
- GHSA-qg78-vmvc-fhjw / CVE-2026-52770 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki: SQL Injection possible through public Bazar entry-listing APIs via numeric `query_
- GHSA-vw42-752g-5mrp / CVE-2026-52769 (yeswiki/yeswiki, untrusted_destination, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki has Unauthenticated Server-Side Request Forgery via ActivityPub `Signature.keyId`_
- GHSA-6x7x-gcmf-7r8x / CVE-2026-52766 (yeswiki/yeswiki, access_control, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki vulnerable to unauthenticated arbitrary page deletion via `{{erasespamedcomments}}_
- GHSA-89v6-j5x6-cmj3 / CVE-2026-52763 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki: SQL injection via the `recentchanges` action `period` argument leads to arbitrary_
- GHSA-65p8-9433-jpcp / CVE-2026-52762 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki: Authenticated (Admin) Server-Side Template Injection to Remote Code Execution via_
- GHSA-w9mx-xmg4-gc4r / CVE-2026-53932 (stefanzweifel/laravel-backup-restore, injection, 2026-07-09): developer tool / CLI, not a web application. _laravel-backup-restore has an OS Command Injection during database restore_
- GHSA-86vw-x4ww-x467 / CVE-2026-56382 (craftcms/cms, injection, 2026-07-09): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: RCE via missing cleanseConfig in FieldsController::actionRenderCardPreview_
- GHSA-c43v-4cr8-6mvp / CVE-2026-56394 (craftcms/cms, path, 2026-07-09): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS has authenticated path traversal in `assets/icon`, allowing local `.svg` file re_
- GHSA-4wj4-79rr-pvff (, output_encoding, 2026-07-08): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Stored CSS injection via Markdown image resize() bypasses prior _
- GHSA-vmwx-m75v-qvch / CVE-2026-53634 (code16/sharp, access_control, 2026-07-08): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) ('create and store endpoints of the Quick Creation Command feature'). _Sharp Missing Authorization Check in Quick Creation Command Endpoints_
- GHSA-c8m7-r2jv-rw63 / CVE-2026-45016 (egroupware/egroupware, path, 2026-07-07): repo over ~150 MB (188 MB per GitHub API). _EGroupware Vulnerable to Local File Inclusion via file:// URI in Mail Compose_
- GHSA-8737-2x9g-xjj7 / CVE-2026-40187 (egroupware/egroupware, injection, 2026-07-07): repo over ~150 MB (188 MB per GitHub API). _EGroupware has Authenticated RCE via Malicious eTemplate Upload_
- GHSA-h9qx-v5xp-ph8p / CVE-2026-27823 (egroupware/egroupware, access_control/injection/path, 2026-07-07): repo over ~150 MB (188 MB per GitHub API). _EGroupware has a Remote Code Execution Vulnerability_
- GHSA-f74w-488g-8x5r / CVE-2026-55794 (craftcms/cms, injection, 2026-07-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Potential authenticated Remote Code Execution via referrer redirect_
- GHSA-xrqc-p465-2xvg / CVE-2026-55793 (craftcms/cms, output_encoding, 2026-07-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Stored XSS via Structure entry title in table view_
- GHSA-24x4-j6x9-rfw5 / CVE-2026-55790 (craftcms/cms, output_encoding, 2026-07-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: DOM XSS via GitHub issue title in CraftSupport widget_
- GHSA-x76w-8c62-48mg / CVE-2026-56384 (craftcms/cms, access_control, 2026-07-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Authenticated "assets/preview-thumb" discloses signed fallback transform previe_
- GHSA-565m-g33j-jq96 / CVE-2026-52889 (verbb/formie, injection, 2026-07-06): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Formie Hidden field defaults vulnerable to Server-Side Template Injection_
- GHSA-rqjw-r5g4-x8qm (, access_control, 2026-07-06): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: Missing authorization check allows non-admin control panel _
- GHSA-3w32-23wj-rxg3 / CVE-2026-50282 (craftcms/cms, access_control, 2026-07-02): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS Vulnerable to Unauthorized Deletion of Destination Folders During Forced Moves_
- GHSA-5hvg-w58j-545m / CVE-2026-9811 (mautic/mautic, output_encoding, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic has Stored Cross-Site Scripting (XSS) in Project Option Selector_
- GHSA-7h65-whp7-rgqf / CVE-2026-9809 (mautic/mautic, output_encoding, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic has Stored Cross-Site Scripting (XSS) in Projects Component_
- GHSA-2jrw-c95w-h43g / CVE-2026-9808 (mautic/mautic, access_control, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic has an Authorization Bypass in API v2 Endpoints_
- GHSA-6r9h-4h75-7q4x / CVE-2026-9559 (mautic/mautic, path, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic vulnerable to Path Traversal via Campaign Import_
- GHSA-9fx4-7cmj-47vg / CVE-2026-9558 (mautic/mautic, injection, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic has Server-Side Template Injection (SSTI) in Theme Templates_
- GHSA-jmv8-8j9j-rcpc / CVE-2026-9557 (mautic/mautic, untrusted_destination, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic Focus component Vulnerable to SSRF_
- GHSA-fcmw-wx57-9p75 / CVE-2026-4776 (mautic/mautic, injection, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic has SQL Injection in API Contact Filtering_
- GHSA-q4rm-m6xh-5pv7 (froxlor/froxlor, access_control, 2026-07-02): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Froxlor customer can create MySQL databases on disallowed servers via Mysqls.add API_
- GHSA-7h62-6v23-v8fm / CVE-2026-50284 (craftcms/cms, access_control, 2026-07-02): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Missing peer-permission check in `AssetsController::actionDeleteFolder` allows _
- GHSA-qh45-9g5p-m2v4 / CVE-2026-50283 (craftcms/cms, access_control, 2026-07-02): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Unauthorized Deletion of Source Assets During File Replacement_
- GHSA-43cq-c2gq-pfpw / CVE-2026-50280 (craftcms/cms, access_control, 2026-07-02): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Authorization bypass in `entries/move-to-section` via missing target-section sa_
- GHSA-qq2c-2q8j-jh27 / CVE-2026-50279 (craftcms/cms, access_control, 2026-07-02): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Authorship spoofing in `entries/save-entry` via pre-check/post-mutation authori_
- GHSA-4h7g-5542-v3fc / CVE-2026-52854 (professionalwiki/maps, output_encoding, 2026-07-02): excluded by name. _mediawiki/maps has stored XSS through the overlays parameter in the display_map parser fun_
- GHSA-j5mc-p8qg-39j7 (kimai/kimai, access_control, 2026-07-02): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai Favorite Timesheet Add and Remove Endpoints Allows Cross-User Bookmark Manipulation_
- GHSA-529h-vh3j-85hq / CVE-2026-49981 (twigphp/twig, access_control, 2026-07-01): general-purpose library: the attacker source lies in the calling application, not in the repo. _Twig: Sandbox filter, tag and function allow-list bypass when sandbox state changes betwee_
- GHSA-hwmc-r6mf-jh83 (spatie/schema-org, output_encoding, 2026-07-01): general-purpose library: the attacker source lies in the calling application, not in the repo. _Schema.org has cross-site scripting (XSS) via script break-out in toScript() output_
- GHSA-2wwr-9x6f-88gp (easycorp/easyadminbundle, output_encoding/path, 2026-07-01): vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository (the advisory says the bundle's own first-party usage is not affected). _EasyAdminBundle has path traversal and reflected XSS in Flag and Icon Twig components_
- GHSA-h8vq-8gpg-mhcg / CVE-2026-48808 (twigphp/twig, access_control, 2026-06-30): general-purpose library: the attacker source lies in the calling application, not in the repo. _Twig: Sandbox property allowlist bypass via the `column` filter under `SourcePolicyInterfa_
- GHSA-8x9c-rmqh-456c / CVE-2026-48807 (twigphp/twig, access_control, 2026-06-30): general-purpose library: the attacker source lies in the calling application, not in the repo. _Twig: Sandbox `__toString()` policy bypass via `Traversable` in `join` and `replace` filte_
- GHSA-5v5v-ww74-355v / CVE-2026-48806 (twigphp/twig, access_control, 2026-06-30): general-purpose library: the attacker source lies in the calling application, not in the repo. _Twig: Sandbox `__toString()` policy bypass via dynamic mapping keys_
- GHSA-5q4q-834j-g8g4 / CVE-2026-47198 (paymenter/paymenter, access_control, 2026-06-30): **taken** as paymenter-checkout-config-override. _Paymenter has URL parameter injection that bypasses paid plan limits at checkout_
- GHSA-7vfx-4246-jcfh (solidinvoice/solidinvoice, access_control, 2026-06-26): repo over ~150 MB (151 MB per GitHub API). _SolidInvoice: IDOR in LiveComponent allows same-company cross-user access to API tokens an_
- GHSA-x8g9-h984-pc36 / CVE-2026-49359 (pontedilana/php-weasyprint, untrusted_destination, 2026-06-26): general-purpose library: the attacker source lies in the calling application, not in the repo. _PhpWeasyPrint vulnerable to SSRF and local file disclosure via the attachment option_
- GHSA-5g9f-cwwg-4p8g / CVE-2026-49358 (pontedilana/php-weasyprint, path, 2026-06-26): general-purpose library: the attacker source lies in the calling application, not in the repo. _PhpWeasyPrint vulnerable to arbitrary file deletion at shutdown via public $temporaryFiles_
- GHSA-2fmj-p74r-3wjm / CVE-2026-49286 (pontedilana/php-weasyprint, deserialization, 2026-06-26): general-purpose library: the attacker source lies in the calling application, not in the repo. _PhpWeasyPrint vulnerable to PHAR deserialization via output filename (CVE-2023-28115 case-_
- GHSA-f5gc-qxf8-mh9g / CVE-2026-49260 (pontedilana/php-weasyprint, injection, 2026-06-26): general-purpose library: the attacker source lies in the calling application, not in the repo. _php-weasyprint: shell command injection via configurable WeasyPrint binary path due to inv_
- GHSA-7mqq-4v55-88gh / CVE-2026-54244 (statamic/cms, access_control, 2026-06-26): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text). _Statamic CMS's incorrect authorization lets view-only users submit Live Preview content re_
- GHSA-v5c4-wcpj-x73m / CVE-2026-54242 (statamic/cms, untrusted_destination, 2026-06-26): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) (Glide image proxy). _Statamic Vulnerable to Server-Side Request Forgery via Glide (DNS rebinding)_
- GHSA-2497-6pwj-pwg7 / CVE-2026-49288 (statamic/cms, access_control, 2026-06-26): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text). _Statamic CMS: Missing authorization on Control Panel fieldtype endpoints allows disclosure_
- GHSA-mmj8-wcvw-6789 / CVE-2026-49262 (aimeos/pagible, untrusted_destination, 2026-06-26): **taken** as pagible-admin-proxy-dns-rebinding-ssrf. _Aimeos Pagible CMS vulnerable to Server Side Request Forgery (SSRF) via DNS rebinding in a_
- GHSA-wpvj-hjcr-h3p2 / CVE-2026-48820 (cakephp/cakephp, path, 2026-06-26): framework `View::element()`; vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository (the advisory: 'if developers are not using user-supplied data in element names, no action is required'). _CakePHP: View::element() is missing a path containment check_
- GHSA-6mmj-jhqj-6c6q / CVE-2026-55542 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT's S3 signature image retrieval lacks authorization before temporary URL _
- GHSA-x667-r589-43m7 / CVE-2026-55519 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has Improper Authorization in File Deletion (IDOR)_
- GHSA-hf68-g98v-wp9g / CVE-2026-55483 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT Vulnerable to Privilege Escalation via Missing admin Permission Check in User Cre_
- GHSA-33g4-646g-qwmm / CVE-2026-55482 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has Multi-Tenancy Bypass via Bulk Asset Update_
- GHSA-6x4j-8954-5hxm / CVE-2026-50550 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has a 2FA reset privilege bypass_
- GHSA-p68w-rgmg-3c2v / CVE-2026-49976 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT Vulnerable to User Account Escalation via CSV Import_
- GHSA-8c6h-7g6x-m5x4 / CVE-2026-49205 (thorsten/phpmyfaq, access_control, 2026-06-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ: Missing userHasPermission() in 4 API write endpoints (CVE-2026-24421 Incomplete _
- GHSA-6f75-x745-xcpr / CVE-2026-48507 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Bulk editing users allowed `ldap_import` and `activated_in` bulk editing users_
- GHSA-44wp-g8f4-f4v5 / CVE-2026-48500 (filamentphp/filament, access_control, 2026-06-23): repo over ~150 MB (582 MB per GitHub API). _Filament: Unauthenticated temporary file upload on auth pages_
- GHSA-52fw-7fw2-fmv5 / CVE-2026-48493 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT Vulnerable to Privilege Escalation for self via API Permissions Assignment_
- GHSA-f3c5-6cw8-fg57 / CVE-2026-48492 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT's selectlist visibility is too permissive_
- GHSA-3fc8-8hp6-6jr4 / CVE-2026-48167 (filamentphp/filament, output_encoding, 2026-06-23): repo over ~150 MB (582 MB per GitHub API). _Filament: Unvalidated ImageColumn and ImageEntry values can be used for XSS_
- GHSA-7cqp-7cfv-6c3q (wwbn/avideo, output_encoding, 2026-06-23): repo over ~150 MB (664 MB per GitHub API). _AVideo Meet plugin: anonymous-to-admin stored XSS via unescaped participant User-Agent in _
- GHSA-wc3f-xc32-435f / CVE-2026-55173 (wwbn/avideo, injection, 2026-06-23): repo over ~150 MB (664 MB per GitHub API). _AVideo has an incomplete fix of CVE-2026-33482: sanitizeFFmpegCommand still allows a singl_
- GHSA-32fw-h446-j4hh (, untrusted_destination, 2026-06-23): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav is Vulnerable to XXE via SVG Upload_
- GHSA-53h4-8rc4-f539 / CVE-2026-48157 (slimphp/slim, output_encoding, 2026-06-23): vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository (only apps that pass request data to `HttpException::setTitle()`/`setDescription()`). _Slim has Reflected XSS in the HtmlErrorRenderer_
- GHSA-8j8m-p79x-g4jm / CVE-2026-33684 (wwbn/avideo, access_control, 2026-06-22): repo over ~150 MB (664 MB per GitHub API). _AVideo's Privilege Escalation via Unguarded Permission Parameters in signUp API Allows Sel_
- GHSA-x93q-x9pc-w5hw / CVE-2026-44585 (paymenter/paymenter, access_control, 2026-06-22): one case per repository (repository taken as paymenter-checkout-config-override). _Paymenter has broken object level authorization via service reference manipulation on tick_
- GHSA-f95g-vm94-46c3 (, output_encoding, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS Vulnerable to Stored XSS in Settings Names and Field Options_
- GHSA-jcmp-jxh2-4jc3 (, path, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS has authenticated path traversal in `assets/icon`, allowing _
- GHSA-f4h3-qhg5-j6mq (, access_control, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: Authorized asset "preview file" requests bypass allows user_
- GHSA-9r7j-7jhg-4f4c (, output_encoding, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS Vulnerable to Stored XSS via User Group Name in User Permiss_
- GHSA-5w9j-w5p8-r4p7 (, output_encoding, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS has Stored XSS in Table Field in its "Row Heading" Column Ty_
- GHSA-xj2c-g5xp-4p47 (, access_control, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: Authenticated "assets/preview-thumb" discloses signed fallb_
- GHSA-pmm4-v8f6-4vpp (, injection, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: RCE via missing cleanseConfig in FieldsController::actionRe_
- GHSA-xj9w-cgqg-q897 (wwbn/avideo, access_control, 2026-06-20): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: AVideo has Unauthenticated PGP Message Decryption via Public Endpoint_
- GHSA-rg7q-4223-phjw (, access_control, 2026-06-20): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: AVideo: Unauthenticated Access to Payment Log DataTables Endpoints Exp_
- GHSA-5c7p-g73q-rpg5 / CVE-2026-55692 (starcitizenwiki/mediawiki-extensions-embedvideo, output_encoding, 2026-06-19): excluded by name. _StarCitizenWiki Extension Embed Video: Stored XSS via malformed src url with $wgEmbedVideo_
- GHSA-7h5p-637f-jfr7 / CVE-2026-55691 (starcitizenwiki/mediawiki-extensions-embedvideo, output_encoding, 2026-06-19): excluded by name. _StarCitizenWiki Extension Embed Video: Stored XSS via unsanitized class passed to template_
- GHSA-c29q-5xm7-5p62 / CVE-2026-55690 (starcitizenwiki/mediawiki-extensions-embedvideo, output_encoding, 2026-06-19): excluded by name. _StarCitizenWiki Extension Embed Video: Stored XSS via unsanitized service name in exceptio_
- GHSA-vm85-hxw5-5432 / CVE-2026-55766 (guzzle/psr7, output_encoding, 2026-06-19): general-purpose library: the attacker source lies in the calling application, not in the repo. _guzzlehttp/psr7: CRLF Injection in HTTP Start-Line Serialization_
- GHSA-9qfv-wgh2-m6p8 / CVE-2026-55374 (jleehr/canto-saas-api, untrusted_destination, 2026-06-19): general-purpose library: the attacker source lies in the calling application, not in the repo. _canto-saas-api: Authenticated API requests can be redirected via unencoded path variables_
- GHSA-p9xj-fpr2-jf2q / CVE-2026-55878 (symfony/ux, path, 2026-06-19): developer tool: the `ux:install` console command, not a web request path. _symfony/ux-toolkit: Path Traversal Allows Arbitrary File Write and Read via Crafted Recipe_
- GHSA-6v8j-33hc-mv84 / CVE-2026-55877 (symfony/ux, output_encoding, 2026-06-19): the source is icon files or a configured Iconify endpoint, not a request; no fix commit referenced. _symfony/ux-icons: XSS via unsanitized SVG content in local files and Iconify on-demand res_
- GHSA-mwqm-4fw3-cjvr / CVE-2026-49216 (symfony/ux, output_encoding, 2026-06-19): the sink is the Stimulus (JavaScript) controller's template literal, not PHP. _symfony/ux-autocomplete: XSS via unescaped AJAX response data_
- GHSA-38x5-rcv4-xf7x / CVE-2026-49210 (symfony/ux, output_encoding, 2026-06-19): **taken** as symfony-ux-live-child-tag-xss. _symfony/ux-live-component: XSS via attacker-controlled child component tag_
- GHSA-r3w8-2c5r-h9j9 / CVE-2026-54005 (getkirby/kirby, access_control, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: `pages.access` permission is not checked in the `site/find` REST API route_
- GHSA-89cp-7p28-jffg / CVE-2026-54004 (getkirby/kirby, access_control, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: Access to files of top-level drafts is not protected by permissions_
- GHSA-wr9h-4r83-f4v6 / CVE-2026-54002 (getkirby/kirby, output_encoding, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: Cross-site scripting (XSS) from incomplete HTML/XML sanitization in `Dom::sanitize(_
- GHSA-4v4h-m2qq-ppgw / CVE-2026-50188 (getkirby/kirby, output_encoding, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: Request header injection in `Http\Remote`_
- GHSA-rhj6-r49h-5932 / CVE-2026-49276 (getkirby/kirby, output_encoding, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: Self cross-site scripting (self-XSS) in the writer field_
- GHSA-23q2-54qv-rq5x / CVE-2026-49274 (getkirby/kirby, access_control, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: `pages.access` permission is not checked in the pages picker for parent pages_
- GHSA-pmf8-g7c8-7v54 / CVE-2026-55890 (getgrav/grav, output_encoding, 2026-06-18): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Stored CSS injection via Markdown image ?style=… reaches MediaObjectTrait::style() —_
- GHSA-7p36-fq2r-4h7r / CVE-2026-11407 (pimcore/pimcore, injection, 2026-06-17): repo over ~150 MB (534 MB per GitHub API). _Pimcore CMS Twig Sandbox Bypass via SecurityPolicy checkMethodAllowed_
- GHSA-m9cv-24rx-8mv7 / CVE-2026-55409 (filamentphp/filament, output_encoding, 2026-06-17): repo over ~150 MB (582 MB per GitHub API). _Filament: Disabled RichEditor field state can be used for XSS_
- GHSA-m557-wrgg-6rp4 / CVE-2026-55599 (phpseclib/phpseclib, untrusted_destination, 2026-06-16): general-purpose library: the attacker source lies in the calling application, not in the repo. _phpseclib: X.509 certificate validation sends attacker-controlled outbound requests (serve_
- GHSA-9mmg-q95p-gp67 / CVE-2026-50886 (firefly-iii/firefly-iii, access_control, 2026-06-15): repo over ~150 MB (313 MB per GitHub API). _Project Firefly III has incorrect access control in the webhook management component_
- GHSA-h5x3-xfc9-m39h / CVE-2026-48784 (symfony/symfony, untrusted_destination, 2026-06-15): repo over ~150 MB (339 MB per GitHub API). _Symfony: UrlGenerator Dot-Segment Encoding Skips Every Other Chained `../` or `./` → Gener_
- GHSA-38cx-cq6f-5755 / CVE-2026-48736 (symfony/symfony, untrusted_destination, 2026-06-15): repo over ~150 MB (339 MB per GitHub API). _Symfony: IpUtils::PRIVATE_SUBNETS Omits IPv6 Transition Forms (6to4, NAT64, Teredo, IPv4-c_
- GHSA-6h46-9jf5-q59x / CVE-2026-48489 (symfony/symfony, access_control, 2026-06-15): repo over ~150 MB (339 MB per GitHub API). _Symfony: Security Firewall Bypass via failure_forward Subrequest: Unauthenticated Access t_
- GHSA-x5qj-865h-mgvm / CVE-2026-48761 (symfony/symfony, output_encoding, 2026-06-15): repo over ~150 MB (339 MB per GitHub API). _Symfony: HtmlSanitizer UrlAttributeSanitizer Misses URL Attributes_
- GHSA-vv4x-qcpq-wgrg / CVE-2026-12198 (microweber/microweber, path, 2026-06-15): repo over ~150 MB (1185 MB per GitHub API). _Microweber vulnerable to Path Traversal_
- GHSA-pjpj-v387-x4vq / CVE-2026-11607 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in its Form Framework_
- GHSA-f34x-rx2w-7pm3 / CVE-2026-47349 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in the Recycler Module_
- GHSA-3p42-w5ch-gg42 / CVE-2026-47347 (typo3/typo3, untrusted_destination, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has an Open Redirect Vulnerability via Core Utilities_
- GHSA-3v8v-4wg6-r7qh / CVE-2026-47343 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS: Destructive Actions on File Mount Folders_
- GHSA-p5j5-4j3q-8mq8 / CVE-2026-47345 (typo3/html-sanitizer, output_encoding, 2026-06-12): general-purpose library: the attacker source lies in the calling application, not in the repo. _TYPO3 HTML Sanitizer allows Cross-site Scripting_
- GHSA-jh32-v29g-68pq / CVE-2026-49741 (typo3/typo3, access_control/injection, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Privilege Escalation & SQL Injection in its Form Framework_
- GHSA-qcmw-6rm2-5x78 / CVE-2026-47350 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in its DataHandler_
- GHSA-hwvq-2w67-rvxp / CVE-2026-47346 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in its Form Framework_
- GHSA-chm7-4vch-h8vr / CVE-2026-49742 (typo3/typo3, path, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in its Media Module_
- GHSA-c78m-c52x-jgwp / CVE-2026-49740 (typo3/typo3, deserialization, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Insecure Deserialization via Core API_
- GHSA-jf56-v8jc-jcc5 / CVE-2026-49738 (typo3/typo3, path, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in its File Abstraction Layer_
- GHSA-2j54-93q2-3hjq / CVE-2026-47352 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in Backend API_
- GHSA-q93m-25xv-94hh / CVE-2026-47351 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS: Broken Access Control in Media Module_
- GHSA-cg75-qfg2-w9hj / CVE-2026-47348 (typo3/typo3, output_encoding, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Cross-Site Scripting in Indexed Search_
- GHSA-jvf5-rxvv-3mcg / CVE-2026-47344 (typo3/html-sanitizer, output_encoding, 2026-06-12): general-purpose library: the attacker source lies in the calling application, not in the repo. _TYPO3 HTML Sanitizer allows Cross-site Scripting_
- GHSA-6jq6-x4cx-qvcm (firefly-iii/firefly-iii, output_encoding, 2026-06-12): repo over ~150 MB (313 MB per GitHub API). _Firefly II has Stored XSS in Audit Log Entry view via piggy bank name (ale.twig)_
- GHSA-7q3w-xqjw-g3cr / CVE-2026-48067 (filamentphp/filament, access_control, 2026-06-11): repo over ~150 MB (582 MB per GitHub API). _Filament has inconsistent scope enforcement for its AttachAction and AssociateAction Selec_
- GHSA-34xg-wgjx-8xph / CVE-2026-48998 (guzzle/psr7, untrusted_destination, 2026-06-11): general-purpose library: the attacker source lies in the calling application, not in the repo. _guzzlehttp/psr7 has Host Confusion via Authority Reinterpretation_
- GHSA-hq7v-mx3g-29hw / CVE-2026-49214 (guzzle/psr7, output_encoding, 2026-06-11): general-purpose library: the attacker source lies in the calling application, not in the repo. _guzzlehttp/psr7 has CRLF Injection via URI Host Component_
- GHSA-q8r6-5hfw-5jff / CVE-2026-53723 (guzzle/guzzle-services, injection, 2026-06-11): general-purpose library (`guzzlehttp/guzzle-services` XML serialization); vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository. _guzzlehttp/guzzle-services' XML Request Serialization Vulnerable to XML Injection via CDAT_
- GHSA-g82f-9pw7-773w / CVE-2026-10721 (concretecms/concretecms, deserialization, 2026-06-10): repo over ~150 MB (303 MB per GitHub API). _Concrete CMS: PHP Object Injection via unserialize() calls_
- GHSA-wmjr-58rf-xgrc / CVE-2026-53675 (buddypress/buddypress, access_control, 2026-06-10): one case per repository (repository taken as wp-buddypress-messages-rest-idor). _BuddyPress: Any authenticated attacker can enumerate another user's complete friend list v_
- GHSA-j3j5-5m8v-7gvc / CVE-2026-53673 (buddypress/buddypress, access_control, 2026-06-10): **taken** as wp-buddypress-messages-rest-idor. _BuddyPress: Authenticated attackers can access arbitrary private message threads via user__
- GHSA-jvc5-6g7q-c843 / CVE-2026-48030 (pheditor/pheditor, injection, 2026-06-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Pheditor: OS Command Injection in terminal handler via unsanitized 'dir' parameter_
- GHSA-pr6f-87hf-hx24 / CVE-2026-50636 (limesurvey/limesurvey, injection, 2026-06-09): repo over ~150 MB (1457 MB per GitHub API). _LimeSurvey has a SQL Injection issue_
- GHSA-87m4-826x-3crx / CVE-2026-45034 (phpoffice/phpspreadsheet, deserialization, 2026-06-08): general-purpose library: the attacker source lies in the calling application, not in the repo. _PHPSpreadsheet has a patch bypass for CVE-2026-34084 _
- GHSA-2g2g-8p8h-fgwm / CVE-2026-47730 (twigphp/twig, output_encoding, 2026-06-05): general-purpose library: the attacker source lies in the calling application, not in the repo. _Twig: XSS in profiler HtmlDumper via unescaped template and profile names_
- GHSA-c3qp-2ggw-xjg7 / CVE-2026-47744 (shopperlabs/shopper, access_control, 2026-06-05): repo over ~150 MB (151 MB per GitHub API). _Shopper: Authorization bypass and RBAC privilege escalation in team settings_
- GHSA-hr9v-r8r2-hg7j / CVE-2026-47743 (shopperlabs/shopper, access_control/output_encoding, 2026-06-05): repo over ~150 MB (151 MB per GitHub API). _Shopper: Multiple data integrity and disclosure issues in admin Livewire components_
- GHSA-fxqw-97cc-7g5c / CVE-2026-47745 (shopperlabs/shopper, access_control, 2026-06-05): repo over ~150 MB (151 MB per GitHub API). _Shopper: Missing per-action authorization on PaymentMethods, Currencies and Carriers admin_
- GHSA-h4mp-g9c6-xwph / CVE-2026-47742 (shopperlabs/shopper, access_control, 2026-06-05): repo over ~150 MB (151 MB per GitHub API). _Shopper: Missing authorization on Product admin Livewire sub-form components_
- GHSA-vg35-5wq7-3x7w / CVE-2026-47761 (tinymce/tinymce, output_encoding, 2026-06-05): repo over ~150 MB (152 MB per GitHub API). _TinyMCE Cross-Site Scripting (XSS) vulnerability using media plugin `data-mce-object` inje_
- GHSA-v98h-vmpc-fpqv / CVE-2026-47762 (tinymce/tinymce, output_encoding, 2026-06-05): repo over ~150 MB (152 MB per GitHub API). _TinyMCE Cross-Site Scripting (XSS) vulnerability through `mce:protected` comments_
- GHSA-q742-qvgc-gc2f / CVE-2026-47759 (tinymce/tinymce, output_encoding, 2026-06-05): repo over ~150 MB (152 MB per GitHub API). _TinyMCE Cross-Site Scripting (XSS) vulnerability using through data-mce- prefixed src, hre_
- GHSA-mh5m-5hw4-5c69 / CVE-2026-47760 (tinymce/tinymce, output_encoding, 2026-06-05): repo over ~150 MB (152 MB per GitHub API). _TinyMCE Cross-Site Scripting (XSS) vulnerability using sanitization bypass through nested _
- GHSA-gq96-5pfx-f4vc / CVE-2026-48013 (shopware/shopware, untrusted_destination, 2026-06-04): repo over ~150 MB (521 MB per GitHub API). _Shopware: SSRF in Media External-Link Endpoint Bypasses IP Validation_
- GHSA-xvhc-gm7j-mhmc / CVE-2026-48015 (shopware/shopware, output_encoding, 2026-06-04): repo over ~150 MB (521 MB per GitHub API). _Shopware: Stored XSS via SVG file upload — no SVG sanitization_
- GHSA-f8q6-3g5w-jjr6 / CVE-2026-48014 (shopware/shopware, access_control, 2026-06-04): repo over ~150 MB (521 MB per GitHub API). _Shopware: Admin API ACL Bypass in Order State Transition Endpoints_
- GHSA-4x3x-869w-xx3m / CVE-2026-48012 (shopware/shopware, untrusted_destination, 2026-06-04): repo over ~150 MB (521 MB per GitHub API). _Shopware SSO referer trust leading to an arbitrary redirect target_
- GHSA-gv8p-48fr-4fxg / CVE-2026-48008 (shopware/shopware, access_control, 2026-06-04): repo over ~150 MB (521 MB per GitHub API). _Shopware: Privilege Escalation via Sync API Integration Admin Flag Bypass_
- GHSA-8whc-2wmv-ww35 / CVE-2026-54458 (wwbn/avideo, output_encoding, 2026-06-04): repo over ~150 MB (664 MB per GitHub API). _WWBN AVideo: Unauthenticated Stored DOM Cross-Site Scripting via Per-Client Metadata Broad_
- GHSA-66q5-cj5g-wrfx / CVE-2026-50183 (wwbn/avideo, output_encoding, 2026-06-04): repo over ~150 MB (664 MB per GitHub API). _WWBN AVideo: Stored XSS via Hostile YouTube Video Title in AVideo YouTubeAPI Gallery Secti_
- GHSA-hgjh-6wj8-gcgf / CVE-2026-50182 (wwbn/avideo, output_encoding, 2026-06-04): repo over ~150 MB (664 MB per GitHub API). _WWBN AVideo: Unauthenticated Reflected XSS via $_GET['search'] in AVideo YouTubeAPI Galler_
- GHSA-2fhx-q92v-5fhv / CVE-2026-49279 (wwbn/avideo, output_encoding, 2026-06-04): repo over ~150 MB (664 MB per GitHub API). _WWBN AVideo: Stored XSS via autoEvalCodeOnHTML Bypass in MessageSQLite WebSocket Handler (_
- GHSA-c8h8-vq34-9fw2 / CVE-2026-47694 (wwbn/avideo, output_encoding, 2026-06-04): repo over ~150 MB (664 MB per GitHub API). _WWBN AVideo: Stored XSS via unescaped Gallery category description_
- GHSA-52pr-7vmf-2w7x / CVE-2026-7888 (concretecms/concretecms, deserialization, 2026-06-03): repo over ~150 MB (303 MB per GitHub API). _Concrete CMS is vulnerable to PHP Object Injection via unserialize() calls in the Workflow_
- GHSA-m8xx-3x29-84h8 / CVE-2022-31114 (laravel-backpack/crud, output_encoding, 2026-06-03): one case per repository (repository taken as backpack-stats-host-header-cmdi). _backpack/crud is vulnerable to Cross-Site Scripting (XSS)_
Passed over (not examined; family quota already met): GHSA-7wwh-xcc3-9fcg (paymenter/paymenter, untrusted_destination, 2026-06-22); GHSA-86hp-hf3j-3m8r (cotonti/cotonti, output_encoding, 2026-06-18); GHSA-hhpq-7wg4-36jm (cakephp/authentication, untrusted_destination, 2026-06-17); GHSA-crmm-hgp2-wgrp (laravel/framework, output_encoding, 2026-06-17); GHSA-gmxh-hjfv-qc2w (benjaminjonard/koillection, untrusted_destination, 2026-06-15); GHSA-p85r-x2wj-mxqj (shlinkio/shlink, untrusted_destination, 2026-06-15); GHSA-wrcg-234w-hfhq (intelliants/subrion, output_encoding, 2026-06-15).
### Lane B: unreviewed advisories with a GitHub fix reference on a PHP repository
- GHSA-84f7-xff5-hx38 / CVE-2022-51019 (akaunting/akaunting, injection, 2026-09-29): republished old vulnerability (CVE-2022-51019, fixed in Akaunting 2.1.31 in 2022), not a 2026 advisory; also advisory names no vulnerable file or function (only 'the alias parameter' of module installation). _Akaunting before 2.1.31 contains an OS command injection vulnerability in the module insta_
- GHSA-rv7j-2hq3-r6fq / CVE-2024-58386 (zoneminder/zoneminder, path, 2026-09-29): repo over ~150 MB (161 MB per GitHub API). _ZoneMinder versions 1.37.0 before 1.38.0 contain a path traversal vulnerability in the fil_
- GHSA-gxmj-vr4j-8j5c / CVE-2026-100884 (krayin/laravel-crm, path, 2026-09-28): the named location is not declarable: the advisory (VulDB text) puts 'the function Storage::download' in `packages/Webkul/Admin/src/Config/acl.php`, a configuration file; the weakness described is an IDOR (CWE-99 'resource identifiers'), not a path. _A vulnerability has been found in Krayin laravel-crm up to 2.2.5. The impacted element is _
- GHSA-7q7h-5g76-8cq4 / CVE-2026-100520 (crivion/laranode, path, 2026-09-26): advisory names no vulnerable file or function (only the `POST /filemanager/upload-file` endpoint and its `path` parameter). _Laranode versions before 1.2.1 contain a path traversal vulnerability in the POST /fileman_
- GHSA-vm72-96wq-2ppv / CVE-2026-96744 (mongodb/laravel-mongodb, injection, 2026-09-24): general-purpose library (`mongodb/laravel-mongodb` cache lock); the attacker source lies in the calling application. _Improper neutralization of special elements in data query logic in the cache lock implemen_
- GHSA-c7hm-qgxh-6vgg / CVE-2026-95701 (misp/misp, path, 2026-09-22): repo over ~150 MB (201 MB). _In MISP, the __statisticsOrgs method in UsersController.php used the organization name dir_
- GHSA-24qr-3pqh-xghm / CVE-2026-95698 (misp/misp, path, 2026-09-22): repo over ~150 MB (201 MB). _The findOrgImage method in MISP's OrgImgHelper constructs a filesystem path by concatenati_
- GHSA-f496-75v4-rv5q / CVE-2026-95693 (misp/misp, path, 2026-09-22): repo over ~150 MB (201 MB). _In MISP, the EventReport::uploadPicture method in processed a caller-supplied tmp_name fie_
- GHSA-qjpg-gmh7-3ch6 / CVE-2026-95667 (misp/misp, path, 2026-09-22): repo over ~150 MB (201 MB). _The MISP installer scripts (for Debian 12, Debian 13, Ubuntu 24.04, and RHEL 9.4) create a_
- GHSA-2jvm-986q-589g / CVE-2026-94401 (misp/misp, path, 2026-09-21): repo over ~150 MB (201 MB). _MISP has a file-handling vulnerability that could let certain authenticated users make the_
- GHSA-7c6f-jc48-6hfq / CVE-2026-94106 (jamesheinrich/getid3, injection, 2026-09-20): general-purpose library (getID3 media parser); the attacker source lies in the calling application. _getID3 before 1.9.26 contains an OS command injection vulnerability in shell-out handlers _
- GHSA-pjph-5h96-886c / CVE-2026-93988 (qloapps/qloapps, path, 2026-09-20): one case per repository (repository taken as qloapps-customermessage-bo-query-sqli). _QloApps through 1.7.0 contains a path traversal vulnerability in the getEmailHTML action o_
- GHSA-3v3g-3pf9-fgcf / CVE-2026-93872 (cotonti/cotonti, deserialization, 2026-09-18): no fix diff: the referenced PR #1897 is not merged, and `plugins/comments/controllers/actions/EditAction.php` has no later commit. _Cotonti 1.0.0 passes the base64-decoded cb parameter to unserialize() without allowed_clas_
- GHSA-q246-734p-8872 / CVE-2026-93533 (spatie/scotty, injection, 2026-09-18): developer CLI (`scotty` SSH task runner, `DoctorCommand`), not a web application; the fix PR 'awaits acceptance'. _A vulnerability was determined in spatie Scotty up to 1.4.4. This impacts the function..._
- GHSA-qqx6-v32j-728q / CVE-2026-89036 (appwrite/appwrite, injection, 2026-09-17): repo over ~150 MB (532 MB). _Appwrite before 2.0.0 contains an argument injection vulnerability that allows authenticat_
- GHSA-2vrr-cgr4-gfmm / CVE-2026-76834 (b2evolution/b2evolution, deserialization, 2026-09-17): repo over ~150 MB (182 MB). _b2evolution CMS versions 6.7.8 through 7.2.5 contain an incomplete fix for CVE-2016-8901 w_
- GHSA-6wrp-rq46-2frr / CVE-2026-92970 (hubzero/hubzero-cms, path, 2026-09-17): repo over ~150 MB (359 MB). _HUBzero CMS through 2.2.32 contains a path traversal vulnerability in project file upload._
- GHSA-xv9c-9hwm-jp8g / CVE-2026-91939 (cotonti/cotonti, deserialization, 2026-09-15): no fix diff: the referenced PR #1889 is not merged. _Cotonti 1.0.0 Comments plugin passes the ci GET parameter to unserialize() without..._
- GHSA-63h9-9gmf-c466 / CVE-2020-15875 (librenms/librenms, injection, 2026-09-13): repo over ~150 MB (369 MB). _An issue was discovered in LibreNMS 1.65. A remote authenticated attacker with normal priv_
- GHSA-6hw9-7gx5-mw88 / CVE-2026-85200 (fitoussi/geo-my-wp, path, 2026-09-12): **taken** as wp-geo-my-wp-info-window-lfi. _The GEO my WP plugin for WordPress is vulnerable to Local File Inclusion in all versions u_
- GHSA-hh3c-hgv7-gg2r / CVE-2026-79362 (woltlab/wcf, injection, 2026-09-11): repo over ~150 MB (203 MB). _Certain Woltlab products are affected by RCE via Cache Poisoning. WCF >= 6.1.0 until < 6.1_
- GHSA-w3w8-r2xc-3rp3 / CVE-2024-58381 (pmmp/pocketmine-mp, deserialization, 2026-09-09): game server (Minecraft Bedrock), not a web application; old vulnerability (CVE-2024-58381); the impact is denial of service. _PocketMine-MP before 5.11.1 contains a denial of service vulnerability in LoginPacket JSON_
- GHSA-cw46-936q-fjcf / CVE-2026-57777 (woocommerce/woocommerce, injection, 2026-09-04): repo over ~150 MB (932 MB). _Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection')..._
- GHSA-xf4v-vq77-2rgv / CVE-2026-84694 (coollabsio/coolify, injection, 2026-09-02): advisory names no vulnerable file or function (only 'environment variable key names in Docker commands'). _Coolify before 4.2.0 fails to properly escape environment variable key names in Docker com_
- GHSA-46j6-w3hw-4r9x / CVE-2026-71981 (cypht-org/cypht, deserialization, 2026-09-01): advisory names no vulnerable file or function (only 'the back_query GET parameter of the logout handler'); the fix is known (`e4aa2f34`, merge of PR #2073) — the nearest alternate for deserialization if the naming rule is relaxed. _Cypht before 2.12.2 contains a PHP object injection vulnerability that allows authenticate_
- GHSA-v64j-pgc3-qgc9 / CVE-2026-82668 (klaussilveira/gitlist, injection, 2026-08-31): **taken** as gitlist-commandline-cmdi. _A security vulnerability has been detected in klaussilveira GitList 2.0.0. Affected by thi_
- GHSA-j8xc-7c27-7wj4 / CVE-2026-81728 (dolibarr/dolibarr, injection, 2026-08-27): repo over ~150 MB (1705 MB). _Dolibarr before 24.0.0 contains a SQL injection in its CSV and XLSX import wizard. The wiz_
- GHSA-xf89-whcw-pq6g / CVE-2020-15878 (librenms/librenms, injection, 2026-08-26): repo over ~150 MB (369 MB). _An issue was discovered in LibreNMS 1.65. A remote authenticated attacker with normal priv_
- GHSA-2658-wj4j-m387 / CVE-2020-15876 (librenms/librenms, injection, 2026-08-26): repo over ~150 MB (369 MB). _An issue was discovered in LibreNMS 1.65. A remote authenticated attacker with normal priv_
- GHSA-6h76-35rh-gxm9 / CVE-2026-80428 (ilias-elearning/ilias, deserialization, 2026-08-26): repo over ~150 MB (3341 MB). _ILIAS deserialises stored session data for an unauthenticated caller. The Shibboleth back-_
- GHSA-4rvx-xf2v-6r64 / CVE-2020-15874 (librenms/librenms, injection, 2026-08-26): repo over ~150 MB (369 MB). _An issue was discovered in LibreNMS 1.65. A remote authenticated attacker with normal priv_
- GHSA-5w6r-2mxc-fq3v / CVE-2026-75497 (qloapps/qloapps, injection, 2026-08-25): **taken** as qloapps-customermessage-bo-query-sqli. _Webkul QloApps does not validate request parameters before a database query. A remote,..._
- GHSA-mj48-jjc7-6wqr / CVE-2026-75498 (qloapps/qloapps, injection, 2026-08-25): one case per repository (repository taken as qloapps-customermessage-bo-query-sqli). _Webkul QloApps does not validate request parameters before a database query. A remote,..._
- GHSA-8x6f-m45x-3rjp / CVE-2026-71905 (volmarg/personal-management-system, injection/path, 2026-08-24): corrupt advisory record: the description is about DrayTek VigorAP routers, not this repository. _Multiple DrayTek VigorAP models contain a command injection vulnerability in the ExportSet_
- GHSA-2c9p-fq48-6545 / CVE-2026-75007 (roundcube/roundcubemail, injection, 2026-08-17): advisory names no vulnerable file or function (only 'the LDAP search filter ... %u/%fu/%d substitution'). _In Roundcube Webmail before 1.6.18 and 1.7.x before 1.7.3, the LDAP search filter was subj_
- GHSA-m462-3x7r-5r7f / CVE-2026-75002 (roundcube/roundcubemail, injection, 2026-08-17): advisory names no vulnerable file or function (only 'mail search and LITERAL+ byte-count desynchronization'). _In Roundcube Webmail before 1.6.18 and 1.7.x before 1.7.3, mail search and LITERAL+ byte-c_
- GHSA-q5rh-mmp4-ppgq / CVE-2026-74997 (roundcube/roundcubemail, injection, 2026-08-17): **taken** as roundcube-markasjunk-cmd-learn-rce. _In Roundcube Webmail before 1.6.18 and 1.7.x before 1.7.3, the cmd_learn driver of the mar_
- GHSA-4x4q-mx3g-g3m2 / CVE-2026-73680 (cockpit-hq/cockpit, injection, 2026-08-14): advisory names no vulnerable file or function in the repository (only the library call `Process::fromShellCommandline()` and 'the FFmpeg integration'). _Cockpit CMS 2.14.0 and prior contains a command injection vulnerability in the FFmpeg inte_
- GHSA-w54w-xf28-wf79 / CVE-2026-71248 (harsh21patel/inventory-management-system-php, injection, 2026-08-05): personal/student project (`harsh21patel/inventory-management-system-php`), not a maintained application. _Inventory-Management-System-PHP's login.php constructs its authentication query via direct_
- GHSA-cv48-22qx-mpmx / CVE-2026-71232 (magicblack/maccms10, injection, 2026-08-05): **taken** as maccms10-template-blacklist-rce. _MacCMS10's admin template editor (application/admin/controller/Template.php) blocks danger_
Passed over (not examined; family quota already met): GHSA-5528-3vrx-6443 (octobercms/october, untrusted_destination, 2026-09-28); GHSA-g8jm-3grw-8wh2 (cotonti/cotonti, untrusted_destination, 2026-09-26); GHSA-gmhw-pxmg-j22w (jamesheinrich/getid3, untrusted_destination, 2026-09-20); GHSA-gp9j-h9gm-p3q2 (cotonti/cotonti, untrusted_destination, 2026-09-18); GHSA-5p43-vwpq-wm4q (cotonti/cotonti, untrusted_destination, 2026-09-18); GHSA-w83g-wg62-w3qm (laradashboard/laradashboard, untrusted_destination, 2026-09-09); GHSA-pmmj-hr44-h52v (ilias-elearning/ilias, path, 2026-08-31); GHSA-wvjm-58fp-5q8g (yaojingang/geoflow, untrusted_destination, 2026-08-31); GHSA-gf7h-mv4h-gg7c (yaojingang/geoflow, path, 2026-08-31); GHSA-jgch-2jr2-87vj (dolibarr/dolibarr, path, 2026-08-27); GHSA-g95v-m725-f6xc (volmarg/personal-management-system, path, 2026-08-27); GHSA-gqr6-9g7c-7g7r (wintercms/winter, path, 2026-08-25); GHSA-j9wc-mrp3-532h (hieventsdev/hi.events, untrusted_destination, 2026-08-24); GHSA-7994-gr8j-h7xp (openemr/openemr, path, 2026-08-19); GHSA-vf28-qp34-hhm8 (roundcube/roundcubemail, untrusted_destination, 2026-08-17); GHSA-qf4h-jxxc-wvr9 (getgrav/grav, path, 2026-08-14); GHSA-qrrp-6pvj-xfvf (rconfig/rconfig, path, 2026-08-12); GHSA-f3gp-9xr4-qmvc (joomla/joomla-cms, path, 2026-08-12); GHSA-7gmf-8rwv-m7jc (ayecode/geodirectory, path, 2026-08-11); GHSA-8879-fh4f-q3qm (krayin/laravel-crm, injection, 2026-08-03); GHSA-pw6j-6h33-fv96 (ayecode/invoicing, path, 2026-08-01); GHSA-rfxf-9pcf-q3rq (leantime/leantime, untrusted_destination, 2026-07-30); GHSA-pjx9-wh57-j26p (leantime/leantime, untrusted_destination, 2026-07-30); GHSA-frjf-677q-65jf (misp/misp, untrusted_destination, 2026-07-28); GHSA-hc3p-4h64-j8wr (getgrav/grav-plugin-api, path, 2026-07-23); GHSA-9wvj-rhhh-8292 (simplemachines/smf, untrusted_destination, 2026-07-14); GHSA-5q79-cfqm-5cxh (roundcube/roundcubemail, untrusted_destination, 2026-07-14); GHSA-cfx5-q5qm-x936 (cockpit-hq/cockpit, path, 2026-07-14); GHSA-9fmc-j924-ghh2 (plank/laravel-mediable, untrusted_destination, 2026-07-13); GHSA-j69f-3cmp-jrjj (ayecode/userswp, path, 2026-07-09); GHSA-7793-hmg3-gh57 (librebooking/librebooking, path, 2026-07-09); GHSA-4crx-2rrv-xxcf (horde/vfs, injection, 2026-07-08); GHSA-qfr5-75q5-g2c7 (kirilkirkov/ecommerce-codeigniter-bootstrap, path, 2026-07-04); GHSA-jf5m-wm8f-5pq9 (kirilkirkov/ecommerce-codeigniter-bootstrap, untrusted_destination, 2026-07-04); GHSA-r5xq-x4r6-7g6x (kirilkirkov/ecommerce-codeigniter-bootstrap, path, 2026-07-04); GHSA-233w-5rfj-8hmj (myvesta/vesta, injection, 2026-07-04); GHSA-cr5v-q482-32fc (phpipam/phpipam, path, 2026-07-04); GHSA-cgfv-42v7-7f53 (notifiarr/dockwatch, injection, 2026-07-02); GHSA-mv7w-qwqh-w2g9 (horde/imp, path, 2026-07-01); GHSA-2852-jv28-2mv8 (dolibarr/dolibarr, injection, 2026-06-30); GHSA-w5j4-x499-pfwp (frontaccountingerp/fa, injection, 2026-06-29); GHSA-8ccm-j5hq-9jhr (frontaccountingerp/fa, injection, 2026-06-29); GHSA-53v3-ffpm-f9hw (frontaccountingerp/fa, injection, 2026-06-29); GHSA-9fmm-h59x-wr53 (frontaccountingerp/fa, path, 2026-06-29); GHSA-7v9f-64q7-x2jg (misp/misp, injection, 2026-06-22); GHSA-qrw9-xj5g-qxw8 (chilek/lms, injection, 2026-06-18); GHSA-765c-7qf8-cw9w (chilek/lms, injection, 2026-06-18); GHSA-c4wp-7485-699w (misp/misp, path, 2026-06-12); GHSA-83v9-496w-54wx (wpusermanager/wp-user-manager, path, 2026-06-06); GHSA-9gjj-gmcx-6fhj (misp/misp, untrusted_destination, 2026-06-04); GHSA-rx96-8w86-5hhh (misp/misp, untrusted_destination, 2026-06-04); GHSA-93rv-mf2v-vv2r (php-censor/php-censor, injection, 2026-06-01).
### Lane C: deserialization (WordPress and other PHP products without a GitHub fix reference)
- GHSA-ggx5-r53p-pf97 / CVE-2026-86841 (WordPress plugin, deserialization, 2026-09-27): advisory names no vulnerable function or file. _The Online Scheduling and Appointment Booking System WordPress plugin before 28.3 does no_
- GHSA-j2rf-55r9-c5ph / CVE-2026-80513 (WordPress plugin, deserialization, 2026-09-24): advisory names no vulnerable function or file (only 'a user-supplied profile field value'). _The wpForo Forum WordPress plugin before 3.1.6 does not restrict which classes may be inst_
- GHSA-jvgp-rj42-qjww / CVE-2026-91827 (WordPress plugin, deserialization, 2026-09-22): advisory names no vulnerable function or file; Ninja Forms has no public development repository. _The Ninja Forms WordPress plugin 3.15.3 does not prevent user-submitted form field values _
- GHSA-wq6q-gx9m-7669 / CVE-2026-19658 (WordPress plugin, deserialization, 2026-09-22): advisory names no vulnerable function or file. _The Give Tributes plugin for WordPress is vulnerable to PHP Object Injection in all versio_
- GHSA-gjf4-862p-rg96 (Softaculous Virtualizor, deserialization, 2026-09-22): proprietary product, no public repository.
- GHSA-mwgw-rr28-m856 (CuteNews 2.1.2, deserialization, 2026-09-21): no fix diff (the advisory links only the master blob of core/core.php and a disclosure repository; cutenews/cutenews-2.0 has no commit since 2026-06).
- GHSA-3wpx-h7q2-mv8g / CVE-2026-85017 (WordPress plugin, deserialization, 2026-09-20): advisory names no vulnerable function or file (an unnamed AJAX action). _The Unlimited Elements For Elementor WordPress plugin before 2.0.20 does not perform a cap_
- GHSA-mqfm-qx5g-96vc / CVE-2026-17086 (WordPress plugin, deserialization, 2026-09-18): advisory names no vulnerable function or file. _The ShortPixel Image Optimizer – Optimize Images, Convert WebP & AVIF plugin for WordPress_
- GHSA-cg5c-mg8j-jqc2 / CVE-2026-78175 (WordPress plugin, deserialization, 2026-09-12): no single identifiable fix diff: in themeum/tutor the withdraw-field fix was committed (`0f0b2b20`), reverted (`0f730b86`), re-applied as an empty commit (`1da79c03`) and followed by `642c8f5c`, all on a branch merged with several unrelated security fixes. _The Tutor LMS – eLearning and online course solution plugin for WordPress is vulnerable to_
- GHSA-4q25-cq4p-f83q / CVE-2026-78006 (WordPress plugin, deserialization, 2026-09-12): repo over ~150 MB (the-events-calendar/the-events-calendar, 153 MB per GitHub API). _The The Events Calendar plugin for WordPress is vulnerable to Remote Code Execution in all_
- GHSA-w3rx-h7m8-2ggf / CVE-2026-84099 (WordPress plugin, deserialization, 2026-09-12): advisory names no vulnerable function or file (an unnamed bundled add-on). _The wpstorecart WordPress plugin through 5.0.7 does not prevent direct, unauthenticated ac_
- GHSA-mv63-gmf8-5x76 / CVE-2026-82845 (WordPress plugin, deserialization, 2026-09-12): advisory names no vulnerable function or file. _The Masteriyo LMS WordPress plugin before 3.4.1 does not prevent user-supplied values hel_
- GHSA-3237-mv84-xmm9 / CVE-2026-82925 (WordPress plugin, deserialization, 2026-09-10): advisory names no vulnerable function or file. _The Site Reviews WordPress plugin before 8.3.0 does not prevent request data from being de_
- GHSA-3xv5-w7r3-9wj8 (FileRun, deserialization, 2026-09-10): proprietary product, no public repository.
- GHSA-7jv8-f84m-pq5j / CVE-2026-11363 (WordPress plugin, deserialization, 2026-09-09): advisory names no vulnerable function or file; no public development repository. _The Ninja Forms – The Contact Form Builder That Grows With You plugin for WordPress is vul_
- GHSA-2phq-cm26-22pm (MaxSite CMS, deserialization, 2026-09-09): no fix diff (maxsite/cms has no fix commit after the 2026-09-08 disclosure).
- GHSA-mmpp-99gg-2rq8 / CVE-2026-16502 (WordPress plugin, deserialization, 2026-09-08): advisory names no vulnerable function or file. _The Live Composer – Free WordPress Website Builder plugin for WordPress is vulnerable to P_
- GHSA-v38w-pf7m-c3pr / CVE-2026-10196 (WordPress plugin, deserialization, 2026-09-05): function named (`handle_form_submission`) but no public development repository (GitHub has only an SVN mirror, common-repository/mail-mint, last push 2024). _The Mail Mint – Email Marketing, Newsletter, Email Automation & WooCommerce Emails plugin _
- GHSA-fq25-6gpx-5vxf / CVE-2026-19887 (WordPress plugin, deserialization, 2026-09-05): function named (`usces_action_acting_transaction`) but no development repository (only SVN mirrors of usc-e-shop). _The Welcart e-Commerce plugin for WordPress is vulnerable to PHP Object Injection in all v_
- GHSA-xw87-h6v2-j87m / CVE-2023-3360 (WordPress plugin, deserialization, 2026-09-02): republished old vulnerability (CVE-2023-3360). _The Weaver Show Posts WordPress plugin before 1.8.1 unserialises the content of an importe_
- GHSA-f6hp-2qj5-48vv / CVE-2026-19116 (WordPress plugin, deserialization, 2026-09-02): advisory names no vulnerable function or file. _The User Frontend WordPress plugin before 4.3.11 does not prevent user-supplied field val_
- GHSA-w8mx-gm67-wrqq / CVE-2026-76547 (WordPress plugin, deserialization, 2026-08-29): advisory names no vulnerable function or file. _The User Profile Builder WordPress plugin before 4.0.1 does not validate the type of data_
- GHSA-j5mw-m2j3-5747 / CVE-2026-14558 (WordPress plugin, deserialization, 2026-08-28): advisory names no vulnerable function or file. _The User Frontend WordPress plugin before 4.3.10 does not properly validate field type de_
- GHSA-rm66-h7v3-77rx / CVE-2026-78572 (WordPress plugin, deserialization, 2026-08-25): advisory names no vulnerable function or file. _The Kalles Addons plugin for WordPress is vulnerable to PHP Object Injection in all versio_
- GHSA-89vc-p8p8-f8q8 / CVE-2026-32563 (WordPress plugin, deserialization, 2026-08-25): advisory names no vulnerable function or file (Patchstack one-line record). _Subscriber PHP Object Injection in ACPT (Pro) - Custom Post Types Plugin for WordPress <= _
- GHSA-mj2c-4qfw-rhhc (TYPO3 extension (unnamed), deserialization, 2026-08-25): advisory names neither the extension's repository nor a function.
- GHSA-r2vh-j53r-xpr5 (TYPO3 extension (Apache Solr for TYPO3), deserialization, 2026-08-25): no function or file named (only content object types), no fix link.
- GHSA-fgg6-3rw5-h3j8 / CVE-2026-0551 (WordPress plugin, deserialization, 2026-08-23): advisory names no vulnerable function or file (only the 'post_protection_roles' parameter). _The PPWP – Password Protect Pages plugin for WordPress is vulnerable to PHP Object Injecti_
- GHSA-hpc2-mqfw-fq4q / CVE-2026-4703 (WordPress plugin, deserialization, 2026-08-22): advisory names no vulnerable function or file. _The WS Form LITE – Drag & Drop Contact Form Builder plugin for WordPress is vulnerable to _
- GHSA-p6x4-h8m9-9mr2 / CVE-2024-13784 (WordPress plugin, deserialization, 2026-08-16): republished old vulnerability (CVE-2024-13784); advisory names no vulnerable function or file. _The Contact Form, Survey, Quiz & Popup Form Builder – ARForms plugin for WordPress is vuln_
- GHSA-v2rq-f773-w46v / CVE-2026-10035 (WordPress plugin, deserialization, 2026-08-16): function named (`wvrbbp_set_to_serialized_values()`) but no development repository (only the SVN mirror common-repository/weaver-for-bbpress). _The Turnkey bbPress by WeaverTheme plugin for WordPress is vulnerable to PHP Object Inject_
- GHSA-q83h-6954-x259 / CVE-2026-16099 (WordPress plugin, deserialization, 2026-08-16): **taken** as wp-podlove-generation-guard-poi. _The Podlove Podcast Publisher plugin for WordPress is vulnerable to arbitrary file deletio_
- GHSA-jxc3-pfg8-hv86 / CVE-2026-16267 (WordPress plugin, deserialization, 2026-08-08): advisory names no vulnerable function or file. _The Newsletters WordPress plugin before 4.16 does not restrict the classes allowed when un_
- GHSA-m7vw-4232-p79x / CVE-2026-16258 (WordPress plugin, deserialization, 2026-08-07): advisory names no vulnerable function or file. _The Ajax Search Lite WordPress plugin before 4.14.5 does not prevent the deserialization _
- GHSA-rm37-v7mc-6795 (Cotonti, deserialization, 2026-08-05): third Cotonti comments deserialization record; no merged fix (see lane B).
- GHSA-jm66-348j-qvv2 (MaxSite CMS, deserialization, 2026-08-04): no fix diff.
- GHSA-5953-j744-chvm / CVE-2026-16297 (WordPress plugin, deserialization, 2026-08-03): advisory names no vulnerable function or file. _The Clearfy Cache WordPress plugin before 2.4.3 does not restrict the classes allowed whe_
- GHSA-w2rh-rmh8-jmwq / CVE-2025-15672 (WordPress plugin, deserialization, 2026-08-03): republished old vulnerability (CVE-2025-15672); advisory names no vulnerable function or file. _The ChamaWP WordPress plugin before 1.0.13 does not properly validate user input before p_
- GHSA-xh9j-gpmv-5c2v / CVE-2026-16062 (WordPress plugin, deserialization, 2026-08-02): advisory names no vulnerable function or file. _The Event Booking Manager for WooCommerce WordPress plugin before 5.3.7 does not prevent _
- GHSA-rpvq-6gf9-58vg / CVE-2026-12720 (WordPress plugin, deserialization, 2026-07-31): advisory names no vulnerable function or file; themeum/kirki has no 6.x code (v2 research). _The Kirki WordPress plugin before 6.0.13 does not restrict which classes may be instantia_
- GHSA-95g3-wxff-cpfp / CVE-2026-1360 (WordPress plugin, deserialization, 2026-07-30): function named (`bp_unserialize_profile_field()`), but no fix commit is identifiable in buddypress/buddypress (history of `src/bp-xprofile/bp-xprofile-functions.php` from 2026-06 shows only coding-standard and PHPDoc commits); the repo is taken anyway by wp-buddypress-messages-rest-idor. _The BuddyPress plugin for WordPress is vulnerable to Deserialization of Untrusted Data in _
- GHSA-4f62-78r3-84fq (Joomla extension Aimy Captcha-Less Form Guard, deserialization, 2026-07-29): commercial extension, no public repository.
- GHSA-2hhm-rw5g-m2vq / CVE-2026-15962 (WordPress plugin, deserialization, 2026-07-26): advisory names no vulnerable function or file. _The Fluent Forms Pro Add On Pack plugin for WordPress is vulnerable to PHP Object Injectio_
- GHSA-m5q4-xwgp-qwg2 / CVE-2026-15008 (WordPress plugin, deserialization, 2026-07-16): fix only inside a whole-release commit (UncannyOwl/Uncanny-Automator `ec5fde67` 'v7.4.0'). _The Uncanny Automator – Easy Automation, Integration, Webhooks & Workflow Builder Plugin p_
- GHSA-cxqr-jhpf-63mm / CVE-2026-12583 (WordPress plugin, deserialization, 2026-07-14): advisory names no vulnerable function or file. _The Newsletters WordPress plugin before 4.15 does not prevent deserialization of untrusted_
- GHSA-ww5c-p3qw-8xxf / CVE-2026-12240 (WordPress plugin, deserialization, 2026-06-30): no development repository on GitHub (only unrelated or stale repositories named export-user-data). _The Export User Data plugin for WordPress is vulnerable to arbitrary file deletion due to _
- GHSA-46p4-p677-cfjc / CVE-2025-69130 (WordPress plugin, deserialization, 2026-06-17): advisory names no vulnerable function or file (Patchstack one-line record, a commercial theme). _Subscriber PHP Object Injection in Entrepreneur - Booking for Small Businesses WordPress T_
- GHSA-v8h4-8wcw-7gjp / CVE-2026-12115 (WordPress plugin, deserialization, 2026-06-17): advisory names no vulnerable function or file. _The Counter Box – Add Countdowns, Timers & Dynamic Counters to WordPress plugin for WordPr_
- GHSA-r622-c48h-h6qq / CVE-2026-8365 (WordPress plugin, deserialization, 2026-06-09): function named (`blocksy_sanitize_post_meta_options()`) but no public development repository for the Blocksy theme was found. _The Blocksy theme for WordPress is vulnerable to PHP Object Injection leading to Remote Co_
- GHSA-4jj4-xwhh-x5m2 / CVE-2026-7566 (WordPress plugin, deserialization, 2026-06-06): excluded by name (LearnPress add-on); advisory names no vulnerable function or file. _The LearnPress – Backup & Migration Tool plugin for WordPress is vulnerable to PHP Object _
- GHSA-h6x4-229g-5f43 / CVE-2026-7654 (WordPress plugin, deserialization, 2026-06-06): **taken** as wp-admin-columns-ids-to-collection-poi. _The Admin Columns plugin for WordPress is vulnerable to PHP Object Injection leading to Re_
+581
View File
@@ -0,0 +1,581 @@
{
"schema_version": 1,
"experiment": "independent-population-v1-evaluation",
"protocol": "protocol-v1.md",
"analyzer_commit": "ebcc2cf7fd847aa6ff7a5d08b6d9dce340f5ed4f",
"started": "2026-09-24T16:45:45+02:00",
"population": "population-v1.toml",
"summary": {
"recall": [
1,
11
],
"recall_wilson95": [
0.016,
0.377
],
"precision": [
1,
12
],
"precision_wilson95": [
0.015,
0.354
],
"fixed_side_alerts": 4,
"benign_alerts": 1,
"cases_with_zero_completed_questions_at_vulnerable_pin": [
"phpmyfaq-captcha-ua-sqli",
"yeswiki-bazar-form-sqli",
"signalk-remote-conn-ssrf",
"saltcorn-sync-sqli",
"mongo-express-bson-rce",
"mockoon-admin-api-cors",
"alerta-query-sqli",
"wger-next-open-redirect"
]
},
"m4_gates": {
"precision>=0.95": false,
"recall>=0.90": false,
"completion>=0.95": false,
"zero_fixed_or_benign_alerts": false
},
"reading": "The first measurement on untouched code fails every M4 gate. One of eleven targets is detected (Signal K's SSRF), eleven of twelve reported findings are false, and four fixed-side and one benign false alerts are raised. The dominant cause is not precision but coverage: at the vulnerable pin eight of eleven cases completed no question at all (Saltcorn asked none), so most targets were never decided. That is the first thing to investigate; nothing here may inform a change before this record is committed.",
"cases": [
{
"id": "wp-ultimate-member-sqli",
"family": "injection",
"language": "php",
"detected": false,
"detected_sites": [],
"unanswered": false,
"instrument_errors": {},
"fixed_side_alerts": [],
"benign_alerts": [],
"completion": {
"vulnerable": [
76,
303
],
"fixed": [
76,
303
],
"benign_base": [
71,
229
],
"benign_tip": [
71,
229
]
},
"seconds": {
"vulnerable": 215.8,
"fixed": 218.0,
"benign_base": 199.3,
"benign_tip": 202.8
}
},
{
"id": "wp-learnpress-sqli",
"family": "injection",
"language": "php",
"detected": false,
"detected_sites": [],
"unanswered": false,
"instrument_errors": {},
"fixed_side_alerts": [],
"benign_alerts": [],
"completion": {
"vulnerable": [
4815,
5000
],
"fixed": [
4815,
5000
],
"benign_base": [
4815,
5000
],
"benign_tip": [
4815,
5000
]
},
"seconds": {
"vulnerable": 414.2,
"fixed": 470.5,
"benign_base": 418.1,
"benign_tip": 467.7
}
},
{
"id": "phpmyfaq-captcha-ua-sqli",
"family": "injection",
"language": "php",
"detected": false,
"detected_sites": [],
"unanswered": true,
"instrument_errors": {},
"fixed_side_alerts": [],
"benign_alerts": [],
"completion": {
"vulnerable": [
0,
5000
],
"fixed": [
0,
5000
],
"benign_base": [
0,
5000
],
"benign_tip": [
0,
5000
]
},
"seconds": {
"vulnerable": 437.7,
"fixed": 426.2,
"benign_base": 416.4,
"benign_tip": 425.7
}
},
{
"id": "yeswiki-bazar-form-sqli",
"family": "injection",
"language": "php",
"detected": false,
"detected_sites": [],
"unanswered": true,
"instrument_errors": {},
"fixed_side_alerts": [],
"benign_alerts": [],
"completion": {
"vulnerable": [
0,
1128
],
"fixed": [
0,
1128
],
"benign_base": [
0,
1128
],
"benign_tip": [
0,
1128
]
},
"seconds": {
"vulnerable": 334.9,
"fixed": 306.9,
"benign_base": 304.3,
"benign_tip": 305.3
}
},
{
"id": "signalk-remote-conn-ssrf",
"family": "untrusted_destination",
"language": "typescript",
"detected": true,
"detected_sites": [
"src/serverroutes.ts:2506:<lambda>1"
],
"unanswered": false,
"instrument_errors": {},
"fixed_side_alerts": [],
"benign_alerts": [
"src/serverroutes.ts:2577:<lambda>1"
],
"completion": {
"vulnerable": [
0,
525
],
"fixed": [
0,
529
],
"benign_base": [
0,
533
],
"benign_tip": [
0,
533
]
},
"seconds": {
"vulnerable": 313.3,
"fixed": 319.5,
"benign_base": 334.3,
"benign_tip": 315.4
}
},
{
"id": "saltcorn-sync-sqli",
"family": "injection",
"language": "javascript",
"detected": false,
"detected_sites": [],
"unanswered": true,
"instrument_errors": {
"vulnerable": "no question was asked",
"fixed": "no question was asked",
"benign_base": "no question was asked",
"benign_tip": "no question was asked"
},
"fixed_side_alerts": [],
"benign_alerts": [],
"completion": {
"vulnerable": [
null,
0
],
"fixed": [
null,
0
],
"benign_base": [
null,
0
],
"benign_tip": [
null,
0
]
},
"seconds": {
"vulnerable": null,
"fixed": null,
"benign_base": null,
"benign_tip": null
}
},
{
"id": "mongo-express-bson-rce",
"family": "injection",
"language": "javascript",
"detected": false,
"detected_sites": [],
"unanswered": true,
"instrument_errors": {},
"fixed_side_alerts": [
"lib/bson.js:32:toString",
"lib/bson.js:8:toBSON"
],
"benign_alerts": [],
"completion": {
"vulnerable": [
0,
40
],
"fixed": [
0,
39
],
"benign_base": [
0,
39
],
"benign_tip": [
0,
39
]
},
"seconds": {
"vulnerable": 47.9,
"fixed": 48.5,
"benign_base": 47.5,
"benign_tip": 50.3
}
},
{
"id": "mockoon-admin-api-cors",
"family": "config_secrets",
"language": "typescript",
"detected": false,
"detected_sites": [],
"unanswered": true,
"instrument_errors": {},
"fixed_side_alerts": [],
"benign_alerts": [],
"completion": {
"vulnerable": [
0,
450
],
"fixed": [
0,
453
],
"benign_base": [
0,
453
],
"benign_tip": [
0,
453
]
},
"seconds": {
"vulnerable": 150.9,
"fixed": 158.5,
"benign_base": 146.4,
"benign_tip": 144.7
}
},
{
"id": "alerta-query-sqli",
"family": "injection",
"language": "python",
"detected": false,
"detected_sites": [],
"unanswered": true,
"instrument_errors": {},
"fixed_side_alerts": [],
"benign_alerts": [],
"completion": {
"vulnerable": [
0,
261
],
"fixed": [
0,
261
],
"benign_base": [
0,
261
],
"benign_tip": [
0,
261
]
},
"seconds": {
"vulnerable": 114.8,
"fixed": 111.7,
"benign_base": 109.9,
"benign_tip": 112.9
}
},
{
"id": "wger-next-open-redirect",
"family": "untrusted_destination",
"language": "python",
"detected": false,
"detected_sites": [],
"unanswered": true,
"instrument_errors": {},
"fixed_side_alerts": [
"wger/core/views/languages.py:131:use_browser_language",
"wger/core/views/user.py:211:trainer_login"
],
"benign_alerts": [],
"completion": {
"vulnerable": [
0,
683
],
"fixed": [
0,
683
],
"benign_base": [
0,
683
],
"benign_tip": [
0,
683
]
},
"seconds": {
"vulnerable": 156.5,
"fixed": 160.0,
"benign_base": 160.8,
"benign_tip": 164.7
}
},
{
"id": "opencve-notification-ssrf",
"family": "untrusted_destination",
"language": "python",
"detected": false,
"detected_sites": [],
"unanswered": false,
"instrument_errors": {},
"fixed_side_alerts": [],
"benign_alerts": [],
"completion": {
"vulnerable": [
14,
27
],
"fixed": [
13,
26
],
"benign_base": [
13,
26
],
"benign_tip": [
13,
26
]
},
"seconds": {
"vulnerable": 129.7,
"fixed": 132.4,
"benign_base": 144.3,
"benign_tip": 139.4
}
}
],
"precision_review": [
{
"case": "signalk-remote-conn-ssrf",
"site": "src/index.ts:916:<lambda>6",
"family": "untrusted_destination",
"rung": "model_entailed",
"witness": "req.headers.host?.split(':')[0] -> res.redirect(`https://${host}:${redirectPort}${req.path}`) in src/index.ts (line 916, 5 steps)",
"reached_from": 1,
"verdict": "FP",
"fp_class": "not_attacker",
"reason": "HTTP->HTTPS redirect built from the request's own Host header, which a browser sets to the real target"
},
{
"case": "signalk-remote-conn-ssrf",
"site": "src/oidc/oidc-auth.ts:303:<lambda>2",
"family": "untrusted_destination",
"rung": "model_entailed",
"witness": "req.headers[\"x-forwarded-proto\"] -> res.redirect(authUrl) in src/oidc/oidc-auth.ts (line 303, 11 steps)",
"reached_from": 1,
"verdict": "FP",
"fp_class": "guarded",
"reason": "authorization URL from provider config; the request redirect is validated by isSafeRelativeUrl"
},
{
"case": "signalk-remote-conn-ssrf",
"site": "src/oidc/oidc-auth.ts:490:<lambda>6",
"family": "untrusted_destination",
"rung": "model_entailed",
"witness": "req.query.redirect -> res.redirect(postLogoutRedirect) in src/oidc/oidc-auth.ts (line 490, 22 steps)",
"reached_from": 1,
"verdict": "FP",
"fp_class": "guarded",
"reason": "postLogoutRedirect = isSafeRelativeUrl(req.query.redirect) ? ... : '/'"
},
{
"case": "signalk-remote-conn-ssrf",
"site": "src/oidc/oidc-auth.ts:501:<lambda>6",
"family": "untrusted_destination",
"rung": "model_entailed",
"witness": "req.query.redirect -> res.redirect(postLogoutRedirect) in src/oidc/oidc-auth.ts (line 501, 22 steps)",
"reached_from": 1,
"verdict": "FP",
"fp_class": "guarded",
"reason": "same isSafeRelativeUrl guard"
},
{
"case": "signalk-remote-conn-ssrf",
"site": "src/oidc/oidc-auth.ts:508:<lambda>6",
"family": "untrusted_destination",
"rung": "model_entailed",
"witness": "req.query.redirect -> res.redirect(postLogoutRedirect) in src/oidc/oidc-auth.ts (line 508, 22 steps)",
"reached_from": 1,
"verdict": "FP",
"fp_class": "guarded",
"reason": "same isSafeRelativeUrl guard"
},
{
"case": "signalk-remote-conn-ssrf",
"site": "src/serverroutes.ts:2506:<lambda>1",
"family": "untrusted_destination",
"rung": "model_entailed",
"witness": "req.body -> protocol.request(options, (response) => {\n let data = ''\n response.on('data', (chunk: string) => {\n data += chunk\n })\n response.on('end', () => {\n resolve({ status: r in src/serverroutes.ts (line 2506, 16 steps)",
"reached_from": 1,
"verdict": "TP",
"fp_class": "",
"reason": "the declared CVE-2026-55591 sink: req.body host/port to protocol.request, unauthenticated by default"
},
{
"case": "signalk-remote-conn-ssrf",
"site": "src/serverroutes.ts:457:<lambda>24",
"family": "untrusted_destination",
"rung": "model_entailed",
"witness": "(_tmp_69 = (req.headers?.host || '')\n .split(':')[0]\n .split('.')[0]).toLowerCase -> res.redirect(app.config.settings.landingPage || landingPage) in src/serverroutes.ts (line 457, 26 steps)",
"reached_from": 1,
"verdict": "FP",
"fp_class": "not_attacker",
"reason": "redirect target is the configured landing page; the Host header only selects nothing"
},
{
"case": "signalk-remote-conn-ssrf",
"site": "src/tokensecurity.ts:677:<lambda>5",
"family": "untrusted_destination",
"rung": "model_entailed",
"witness": "req.body.destination -> res.redirect(getSafeDestination(req.body.destination)) in src/tokensecurity.ts (line 677, 12 steps)",
"reached_from": 1,
"verdict": "FP",
"fp_class": "guarded",
"reason": "getSafeDestination allows only relative paths and rejects //"
},
{
"case": "yeswiki-bazar-form-sqli",
"site": "includes/entities/ConfigurationFile.php:69:load",
"family": "injection",
"rung": "model_entailed",
"witness": "$this->_file -> eval($content) in includes/entities/ConfigurationFile.php (line 69, 5 steps)",
"reached_from": 1,
"verdict": "FP",
"fp_class": "not_attacker",
"reason": "eval of the site's own configuration file"
},
{
"case": "yeswiki-bazar-form-sqli",
"site": "includes/services/DbService.php:46:initSqlConnection",
"family": "injection",
"rung": "model_entailed",
"witness": "$this->params -> mysqli_query($this->link, \"SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci\") in includes/services/DbService.php (line 46, 9 steps)",
"reached_from": 2,
"verdict": "FP",
"fp_class": "constant_query",
"reason": "the SQL text is a constant SET NAMES; taint reaches only the connection handle"
},
{
"case": "yeswiki-bazar-form-sqli",
"site": "setup/install.php:113:<global>",
"family": "injection",
"rung": "model_entailed",
"witness": "$_POST[\"config\"] -> mysqli_query($dblink, \"SET NAMES utf8mb4 COLLATE utf8mb4_general_ci\") in setup/install.php (line 113, 19 steps)",
"reached_from": 1,
"verdict": "FP",
"fp_class": "constant_query",
"reason": "constant SET NAMES; taint reaches only the connection handle"
},
{
"case": "yeswiki-bazar-form-sqli",
"site": "setup/install.php:54:<global>",
"family": "injection",
"rung": "model_entailed",
"witness": "$_POST[\"config\"] -> mysqli_query($dblink, \"CREATE DATABASE \" . $config[\"mysql_database\"]) in setup/install.php (line 54, 16 steps)",
"reached_from": 1,
"verdict": "FP",
"fp_class": "no_boundary",
"reason": "installer: whoever submits it must already supply the database password; borderline, counted false"
}
]
}
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More