mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 20:49:28 +00:00
Compare commits
293
Commits
v1.2.0-alpha.1
...
v2.0.0
@@ -0,0 +1,27 @@
|
||||
# Build context for the root Dockerfile. The image copies pyproject.toml, README.md, src/, benchmarks/ and
|
||||
# ops/frontend-retention/install.py; everything below is either never copied or is evidence the image does not
|
||||
# need. plane/Dockerfile.runner has its own plane/Dockerfile.runner.dockerignore.
|
||||
|
||||
# Local state, never part of an image (agent worktrees alone run to hundreds of MB; .env holds keys).
|
||||
.git
|
||||
.claude
|
||||
.venv
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
**/__pycache__
|
||||
**/*.pyc
|
||||
.mypy_cache
|
||||
.ruff_cache
|
||||
.pytest_cache
|
||||
.openultrasast
|
||||
reports
|
||||
*.sarif
|
||||
|
||||
# The measurement archive (about 20 MB) is evidence, not runtime input. These records stay:
|
||||
# ops/smoke_report.py reads the two 2026-09-13 ones from /app inside the image (ops/README.md).
|
||||
benchmarks/measurements/*
|
||||
!benchmarks/measurements/2026-09-13-delta-comparison-smoke.json
|
||||
!benchmarks/measurements/2026-09-13-admission-policy-smoke.json
|
||||
# A label source of `ousast learn` (src/openultrasast/learn/sources.toml).
|
||||
!benchmarks/measurements/2026-09-30-harnessx-removal-baseline/pairs.json
|
||||
+21
-9
@@ -1,12 +1,24 @@
|
||||
# Copy to .env and fill in. .env is gitignored.
|
||||
OPENROUTER_API_KEY=
|
||||
# Copy to .env and fill in. .env is gitignored and never committed.
|
||||
# Note: values in .env never override variables already exported in your shell
|
||||
# (e.g. from ~/.bashrc). Unset a stale export, or the shell value wins.
|
||||
|
||||
# Prove-budget and `ousast index` embeddings via OpenRouter /embeddings.
|
||||
# LLM calls (verify, roles, the decision engine): DeepSeek.
|
||||
DEEPSEEK_API_KEY=
|
||||
# DEEPSEEK_BASE_URL=https://api.deepseek.com
|
||||
|
||||
# Embeddings only: OpenAI text-embedding-3-small through OpenRouter.
|
||||
OPENROUTER_API_KEY=
|
||||
OPENROUTER_EMBEDDING_MODEL=openai/text-embedding-3-small
|
||||
|
||||
# Any OpenAI-compatible chat endpoint works for the hunter: set the key above to that
|
||||
# provider's key and override the base URL. Example for DeepSeek:
|
||||
# OPENROUTER_API_KEY=sk-deepseek-...
|
||||
# OPENROUTER_BASE_URL=https://api.deepseek.com/v1
|
||||
# Then set [models].hunter in openultrasast.toml (or --hunter-model) to that provider's model id.
|
||||
OPENROUTER_BASE_URL=
|
||||
# Plane memory store. Default is a local FileStore under ~/ousast-results/plane/memory.
|
||||
# For an S3-compatible server (RustFS is the tested one): OUSAST_MEMORY=s3://<bucket>[/<prefix>], or
|
||||
# OUSAST_MEMORY=s3:// to use S3_BUCKET. Needs the s3 extra (boto3). Example for a RustFS server:
|
||||
# OUSAST_MEMORY=s3://sast-memory
|
||||
# S3_ENDPOINT=https://files.example.com
|
||||
# S3_REGION=us-east-1
|
||||
# S3_BUCKET=sast-memory
|
||||
# AWS_ACCESS_KEY_ID=
|
||||
# AWS_SECRET_ACCESS_KEY=
|
||||
# AWS_SESSION_TOKEN=
|
||||
# The bucket must have versioning enabled, a lifecycle rule expiring runs/, S3 Select and readable object tags
|
||||
# (see docs/rustfs.md); the store verifies them at startup and refuses to run otherwise.
|
||||
|
||||
@@ -33,3 +33,6 @@ vector-store/
|
||||
# own scratch dir so it never lands here, but a maintainer querying a CPG by hand from the repo root
|
||||
# will create one.
|
||||
workspace/
|
||||
|
||||
# MkDocs build output (mkdocs build)
|
||||
site/
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
# Brief: an AI service plane for OpenUltraSAST (Kubernetes-style, at the model level)
|
||||
|
||||
Status: brief for maintainer decision (2026-09-29). Not a spec yet; no approvals reset.
|
||||
|
||||
## Why now
|
||||
|
||||
Two measurements say the current pipeline cannot be developed further at its token cost:
|
||||
|
||||
- Development: 90% of the tokens of the last session were tool I/O (inline scripts and their output), because
|
||||
every stage lives in ad-hoc scripts run from one long conversation (`dev-tokens-are-tool-io` memory).
|
||||
- Pipeline: verification costs $0.009-0.022 per candidate for two passes because every hunt re-derives the same
|
||||
per-repository facts and every pass re-sends its whole conversation; resume, budgets and pass agreement are
|
||||
hand-rolled per script (`results-v2-model-pipeline.json`).
|
||||
|
||||
Google's `ax` (open-sourced 2026-09; Apache-2, Go, early) frames agent work as a Kubernetes-style service:
|
||||
declarative `Task` / `Workspace` / `Model` manifests, a reconciler, sandboxed isolated execution, suspend and
|
||||
resume, and a runner contract any binary can honour. Its software needs Kubernetes and Agent Substrate, lists
|
||||
only Google and Anthropic providers, and has no pipelines, artifacts or budgets yet. Its *design* is what we
|
||||
adopt; its software is a later deployment target.
|
||||
|
||||
## The model
|
||||
|
||||
| ax primitive | OpenUltraSAST meaning |
|
||||
|---|---|
|
||||
| `Workspace` | one pinned checkout: repository, commit, product-file filter; materialised once, bound by many tasks |
|
||||
| `Model` | provider, model id, credentials, prices, per-task budget (usd, calls); DeepSeek via an adapter |
|
||||
| `Task` | one concern, isolated: declared inputs (artifacts), declared outputs, resource limits, idempotent per unit, resumable |
|
||||
| runner contract | our Python entrypoint reads the task and workspace YAML from the environment and writes artifacts |
|
||||
| reconciler | a local loop over manifests today (one engine container at a time on the 7 GB host); `ax` later, unchanged manifests |
|
||||
|
||||
What ax lacks and we add as a thin layer: a `Run` manifest that lists tasks with dependencies (a DAG), artifact
|
||||
locations, and budgets. This layer is what the reconciler executes; the tasks themselves are ax-shaped.
|
||||
|
||||
## The concerns as tasks
|
||||
|
||||
1. `classify-sinks` -- workspace -> `candidates.json` (model-read, no tools). Exists: `model_sinks.py`.
|
||||
2. `triage` -- candidates -> `kept.json` with `data_origin` per function (no tools). Exists inside `verify_batched.py`.
|
||||
3. `repo-facts` -- workspace -> `facts.json`: per-file callers and entry points, computed once, no model. New;
|
||||
this is the memory layer, and it is an artifact, not a cache.
|
||||
4. `verify` -- kept + facts, `pass=n` -> `verdicts.jsonl`, one hunt per file, budgeted. Exists as a script.
|
||||
5. `judge` -- verdicts -> `agreed.json`: agreement across passes, then a no-tools judgement over each agreed
|
||||
finding's rationale and code. Agreement exists; the judge does not.
|
||||
6. `recheck-fixed` -- fixed workspace + agreed -> `fixed_alerts.json`. Exists (`--fixed`).
|
||||
7. `score` -- all of the above + the population record -> `score.json`. Exists (`score_batched.py`).
|
||||
|
||||
Each task: one manifest, one entrypoint, one fixture test, one budget. A task that fails leaves no partial
|
||||
truth: it is unfinished, never "clean" (the 402 lesson).
|
||||
|
||||
## What it buys, measured or estimated
|
||||
|
||||
- Tokens per task bounded by declared inputs; `repo-facts` replaces the 1-2 tool turns per hunt that today
|
||||
re-grep callers (each turn re-sends the whole context); estimated 20-40% per hunt, to be measured on the
|
||||
46-candidate validation set.
|
||||
- Two passes and the judge become tasks with their own budgets instead of loops inside a script.
|
||||
- Development in small contexts: one task type at a time, against a fixture; the transcript measure is the
|
||||
check.
|
||||
- Portability: the same manifests run on the laptop reconciler now and on `ax` on a cluster later.
|
||||
|
||||
## What it does not buy
|
||||
|
||||
Recall and precision come from the model and the questions asked, not from orchestration. The open problems
|
||||
recorded in the strategy review (judge quality, the config-family question, pass stability) remain the
|
||||
detection work; this design makes them affordable to attack.
|
||||
|
||||
## Risks
|
||||
|
||||
- `ax` is early ("heavy development", "major breaking changes"); we depend on its shape, not its releases.
|
||||
- Its provider list excludes DeepSeek; an adapter task image is ours to keep.
|
||||
- A local reconciler is new code; it must stay thin (manifests, DAG, artifacts, budgets, resume) or it becomes
|
||||
the next thing that eats tokens.
|
||||
|
||||
## Decisions needed
|
||||
|
||||
1. Adopt this as a spec (`/kiro-spec-init` on this brief) and pause further script-level pipeline changes.
|
||||
2. Manifest format: ax's `ax.io/v1alpha1` kinds verbatim plus our `Run` kind, so the later move is a rename of
|
||||
the reconciler, not of the manifests.
|
||||
3. First increment: `repo-facts` + `verify` as tasks under the local reconciler, measured against the
|
||||
46-candidate set for cost and recall before the other concerns move.
|
||||
@@ -0,0 +1,254 @@
|
||||
# Design Document
|
||||
|
||||
## Overview
|
||||
|
||||
A thin service plane under `src/openultrasast/plane/`: manifest schemas in ax's shape plus a `Run` kind, an
|
||||
ax-backed reconciler that submits a Run's tasks to ax on this host's kind cluster in dependency order with
|
||||
persisted status and resume, a runner that honours ax's contract and delivers artifacts, and task entrypoints.
|
||||
ax (with Agent Substrate) is the only executor; there is no local subprocess path. The pipeline's logic lives only in the task entrypoints; the
|
||||
reconciler and runner know nothing about prompts, models or scoring (Requirement 3.4). The first increment moves
|
||||
`repo-facts` and `verify` (plus a model-free `agree` step) and measures them on the 46-candidate set
|
||||
(Requirement 6); the other concerns stay as scripts until that measurement is recorded.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
plane/ manifests (YAML), checked in
|
||||
models/deepseek-flash.yaml Model
|
||||
workspaces/<case>-<pin>.yaml Workspace, generated from population-v2.toml by `ousast plane workspaces`
|
||||
tasks/repo-facts.yaml Task
|
||||
tasks/verify.yaml Task
|
||||
runs/validation-46.yaml Run: DAG, artifacts, budgets
|
||||
src/openultrasast/plane/
|
||||
manifests.py load + validate Task / Workspace / Model / Run (PyYAML, declared dependency)
|
||||
reconciler.py Run execution, status, resume (<= 500 lines, test-enforced)
|
||||
runner.py ax runner contract: AX_TASK_YAML / AX_WORKSPACES_YAML -> entrypoint
|
||||
doctor.py `ousast plane doctor`: the four checks of Requirement 3.5
|
||||
budget.py spend and call metering from provider usage fields
|
||||
tasks/repo_facts.py source-only facts artifact
|
||||
tasks/verify.py one hunt per file with facts in the prompt; per-unit resume
|
||||
tasks/agree.py pass agreement, no model
|
||||
~/ousast-results/plane/<run>/ state.json + one directory per task with summary.json and artifacts
|
||||
```
|
||||
|
||||
Locally the reconciler runs a task as a subprocess of the same interpreter with the ax environment set; in a
|
||||
container the same module is PID 1 (`ax-task-runner`), built as a target of the root `Dockerfile` that extends
|
||||
`openultrasast:dev`. Nothing in a task depends on which of the two started it.
|
||||
|
||||
## Components and Interfaces
|
||||
|
||||
### Manifests (`manifests.py`)
|
||||
|
||||
- `Task`, `Workspace`, `Model`: `apiVersion: ax.io/v1alpha1`, fields exactly as ax documents; unknown fields
|
||||
are rejected with the path named. `Model.spec.provider` accepts ax's `google` and `anthropic`, and `deepseek`
|
||||
only when `metadata.annotations["openultrasast.io/provider-extension"] == "true"` (Requirement 1.4): the
|
||||
deviation is declared, not hidden in another field.
|
||||
- Field names are ax's verbatim (`ax.proto`; its API server decodes with `protojson`, so an unknown field fails
|
||||
`ax apply`): `metadata` is `name` and `atespace` (not `namespace`), and a Workspace `git` entry is `name`,
|
||||
`repo`, `branch`, `dir`, `depth` (no `commit`; a `commit` there is rejected with a pointer to the annotation).
|
||||
`metadata.annotations` is this project's only extension, and nothing under it reaches ax: the reconciler strips
|
||||
it and carries it to the task as env. The Model binding (`openultrasast.io/model`) becomes `OUSAST_MODEL` and
|
||||
`OUSAST_MODEL_PARAMS`. Commit pins (`openultrasast.io/git-commits: "<git name>=<40-hex sha>,..."` on a
|
||||
Workspace) become `OUSAST_GIT_PINS` = `{"<workspace>/<git name>": "<sha>"}`, which the runner checks out after
|
||||
fetching the way ax's `internal/workspace/setup.go` does, with the same `dir` and `depth` rules.
|
||||
- `Run`: `apiVersion: openultrasast.io/v1alpha1`. `spec.tasks[]`: `name`, `task` (a Task manifest name),
|
||||
`dependsOn[]`, `inputs{name: "<task>/<artifact>"}`, `outputs[]`, `budget{usd, calls}`, `serialize` (a label
|
||||
for tasks that must not overlap; engine-bearing tasks carry `engine`). The reconciler passes budgets and
|
||||
artifact paths to a task as `env` entries (`OUSAST_BUDGET_USD`, `OUSAST_BUDGET_CALLS`, `OUSAST_INPUT_<NAME>`,
|
||||
`OUSAST_OUTPUT_DIR`), which is ax's own mechanism for task configuration, so a Run never adds a field to a
|
||||
Task. Inputs another task produced are never rendered into a Workspace (ax puts every bound Workspace into one
|
||||
env value, `AX_WORKSPACES_YAML`, and Agent Substrate rejects values over 32768 characters): the Task carries
|
||||
`OUSAST_INPUTS` = `{"<NAME>": "<producer>/<artifact>"}`, and after the start request the runner fetches each
|
||||
from the receiver's `GET /inputs/<producer>/<artifact>` (same headers, dial and retries as delivery; only a
|
||||
running task's declared inputs are served) to its `OUSAST_INPUT_<NAME>` path; a failed fetch delivers a
|
||||
failed summary naming the input and the command never runs.
|
||||
|
||||
### Runner (`runner.py`)
|
||||
|
||||
Reads `AX_TASK_YAML` and `AX_WORKSPACES_YAML`, materialises each workspace's `git` entries at the pinned commit
|
||||
into the bound path (locally a `git archive` from the case cache, as `evaluate.export` does today), writes
|
||||
`spec.files`, then runs `spec.command` as ax's runner contract requires (ax `docs/runner.md`,
|
||||
`docs/sandbox.md`): a child process `python -m openultrasast.plane.tasks.<name> <args>` in its own process group,
|
||||
with the first workspace as working directory and `AX_METADATA_URL` pointing at the runner's own server. It serves
|
||||
`/healthz`, `/readyz` (503 until workspaces are ready; ax probes `/readyz?check=workspace`) and ax's two metadata
|
||||
paths on port 80 (Requirement 4.2). Workspaces are prepared on the first boot only; `/workspace` survives suspend
|
||||
and resume. After the child exits the runner reads `summary.json` (a crash without one becomes `failed` with the
|
||||
exit code and stderr tail), delivers the output directory (Requirement 4.4), writes the completion marker
|
||||
`/workspace/.ousast-state/<task>.done.json` (`exit`, `status`, `delivered`) and then keeps running as PID 1 until
|
||||
SIGTERM: an exiting runner stops the container, which the first live run showed as every resume timing out. A
|
||||
boot that finds the marker does not rerun the command (a resumed actor must not repeat billed work); a marker
|
||||
with a failed delivery retries the delivery only. SIGTERM (ax's stop and suspend) is forwarded to the command's
|
||||
process group, followed by SIGKILL after `OUSAST_TERM_GRACE` (10 s), and the runner exits 0.
|
||||
`AX_RUNNER_EXIT_AFTER_COMMAND=1` is a unit-test escape that exits with the task's code after delivery.
|
||||
|
||||
The command never starts by itself (Requirements 4.5, 4.6). Once the workspaces are ready the runner serves
|
||||
`POST /ousast/v1/start` on port 80 with the body `{"run", "task", "credentials"}`: 503 before the workspaces are
|
||||
ready (the reconciler retries), 409 when `run`/`task` differ from `OUSAST_RUN` and the Task's name, 409 for a
|
||||
second request on the same boot, 409 with the reason when the completion marker records a delivered run, 400 for
|
||||
a malformed body or a credential name outside `^[A-Z][A-Z0-9_]*$` (or under `AX_`/`OUSAST_`), and 202 once.
|
||||
Only then does the command run -- or, when the marker records a failed delivery, only the delivery is retried; a
|
||||
runner failure (bad manifests, a workspace that cannot be materialised) is likewise delivered only after a start,
|
||||
or refused with its reason. The credentials stay in the runner's memory and are merged into the child's
|
||||
environment only; they are never logged or written, and their values are redacted from the child's echoed
|
||||
stderr and the stderr tail a crash summary keeps. `AX_RUNNER_AUTOSTART=1` (unit tests only; required with
|
||||
`AX_RUNNER_HTTP=0`) starts without a request.
|
||||
|
||||
**Golden snapshot.** Agent Substrate boots every new ActorTemplate once as a temporary "golden" actor (atespace
|
||||
`ate-golden`, a UUID name) to capture a snapshot. ax bakes `AX_TASK_YAML`, `AX_WORKSPACES_YAML` and `spec.env`
|
||||
into the template, and inside the sandbox the hostname is always `actor` with no identity variable, so the golden
|
||||
boot and the real one are indistinguishable from within (observed on the live cluster, 2026-09-29: the command
|
||||
ran in both). The distinction exists only outside: Substrate's router (`svc/atenet-router` in `ate-system`)
|
||||
routes on `ate-target-actor: <atespace>/<name>` to the task's actor, resuming it if suspended, and never to the
|
||||
golden one. Hence the start request, and hence credentials only in it: a template, a rendered Task or an ax
|
||||
manifest is visible to the golden boot and to anyone who can read the cluster's objects.
|
||||
|
||||
### Reconciler (`reconciler.py`, ax-backed)
|
||||
|
||||
`ousast plane run <Run.yaml>`, `ousast plane status <run>` and `ousast plane doctor`. State: `state.json` with
|
||||
per-task status in `pending | running | suspended | done | failed | unfinished`, timestamps, spend. Execution:
|
||||
topological order; for each ready task the reconciler renders the Task manifest with the run's env
|
||||
(`OUSAST_BUDGET_*`, `OUSAST_INPUT_*`, `OUSAST_OUTPUT_DIR`, `OUSAST_ARTIFACT_URL`), applies it and its
|
||||
Workspaces through the `ax` CLI (`ax apply`, `ax resume`, `ax get`, `ax delete`; the CLI path is injectable so
|
||||
tests use a fake). ax creates a Task `Suspended`; it runs only after `ax resume task`, and the first resume of a
|
||||
new image can fail with `DeadlineExceeded` while Agent Substrate builds its golden snapshot (ax then reports
|
||||
`Failed`), so the reconciler retries a DeadlineExceeded/Unavailable resume with backoff for up to
|
||||
`OUSAST_RESUME_TIMEOUT` (900 s). ax has no Completed phase and never reports that a command exited: completion is
|
||||
the artifact delivery. `Failed` after a resume went through, or the task disappearing, before delivery fails the
|
||||
task with ax's condition message; no delivery within `OUSAST_TASK_TIMEOUT` (7200 s) fails it too; `ax delete task`
|
||||
follows either way. Artifacts: the reconciler serves a small HTTP receiver on the host (reachable from actors
|
||||
by name through the egress gateway, see Egress below); the runner posts the output directory there after its command exits. Tasks sharing a
|
||||
`serialize` label never overlap; everything else may run in parallel up to `--workers`. A task's outcome is
|
||||
read from its delivered `summary.json`: `done` only when the task says every unit finished; `unfinished` on a
|
||||
budget stop; `failed` on an account or authentication error, a crash, or a missing delivery. `status` prints
|
||||
the token attribution table (Requirement 7.3) from the delivered summaries and writes `attribution.json`. On rerun, `done` tasks are skipped and
|
||||
the others re-executed; per-unit resume is the task's own duty (Requirement 2.2). No prompts, scoring or model
|
||||
calls anywhere in this module; a test counts its lines.
|
||||
|
||||
Start (Requirements 4.5, 4.6; `router.py`, kept out of the reconciler's line budget): for the whole run the
|
||||
reconciler holds one `kubectl --context kind-ousast -n ate-system port-forward svc/atenet-router :80` (random local
|
||||
port read from kubectl's `Forwarding from 127.0.0.1:NNNN`; the child is terminated when the run ends;
|
||||
`OUSAST_ROUTER_URL` replaces it, as in the tests). When ax reports `Running` after an accepted resume -- again
|
||||
after a re-resume of a suspended task, where an "already started on this boot" answer means the command
|
||||
survived and still runs -- it posts the start body with `ate-target-actor: <atespace>/<task>`,
|
||||
retrying 502/503/504 and connection errors with backoff up to `OUSAST_START_TIMEOUT` (300 s); a 409 or the timeout
|
||||
fails the task with the response text unless it already delivered. Credentials come from the Model binding: the
|
||||
bound Model's `spec.secretKey.key` names the variable (`DEEPSEEK_API_KEY` for `deepseek-flash`), read from the
|
||||
reconciler's own environment (`ousast` loads `.env` at startup); a Model-bound task whose variable is unset fails
|
||||
before `ax apply`, and a task without a Model gets none. The value is never rendered, logged or written.
|
||||
|
||||
#### Egress (`egress.py`)
|
||||
|
||||
Every actor connection except DNS goes through Agent Substrate's egress gateway (`atenet-egress`), which denies
|
||||
an actor without an `EgressPolicy`. Three pieces make the plane work under that rule:
|
||||
|
||||
- **Receiver by name.** `ops/ax/receiver-service.yaml.tmpl` (applied by `up.sh`) is a selector-less Service
|
||||
`ousast-receiver` in `ax-system`, port 80, with an EndpointSlice on the kind gateway at `OUSAST_ARTIFACT_PORT`
|
||||
(fixed, 18090), where the reconciler binds its receiver. Tasks get
|
||||
`OUSAST_ARTIFACT_URL=http://ousast-receiver.ax-system.svc.cluster.local/` plus `OUSAST_ARTIFACT_DIAL=<the
|
||||
Service's ClusterIP>:80` (looked up with kubectl unless set): the actor cannot resolve cluster names, so the
|
||||
runner connects to the dial address and sends the URL's host as `Host`. The gateway decides the policy on
|
||||
`Host` but connects to the address the actor dialled (measured: dialling the kind gateway on port 80 was allowed
|
||||
and then refused upstream), so the dial address is the ClusterIP, which kube-proxy maps to the host.
|
||||
`OUSAST_ARTIFACT_HOST` set keeps the old direct URL (tests).
|
||||
- **Per-task policy.** `policy_for` grants `http` on port 80 to the receiver name, and `tls_passthrough` on 443
|
||||
to the Git hosts of the bound Workspaces' `https` repos and to the bound Model's
|
||||
`openultrasast.io/egress-hosts` (`api.deepseek.com` for `deepseek-flash`, validated as a plain DNS name).
|
||||
Nothing else: no IP address or wildcard, and passthrough only -- an `https` rule intercepts TLS and the actor
|
||||
would need the gateway CA, which ax templates do not mount.
|
||||
- **Order.** `ax apply` creates the actor; the reconciler writes the policy with `kubectl ate create
|
||||
egress-policy` (update when one exists), reads it back, waits `OUSAST_EGRESS_SETTLE` (11 s: the gateway caches a
|
||||
missing policy as deny for 10 s), then resumes. kubectl-ate has no delete verb; Substrate's store deletes the
|
||||
policy with its actor, so after `ax delete task` the reconciler confirms it is gone.
|
||||
|
||||
### Budget (`budget.py`)
|
||||
|
||||
Wraps a chat client: sums the provider's usage fields per call (`prompt_tokens`, `prompt_cache_hit_tokens`,
|
||||
`completion_tokens`), prices them from the Model's parameters, raises `BudgetExhausted` at the ceiling and
|
||||
`AccountError` on 401/402 or an "insufficient balance" body. A model without prices makes the meter report
|
||||
`unpriced` and refuse to run under a `usd` budget (Requirement 2.4).
|
||||
|
||||
### Task contract (all tasks)
|
||||
|
||||
Inputs from `OUSAST_INPUT_*` paths and the bound workspace only; outputs under `OUSAST_OUTPUT_DIR`; progress in a
|
||||
`units.jsonl` appended per finished unit and read back on start; `summary.json` written last with
|
||||
`status`, `units_done`, `units_total`, `usd`, `calls`, `usage`. A task never writes `summary.json` with
|
||||
`status: done` unless `units_done == units_total`.
|
||||
|
||||
### `repo-facts` (`tasks/repo_facts.py`)
|
||||
|
||||
Source only, no model. For every product file (the `enumerate_source_files` + product filter of
|
||||
`model_sinks.py`): the functions it defines (the per-language declaration patterns of `sink_candidates.py`),
|
||||
and for each function name in the candidate list (input `candidates.json`) the call sites in other product
|
||||
files: `path`, `line`, the enclosing function, capped at 40 per name. Output `facts.json`, keyed by file,
|
||||
sorted, deterministic for a workspace (Requirement 5.2).
|
||||
|
||||
### `verify` (`tasks/verify.py`)
|
||||
|
||||
The hunt of `verify_batched.py` (system prompt, one hunt per file, up to 6 candidates, 6 steps, candidate-
|
||||
anchored sites) moved into the package, with two changes: the prompt lists each candidate's known call sites
|
||||
from `facts.json` ("Known callers: a.py:12 in handler(), ..."), and each hunt records its tool turns and usage
|
||||
in `units.jsonl`. `pass` is an env value from the Run, so two passes are two task instances with distinct output
|
||||
directories. Candidates and triage come from the existing `independent-v2-modelsinks` and batched-check
|
||||
artifacts for the increment; the `classify` and `triage` tasks are later specs.
|
||||
|
||||
### `agree` (`tasks/agree.py`)
|
||||
|
||||
Reads the pass outputs, emits `agreed.json` and `disputed.json` by the same rule as `verify_batched.agreement`,
|
||||
plus the per-candidate cost, turns and the declared-site match (`evaluate.matches_v2`) for the measurement.
|
||||
The rule is reproduced in the package (`agree.matches_declared`, parity-tested against `matches_v2` on the recorded
|
||||
reference outputs) and the generator writes the fix ranges it matches against into each case's `case.json`, so
|
||||
the runner image carries no benchmark tree.
|
||||
|
||||
**Measurement.** Requirement 6.2's cost is judged on `cost_per_candidate_with_recorded_triage`: the plane's two
|
||||
verify passes plus the recorded triage cost of the case, over the candidates before triage, summed across the run.
|
||||
The reference's $0.022 (`verifier-batched-check-2026-09-29.json`) paid for per-file triage and the hunts while the
|
||||
plane applies the recorded triage without a model call, so `cost_per_candidate` (plane spend alone) is reported but
|
||||
not judged; the triage share is derived per file from the recorded spend minus the priced hunt usage (about $0.087
|
||||
of the $1.01) and labelled as derived in `case.json`. Sites are judged as `declared_sites_matched_agreed` against
|
||||
`declared_sites_in_set`, the reference's 16 of 20.
|
||||
The tie-break (Requirement 6.4) appends per case `<case>-vc`, `verify` pass c restricted by `OUSAST_INPUT_ONLY`
|
||||
to the agree task's `disputed.json` (an empty list is `done` with no model call), and `<case>-final`, `agree` over
|
||||
passes a, b and c, which decides 2-of-3 where pass c asked and keeps the a/b rule elsewhere, so pass c alone never
|
||||
decides and `metrics.tiebreak` counts the flips; the first 60 entries stay byte-identical, so a rerun keeps their
|
||||
`done` state, executes only the 30 new tasks, and pass c's cost adds to `usd_total` for the 6.2 judgement.
|
||||
|
||||
## Data Models
|
||||
|
||||
- `facts.json`: `{ "<path>": { "functions": ["name", ...], "callers": { "<fn>": [ {"path", "line", "in"} ] } } }`
|
||||
- `units.jsonl` (verify): one row per file hunt: `{"path", "candidates", "flagged": [...], "turns", "usage",
|
||||
"usd"}`; rows with `"error"` are re-asked on resume.
|
||||
- `summary.json`: `{"status", "units_done", "units_total", "usd", "calls", "usage", "reason"?}`
|
||||
- `state.json` (run): `{"run", "started", "tasks": {"<name>": {"status", "started", "finished", "usd"}}}`
|
||||
|
||||
## Error Handling
|
||||
|
||||
- Manifest errors stop before any task starts, naming the manifest, field and rule.
|
||||
- `AccountError` inside a task: the task writes `summary.json` with `status: failed` and the provider's
|
||||
message, exits 2; the reconciler marks the run `failed` and starts nothing further (Requirement 2.3).
|
||||
- `BudgetExhausted`: `status: unfinished`, exit 3; dependants are not started; a rerun with a larger budget
|
||||
continues from `units.jsonl`.
|
||||
- A crash: `failed`, traceback in `summary.json`; `units.jsonl` is still valid for resume.
|
||||
- Two runs of the same Run name never share a directory: the reconciler refuses to start a second run while one
|
||||
is `running` (a lock file with the PID), which also keeps engine tasks to one at a time on this host.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- `tests/test_plane_manifests.py`: valid ax manifests load; an unknown field, a bad provider without the
|
||||
annotation, and a Run with a cycle are rejected with the field named.
|
||||
- `tests/test_plane_reconciler.py`: a fake `ax` CLI with ax's real lifecycle (apply -> `Suspended`, a first
|
||||
resume failing DeadlineExceeded, then `Running` forever, and a scripted `summary.json` delivery only after a
|
||||
resume went through) drives a DAG: order, `serialize` non-overlap, skip-on-rerun, `unfinished` and `failed`
|
||||
propagation, resume retry and timeout, `Failed` with a condition message, a vanished task, the task timeout
|
||||
without delivery, the lock, the attribution table; a
|
||||
test asserts the module is under 500 lines. `tests/test_plane_ax_live.py` (marker `ax`, skipped unless
|
||||
`ousast plane doctor` passes) applies one trivial Task on the real cluster.
|
||||
- `tests/test_plane_runner.py`: the env contract on a fixture workspace; `/readyz` 503 then 200; the command as a
|
||||
child in its own group in the first workspace; the runner serving on after the command; SIGTERM reaching the
|
||||
command's group; the completion marker preventing a rerun and retrying a failed delivery alone.
|
||||
- `tests/test_plane_repo_facts.py`: a three-file tree; callers found across files, none from tests or vendor;
|
||||
identical output on a second run.
|
||||
- `tests/test_plane_verify.py`: a scripted client; the prompt carries the callers; `units.jsonl` resume asks
|
||||
only the unfinished file; a scripted 402 yields `failed`.
|
||||
- Measurement (Requirement 6): `plane/runs/validation-46.yaml` executed twice (passes a and b); the result
|
||||
compared with `verifier-batched-check-2026-09-29.json` and recorded in `benchmarks/independent/`, with the
|
||||
transcript measure of the increment's development session next to it.
|
||||
@@ -0,0 +1,157 @@
|
||||
# Requirements Document
|
||||
|
||||
## Introduction
|
||||
|
||||
The model-driven pipeline (classify sinks, triage, verify, judge, recheck, score) is a set of scripts run from
|
||||
one long conversation, each with its own resume, budget and pass logic. Measured 2026-09-29: 90% of development
|
||||
tokens are tool I/O around those scripts, and verification re-derives the same per-repository facts in every
|
||||
hunt and every pass. This feature restructures the pipeline as a Kubernetes-style service plane at the model
|
||||
level, in the shape of google/ax: declarative `Task`, `Workspace` and `Model` manifests, a reconciler that
|
||||
executes them in isolation with resume, and a runner contract. The plane runs on ax itself: a `kind` cluster
|
||||
with Agent Substrate and the ax controller on this host (maintainer decision 2026-09-29: "switch to ax
|
||||
completely now"). What ax does not yet carry -- task dependencies, artifacts, token budgets (on ax's roadmap),
|
||||
a DeepSeek provider -- is carried by our thin `Run` layer and dissolves into ax as ax gains it.
|
||||
|
||||
## Boundary Context
|
||||
|
||||
- **In scope**: the manifest kinds and a `Run` kind for dependencies, artifacts and budgets; bring-up of ax on
|
||||
this host (kind + Agent Substrate + ax controller) with a doctor command; an ax-backed reconciler that
|
||||
submits Tasks to ax (one engine-bearing task at a time) and is the only executor; the runner contract and a
|
||||
runner image; the first two tasks, `repo-facts` and
|
||||
`verify`, with the existing scripts as reference behaviour; measurement on the 46-candidate validation set.
|
||||
- **Out of scope**: a remote or multi-node cluster; contributing providers or budgets to ax upstream (a
|
||||
follow-on once the increment is measured); changing what the model is asked (prompts, families, pass rules)
|
||||
except as required to consume `repo-facts`; the judge stage, the config-family question and population v3
|
||||
(they become tasks later, under their own specs); any change to `evaluate.py`'s protocol scoring.
|
||||
- **Adjacent expectations**: the independent-population protocols and records stay as they are; `finding_dump`
|
||||
and the engine path are untouched; `vulnerabilities-over-plumbing` applies -- this is plumbing that must pay
|
||||
for itself in measured tokens and unchanged or better recall.
|
||||
- **Target state (maintainer, 2026-09-29)**: this plane replaces HarnessX as the agentic plane; HarnessX is
|
||||
removed by its own spec (`harnessx-removal`) once Requirement 6's measurement shows value; the `Run` layer
|
||||
shrinks as ax gains dependencies, budgets and artifacts. No new HarnessX feature is added meanwhile.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement 1: Manifests are ax's kinds plus a Run kind
|
||||
|
||||
**User Story:** As the maintainer, I want the pipeline declared in manifests that ax would accept, so that
|
||||
moving to a cluster later changes the reconciler and not the pipeline.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. `Task`, `Workspace` and `Model` manifests use `apiVersion: ax.io/v1alpha1` and only fields ax documents
|
||||
(`spec.image`, `spec.command`, `spec.env`, `spec.resources`, `spec.workspaces`; `spec.git`, `spec.files`;
|
||||
`spec.provider`, `spec.model`, `spec.secretKey`, `spec.parameters`).
|
||||
2. A `Run` manifest (our kind) lists tasks with `dependsOn`, the artifacts each task produces and consumes, and a
|
||||
per-task budget (`usd`, `calls`); it contains nothing a task needs to execute.
|
||||
3. A manifest that violates its schema is rejected before anything runs, with the field named.
|
||||
4. A DeepSeek model is expressed as a `Model` with a provider adapter noted in the manifest; no ax field is
|
||||
repurposed to carry it.
|
||||
|
||||
### Requirement 2: A task is isolated, idempotent and resumable
|
||||
|
||||
**User Story:** As the maintainer, I want a stopped or failed run to resume without repeating finished work and
|
||||
without ever recording partial work as a result.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. A task reads only its declared inputs and the bound workspace, and writes only its declared artifacts.
|
||||
2. Work is recorded per unit (file, candidate) as it completes; a restarted task skips completed units and
|
||||
repeats unverified ones.
|
||||
3. A task whose model calls fail on account or authentication errors stops at once with a distinct status;
|
||||
its artifacts are marked unfinished, never complete.
|
||||
4. A task's spend is metered from the provider's usage fields and stops the task at its budget; a task with no
|
||||
priced model reports "unpriced", not zero.
|
||||
|
||||
### Requirement 3: An ax-backed reconciler executes a Run on this host's ax
|
||||
|
||||
**User Story:** As the maintainer, I want to run a whole pipeline with one command on the laptop, with every
|
||||
task executed by ax on the local cluster, so that the same manifests move to a bigger cluster unchanged.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. The reconciler submits each task to ax as a `Task` (with its bound `Workspace`s and `Model`) in dependency
|
||||
order, at most one engine-bearing task at a time, reads task outcome from ax's status plus the task's
|
||||
delivered `summary.json`, and passes artifacts between tasks by placing a producer's outputs where the
|
||||
consumer's manifest expects them. It never executes a task as a local subprocess; ax is the only executor.
|
||||
2. Task status (`pending`, `running`, `suspended`, `done`, `failed`, `unfinished`) is persisted and shown by
|
||||
one status command.
|
||||
3. A suspended or interrupted run resumes from persisted status (Requirement 2.2) with the same command.
|
||||
4. The reconciler is under 500 lines and holds no pipeline logic: no prompts, no scoring, no model calls.
|
||||
5. `ousast plane doctor` verifies the host's ax in one command: kind cluster reachable, Agent Substrate control
|
||||
API up, ax controller ready, runner image loaded; each check names what is missing and how to bring it up.
|
||||
6. Bring-up on this host (kind + Agent Substrate + ax) is scripted, documented with its measured memory and
|
||||
CPU footprint, and smoke-tested with one trivial Task before any pipeline task runs.
|
||||
|
||||
### Requirement 4: The runner contract is ax's
|
||||
|
||||
**User Story:** As the maintainer, I want each task entrypoint to run unchanged inside an ax task container
|
||||
later.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. An entrypoint receives its task and workspace manifests as YAML in `AX_TASK_YAML` and `AX_WORKSPACES_YAML`
|
||||
and takes nothing else from the command line except what the manifest's `command` states.
|
||||
2. The runner image serves `/healthz` and `/readyz` on port 80, `/readyz` returning 503 until workspaces are
|
||||
materialised, as ax's runner contract requires; the image contains this package and is loaded into kind.
|
||||
3. One fixture test per task type runs the entrypoint against a small workspace and checks its artifacts.
|
||||
4. Because ax has no artifact channel, the runner delivers the task's output directory (including
|
||||
`summary.json` and `units.jsonl`) to the reconciler's artifact endpoint named in `OUSAST_ARTIFACT_URL`
|
||||
before it reports completion; a delivery failure is reported as `failed`, never as done.
|
||||
5. The command never runs in Agent Substrate's golden-snapshot boot, which executes the same image with the same
|
||||
task environment (observed on the live cluster, 2026-09-29): the runner prepares workspaces, reports ready and
|
||||
waits; the reconciler starts the command with one request to the task's actor through Substrate's router
|
||||
(`ate-target-actor: <atespace>/<task>`), which never addresses the golden actor. A second start request for
|
||||
the same task is refused.
|
||||
6. Provider credentials reach a task only in that start request, held in the runner's memory and passed to the
|
||||
command's environment; they never appear in a manifest, a rendered Task, an ax template or any file.
|
||||
|
||||
### Requirement 5: `repo-facts` is the memory layer
|
||||
|
||||
**User Story:** As the maintainer, I want per-repository facts computed once and reused by every hunt and pass,
|
||||
so that verification stops paying tool turns for the same lookups.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. `repo-facts` produces, without model calls, per product file: the functions defined, and for each candidate
|
||||
function the call sites in other product files (path and line) -- from source text only, never from git.
|
||||
2. The artifact is deterministic for a given workspace and is reused across tasks and passes unchanged.
|
||||
3. `verify` includes each candidate's callers from `repo-facts` in its prompt and reports the number of tool
|
||||
turns per hunt.
|
||||
|
||||
### Requirement 6: The first increment is measured before the rest moves
|
||||
|
||||
**User Story:** As the maintainer, I want proof that the restructuring saves tokens without losing recall
|
||||
before the remaining concerns are converted.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. `repo-facts` + `verify` run on the 46-candidate validation set through the reconciler on ax, two passes, and
|
||||
report cost per candidate, tool turns per hunt and declared sites agreed.
|
||||
2. Acceptance: declared sites agreed at least 16 of 20 (the current value) and cost per candidate below the
|
||||
current $0.022 for two passes; otherwise the increment is reported as not met and the design revisited.
|
||||
3. The development transcript measure (`benchmarks/dev/token_report.py`) is run at the end of the increment and
|
||||
recorded next to the result.
|
||||
4. After the first measurement (plane-increment-1.json: 14/20 agreed, 19/20 in either pass), agreement is decided
|
||||
2-of-3 with a third pass run only on candidates passes a and b dispute; the gate of 6.2 is re-evaluated on that
|
||||
decision, and the result is recorded as plane-increment-2.json next to the first.
|
||||
|
||||
### Requirement 7: Routing is a declared concern of the plane
|
||||
|
||||
**User Story:** As the maintainer, I want which model serves which concern to be a manifest decision with a
|
||||
budget, not a prompt trick inside one conversation, so that token spend is structured by component.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. Every `Task` in a `Run` binds exactly one `Model`; a Run may bind different Models to different tasks
|
||||
(a cheap model for reads and triage, a stronger one for verification and judgement), and the reconciler
|
||||
passes the bound Model's identity and prices to the task.
|
||||
2. A task never chooses or switches its model at runtime; the Model it was bound to is recorded in its
|
||||
`summary.json`.
|
||||
3. `ousast plane status <run>` prints a token attribution table: one row per task and Model binding with
|
||||
calls, input tokens, cache-hit tokens, output tokens and USD, a total row, and a per-Model subtotal; every
|
||||
number comes from the provider's usage fields recorded in the tasks' `summary.json`, never estimated. A
|
||||
`--units` flag breaks a task down per unit (file or candidate) from `units.jsonl`. The same table is
|
||||
written to `attribution.json` in the run directory so runs can be compared.
|
||||
4. A task that needs a second model for a sub-step (a judge over a verifier's output) is a separate task with
|
||||
its own Model binding, never a second client inside one task.
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"feature_name": "ai-service-plane",
|
||||
"created_at": "2026-09-29T15:40:00Z",
|
||||
"updated_at": "2026-09-29T17:55:00Z",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
},
|
||||
"design": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
},
|
||||
"tasks": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": true
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
# Implementation Plan
|
||||
|
||||
Order follows the dependency chain: ax bring-up (parallel with 1-2) -> manifests -> budget -> runner ->
|
||||
ax-backed reconciler -> tasks -> increment run on ax -> measurement. Every task ships with its tests; a task is done when its tests pass in the host suite, ruff and
|
||||
mypy are clean, and the commit is gated on pytest's own exit code. Script-level pipeline work stays paused.
|
||||
|
||||
- [x] 0. ax bring-up on this host
|
||||
- Install `kind`, `kubectl`, the `ax` CLI and `ko` (user-local). `ops/ax/up.sh`: create the kind cluster and
|
||||
install Agent Substrate with its `hack/create-kind-cluster.sh` and `hack/install-ate-kind.sh
|
||||
--deploy-ate-system`, deploy the ax control plane with `ko` to the kind registry, apply one trivial Task
|
||||
and wait for it to finish; `ops/ax/down.sh` tears it down. Record the measured idle footprint (RSS, CPU)
|
||||
in `ops/ax/README.md` next to the 7 GB host constraint.
|
||||
- `ousast plane doctor` (in `reconciler.py` or a sibling `doctor.py`): the four checks of Requirement 3.5.
|
||||
- Egress: `up.sh` applies Substrate's agentgateway egress gateway (`atenet-egress`, prebuilt image, rendered
|
||||
with `kubectl kustomize`) when absent, and the receiver Service `ousast-receiver.ax-system`
|
||||
(`receiver-service.yaml.tmpl`, endpoint the kind gateway at port 18090). The reconciler writes each task's
|
||||
egress policy (`egress.py`: receiver over HTTP, Git and Model hosts as TLS passthrough) between `ax apply` and
|
||||
resume; the runner delivers to `OUSAST_ARTIFACT_DIAL` (the Service's ClusterIP, port 80) with the receiver's name as
|
||||
`Host`.
|
||||
- Stop rule: if the cluster does not come up within one working session, report the exact failure and stop;
|
||||
the decision to work around it is the maintainer's.
|
||||
- _Requirements: 3.5, 3.6_
|
||||
|
||||
- [x] 1. Manifest schemas and validation
|
||||
- Add `pyyaml` as a declared dependency in `pyproject.toml`.
|
||||
- `src/openultrasast/plane/manifests.py`: dataclasses and loaders for `Task`, `Workspace`, `Model`
|
||||
(`ax.io/v1alpha1`, ax's fields only) and `Run` (`openultrasast.io/v1alpha1`: tasks, dependsOn, inputs,
|
||||
outputs, budget, serialize); reject unknown fields, a provider outside ax's two without the
|
||||
`openultrasast.io/provider-extension` annotation, and a Run with a cycle or an input naming no task.
|
||||
- `tests/test_plane_manifests.py`: valid manifests load; each rejection names the field.
|
||||
- _Requirements: 1.1, 1.2, 1.3, 1.4_
|
||||
|
||||
- [x] 2. Budget metering
|
||||
- `src/openultrasast/plane/budget.py`: a client wrapper that sums provider usage per call, prices from the
|
||||
Model's parameters, raises `BudgetExhausted` at the ceiling and `AccountError` on 401/402 or an
|
||||
"insufficient balance" body; an unpriced model reports `unpriced` and refuses a `usd` budget.
|
||||
- Tests with a scripted client: ceiling stop, account error, unpriced refusal, cache-hit pricing.
|
||||
- _Requirements: 2.3, 2.4_
|
||||
|
||||
- [x] 3. Runner contract
|
||||
- `src/openultrasast/plane/runner.py`: read `AX_TASK_YAML` / `AX_WORKSPACES_YAML`; materialise `git`
|
||||
entries at the pinned commit from the local case cache and write `spec.files` (first boot only); run
|
||||
`spec.command` as a child `python -m openultrasast.plane.tasks.<name>` in its own process group, cwd the
|
||||
first workspace, `AX_METADATA_URL` set; `/healthz` `/readyz` and ax's metadata paths on port 80 (503 until
|
||||
workspaces are ready; disabled only under `AX_RUNNER_HTTP=0` for unit tests); after the command exits,
|
||||
POST the output directory as a tar stream to `OUSAST_ARTIFACT_URL` (report `failed` if delivery fails),
|
||||
write `/workspace/.ousast-state/<task>.done.json`, and keep serving until SIGTERM (ax's contract for PID 1);
|
||||
a boot with the marker never reruns the command and retries only a failed delivery; SIGTERM goes to the
|
||||
command's group, SIGKILL after `OUSAST_TERM_GRACE`, exit 0. `AX_RUNNER_EXIT_AFTER_COMMAND=1` (tests only)
|
||||
exits after delivery with 0 done, 2 failed, 3 unfinished. The command never starts by itself: after the
|
||||
workspaces are ready the runner waits for `POST /ousast/v1/start` (`{run, task, credentials}`; 503 before
|
||||
ready, 409 on a run/task mismatch, a second request, or a marker recording a delivered run; 202 once),
|
||||
merges the credentials into the child's environment only and redacts them from its echoed stderr, so
|
||||
Agent Substrate's golden-snapshot boot runs nothing (Req 4.5, 4.6). `AX_RUNNER_AUTOSTART=1` (tests only)
|
||||
starts without a request. `plane/Dockerfile.runner`: this package on Python 3.12, installed to
|
||||
`/usr/local/bin/ax-task-runner`, loaded into kind by `ops/ax/up.sh`.
|
||||
- `tests/test_plane_runner.py`: the env contract on a fixture workspace; readyz 503 then 200; delivery to a
|
||||
local receiver; delivery failure yields `failed`; the child's cwd and `AX_METADATA_URL`; the runner stays
|
||||
up after the command; SIGTERM kills a long-running command's group; the marker prevents a rerun; no command
|
||||
before a start request, 409 on a wrong run/task and on a second start, the credential reaches the child
|
||||
(its hash in the artifact) and no file or log record, a crash printing it is redacted, and of two runners
|
||||
with identical env only the addressed one runs (the golden scenario).
|
||||
- _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5, 4.6_
|
||||
|
||||
- [x] 4. ax-backed reconciler and CLI
|
||||
- `src/openultrasast/plane/reconciler.py` (under 500 lines, no pipeline logic, no local subprocess
|
||||
executor): `state.json`, statuses, topological submission through the `ax` CLI (path injectable),
|
||||
`ax resume task` after `ax apply` (a created Task is `Suspended`), retrying DeadlineExceeded/Unavailable
|
||||
with backoff up to `OUSAST_RESUME_TIMEOUT`; completion is the artifact delivery, never an ax phase (ax has
|
||||
none for a finished command); `Failed` or a vanished task before delivery, or no delivery within
|
||||
`OUSAST_TASK_TIMEOUT`, fails the task with ax's condition message; the artifact HTTP receiver, `serialize` labels never overlapping, `--workers`, budgets and
|
||||
artifact env passed in the rendered Task manifest, outcome read from the delivered `summary.json`, skip
|
||||
`done` on rerun, PID lock per run, `ax delete` of finished Tasks. Once ax reports `Running` after a
|
||||
resume, one start request per resume through Agent Substrate's router (`src/openultrasast/plane/router.py`:
|
||||
`kubectl port-forward svc/atenet-router` for the run, or `OUSAST_ROUTER_URL`; header
|
||||
`ate-target-actor: <atespace>/<task>`; 502/503/504 and connection errors retried up to
|
||||
`OUSAST_START_TIMEOUT`, a 409 final), carrying the bound Model's `secretKey.key` variable from the
|
||||
reconciler's environment; a Model-bound task whose variable is unset fails before `ax apply`.
|
||||
Task-produced inputs travel as `OUSAST_INPUTS`, not as a rendered Workspace: the runner fetches them after
|
||||
the start from the receiver's authorised `GET /inputs/<producer>/<artifact>` (a 182 KB `facts.json` in
|
||||
`AX_WORKSPACES_YAML` exceeded Substrate's 32768-character env limit on the live cluster, 2026-09-29).
|
||||
- `ousast plane run <Run.yaml>`, `ousast plane status <run>` (token attribution table per task and Model
|
||||
from the tasks' `summary.json` usage; `--units` from `units.jsonl`; also written to `attribution.json`),
|
||||
`ousast plane workspaces <population.toml>` (generates Workspace manifests per case pin).
|
||||
- `tests/test_plane_reconciler.py`: a fake `ax` CLI with ax's real lifecycle (Suspended, a first resume
|
||||
failing DeadlineExceeded, Running without a Completed phase) drives a DAG: order, non-overlap,
|
||||
skip-on-rerun, `unfinished` and `failed` propagation, resume retry and timeout, `Failed` with a condition
|
||||
message, the task timeout without delivery, the lock, the attribution table, and the
|
||||
line-count assertion; with a fake router: one start per task after the accepted resume and before the
|
||||
delivery, the credential in the verify start only and in no file under the run or tmp dirs, a missing
|
||||
secret failing the task, a refused start failing it. `tests/test_plane_router.py`: port parsing from a
|
||||
fake kubectl and its termination on exit, retry on 503/502 then 202, 409 raised with redacted text, the
|
||||
start timeout. `tests/test_plane_ax_live.py` (marker `ax`, skipped without a live cluster): one
|
||||
trivial Task end to end.
|
||||
- _Requirements: 3.1, 3.2, 3.3, 3.4, 2.2, 4.5, 4.6, 7.1, 7.3_
|
||||
|
||||
- [x] 5. `repo-facts` task
|
||||
- `src/openultrasast/plane/tasks/repo_facts.py`: product files, functions per file (per-language
|
||||
declaration patterns), call sites per candidate function across product files (path, line, enclosing
|
||||
function, cap 40), sorted and deterministic; `summary.json` per the task contract.
|
||||
- `tests/test_plane_repo_facts.py`: three-file tree; cross-file callers found, tests and vendor excluded,
|
||||
identical output on a second run.
|
||||
- _Requirements: 5.1, 5.2, 2.1_
|
||||
|
||||
- [x] 6. `verify` and `agree` tasks
|
||||
- `src/openultrasast/plane/tasks/verify.py`: the batched hunt moved from `verify_batched.py` (one hunt per
|
||||
file, up to 6 candidates, 6 steps, candidate-anchored sites) with known callers from `facts.json` in the
|
||||
prompt, tool turns and usage per hunt in `units.jsonl`, per-unit resume, `pass` from env, budget wrapper
|
||||
from task 2; `summary.json` never `done` unless every unit finished, and records the bound Model and
|
||||
the summed usage fields (prompt, cache-hit, output tokens, calls, usd).
|
||||
- `src/openultrasast/plane/tasks/agree.py`: agreement across pass outputs, per-candidate cost and turns,
|
||||
declared-site matching via `evaluate.matches_v2`.
|
||||
- `tests/test_plane_verify.py`: scripted client; prompt carries the callers; resume asks only the
|
||||
unfinished file; a scripted 402 yields `failed`; agree on two scripted passes.
|
||||
- _Requirements: 5.3, 2.1, 2.2, 2.3, 7.2_
|
||||
|
||||
- [x] 7. Manifests for the increment
|
||||
- `plane/models/deepseek-flash.yaml` (annotated extension, prices in parameters);
|
||||
`plane/tasks/repo-facts.yaml`, `plane/tasks/verify.yaml`, `plane/tasks/agree.yaml`;
|
||||
`plane/runs/validation-46.yaml`: repo-facts -> verify pass a, verify pass b -> agree, budgets per task,
|
||||
inputs from the recorded candidate and triage artifacts.
|
||||
- Workspaces generated for the 15 validation cases' vulnerable pins; the runner image rebuilt and loaded.
|
||||
- _Requirements: 1.1, 1.2, 6.1_
|
||||
|
||||
- [x] 8. Measurement of the first increment
|
||||
- Run `validation-46` through the reconciler on ax (needs DeepSeek credit); record cost per candidate, tool turns
|
||||
per hunt, declared sites agreed, against `verifier-batched-check-2026-09-29.json`.
|
||||
- Run `benchmarks/dev/token_report.py` on the increment's development session.
|
||||
- Record `benchmarks/independent/plane-increment-1.json` with the gate result (met or not met) and commit;
|
||||
if not met, revise the design before task 9.
|
||||
- _Requirements: 6.1, 6.2, 6.3_
|
||||
|
||||
- [x] 8.1 Tie-break pass and re-measurement
|
||||
- `verify` takes `OUSAST_INPUT_ONLY` (pass c on the disputed candidates only); `agree` takes
|
||||
`OUSAST_INPUT_PASS_C` and decides 2-of-3 where pass c asked, the a/b rule elsewhere (`metrics.tiebreak`).
|
||||
- `validation-46` gains `<case>-vc` and `<case>-final` per case, appended so the first 60 entries keep their
|
||||
state; an offline replay with an empty pass c reproduces the recorded 14/20.
|
||||
- Rerun `validation-46` on ax (only the new tasks run), re-evaluate the 6.2 gate on the final decision and record
|
||||
`benchmarks/independent/plane-increment-2.json` next to the first.
|
||||
- _Requirements: 6.2, 6.4_
|
||||
|
||||
- [x] 9. Handover
|
||||
- `ops/README.md`: how to run a Run locally and what an ax deployment would need (Kubernetes, Agent
|
||||
Substrate, the runner image, the provider extension); memory note updated; scripts that the tasks replaced
|
||||
marked as reference-only in their docstrings.
|
||||
- _Requirements: 4.2_
|
||||
@@ -1,5 +1,7 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** dormant. Tasks were approved but none of the 24 was started; the absence-bug case it targets is handled in practice by the dominance arbiter (`model/dominance.py`) and `semantic/obligations/`. Reopen it only with a measured gap the arbiter misses.
|
||||
|
||||
## Introduction
|
||||
|
||||
AppSec engineers and maintainers of OpenUltraSAST scan web code, increasingly vibe-coded or agent-written, in which the dominant bugs are absences rather than flows: a handler queries a user-owned record without constraining it to the caller, a privileged route has no guard on its path, an identity is read from the request body instead of the authenticated context, a security control stays at its permissive default. Today the detector, the overlay, and the variant search model only source-to-sink flows, so these pairs score zero and cannot teach a mechanism (6 of 19 non-teaching vibe-py pairs; 15 of 29 agent-vfc rows carry absence labels). This spec adds obligations: an operation that must be discharged by a guard, an identity constraint, or a safe value before it is reached. The application's own policy, recovered from the consistency of its sibling handlers and optionally declared in a versioned file, says which operations carry which obligations. Violations are reported with the missing discharger and its evidence, measured leave-one-out on the absence pairs, and admitted into the detector through the existing improve-loop gate.
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** superseded by `model-grounded-detection` (spec.json `superseded_by`). 4 of 20 tasks were done and the tasks were never approved; none will be. Kept as decision history; nothing in src, tests or CI points here.
|
||||
|
||||
## Introduction
|
||||
|
||||
The detector's instability is structural, not statistical. Measured on 2026-09-06 over twelve injection pairs at
|
||||
|
||||
@@ -51,6 +51,38 @@ for someone who does not have Joern.
|
||||
- Building a sandbox. Req 10 of the predecessor stands: adopt, never build.
|
||||
- Changing the gates, the taxonomy, or the fact-table schema.
|
||||
|
||||
## The fact tables are an ontology, and it is general-purpose — 2026-09-21
|
||||
|
||||
The `[[source]]`, `[[sink]]`, `[[sanitizer]]`, `[[operation]]`, `[[discharger]]` and `[[layout]]` rows
|
||||
are not a pattern list. They are an ontology of program elements and their security relations: what
|
||||
carries untrusted input, what is dangerous under which weakness class, what discharges which
|
||||
obligation, what a value's shape has to be for a call to mean what the table says. The engine reasons
|
||||
over that ontology; it is where nearly all of this tool's accuracy lives, and the 2026-09 family
|
||||
census measured that directly. Five of seven families established nothing, and every one of them
|
||||
failed for an ontology reason with the flow engine ready and nothing to aim at: no HTTP client for a
|
||||
server-side request forgery, no collection read for an object-reference defect, no WordPress spelling
|
||||
for a deserialization, and only browser sinks for a server-rendered injection.
|
||||
|
||||
**This is a general-purpose pre-commit hook, not an application-specific one.** The ontology describes
|
||||
LIBRARIES, FRAMEWORKS and LANGUAGE CONSTRUCTS. It does not describe repositories. That distinction is
|
||||
the difference between a tool that transfers and a set of rules that happen to fit whatever was last
|
||||
measured, and it constrains how the ontology may grow:
|
||||
|
||||
- An entry names an API and is justified by what that API DOES. `needle.get` is admissible because
|
||||
needle is an HTTP client and the argument is a destination, not because NodeGoat calls it.
|
||||
- An entry carries provenance and a weakness class, so a reader can check the claim without trusting
|
||||
the person who added it.
|
||||
- An entry must not encode a path, directory name or identifier peculiar to one project. A plugin's
|
||||
`includes/lib/` was measured and declined on exactly this ground: it would have fixed one subject and
|
||||
claimed something false about every other.
|
||||
- A measured gap is a CANDIDATE, never an answer. The coverage instrument below reports what a
|
||||
repository calls that nothing models; each candidate must then be resolved to a library API with
|
||||
provenance before it enters the ontology. Accretion by subject is how a general tool becomes a
|
||||
fixture of its benchmark.
|
||||
|
||||
The corollary is that ontology growth should be driven by measurement rather than by whatever
|
||||
repository is in front of the maintainer, which is what task 7.1 exists to make possible.
|
||||
|
||||
## Boundary Commitments
|
||||
|
||||
### This Spec Owns
|
||||
|
||||
@@ -89,3 +89,19 @@ The partial-census correction now preserves actual counts and missing filenames,
|
||||
with missing/empty/malformed census still preventing completed questions. Runtime amortization and VAmPI context/witness evidence remain open.
|
||||
|
||||
Source: [Joern 4.0.625 AstGenRunner](https://raw.githubusercontent.com/joernio/joern/v4.0.625/joern-cli/frontends/jssrc2cpg/src/main/scala/io/joern/jssrc2cpg/utils/AstGenRunner.scala).
|
||||
|
||||
## 2026-09-14 post-census measurement and lifecycle experiment
|
||||
|
||||
The full new profile establishes compatible graph/query reuse on identical tips
|
||||
(18.521–18.874s), but all 18 other transactions time out and 0/24 target checks complete.
|
||||
The isolated local-session census probe is promising: four warm requests take 0.221–0.467s
|
||||
versus 14.292s for disposable execution, with confirmed hanging-operation cancellation
|
||||
and process-group reaping in 0.043s. Startup and load still cost 12.805s and 3.636s.
|
||||
The first transport attempt correctly failed on HTTP success without a response payload;
|
||||
all attempts are retained. No production session integration or query equivalence is claimed.
|
||||
|
||||
[Remediation research](release-remediation-research.md) records the exact results and
|
||||
the shared context-contract gaps: missing non-taint family producers, global vendor and
|
||||
line-correspondence vetoes, and unresolved query-to-operation identity. Preserve the gaps
|
||||
until actual question-owned dependency/correspondence evidence can resolve them. Task 8.3
|
||||
and independent qualification remain blocked; the alpha's eligibility remains disabled.
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
# Release remediation research — 2026-09-14
|
||||
|
||||
Status: investigation under the maintainer's instruction to continue towards release.
|
||||
This record does not approve rollout, replace the existing design, or mark a production
|
||||
remediation complete. The published candidate is `v1.2.0-alpha.1`, commit `04f8880`.
|
||||
|
||||
## Summary
|
||||
|
||||
Two independent constraints remain after task 2.19 restored Node's complete named census:
|
||||
engine lifecycle cost and the scope of completeness evidence. A faster engine cannot
|
||||
repair missing context; suppressing completeness gaps cannot prove useful detection.
|
||||
Both belong in the existing contributor-scan boundary, with comparison/admission changes
|
||||
owned by pre-push-safety-net. The ranker continues to select all analysis work.
|
||||
|
||||
## Engine lifecycle investigation
|
||||
|
||||
The earlier task 2.3 decision rejected server mode on the measured large C workload:
|
||||
evaluation dominated its elapsed time. That decision remains valid for that measurement.
|
||||
It does not establish the best lifecycle for small, frequently changed API repositories.
|
||||
|
||||
The fresh Node priming transaction reopened the declared inputs and completed in 338.157s.
|
||||
Its two frontends took 5.536s and 5.451s; overlays 15.779s and 15.239s; standalone census
|
||||
13.716s and 13.708s. Each of eight query invocations took another 13.264–22.783s. These
|
||||
are individual process receipts, not estimates of exclusive JVM startup time. The remaining
|
||||
transaction time includes Python orchestration, cache publication and other work and must
|
||||
not be attributed to the JVM without a separate measurement.
|
||||
|
||||
The first identical-tip replay took 18.874s with two graph hits, 1,512 query hits, and no
|
||||
engine subprocesses. Thus compatible persistence now works on this workload. Every head
|
||||
question still has incomplete change context; base questions have `graph_incomplete` from
|
||||
the vendor boundary described below. Identical-tip timing is not changed-code qualification.
|
||||
|
||||
Joern documents a local interpreter server with synchronous and asynchronous query APIs:
|
||||
[official server contract](https://docs.joern.io/server/). This supplies a candidate transport,
|
||||
not a sandbox. Before adopting it, probe the actual pinned 4.0.625 image in a disposable,
|
||||
network-disabled container with no published ports, a private graph copy and process-group
|
||||
cleanup. Compare an actual named census against disposable script execution, record startup,
|
||||
load and repeated request times separately, and test cancellation of an in-flight request.
|
||||
A census microbenchmark cannot establish security-query equivalence or hook p95.
|
||||
|
||||
### Completed lifecycle probe
|
||||
|
||||
[Full receipts and all three instruments](../../../benchmarks/measurements/2026-09-14-joern-session-lifecycle.json)
|
||||
retain the failed first attempt as well as both successful file-receipt attempts.
|
||||
The first attempt received HTTP `success=true` with an empty stdout for the census;
|
||||
the `println` payload went to the process log. It failed rather than interpreting that
|
||||
response as a zero. The repaired instrument writes a fresh private result file per
|
||||
request and checks a unique receipt before accepting its census.
|
||||
|
||||
The final attempt reopened all 44 source files (95,411 bytes), verified the cached graph
|
||||
digest, loaded a private copy, and matched names, counts and overlays against disposable
|
||||
execution. It confirmed entry into a deliberately hanging operation before cancellation,
|
||||
then verified the entire process group disappeared. The cached graph remained unchanged.
|
||||
|
||||
| Final attempt phase | Observed seconds |
|
||||
|---|---:|
|
||||
| Disposable census invocation | 14.292 |
|
||||
| Server startup through usable request | 12.805 |
|
||||
| Graph load | 3.636 |
|
||||
| First census request | 1.402 |
|
||||
| Four warm census requests | 0.339, 0.467, 0.353, 0.221 |
|
||||
| Kill and process-group reaping | 0.043 |
|
||||
|
||||
Decision: a transaction-owned local session is a promising candidate for the owner
|
||||
implementation. The adapter needs explicit request receipts; HTTP status alone is not
|
||||
an answer contract. This probe does not cover full taint/dominance/config equivalence,
|
||||
multiple graph epochs, concurrent transactions, overlays in-session or production
|
||||
failure recovery. It neither overturns the large-C evaluation result nor proves 30s p95.
|
||||
|
||||
### Completed full runtime profile
|
||||
|
||||
The [post-census scorecard](../../../benchmarks/measurements/2026-09-14-nodegoat-post-census-feasibility.json)
|
||||
and its compressed 115-file evidence bundle retain all 21 planned transactions plus
|
||||
priming, raw scans, process traces, CLI output and exact instruments. All 22 scan artifacts
|
||||
match the frozen runtime/core provenance; recomputing the scorecard reproduced its result.
|
||||
There were no missing cases or instrument failures. Eighteen transactions timed out;
|
||||
the three identical-tip cases took 18.521–18.874s. Completion remains 0/24 target checks;
|
||||
maximum observed cancellation/report overrun was 0.8732s. Runtime and rollout remain NO-GO.
|
||||
|
||||
Verification: seven runtime-scorecard tests pass; JSON/bundle round-trip, hashes, population,
|
||||
provenance and scorecard recomputation pass. Production source is unchanged from the
|
||||
released alpha; this is evidence validation, not a new full implementation verification.
|
||||
|
||||
## Completeness diagnosis
|
||||
|
||||
### Missing family context producer
|
||||
|
||||
`model/scan.py:_evidence_pass` dispatches only taint queries and records their context by
|
||||
the exact family-specific question identity. `affected_context` requires `context_summary`
|
||||
and readable method extents for every question. Neither `dominance.sc` nor `config.sc`
|
||||
exports that contract. Consequently access-control and configuration questions cannot
|
||||
complete change context through the production path. This is deterministic missing
|
||||
integration, not evidence that the ranker selected the wrong region.
|
||||
|
||||
Do not copy taint context into other families: dominance deliberately evaluates sibling
|
||||
operations across a file; configuration has different scope; source, sink and guard
|
||||
summaries have different obligations. A producer must describe the actual scope of the
|
||||
query it accompanies. Missing extents, unresolved external calls and unsupported reasoning
|
||||
remain explicit. Context relationships alone never prove an exploitable flow or guard loss.
|
||||
|
||||
### Exclusion is currently a global completion veto
|
||||
|
||||
`model/partitions.py` records `vendor_semantics_unresolved` when it excludes any vendor
|
||||
path. `model/scan.py` includes that reason in `graph_gaps`, changing every otherwise
|
||||
completed outcome to `graph_incomplete`. In the fresh Node prime, this is the only base
|
||||
degradation, despite a complete 44-file first-party census and successful query receipts.
|
||||
|
||||
Vendor exclusion must remain physical. The missing contract is which question actually
|
||||
requires excluded semantics. Distinguish input-integrity failures, which invalidate the
|
||||
affected graph, from an excluded dependency whose relevance needs query evidence. Only
|
||||
an explicit, complete dependency proof or versioned summary can discharge that question's
|
||||
boundary; lack of a recorded dependency is not such proof.
|
||||
|
||||
### Line ambiguity propagates beyond the affected correspondence
|
||||
|
||||
`push/snapshot.py` conservatively omits non-unique unchanged lines and records a file-level
|
||||
`ambiguous_line_correspondence`. `model/regions.py:affected_context` inherits any existing
|
||||
boundary into every question. `push/policy.py:compare_evidence` also rejects a candidate
|
||||
when any transaction context boundary exists. Repeated unchanged lines can therefore
|
||||
invalidate an otherwise distinct, uniquely mapped operation elsewhere.
|
||||
|
||||
Retain ambiguous mappings as unavailable. A future repair must bind the required base/head
|
||||
correspondence to the specific operation and dependency context, with absent, renamed,
|
||||
removed-guard and ambiguous-operation controls. Aggregate coverage must retain every gap.
|
||||
Do not delete the ambiguity check or treat lexical alignment as a semantic proof.
|
||||
|
||||
Comparison currently also requires all selected base questions to complete, no base
|
||||
degradations, and no non-tier-zero deferred question. That prevents a complete target
|
||||
comparison from surviving an unrelated unsupported family. Any amendment must establish
|
||||
the complete base counterpart and its required scope explicitly; a missing raw finding
|
||||
in an incomplete base scan cannot establish novelty. This change belongs to the push
|
||||
comparison policy, consuming evidence ownership supplied by the shared scan contract.
|
||||
|
||||
### Query identity and witness identity differ
|
||||
|
||||
The Node contribution target is selected through a file-fallback injection question
|
||||
(`function=None`), while raw rows report `<lambda>2` at lines 32–34. The declared benchmark
|
||||
target is `handleContributionsUpdate`. VAmPI similarly has a raw SQLI site without an exact
|
||||
function target question. The benchmark correctly refuses exact-target completion today.
|
||||
Production normalization must carry verified query-to-operation provenance; do not inject
|
||||
benchmark function names into regions or change the denominator to count coincident sites.
|
||||
|
||||
## Bounded remediation sequence
|
||||
|
||||
1. Finish and preserve the unchanged seven-class runtime profile, including all failures.
|
||||
Run the isolated lifecycle probe afterwards so the experiments do not contend for CPU.
|
||||
2. Use those results to choose one engine lifecycle amendment. If a session is promising,
|
||||
keep it transaction-owned inside the existing backend: one deadline, serialized requests,
|
||||
explicit graph identity/epoch, private leases, failure poisoning, kill-and-reap cancellation,
|
||||
and fresh census on each answer. No resident laptop service is implied. Compare every
|
||||
shipped query kind with the disposable path before any production switch.
|
||||
3. Specify question-owned completeness across scan and comparison boundaries. First add
|
||||
authentic family context producers, then dependency/correspondence ownership, with paired
|
||||
PHP/JavaScript normalized controls and VAmPI regressions. Unknown ownership remains a
|
||||
blocker. A family that cannot provide this evidence stays experimental.
|
||||
4. Refreeze runtime and semantic identities. Rerun changed-code latency/completion and the
|
||||
independent reviewed vulnerable/fixed/benign populations together. Regenerate eligibility
|
||||
only after the existing joint gates pass; then repeat packaged acceptance validation.
|
||||
|
||||
This sequence separates runtime transport from evidence semantics. It adds no detector,
|
||||
second IR, language-specific rank boost, vendor graph expansion or differential CPG mutation.
|
||||
|
||||
## Verification needed for implementation
|
||||
|
||||
| Boundary | Required evidence |
|
||||
|---|---|
|
||||
| `cpg/backend.py`, candidate local session transport | Actual pinned launch/API; full query equivalence; malformed/absent response; switched graph isolation; concurrent transactions; deadline, crash and orphan cleanup |
|
||||
| `cpg/queries/{taint,dominance,config}.sc`, `model/scan.py` | Context from actual query scope; authentic per-family methods; answered-empty distinct from unavailable; unchanged rank decisions |
|
||||
| `model/contracts.py`, `model/regions.py`, `model/partitions.py` | Explicit gap ownership; first-party input failures remain blocking; dependent vendor/unknown paths unresolved; unrelated exclusion never silently proves completeness |
|
||||
| `push/snapshot.py`, `push/policy.py` | Exact operation correspondence; guard deletion; same-named and ambiguous operations; incomplete base cannot establish novelty; aggregate partial coverage retained |
|
||||
| Evaluation and eligibility | Frozen identities and populations; unchanged 30s/2s and precision/recall/completion gates; no capability enabled by a microbenchmark |
|
||||
|
||||
Production changes require task-local review and fresh ordinary scan gates, runtime smoke,
|
||||
and downstream acceptance evidence. This research record is not that verification.
|
||||
@@ -1210,3 +1210,92 @@ checkout that can actually demonstrate detection.
|
||||
pre-push-safety-net task 3.4: pinned JavaScript frontend filters remove declared
|
||||
first-party tests. The maintainer approved its dependency adaptation and acceptance checks;
|
||||
task 2.18 now implements it before downstream partition completion.
|
||||
|
||||
|
||||
## Group 7 — Grow the ontology by measurement, not by subject (2026-09-21)
|
||||
|
||||
Named by the family detection census and the week of repairs after it. Five of seven families
|
||||
established nothing and every one failed for an ontology reason, so this group is where detection
|
||||
efficiency actually lives. The design amendment above states the constraint these tasks work under:
|
||||
the ontology describes libraries, frameworks and language constructs, never repositories.
|
||||
|
||||
- [ ] 7.1 Report what a repository calls that the ontology does not model
|
||||
- Census the call names, and the receiver-qualified spellings, that appear in a repository's first-party graph; subtract everything any fact table names; rank the remainder by how often it is called and by how close it sits to a modelled source. The output is a measured worklist of what this tool is blind to here.
|
||||
- It needs no dataflow, which is why it is cheap: the same structural pass that answers 17,700 requests in 44 s already walks every call.
|
||||
- **It reports candidates, not entries.** A candidate becomes ontology only when someone resolves it to a library API, states the weakness class and records provenance. Nothing may be added because a subject called it.
|
||||
- Three defects this would have named in one run rather than three investigations: `needle.get` absent while the family found the redirect two files away, `find` absent so the documented object-reference defect raised no obligation, and `maybe_unserialize` absent so the only deserialization site in 1,164 PHP files was invisible.
|
||||
- Done when the instrument reports a ranked gap list for each measured subject, every entry carries how often it is called, and adding nothing to the ontology changes no result.
|
||||
- _First run 2026-09-21 on NodeGoat:_ 18 of 175 distinct call names modelled, covering 12% of 1,185 calls. The top product candidate is `res.render`, 23 calls with five in a method reading untrusted input — the same gap task 14.3 reasoned its way to by hand, found here in one run, which is the validation that matters. It also surfaced `Math.random` in the user data-access module: weak randomness is a weakness class this ontology has no family for, so no census could have found it, because the family list is itself the blind spot.
|
||||
- _It had to be taught what is not an API._ The first run ranked `_tmp_3.push` and `__ecma.Array.factory` first, which is how the frontend lowers a destructuring and an array literal, and a Cypress suite and a Gruntfile held four of the next five. Both are now separated, because noise at the top of a worklist is how a worklist stops being read. Evidence: `benchmarks/measurements/2026-09-21-ontology-coverage-nodegoat.json`.
|
||||
- _Boundary: shipped semantic facts, evaluation harness_
|
||||
- _Requirements: 6.2, 8.1_
|
||||
- [ ] 7.2 Key sanitizers to the weakness they actually prevent
|
||||
- The sanitizer list is flat across families, and that forces a false choice which is now measured: adding WordPress's `esc_attr` removes eighteen output-encoding false positives on one plugin and simultaneously hides a real SQL injection, because no HTML escaper stops SQL injection. `php.toml` has carried a comment predicting exactly this since before the measurement existed.
|
||||
- Key each sanitizer to the weakness class it discharges, so a family consults the cleansers that apply to it. Do not solve it by choosing which defect to keep.
|
||||
- Done when an HTML escaper clears an output-encoding flow and leaves an injection flow standing, on the measured plugin, with both outcomes recorded.
|
||||
- _Boundary: shipped semantic facts_
|
||||
- _Requirements: 6.2, 8.1_
|
||||
- [ ] 7.3 Do not arbitrate a question whose evidence says there is nothing to find
|
||||
- The structural pass answers 17,700 requests in 44 s; arbitration costs one to two seconds per question, and a 673-file plugin asks 2,526 of them. Gate arbitration on the evidence vector already computed: a question whose scope holds no modelled sink, or no reachable source, cannot produce a flow and does not need the expensive pass.
|
||||
- This attacks the scale limit from the opposite side to graph reuse and can be sized today from data already on disk, by counting how many of those 2,526 questions had any sink in scope at all.
|
||||
- Done when the arbitrated count falls without any finding being lost on the measured subjects, and the questions skipped are recorded as skipped rather than as answered.
|
||||
- _Implemented and measured 2026-09-22, and the mechanism already existed._ Tier-zero pruning defers a question whose evidence vector says no sink of its family is in reach. It was switched off for the WHOLE SCAN whenever any of twelve degradation reasons appeared, and a multi-language repository always carries two of them, so the measured plugin arbitrated every question it could ask. Third instance this week of the same blast radius. A prune blocker that names a partition now withholds pruning from that partition's questions and nobody else's, and `cross_partition_semantics_unresolved` leaves the blocker set entirely for the reason it left the integrity gaps.
|
||||
- _The arbitrated count falls and no finding is lost._ At 25 regions: 127 questions to 34, with 93 recorded deferred as `tier_zero`, and 19 findings before and after, identical family by family. At 500: 2,526 to 406, with 2,120 deferred. A family whose every question was pruned now reports "no sink of this family in reach" rather than "region offered, no question asked", because nothing to ask is a different statement from never offered.
|
||||
- **It is not sufficient, and that is the useful half of the result.** The 500-region run still fails in the same way at the same cost, because 406 questions at roughly one and a half seconds each still exceed the 300-second per-batch ceiling. This lever moved the volume; the ceiling is a separate limit and pre-push 13.3 still cannot reach the population that holds its target without it. Evidence: `benchmarks/measurements/2026-09-22-tier-zero-pruning.json`.
|
||||
- _Boundary: contributor-scan scan budget_
|
||||
- _Requirements: 4.1, 6.3, 8.1_
|
||||
- [ ] 7.4 One matcher, used by every query
|
||||
- Four query files carry near-copies of the same token matcher and each acquired the same defect separately: the configuration matcher needed a receiver check the dominance matcher already had, and the sink matcher searched a whole node's text including its comments. Six instances of one bug class are listed in the pre-push design's invariants.
|
||||
- Extract one matcher with callee confinement and word boundaries built in, and have taint, dominance and config use it. A rule in a document gets rediscovered; a rule in a constructor does not.
|
||||
- Done when one implementation is shared, each caller's behaviour is unchanged on the measured subjects, and a new query cannot match text that merely contains a name.
|
||||
- _Boundary: contributor-scan queries_
|
||||
- _Requirements: 6.2, 8.1_
|
||||
- [ ] 7.5 A degradation must declare the scope it reaches
|
||||
- Five instances of one bug class are listed in the pre-push design's invariants, two of them found this week, and two of those cost the declared target its entire admissibility. A degradation is currently a bare dict, so nothing forces a new reason to say whether it reaches one question, one file, one partition or the whole read.
|
||||
- Make it a record with a required scope, and let the demotion and the admission policy read that field rather than a hand-maintained membership set.
|
||||
- Done when a degradation cannot be constructed without a scope, the existing reasons carry theirs, and the measured subjects' outcomes are unchanged.
|
||||
- _Boundary: contributor-scan degradation vocabulary, pre-push admission_
|
||||
- _Requirements: 6.3, 8.1_
|
||||
- [ ] 7.6 Golden findings per subject, so a silent change cannot pass
|
||||
- Nothing fails today when a change alters what NodeGoat or VAmPI report. Four changes in one week altered detection and each time the census had to be re-run from memory; one false positive was caught by reading a witness rather than by a test.
|
||||
- Commit an expectation per subject at the finding level, site and family and rung, and diff against it. Keep it honest about what it is: a regression guard on development subjects, never evidence of qualification.
|
||||
- Done when a deliberate change to a matcher or a fact table fails the diff, and the expectation records the ruleset and engine identities it was taken under.
|
||||
- _Boundary: evaluation harness_
|
||||
- _Requirements: 8.1, 8.2_
|
||||
|
||||
- [ ] 7.7 Budget a batch by the sinks it implies, not by the requests it holds
|
||||
- **Named by the 2026-09-22 identification.** A taint batch costs what its matched sink NODES cost, because `reachableByFlows` is asked per sink. Measured at a fixed 100 requests on one graph: SQL injection 147 s, path 194 s, output encoding 704 s — and output encoding declares seven sink names against path's sixteen. What separates them is that `echo` and `print` match thousands of calls in a WordPress plugin.
|
||||
- The driver currently splits a failed batch into eight parts by request count, which cannot help: each part carries the same everywhere-sinks.
|
||||
- The count needed is already computed. `taint.sc` memoises its candidate sink list per sink set, and the evidence pass reports the in-scope sink list per request before any dataflow runs. A driver that reads it can size a batch by expected work, and can report a family whose sinks are everywhere as EXPENSIVE rather than as failed.
|
||||
- Done when a batch is sized by the sink nodes it implies, the measured plugin's arbitration completes within its ceiling, and a family too expensive to arbitrate is reported as such rather than as a timeout.
|
||||
- _Implemented and measured 2026-09-22._ On the 500-region population where every earlier run produced nothing: **153 of 406 questions completed and 125 findings, inside the deadline**, against 0 and 0. 380 requests carrying 5,807 sink visits split into 11 batches, each answering in 80 to 197 s, none near the 300 s ceiling. Every family that can speak on this subject now does: injection 49 completed and 24 findings, path 64 and 15, output encoding 32 and 70, untrusted destination 8 and 16. Eight batches were left unasked to keep time for arbitration, and that is recorded rather than silent.
|
||||
- _Four defects, each hidden by the one before it, and three of them the same rule._ Batches sized by request count when cost is per matched sink. Packing heaviest-first, which starved injection — the one family with a true positive here — of all 114 of its questions. Any per-query timeout cancelling the whole scan, abandoning batches with 33 minutes of budget left. And no budget kept for arbitration, which removing that cancellation exposed: rows are not findings, so the query phase ran the full deadline and completed nothing, where the cascade had been an accidental guard.
|
||||
- _The reserve had to be checked per BATCH._ Checked between query kinds it changed nothing at all, because one kind is a single loop iteration that runs every one of its batches inside it. The granularity of a budget check has to match the granularity of the work it guards. Evidence: `benchmarks/measurements/2026-09-22-sink-weighted-batching.json`.
|
||||
- _What it does not claim:_ 253 of 406 questions still went unanswered and one batch still records a timeout; the raw finding count is dominated by output encoding, the family already measured to be imprecise on PHP; and the reserve share of a fifth is a first value, not a tuned one.
|
||||
- _Boundary: contributor-scan scan budget, cpg backend_
|
||||
- _Depends: 7.3_
|
||||
- _Requirements: 4.1, 6.3, 8.1_
|
||||
|
||||
- [ ] 7.8 Portion the work instead of capping it: a kill keeps what finished
|
||||
- **Asked for by the maintainer on 2026-09-22, after 7.7.** A hard per-invocation ceiling is all-or-nothing: a batch killed at 301 s loses every answer it had already computed, and the retry then pays a fresh JVM start per part to ask them all again. Measured on the plugin: 116 requests killed at 300 s, eight restarts, and the same everywhere-sinks in every part.
|
||||
- Three coordinated changes replace the cap as the thing that governs cost. The taint query STREAMS: the census first, then one line per request as it is answered, flushed as it goes, inside the unchanged fence. The driver harvests a killed process's stdout and assembles every complete line, marking the result partial; the retry then asks only the remainder. And portion sizing ADAPTS: after each portion the observed seconds per sink visit, less the fixed start, resizes the rest toward a target duration, and a portion that did not answer halves the budget rather than being fitted. The ceiling stays as a safety net that the sizing is meant to make rare.
|
||||
- Legacy single-document payloads must keep parsing unchanged; the other queries do not stream and need not. A census line is one whose only key is the census — a first cut misread a legacy document that carried the census beside its answers as a census line and dropped every answer it sat beside, which a control caught.
|
||||
- Done when a killed taint batch hands back the answers it finished and the retry asks only the rest; portion sizes follow observed cost; the streamed and legacy forms both parse; and the plugin population completes more questions per unit of budget than under 7.7 with no finding lost.
|
||||
- _Status 2026-09-22:_ implemented; host controls pass for the assembler in both forms and under a kill, for the remainder-only retry, and for the sizer; the joern-gated check passes on the real engine — the taint query streams one line per request, and the comment-sink and literal-destination controls hold under the streamed form. The plugin measurement is running on a frozen export of the committed source.
|
||||
- _Two things the measurement exposed on the way, both fixed:_ the evidence pass was one unportioned batch protected only by the ceiling — 22,125 requests on a plugin push transaction — and now runs in portions of 4,000; and a first cut of the harvest read a timed-out process's pipe BEFORE killing it, which blocks forever because a pipe whose writer is alive never reaches end-of-file. The suite's own hanging-engine control caught that after twenty minutes of what I mistook for CPU starvation.
|
||||
- _A stated limit:_ the reuse layer's re-query of `missing` ids calls the backend batch directly and so bypasses sink-weighted portioning. Those re-queries are the small remainder of a cached scan, and streaming still keeps what a killed one finished, so it is recorded here rather than fixed. _Corrected 2026-09-23:_ the limit was stated backwards and was worse. The reuse layer wrapped only `run_batch`, and the portioner asks `run_batch_once`, so every portioned query bypassed the reuse cache — and its source-root normalization — entirely. Fixed under 7.9: both paths go through the cache.
|
||||
- _End-to-end tuning, 2026-09-23, the honest version._ The first end-to-end run REGRESSED completion (64 of 406 against 7.7's 153) while finding as many defects, and each fix exposed the next: the adaptive sizer read one portion at 953 s because it timed `run_batch`, whose own split-retry folded in, and collapsed; giving the portioner a single-attempt path (`run_batch_once`) fixed the timing but only after it was wired on the PARTITIONED graph too, which every plugin uses and which the first wiring missed; and even then completion sat at 56 of 406 because the portioner iterated a fixed list and dropped the remainders of killed portions, stopping at 1,263 s of a 3,600 s budget. The queue re-asks those remainders. The sizer now adapts both ways in the trace — 268 up to 535 after a cheap portion, halved to 267 to 133 on kills — and re-asks what a kill left. Each step is unit-tested: the assembler in both forms and under a kill, the single-attempt path, coverage-halving, and queue termination. _Final end-to-end 2026-09-23, mixed and told straight._ Against the hard-cap-plus-reserve baseline on the same 500-region PHP run: findings IDENTICAL at 125 either way, completion slightly LOWER under portioning (131 of 406 against 153), wall time higher (2,982 s against 2,338). Portioning does not beat the cap on throughput here; it matches it on what it finds. What it buys is kill-safety -- the maintainer's actual ask -- and reaching the same 125 findings is the evidence that streaming loses nothing the cap kept while no longer discarding a near-complete batch when the ceiling falls mid-answer. Completion did not rise because the requeue re-asks a killed portion's remainder, and on this subject those are the expensive output-encoding questions that keep hitting the ceiling in smaller pieces; telling a transient kill from a genuinely-too-expensive family is the next lever, not more portioning. Neither approach is release-ready regardless: 32-38% completion against a 95% gate, on development data that qualifies nothing. Evidence: `benchmarks/measurements/2026-09-23-portioning-vs-hard-cap.json`.
|
||||
- _Boundary: cpg backend, contributor-scan scan budget, taint query_
|
||||
- _Depends: 7.7_
|
||||
- _Requirements: 4.1, 6.3, 8.1_
|
||||
|
||||
- [x] 7.9 Size portions from what each answer cost, and tell a transient kill from an expensive question
|
||||
- **The next lever named by 7.8, and the trace said it was a different lever.** Re-read on 2026-09-23, the 7.8 run's sizer did not lose completion to expensive families so much as to a death spiral: after two kills the budget fell to 3 sink visits per portion and never recovered. Portions of 3–6 visits finished in 25–30 s, which under the assumed 70 s fixed start read as "no information", so the budget could shrink and never grow — and the rest of the run paid a JVM start per handful of requests.
|
||||
- The instrument first: the taint query streams each answer's own cost (`ms`), carried as a reserved `__timing__` list through shards, partitions and reuse. The sizer then MEASURES what it used to assume — the fixed start as the wall time the answers do not explain (median over whole portions), and a seconds-per-sink-visit rate per family from that family's own answers. Portions are cut in predicted seconds and grow request by request from the head of the queue.
|
||||
- A kill names its culprit. Answers stream in request order, so the first unanswered id was running at the kill and every id after it never started: those go straight back to the queue, the culprit is asked alone after everything else, and its family is costed at the lower bound it proved. A request that fails alone after more than a portion's target is `query_too_expensive`, reported apart from `query_failed`.
|
||||
- Two defects fixed on the way: a scan small enough for one portion took a single attempt and lost whatever a kill cut off (it now goes through the queue); and requests the arbitration reserve held back were reported as `query_failed` although they were never asked (now `query_budget_reserved`).
|
||||
- _Measured 2026-09-23, same 500-region PHP run, 3,600 s, frozen at `1b28465`:_ **281 of 406 questions completed, against 153 (hard cap) and 131 (7.8)**, in 2,987 s. Findings per family never fell: injection 24→33, output encoding 70→125, path 15→35, untrusted destination 16→31; 224 in total against 125. The declared target `classes/class.memberorder.php:936:getMemberOrderByCode` is found at `model_entailed`. The measured fixed start was 22–24 s throughout. **No portion was killed in the whole run** — every one answered whole — so the gain is the sizing, and the culprit path is covered by host controls but was NOT exercised end to end here.
|
||||
- _Limits, stated:_ one subject, one budget. The census keeps ten examples per family, so it shows per-family counts did not fall, not that the earlier findings are a strict subset. The extra findings are unadjudicated: output encoding on PHP `echo` is the family most likely to carry false positives, and 224 is a count, not a precision. 99 taint questions were still held for arbitration, and access control, config and prototype were never reached (0 of 39) — the ranker puts them last, and the reserve cuts the tail. Completion is 69%, still short of the 95% M4 gate; this is development data and qualifies nothing. Evidence: `benchmarks/measurements/2026-09-23-measured-portioning.json`.
|
||||
- _Boundary: cpg backend, taint query, contributor-scan scan budget, push reuse_
|
||||
- _Depends: 7.8_
|
||||
- _Requirements: 4.1, 6.3, 8.1_
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** complete, 17 of 17 tasks (spec.json phase corrected to `implementation-complete`). Its `[variants]` config keys were retired on 2026-10-02: the variant search they configured (`semantic/variants.py`) had already been deleted.
|
||||
|
||||
## Introduction
|
||||
|
||||
OpenUltraSAST's labeled pair corpus measures the detector but does not improve it, and its mechanism memory is empty because only sandbox-proven findings may write to it. This spec turns every trusted pair into a mechanism record, adds a structural variant search that proposes matches on any scanned tree, measures the corpus's own detection rate leave-one-out, and gives the improve loop a lever that can raise recall under the existing false-positive ceiling and per-profile holdout gate.
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
{
|
||||
"feature_name": "corpus-seeded-mechanisms",
|
||||
"created_at": "2026-09-05T06:18:31Z",
|
||||
"updated_at": "2026-09-05T16:30:00Z",
|
||||
"updated_at": "2026-10-02T00:00:00Z",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"phase": "implementation-complete",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": true,
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Brief: flow-aware-ranking — spend the expensive query where the cheap evidence is strongest
|
||||
|
||||
> **Status 2026-10-02:** dormant. Brief only and unapproved; its 2026-09-12 amendment placed ranking inside the pre-push safety net, where the ranker now lives.
|
||||
|
||||
**Status:** brief only, unapproved.
|
||||
|
||||
## Current amendment — 2026-09-12
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
# Brief: remove HarnessX after the plane proves value
|
||||
|
||||
Status: brief only, gated. Not started until `ai-service-plane` Requirement 6 records a met gate.
|
||||
|
||||
## Inventory (read-only, 2026-09-29)
|
||||
|
||||
- Code: `src/openultrasast/harness_ext.py` (97 lines) and `hunter_harness.py` (197) exist only for HarnessX;
|
||||
guarded branches in `cli.py` (372-512), `fusion.py` (LLM panels, ~100 lines), `verify_judge.py` (judge path,
|
||||
~90), `stage_processors.py` (`host_under_harnessx`, ~40), `config.py` (`HarnessxConfig`). About 575 of
|
||||
30,243 lines (1.9%). Every path falls back deterministically when the extra is absent and records a
|
||||
degradation (`harnessx_extra_unavailable`; fallbacks `run_hunter_pool`, `structural_verifier`,
|
||||
`deterministic_panels`). `evolve`, `regress`, `benchmark`, `gate` and quick mode do not depend on it.
|
||||
- Comment-only mentions (re-verified 2026-09-30): `gate.py:174`, `improve/evolve.py:9`, `slot_contract.py:7-13`.
|
||||
- Packaging: `pyproject.toml` extra `harnessx` pinned to a git commit; mypy override; `uv.lock` entry.
|
||||
- Tests: 14 files touch it; dedicated `test_harness_ext.py`, `test_hunter_harness.py`, `test_cli_hx_dispatch.py`,
|
||||
`test_verify_judge.py`; `tests/conftest.py` fixture `assert_cold_of_harnessx`.
|
||||
- Docs: README sections "Configuring the HarnessX agentic plane" and "The HarnessX self-improving cycle";
|
||||
`docs/examples.md` section 6; `docs/threat-model.md` egress and spend caps.
|
||||
- Specs: `harnessx-self-improving-rulesets` (approved, 43/43 done, Requirement 1 is direct adoption);
|
||||
`learning-harness` (30/33); `three-stage-scan` framed as its follow-up; `openrouter-sast-harness` (no
|
||||
spec.json); mentions in four more; steering `overview.md:175`.
|
||||
|
||||
## Shape of the removal
|
||||
|
||||
1. Replace the three guarded paths with plane tasks (hunter pool -> `verify`; judge -> `judge`; panels ->
|
||||
`judge`/`agree`) so their fallbacks become the only path, then delete the two modules, the config block,
|
||||
the extra, the override and the fixture.
|
||||
2. Retire the four specs' HarnessX requirements with a dated note, not a rewrite; update README and docs.
|
||||
3. Gate: the plane increment met (recall and cost), and the replaced features measured no worse on the
|
||||
development pins before deletion.
|
||||
@@ -0,0 +1,507 @@
|
||||
# Design Document
|
||||
|
||||
## Overview
|
||||
|
||||
HarnessX leaves the product in one reviewable sequence, and the plane's memory layer takes over what HarnessX was
|
||||
meant to drive. Three facts from the code shape the design:
|
||||
|
||||
- **No HarnessX capability is measured, and one of them never existed.** The LLM paths are reached only through
|
||||
config keys, not CLI flags (`scan` has `path`, `--mode`, `--config`, `--fail-on`; `cli.py:163-167`), and
|
||||
`benchmarks/` holds no run of any of them (no `hx-hunter:` finding, no `llm-panels` decision). The
|
||||
`MetaAgent.evolve` proposer is only a docstring (`improve/evolve.py:8-9`, `harness_ext.py:7-8`). No code calls it.
|
||||
- **The plane already does the LLM work better, and outside `ousast scan`.** `verify` (two passes, known callers
|
||||
from `repo-facts`) plus `agree` (2-of-3 with the tie-break) met the ai-service-plane gate
|
||||
(`benchmarks/independent/plane-increment-2.json`: 16/20 declared sites agreed, $0.0205 per candidate). The
|
||||
standard-mode LLM hunter that users actually reach, the tool hunter at the MAP stage (`cli.py:631-640`,
|
||||
`tool_hunter.resolve_hunter_client`), is not HarnessX and stays.
|
||||
- **Everything deterministic already runs without the extra.** Every guarded branch falls back to a deterministic
|
||||
function (`run_hunter_pool`, `verify_findings`, `fuse_finding`). Removal therefore makes the fallback the only
|
||||
path. Quick mode, `benchmark`, `pairs`, `improve` and the gates never enter a guarded branch.
|
||||
|
||||
So every HarnessX capability in `ousast scan` is **retired** with its deterministic path kept. The plane's `verify`
|
||||
and `agree` tasks are the documented agentic path (`ousast plane run <Run>`). The proposer is **carried by the
|
||||
memory layer**: a persistent store, keyed by repository and pin, that plane Runs write through a model-free
|
||||
`remember` task. From that store a deterministic generator emits `RuleStatusEdit` proposals into the unchanged
|
||||
`EvolveValidator` and gate. The improvement loop becomes a plane Run whose only Model-bound tasks are the verify
|
||||
passes.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Before After
|
||||
ousast scan --mode standard ousast scan --mode standard
|
||||
hunter_pool: HxScanOrchestrator | run_hunter_pool hunter_pool: run_hunter_pool (only path)
|
||||
verify: verify_judge (llm-judge) | verify_findings verify: verification.verify_findings
|
||||
fusion: llm panels | deterministic panels fusion: deterministic panels (fuse_findings)
|
||||
MAP: tool hunter ([models] hunter, DeepSeek) MAP: tool hunter (unchanged)
|
||||
harness_ext.py, hunter_harness.py, verify_judge.py deleted
|
||||
stage_processors.host_under_harnessx deleted; deterministic pipeline kept
|
||||
[harnessx] config, extra, mypy override, lock one load warning; extra/override/lock gone
|
||||
improve: benchmark per-rule proposer (+ MetaAgent improve: benchmark proposer + memory proposer (--memory)
|
||||
"plugs in" -- never implemented)
|
||||
plane Run (per case)
|
||||
facts -> va, vb -> agree -> vc -> final (existing)
|
||||
alerts (new, no model) ------------------+
|
||||
remember (new, no model) <- facts, final, alerts, va/vb/vc
|
||||
plane Run (loop tail, once)
|
||||
measure -> propose -> improve(validate+gate) (no model)
|
||||
~/ousast-results/plane/memory/ the store: facts by content, rows by repo + pin
|
||||
```
|
||||
|
||||
New code: `src/openultrasast/plane/memory.py` (store, ingest, seed; host side),
|
||||
`src/openultrasast/plane/tasks/{alerts,remember,measure,propose,improve}.py`,
|
||||
`src/openultrasast/improve/memory.py` (deterministic proposal rules). None of it is in the reconciler. The
|
||||
reconciler gains only `mark_done` (below). It is 465 lines today, and `tests/test_plane_reconciler.py:759-761`
|
||||
keeps it under 500.
|
||||
|
||||
## Components and Interfaces
|
||||
|
||||
### 1. Fate of each HarnessX capability (Requirement 1)
|
||||
|
||||
| Capability | Where today | Fate | What the user loses |
|
||||
| --- | --- | --- | --- |
|
||||
| LLM hunter pool (`HxScanOrchestrator`) | `hunter_harness.py:49-185`; gated in `cli.py:400-456` (`hx_hunter` at :405, orchestrator at :438-448, degradation at :452-455) | **Retired.** `run_hunter_pool` becomes the only standard-mode hunter pool. LLM hunting of a repository stays available in two places: the MAP-stage tool hunter in `ousast scan --mode standard/deep` with `[models] hunter` (`cli.py:631-640`, not HarnessX), and plane `verify` via `ousast plane run plane/runs/validation-46.yaml` | `hx-hunter:<path>:<line>` findings from an anthropic/openai/litellm agent (`hunter_harness.py:155`). No run of this path is recorded under `benchmarks/`, so no measured TP is lost. |
|
||||
| LLM judge | `verify_judge.py:22-63`; gated in `cli.py:406, 499-511` | **Retired.** `verification.verify_findings` is called directly and `verify_judge.py` is deleted (it is only the dispatcher and the judge). The plane's LLM verification is `verify` + `agree` (+ `vc`/`final` tie-break) | Per-finding `plausible/unsupported` verdicts that set `ACCEPTED/REJECTED` (`verify_judge.py:83-88`). Unmeasured. `[models] judge` (model-grounded-detection, `model/endpoint.py:195`) is a different capability and stays. |
|
||||
| LLM fusion panels | `fusion.py:194-326` (`_fuse_with_llm_panels`, `_run_llm_panels`); gated in `cli.py:519-541` | **Retired.** The deterministic panels (`_panel_a`, `_panel_b`, `_reconcile`, `fusion.py:101-167`) become the only path. `fuse_findings_dispatch` is renamed `fuse_findings(findings, verifications, *, high_assurance, mitigated_ids)`. The plane counterpart of independent cross-examination is `agree` across independent passes with the 2-of-3 tie-break | Steel-man LLM panels (`decision_source: "llm-panels"`). Unmeasured. `fusion.json` for deterministic decisions is unchanged. |
|
||||
| Deterministic stages hosted under HarnessX | `stage_processors.py:129-156` (`host_under_harnessx`); only caller is the skipped test `test_slot_contract.py:119-133` | **Retired, no replacement.** ax runs whole tasks, so a model-disabled in-loop host has nothing to host. `stage_processors`/`slot_contract` stay as the standalone deterministic pipeline that `test_gate.py:147-148` pins | Nothing. The function has no production caller. |
|
||||
| LLM evolution proposer (`MetaAgent.evolve`) | docstring only: `improve/evolve.py:8-9`, `harness_ext.py:7-8` | **Carried by the memory layer** (Requirement 6): `improve/memory.py` rules over the store, fed into `run_round` as extra proposals | Nothing implemented is lost. |
|
||||
|
||||
No CLI flag is HarnessX-specific, so Requirement 1.3 applies to config keys. A key that asked for a retired
|
||||
capability makes `load_config` raise `RetiredConfigError(ValueError)`, and the CLI prints the message and exits 2.
|
||||
A silent downgrade is not allowed:
|
||||
|
||||
| Key | Message (abridged) |
|
||||
| --- | --- |
|
||||
| `[models] verifier` | "`[models] verifier` was retired with HarnessX on <date>: LLM verification runs on the plane (`verify` + `agree`, `ousast plane run`, see ops/ax/README.md). Remove the key." |
|
||||
| `[fusion] panel_model`, `[fusion] decider_model` | "... LLM fusion panels were retired; fusion is deterministic; independent LLM agreement is the plane's `agree` task ..." |
|
||||
|
||||
`[harnessx]` is different (Requirement 3.3). It only tuned budgets and never requested a capability, so it loads
|
||||
with **one** `logging.warning` on logger `openultrasast.config` that names the plane ("`[harnessx]` is ignored:
|
||||
HarnessX was removed; agentic work runs on the ax plane with per-task budgets, see ops/ax/README.md"). The section
|
||||
is otherwise ignored. `[models] hunter` keeps its meaning (tool hunter, obligations intent adjudication at
|
||||
`cli.py:1237`), so it gets no message. With the extra installed it no longer starts a second, HarnessX hunter. The
|
||||
release notes say so.
|
||||
|
||||
Requirement 1.2 (moved capabilities have a CLI path, a Model per task and a fixture test) is met by the existing
|
||||
`verify`/`agree` tasks (`ousast plane run`; Model `deepseek-flash` bound per Task; `tests/test_plane_verify.py`,
|
||||
`tests/test_plane_scoring_parity.py`, `tests/test_plane_tiebreak.py`), and by the new memory and loop tasks below, each with its own fixture test. No
|
||||
capability is moved into a new Model-bound task.
|
||||
|
||||
### 2. Removal mechanics (Requirement 2)
|
||||
|
||||
**Source.**
|
||||
|
||||
- Delete `harness_ext.py` (97 lines), `hunter_harness.py` (197) and `verify_judge.py` (130).
|
||||
- `cli.py`: drop the imports at :44, :46 and :94. Drop `harnessx_present`, `hx_hunter` and `hx_verify` (:400-406),
|
||||
the orchestrator branch and its degradation (:438-456), and the verify degradation (:499-502). Call
|
||||
`verify_findings(findings)` at :503-511. Drop `hx_fusion` and its degradation (:521-527). Call
|
||||
`fuse_findings(findings, verifications, high_assurance=config.fusion.high_assurance)`. The `verifier_model`
|
||||
local goes away.
|
||||
- `fusion.py`: drop the import at :24, the `panel_model`/`decider_model`/`provider`/`use_harnessx` parameters, and
|
||||
`_with_degradation`, `_fuse_with_llm_panels`, `_run_llm_panels` and `_text` (:229-338). Rewrite the module
|
||||
docstring (:10-14) and update `__all__`.
|
||||
- `stage_processors.py`: drop `host_under_harnessx` (:129-156) and its `__all__` entry, and rewrite the module
|
||||
docstring (:10-11). `slot_contract.py:7-14` becomes "the zero-dependency substrate for the deterministic
|
||||
pipeline".
|
||||
- `config.py`: drop `HarnessxConfig` (:99-107), `ResolvedConfig.harnessx` (:201), `_load_harnessx` (:345-351) and
|
||||
its call (:230). Drop `ModelConfig.verifier` (:37), `FusionConfig.panel_model/decider_model` (:123-124) and
|
||||
their loaders (:266, :366-367). Add `RETIRED_KEYS` and the one-warning rule above. `harness.py:183` keeps the
|
||||
`"verifier": None` literal in `model_roles`, so `harness.json` keeps its schema byte for byte (the key has always
|
||||
been `null` in the baseline config).
|
||||
- Comment-only mentions: `gate.py:174` ("a model-backed stage"), and `improve/evolve.py:8-9`, which now names the
|
||||
memory proposer.
|
||||
- `plane/Dockerfile.runner:23` loses "no HarnessX".
|
||||
|
||||
**Packaging.**
|
||||
|
||||
- Delete the `harnessx` extra and its comment (`pyproject.toml:14-22`) and the `harnessx.*` mypy override and its
|
||||
comment (:67-71).
|
||||
- `uv lock` regenerates `uv.lock`. The `harnessx` package (:1062), the extra (:2145-2146), the requirement
|
||||
(:2169) and `provides-extras` (:2178) go.
|
||||
- Extras that remain: `semantic`. There is no `all` extra. `pip install .` and `pip install '.[semantic]'` in a
|
||||
fresh venv are the Requirement 2.2 check.
|
||||
- No source or test imports a HarnessX transitive dependency. The only third-party imports are `yaml`,
|
||||
`tree_sitter` and `pytest` (checked with an import scan), so dropping the lock entry cannot break an import.
|
||||
|
||||
**Tests** (14 files plus `conftest.py`):
|
||||
|
||||
| File | Action |
|
||||
| --- | --- |
|
||||
| `test_harness_ext.py`, `test_hunter_harness.py`, `test_verify_judge.py`, `test_cli_hx_dispatch.py` | **Delete.** Each exercises only HarnessX or its dispatcher. |
|
||||
| `conftest.py` | Rename fixture `assert_cold_of_harnessx` to `assert_cold_import(code)`: a fresh interpreter runs `code`, with no module-name guard. `test_semantic_extra.py:28-35` is its remaining user (the tree-sitter check brings its own assertion). |
|
||||
| `test_gate.py` | Delete `test_harnessx_stage_stays_within_tolerance_of_baseline` (:124-150) and the `has_harnessx` import (:23). Replace the two "never import" tests (:155-175) with `test_governance_planes_import_only_stdlib_and_declared_dependencies`, an allow-list (stdlib, `openultrasast`, `yaml`) over module-level imports of `GOVERNANCE_SCORING_BENCHMARK_PLANES`. It covers any extra without naming one. Update the docstring at :5-6. |
|
||||
| `test_scan_pipeline_integrity.py` | Rename `test_standard_scan_extra_absent_matches_deterministic_baseline` to `test_standard_scan_with_a_hunter_model_keeps_deterministic_findings` (config `hunter` only). Rename `test_quick_scan_is_unaffected_by_the_extra` to `test_quick_scan_records_no_degradation`. Drop the `has_harnessx` monkeypatch and the `harnessx` parameter. Recompute the expected degradation set from the post-removal run, since `hunter_pool`/`verify` entries no longer exist. |
|
||||
| `test_slot_contract.py` | Delete `test_host_under_harnessx_disables_the_model` and the imports at :16 and :22. Keep the rest. |
|
||||
| `test_config.py` | Replace `test_harnessx_defaults_when_section_absent` and `test_load_config_reads_harnessx_section` with `test_retired_agentic_section_loads_with_one_warning` (caplog: exactly one record naming the plane) and `test_retired_llm_keys_fail_naming_the_replacement` (verifier, panel_model, decider_model). |
|
||||
| `test_fusion.py` | Delete `test_dispatch_records_degradation_when_llm_requested_but_extra_absent` (:142-146) and the import at :16. Rename the dispatch calls to `fuse_findings`. |
|
||||
| `test_tool_hunter.py` (:229, :266), `test_scan_deep.py` (:144) | Drop the `has_harnessx` monkeypatch line. |
|
||||
| `test_semantic_extra.py` | Use `assert_cold_import`. |
|
||||
| `test_module_audit.py` | The subsystem tuple at :31 becomes `("fusion", "mcp", "skills")`. A new assertion checks that the three deleted modules are absent from the audit. |
|
||||
|
||||
**Module audit.** `model/audit.py:76-77` (`harness_ext`, `hunter_harness`) are deleted, and `:78`
|
||||
(`stage_processors`) is reworded to "deterministic slot-contracted scan stages; pinned by the zero-dependency
|
||||
guard in test_gate". Regenerate `benchmarks/measurements/2026-09-08-module-audit.json` in place with `to_dict(audit(SRC))`.
|
||||
`test_the_manifest_matches_the_tree` pins that path. The diff shows three rows removed and no orphan.
|
||||
|
||||
**Search (Requirement 2.4).** `grep -rni harnessx src tests pyproject.toml README.md` may match only lines tagged
|
||||
`retired 2026-..`:
|
||||
|
||||
- the `RETIRED_KEYS` entry and warning text in `config.py`
|
||||
- the two retirement tests in `test_config.py`
|
||||
- the one README line in the plane section
|
||||
|
||||
Requirement 3.3 cannot be implemented or tested without naming the section, so these lines count as retirement
|
||||
notes. The check is a test (`test_no_harnessx_outside_retirement_notes`) that fails on any untagged match. It
|
||||
builds its pattern at runtime, so it does not match itself.
|
||||
|
||||
### 3. Equality proof (Requirement 3)
|
||||
|
||||
The spec's "`ousast regress`" is not a subcommand (`cli.py:133-251`). Regress exists only as the deep-mode
|
||||
sandbox stage. On this project the regression command over the development pins is `ousast pairs`
|
||||
(README.md:38): the vuln-vs-fixed replay over local fixtures and vendored pinned VFC pairs, offline by default.
|
||||
The benchmark replay is README.md:35's manifest. The baseline also covers the two paths the removal touches
|
||||
directly: one standard-mode scan (verify and fusion) and one `improve --dry-run` (evolve). Everything is
|
||||
quick or standard mode with no Joern on the host, so each command takes minutes, not hours, on this 7 GB host.
|
||||
|
||||
**Environment (instrument first).**
|
||||
|
||||
- Run from a `git archive HEAD` export in the scratchpad (the source is frozen, per the long-runs rule).
|
||||
- `uv sync --frozen --extra semantic`, which drops the installed `harnessx`. It is installed in `.venv` today.
|
||||
- `OPENULTRASAST_SKIP_DOTENV=1` and `env -u DEEPSEEK_API_KEY -u OPENROUTER_API_KEY -u ANTHROPIC_API_KEY -u OPENAI_API_KEY`,
|
||||
plus an empty `--config`, so no command can reach a model.
|
||||
|
||||
The harness asserts `find_spec("harnessx") is None`, `find_spec("tree_sitter") is not None`, and a
|
||||
`uv pip freeze` hash. It also prints the byte size of each input manifest, and fails on any non-zero exit code.
|
||||
|
||||
**Commands** (one script, `benchmarks/harnessx_removal_equality.py record|compare <dir>`):
|
||||
|
||||
1. `pytest -q -p no:cacheprovider --junitxml=suite.xml`, which gives outcome per test id.
|
||||
2. `ousast pairs --json > pairs.json`. Instrument check: pair count > 0 and at least one TP.
|
||||
3. `ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick`. Keep
|
||||
`benchmark_result.json`, `calibration_records.json` and `external_baseline_deltas.json`. Instrument check:
|
||||
`expected > 0`.
|
||||
4. `ousast scan benchmarks/fixtures/python-vulnerable --mode standard --config empty.toml`. Keep
|
||||
`findings.json`, `verification.json`, `fusion.json`, `report.sarif`, `harness.json` and `manifest.json`.
|
||||
5. `ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --no-pair-gate`, stdout plus
|
||||
the dry-run journal.
|
||||
|
||||
**Stored.** `benchmarks/measurements/<date>-harnessx-removal-baseline/`: the normalized outputs, and `env.json`
|
||||
(commit, freeze hash, instrument checks). The script's exclusion list is written into that directory's
|
||||
`README` line.
|
||||
|
||||
**Comparison.** JSON is parsed, normalized and re-serialized with sorted keys. Everything else is compared as
|
||||
raw bytes. Excluded:
|
||||
|
||||
- keys `runtime_seconds`, `scan_id`, `timestamp`, `started`, `finished`, `created_at`, `generated_at`, and any
|
||||
key ending `_seconds` or `_ms`
|
||||
- run directory names (timestamped, `benchmark.py:186`)
|
||||
- `trace/events.jsonl` as a whole (timings)
|
||||
- the export's absolute root path, replaced by `<ROOT>`
|
||||
|
||||
For the suite, every test id that survives must keep its outcome. Deleted and renamed ids are compared through
|
||||
the map in the Tests table above, and any other difference is a failure. Any difference in 2-5 blocks the merge.
|
||||
|
||||
### 4. Memory layer (Requirement 6.1, 6.4)
|
||||
|
||||
**Store** (`plane/memory.py`). The store lives at `results_root()/memory/` (`~/ousast-results/plane/memory/`,
|
||||
honouring `OUSAST_RESULTS`, `reconciler.py:58-59`). It is JSON files, not SQLite: the rest of the plane is JSON,
|
||||
diffable and greppable, and one run at a time holds the run lock, so SQLite's concurrency buys nothing.
|
||||
|
||||
```
|
||||
memory/
|
||||
index.jsonl one row per ingested (run, task): sha256 of its memory.jsonl, row count
|
||||
facts/<sha256>.json facts.json by content (identical facts stored once)
|
||||
repos/<host>__<owner>__<name>/<pin>.jsonl the rows of one repository + 40-hex pin
|
||||
```
|
||||
|
||||
Ingest is idempotent: an `index.jsonl` hit skips the delivery, and rows carry a deterministic `id`. Ingest also
|
||||
refuses to write when `shutil.disk_usage` reports less than 1 GiB free. The disk is at 98% today, so it names
|
||||
the store path in the error.
|
||||
|
||||
**Storage backends (maintainer, 2026-09-30: asked to use the object-store features of the S3 server he runs,
|
||||
including its JSON support).** The store is written against one interface, `MemoryStore` (`put_row`, `put_facts`, `get_facts`,
|
||||
`rows(repo=..., pin=..., kind=..., where=...)`, `index`), with two backends of identical behaviour:
|
||||
|
||||
- `FileStore`: the layout above, for this laptop and the unit tests.
|
||||
- `S3Store`: an S3-compatible bucket (boto3; RustFS is the tested server), and the store of record once the plane runs on a separate
|
||||
Kubernetes cluster. The layout maps onto one bucket with the same key paths:
|
||||
- **Object metadata and tags** on every row object: `repo`, `pin`, `kind`, `family`, `run`, `population`,
|
||||
`split`. Rules list by prefix (`repos/<repo>/`) and filter by tag instead of scanning the bucket.
|
||||
- **Server-side JSON queries**: `rows(where=...)` pushes the filter down with S3 Select
|
||||
(`SelectObjectContent`, JSON Lines input, JSON output), so a proposal rule reads matching rows, not whole
|
||||
files. Support is probed once per process against the server (a small Select on a known object); when the
|
||||
server lacks it, the backend fetches and filters locally, logs the
|
||||
fallback once, and returns the same rows. A query never silently returns nothing because a feature is
|
||||
missing.
|
||||
- **Versioning** on the bucket: each proposal's provenance records the object key and version id of every
|
||||
memory row it cites, so its evidence cannot change underneath it (Req 6.4). Object lock (governance mode) is
|
||||
optional for the `proposals/` prefix.
|
||||
- **Lifecycle rules**: raw run outputs under `runs/` expire after a configurable number of days; `repos/`,
|
||||
`facts/` and `proposals/` are kept. The near-full laptop disk stops being the store's limit.
|
||||
- **Presigned URLs** (the step that removes the host receiver for the Kubernetes move, not built in this
|
||||
spec): the reconciler can hand a task presigned PUT URLs for its outputs and GET URLs for its inputs in the
|
||||
start request, so tasks write to the bucket directly and never hold credentials. The interface keeps
|
||||
`presign_put`/`presign_get` so that follow-on changes the reconciler, not the store.
|
||||
- **Bucket notifications** (follow-on): an event on new `repos/` rows can trigger the loop Run
|
||||
(event-driven loop engineering).
|
||||
- **Configuration**: `OUSAST_MEMORY` selects the backend (`file:///path` or `s3://<bucket>[/<prefix>]`); the S3
|
||||
endpoint and credentials come from `.env` (`S3_ENDPOINT`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`,
|
||||
optional `S3_REGION`), never from a manifest, never printed. Client: boto3 as an optional extra
|
||||
(`openultrasast[s3]`); the core install stays PyYAML-only.
|
||||
- **Tests**: the same contract tests run against `FileStore` and against `S3Store` pointed at an S3-compatible
|
||||
endpoint (skipped unless `OUSAST_MEMORY_TEST_S3` is set), including the Select probe with its fallback
|
||||
forced, versioned provenance, and metadata filters. The disk-space refusal applies to `FileStore` only.
|
||||
|
||||
**`remember` task** (`plane/tasks/remember.py`, no Model, budget `{usd: 0, calls: 0}`, one per case). It reads the
|
||||
case's delivered artifacts as inputs: `facts.json`, the `units.jsonl` of `va`/`vb`/`vc`, `final/agreed.json` and
|
||||
`final/disputed.json` with the per-candidate rows, and `alerts.jsonl`. It takes the repository URL and pin from
|
||||
the bound Workspace (`AX_WORKSPACES_YAML`, `OUSAST_GIT_PINS`) and the population from the Run annotation
|
||||
`openultrasast.io/population`. It writes `memory.jsonl` and passes `facts.json` through. Task code interprets
|
||||
the artifacts, so the reconciler and runner stay content-blind (ai-service-plane Req 3.4). After `reconciler.run`
|
||||
returns, `cli._plane` calls `memory.ingest(run_dir)`. `ousast plane remember <run>` does the same by hand.
|
||||
|
||||
**Row kinds** (every row has `id`, `kind`, `repo`, `pin`, `run`, `task`, `population`, `split`, `image`):
|
||||
|
||||
- `facts`: `sha256`, `candidates_digest` (`repo_facts._digest`, `repo_facts.py:188`), `files`.
|
||||
- `verdict`: `candidate` (`path::function`), `family`, `site`, `passes {a, b, c}` (true/false/null), `final`
|
||||
(`agreed | rejected | disputed`), `tiebreak` (bool), `site_match` (bool/null), `usd`, `turns`.
|
||||
- `unit_cost`: `pass`, `path`, `usd`, `calls`, `usage`, `turns`, `model`.
|
||||
- `alert`: `rule_id`, `rule_status` (`enabled | shadow`), `path`, `line`, `function`, `pin_role`
|
||||
(`vulnerable | fixed`).
|
||||
- `proposal_outcome`: `edit_key`, `evidence_digest`, `outcome` (`accepted | reverted | rejected`), `gate_run`.
|
||||
|
||||
**Fact reuse.** A facts entry is valid for the key (repo, pin, candidates digest, runner image digest). The image
|
||||
is in the key because the host's source and the image's source can differ, and facts computed by other code are
|
||||
not the same facts. `ousast plane workspaces --validation-set` writes the key as the Task annotation
|
||||
`openultrasast.io/memory-key`. Annotations are already stripped before ax (`manifests.py:4-6`). Before
|
||||
`reconciler.run`, `cli._plane` calls `memory.seed(run_manifest)`. For each Run entry whose Task carries a key with
|
||||
stored facts and no state yet, seed writes `<run>/<task>/facts.json` and a `summary.json`
|
||||
(`status: done, units_done == units_total == 1, reused: {"sha256", "from_run"}`). It then calls
|
||||
`reconciler.mark_done(run, task, reused=...)`, a public helper of about 10 lines that takes the run lock, sets
|
||||
the state and releases it. The reconciler already skips `done` tasks (`reconciler.py:342`). The receiver already
|
||||
serves any regular file under the run directory as a declared input (`egress.py:242-246`). A changed pin, a
|
||||
changed candidate set or a rebuilt image recomputes.
|
||||
|
||||
### 5. Proposals from memory (Requirement 6.2, 6.4)
|
||||
|
||||
**Interface.** The proposal surface HarnessX would have used is `run_round` (`improve/evolve.py:149-247`). It
|
||||
builds `RuleStatusEdit`s (`improve/validator.py:36-47`), filters them through the journal's novelty gate
|
||||
(`journal.reverted_edit_keys`, `journal.py:26-38`), validates each with `EvolveValidator.validate`
|
||||
(`validator.py:75-81`, which accepts only `RuleStatusEdit` and `PolicyConstantEdit`), smoke-replays and gates.
|
||||
|
||||
`improve/memory.py` exposes:
|
||||
|
||||
```
|
||||
propose_from_memory(rows, ruleset_by_id, current_ledger, journal_rounds, *, excluded) -> list[MemoryProposal]
|
||||
MemoryProposal = (edit: RuleStatusEdit, provenance: dict)
|
||||
```
|
||||
|
||||
`run_round` gains `proposals: Sequence[MemoryProposal] = ()`. It appends their edits after the benchmark's edits
|
||||
and keeps the first edit per `key()`. The validator and the gate are untouched. `ousast improve` gains
|
||||
`--memory [DIR]` (off by default, so the deterministic `improve` output is byte-identical) and
|
||||
`--qualify-population NAME` (repeatable).
|
||||
|
||||
**Rules** (the thresholds are module constants and part of the provenance record). Join key: an `alert` and a
|
||||
`verdict` match when repo, pin and `path::function` are equal. A `disputed` final counts as no evidence in
|
||||
either direction.
|
||||
|
||||
- **M1, repeated false alerts** (`enabled -> shadow`). Rule R qualifies when both hold:
|
||||
- at least 3 R alerts sit at candidates whose final is `rejected`, or on a `fixed` pin inside the case's declared
|
||||
fix range, across at least 2 distinct repositories;
|
||||
- no R alert sits at an `agreed` candidate or a `site_match` site.
|
||||
- **M2, missed declared sites** (`shadow -> enabled`). Shadow rule R qualifies when all three hold:
|
||||
- its shadow alerts hit at least 2 `agreed` candidates with `site_match == true`, across at least 2
|
||||
repositories;
|
||||
- no enabled rule alerts at those candidates;
|
||||
- R has no alert on a `fixed` pin at the same function and at most 1 alert at a `rejected` candidate.
|
||||
- **M3, frequently disputed families.** Families with at least 5 candidates and a dispute rate of at least 0.3 go
|
||||
to `signals.json` as advisory input for plane routing (for example, scheduling pass `c` up front). They are not
|
||||
an edit and never reach the validator.
|
||||
|
||||
**Provenance.** Each proposal's `rationale` is `memory:<proposal_id>`, where `proposal_id` = sha256 of
|
||||
(rule, edit key, sorted evidence row ids, thresholds). The sidecar `memory_proposals.jsonl`, next to the journal,
|
||||
holds:
|
||||
|
||||
- `proposal_id`, `edit_key`, `rule_id`
|
||||
- `evidence` (the row ids plus their repo, pin, run and task)
|
||||
- the counts that met each threshold
|
||||
- `excluded` (populations and pins removed by the guard, with counts)
|
||||
- the store's `index.jsonl` digest at proposal time
|
||||
|
||||
`evolve._record` (`evolve.py:366-396`) adds `"evidence": proposal_id` to an edit's journal entry only when the
|
||||
edit came from memory. Otherwise the journal is unchanged byte for byte. A reader follows journal, then sidecar,
|
||||
then store rows, then the run directory.
|
||||
|
||||
**Novelty.** `journal.reverted_edit_keys` does not block a rule edit that carries a rationale (`journal.py:36`), so
|
||||
a memory proposal would otherwise come back every round. The proposer itself skips any key whose reverted journal
|
||||
entry has the same evidence digest. New evidence is the "new rationale" that allows a retry.
|
||||
|
||||
**Train-on-test guard.** A row is excluded from proposals when any of these holds:
|
||||
|
||||
- (a) its `population` is one of `--qualify-population`, which is also where the proposal will be gated or
|
||||
qualified;
|
||||
- (b) its (repo, pin) is a case of the benchmark manifest being gated;
|
||||
- (c) its (repo, pin) is a `split = "holdout"` pair of the pair catalog, which the per-profile holdout clause
|
||||
(`evolve.py:204-214`) measures.
|
||||
|
||||
This is the rule the closed-loop leak broke, where a holdout pair taught the shape that "recovered" it. Here the
|
||||
filter runs before any rule sees a row. Concretely, rows from `validation-46` (population
|
||||
`independent-v2/validation`) never propose edits that are then qualified on `independent-v2/validation`. The
|
||||
tie-break was designed on that set, so it is development evidence only.
|
||||
|
||||
### 6. The loop as a plane Run (Requirement 6.3)
|
||||
|
||||
`ousast plane workspaces --validation-set ... --loop` extends the generated Run. It also emits a Workspace for each
|
||||
case's fixed pin (the population's `fixed` sha, already read by `generate.fix_ranges`, `generate.py:89`):
|
||||
|
||||
| Task (per case unless noted) | Model | Budget | Inputs -> outputs |
|
||||
| --- | --- | --- | --- |
|
||||
| `<case>-facts`, `-va`, `-vb`, `-agree`, `-vc`, `-final` | verify passes: `deepseek-flash`; others none | as generated today (`generate.verify_budget`) | unchanged |
|
||||
| `<case>-alerts` (new) | none | `{0, 0}`, `serialize: engine` | vulnerable + fixed Workspaces -> `alerts.jsonl` (quick-mode `quick_scan_findings` with enabled and shadow rules, function-attributed) |
|
||||
| `<case>-remember` (new) | none | `{0, 0}` | facts, va, vb, vc, final, alerts -> `memory.jsonl`, `facts.json` |
|
||||
| `loop-measure` (once) | none | `{0, 0}` | every `remember` + `memory-snapshot/rows.jsonl` -> `measure.json` (the increment metrics: declared sites agreed, cost per candidate, dispute rates) |
|
||||
| `loop-propose` (once) | none | `{0, 0}` | measure, snapshot, journal -> `proposals.jsonl`, `memory_proposals.jsonl`, `signals.json` |
|
||||
| `loop-improve` (once) | none | `{0, 0}`, `serialize: engine` | proposals; Workspace = this repository at the commit under test -> `gate.json` (the `RoundOutcome`), `journal.json`, `rule_policy.json` (only when accepted) |
|
||||
|
||||
Before the Run starts, `memory-snapshot` is seeded into the run directory like reused facts. It holds the store
|
||||
rows after the guard, since a sandboxed task cannot read the host store. A `{usd: 0, calls: 0}` budget makes
|
||||
`budget.py` refuse any call, so a model-free task that tries one fails loudly rather than spending. Attribution
|
||||
per task is the existing `ousast plane status` table.
|
||||
|
||||
**The gate's verdict** lands in `~/ousast-results/plane/<run>/loop-improve/gate.json` and is ingested as
|
||||
`proposal_outcome` rows. An accepted `rule_policy.json` is adopted into the repository only by a maintainer
|
||||
commit, as any ledger change is today (README "Rule-level loop").
|
||||
|
||||
### 7. Documentation and specs (Requirement 4)
|
||||
|
||||
- **README.** Replace "Configuring the HarnessX agentic plane" (README.md:293-329) with "The agentic plane (ax)":
|
||||
what runs there (repo-facts, verify, agree, memory), `ousast plane run|status|remember`, a pointer to
|
||||
`ops/ax/README.md`, and one tagged line saying HarnessX was retired. Rename "The HarnessX self-improving cycle"
|
||||
(:330) to "The self-improving cycle". Its calibration ledger and `ousast improve` content is not HarnessX and
|
||||
stays. The MetaAgent paragraph (:393-400) is replaced by the memory proposer (`--memory`, provenance, guard).
|
||||
Drop the `panel_model`/`decider_model` lines at :227-228.
|
||||
- **`docs/examples.md` section 6** becomes "Running the agentic plane", with the plane doctor, run and status
|
||||
commands.
|
||||
- **`docs/threat-model.md:17-20, 32-34`**: egress is the per-task EgressPolicy (receiver, Git hosts, Model
|
||||
`egress-hosts`, passthrough only). Credentials travel only in the start request. Spend is the per-task
|
||||
`usd`/`calls` budget in the Run.
|
||||
- **Retirement notes.** A dated note ("Retired 2026-..: HarnessX was removed by `harnessx-removal`; the agentic
|
||||
plane is ax (`ai-service-plane`), proposals come from the plane memory. History below is unchanged.") goes at
|
||||
the top of the requirements (and design, where present) of `harnessx-self-improving-rulesets`,
|
||||
`learning-harness`, `three-stage-scan` and `openrouter-sast-harness`, and in `.kiro/steering/overview.md` near
|
||||
:175. The other specs that mention HarnessX in passing keep their history untouched.
|
||||
- `RELEASE_NOTES.md` gets one entry listing the retired keys. Kiro tooling stays out of user docs.
|
||||
|
||||
## Data Models
|
||||
|
||||
- `memory.jsonl` (remember output) and `repos/<repo>/<pin>.jsonl`: the rows above, one JSON object per line,
|
||||
sorted keys, stable `id` = sha256(kind, run, task, candidate|path|rule_id+line).
|
||||
- `index.jsonl`: `{"run", "task", "sha256", "rows", "ingested"}`.
|
||||
- `memory_proposals.jsonl`: `{"proposal_id", "edit_key", "rule_id", "rule": "M1|M2", "evidence": [...],
|
||||
"counts": {...}, "thresholds": {...}, "excluded": {...}, "store_digest"}`.
|
||||
- Journal edit entry: the existing fields (`evolve.py:381-384`) plus `"evidence"` only for memory edits.
|
||||
- Equality record: `env.json` `{"commit", "freeze_sha256", "harnessx_importable": false, "tree_sitter": true,
|
||||
"inputs": {path: bytes}, "exclusions": [...]}`.
|
||||
|
||||
## Error Handling
|
||||
|
||||
- `RetiredConfigError`: exit 2 with the key and its replacement, before any stage runs.
|
||||
- A `[harnessx]` section produces exactly one warning, and loading continues.
|
||||
- `memory.ingest` on a run without `remember` outputs is a no-op that says so. A malformed row fails the ingest
|
||||
and names the file and line, and nothing partial is written: rows are staged in a temporary file and renamed.
|
||||
- A low-disk refusal names the store and the free space. The plane run's result is unaffected, and ingest can be
|
||||
repeated with `ousast plane remember`.
|
||||
- `seed` never overwrites an existing task directory or a non-pending state. A facts file whose sha256 no longer
|
||||
matches is dropped from the store with a warning, and that task recomputes.
|
||||
- `propose_from_memory` with an empty store after the guard returns no proposals, and `run_round` reports
|
||||
`no_proposals` as today.
|
||||
- The equality script exits non-zero on the first failed instrument check. It also exits non-zero on any
|
||||
difference outside the exclusion list and prints the first differing path and key.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- `tests/test_config.py`: the one-warning section and the retired keys (above).
|
||||
- `tests/test_no_harnessx_references.py`: the tagged-only search of Requirement 2.4.
|
||||
- `tests/test_plane_memory.py`: ingest is idempotent and content-addresses identical facts. `seed` marks a facts
|
||||
task done and serves its file, and does not for a changed pin, candidate digest or image. Low-disk refusal
|
||||
(monkeypatched `disk_usage`).
|
||||
- `tests/test_plane_remember.py`, `test_plane_alerts.py`, `test_plane_loop.py`: fixture run directories built from
|
||||
the recorded validation-46 shapes. `remember` rows match `agree`'s per-candidate rows. `alerts` attributes
|
||||
functions. `measure` reproduces the increment's recorded metrics on the recorded artifacts. `propose` and
|
||||
`improve` work end to end on a fixture manifest.
|
||||
- `tests/test_improve_memory.py`:
|
||||
- M1 and M2 fire at their thresholds and not below.
|
||||
- A disputed candidate counts neither way.
|
||||
- The guard removes rows by population, manifest case and holdout pair, so a proposal that exists only because
|
||||
of a holdout row disappears.
|
||||
- Provenance ids are stable across runs.
|
||||
- A reverted memory edit is not re-proposed on the same evidence, and is re-proposed on new evidence.
|
||||
- `run_round` without proposals writes a journal byte-identical to today's.
|
||||
- `tests/test_plane_reconciler.py`: `mark_done` under the lock. The existing under-500-lines test stays.
|
||||
- The existing gate and pair tests, `test_benchmark_output_is_byte_identical_to_golden_baseline`
|
||||
(`test_gate.py:104`) and the equality script (section 3) are the regression net.
|
||||
|
||||
## Requirement Traceability
|
||||
|
||||
| Requirement | Design section | Verified by |
|
||||
| --- | --- | --- |
|
||||
| 1.1 fate of every capability | 1 (table) | this document; commit 5 |
|
||||
| 1.2 moved capabilities reachable, Model per task, fixture test | 1 (existing `verify`/`agree`), 4-6 (new tasks are Model-free) | `test_plane_verify.py`, `test_plane_tiebreak.py`, new plane task tests |
|
||||
| 1.3 retired keys fail naming the replacement | 1 (`RetiredConfigError`) | `test_config.py` |
|
||||
| 2.1 modules and guards gone | 2 Source | commit 6; audit test |
|
||||
| 2.2 extra, override, lock gone; installs succeed | 2 Packaging | fresh-venv `pip install .` / `'.[semantic]'` |
|
||||
| 2.3 tests deleted or renamed; audit regenerated | 2 Tests, Module audit | `test_module_audit.py` |
|
||||
| 2.4 search returns only retirement notes | 2 Search | `test_no_harnessx_references.py` |
|
||||
| 3.1 baseline without the extra | 3 | commit 1 record |
|
||||
| 3.2 byte-identical after, exclusions written | 3 Comparison | commit 7 record |
|
||||
| 3.3 `[harnessx]` loads with one warning | 1 | `test_config.py` |
|
||||
| 4.1-4.3 docs, retirement notes, Kiro out of user docs | 7 | commit 8 review |
|
||||
| 5.1-5.2 order, green steps, no deletion before replacement | Commit sequence | per-commit gates |
|
||||
| 6.1 store keyed by repo + pin, fact reuse | 4 | `test_plane_memory.py`, `test_plane_remember.py` |
|
||||
| 6.2 deterministic proposals through validator and gate | 5 | `test_improve_memory.py` |
|
||||
| 6.3 loop as a plane Run with per-task Models and budgets | 6 | `test_plane_loop.py`; `ousast plane status` attribution |
|
||||
| 6.4 provenance and train-on-test guard | 5 | `test_improve_memory.py` |
|
||||
|
||||
## Commit sequence (Requirement 5), mapped to future tasks
|
||||
|
||||
Each commit is gated on the full suite's own exit code, `ruff` and `mypy`.
|
||||
|
||||
1. **Baseline** (3.1): the equality script and `benchmarks/measurements/<date>-harnessx-removal-baseline/`,
|
||||
recorded with the extra uninstalled.
|
||||
2. **Memory store and `remember`** (6.1): `plane/memory.py`, `tasks/remember.py`, `reconciler.mark_done`, seed and
|
||||
ingest wiring in `cli._plane`, generator annotations, fixture tests.
|
||||
3. **Memory proposer** (6.2, 6.4): `improve/memory.py`, `run_round(proposals=...)`, `ousast improve --memory`,
|
||||
sidecar and journal evidence, guard tests.
|
||||
4. **Loop Run** (6.3): `alerts`, `measure`, `propose` and `improve` tasks, `--loop` generation, fixed-pin
|
||||
Workspaces, fixture tests.
|
||||
5. **Retire the scan paths** (1, 3.3): the `cli.py`, `fusion.py`, `verify_judge.py` and `stage_processors.py`
|
||||
changes, `RetiredConfigError`, the one warning, and the test renames that follow. At this point nothing imports
|
||||
`harness_ext`/`hunter_harness`. Their replacements (plane tasks, and the memory proposer from 2-4) have landed.
|
||||
6. **Delete code and packaging** (2): the two modules, the extra, the mypy override, `uv.lock`, the deleted tests,
|
||||
the fixture rename, audit rows and the manifest regeneration, and the reference search test.
|
||||
7. **Equality proof** (3.2): rerun the script, commit the comparison record next to the baseline, and link it in
|
||||
the commit message.
|
||||
8. **Docs and specs** (4): the README, `docs/examples.md`, `docs/threat-model.md`, retirement notes, steering and
|
||||
release notes.
|
||||
|
||||
## Risks
|
||||
|
||||
- **Existing users of the LLM scan paths.** There is no `--llm` flag. Users who set `[models] verifier` or
|
||||
`[fusion] panel_model` get a hard error, which is intended (1.3). Users who installed the extra and set
|
||||
`[models] hunter` lose the second (HarnessX) hunter without an error. Only the release note covers that,
|
||||
because the key is still valid. No measurement suggests either path found anything the remaining paths miss.
|
||||
The absence of a measurement is itself the risk.
|
||||
- **Plane precision is unsolved.** `plane-increment-2.json` agrees 28 candidates, of which 16 are declared
|
||||
sites. The other 12 are unadjudicated and several may be false. M1 treats a `rejected` final as negative
|
||||
evidence, while plane recall is 16/20, so "rejected" is not "safe". The thresholds (2 or more repositories, zero
|
||||
agreed hits) and the unchanged recall/FP gate are the guard. Precision work (judge design) remains out of scope.
|
||||
- **Evidence is thin.** Today's store would hold 15 cases from one validation population, which is exactly the
|
||||
population the guard excludes for its own qualification. Expect `no_proposals` until a second population has
|
||||
run through the loop.
|
||||
- **Disk.** The home volume is 98% full (2.4 GB free). One validation-46-sized Run adds about 3 MB of facts
|
||||
(content-addressed; about 0.1 MB when pins are unchanged) plus about 0.5 MB of rows. Ingest refuses below 1 GiB
|
||||
free. The store has no eviction in this spec.
|
||||
- **The baseline measures the environment as well as the code.** `.venv` has `harnessx` installed today. A
|
||||
baseline taken without re-syncing would include it. The script's `find_spec` and freeze checks exist for that.
|
||||
- **Seeding writes reconciler state from outside the reconciler.** It is confined to `mark_done` under the run
|
||||
lock. The reconciler's line budget (465 of 500) leaves no room for fact reuse inside the reconciler.
|
||||
@@ -0,0 +1,125 @@
|
||||
# Requirements Document
|
||||
|
||||
## Introduction
|
||||
|
||||
HarnessX is the optional agentic plane (`openultrasast[harnessx]`): an LLM hunter pool, an LLM judge and LLM
|
||||
fusion panels, each behind a capability guard with a deterministic fallback, plus a lazy bridge that hosts
|
||||
deterministic stages under a HarnessX processor. The ai-service-plane increment met its gate on google/ax
|
||||
(`benchmarks/independent/plane-increment-2.json`), which is the condition the maintainer set on 2026-09-29:
|
||||
"goal is to remove harnessx and to shift to ax fully once we establish we have value." This feature removes
|
||||
HarnessX from the product and moves every LLM capability it carried either onto the plane or into a recorded
|
||||
retirement. Inventory: `brief.md` (verified 2026-09-30).
|
||||
|
||||
Maintainer, 2026-09-30: "Please remove harnessx completely. the memory layer that saves tokens takes over the
|
||||
evolutionary improvements as well as loop engineering." So nothing of HarnessX stays, not even as an optional
|
||||
path, and the self-improving cycle HarnessX was meant to drive (the LLM `MetaAgent.evolve` proposer plugged into
|
||||
`improve/evolve.py`) is carried by the plane's memory layer instead: what runs record (repository facts,
|
||||
verdicts, disputes, costs) becomes the input to proposals and to the run-measure-propose-gate loop.
|
||||
|
||||
## Boundary Context
|
||||
|
||||
- **In scope**: the HarnessX-only modules (`harness_ext.py`, `hunter_harness.py`); the guarded branches in
|
||||
`cli.py` (scan `--llm` path, `HxScanOrchestrator`), `verify_judge.py` (judge path), `fusion.py` (LLM panels),
|
||||
`stage_processors.py` (`host_under_harnessx`), `config.py` (`HarnessxConfig`); the `harnessx` extra, its mypy
|
||||
override and lock entry; the tests that exercise HarnessX; README and docs sections; dated retirement notes in
|
||||
the specs and steering that name HarnessX.
|
||||
- **Also in scope**: the plane memory layer as the source of evolutionary proposals and the loop (Requirement 6),
|
||||
replacing the HarnessX proposer.
|
||||
- **Out of scope**: changing the deterministic engine, rulesets or `evolve`'s validator and gate (proposals change
|
||||
source, not how they are validated and gated); new detection logic; qualification on an independent population; the plane's own precision work (judge design) --
|
||||
where a capability moves to the plane, this spec wires it, it does not improve it.
|
||||
- **Adjacent expectations**: every deterministic path (quick mode, `regress`, `benchmark`, `gate`, `evolve`)
|
||||
produces byte-identical output before and after; `vulnerabilities-over-plumbing` applies, so capabilities are
|
||||
kept or retired by what they detect, reported in TP/FP terms where a measurement exists.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement 1: HarnessX is removed completely; the plane is the only agentic path
|
||||
|
||||
**User Story:** As the maintainer, I want no HarnessX path left, optional or not, and every LLM capability either
|
||||
on the plane or explicitly retired, so that nothing is lost silently.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. No HarnessX code path remains, optional or guarded. The design lists each capability -- LLM hunter pool
|
||||
(`ousast scan --llm`), LLM judge (`verify_judge`), LLM fusion panels (`fusion`), deterministic stages hosted
|
||||
under HarnessX (`host_under_harnessx`), the LLM evolution proposer -- with its fate: carried by a plane task
|
||||
(named), carried by the memory layer (Requirement 6), or retired with the deterministic path kept.
|
||||
2. A capability moved to the plane is reachable from the CLI through the plane (`ousast plane run` with a
|
||||
documented Run, or a CLI flag that submits one), binds its own Model per task (ai-service-plane Req 7), and
|
||||
has a fixture test like the other plane tasks.
|
||||
3. A retired capability's CLI flags and config keys fail with a message that names the replacement or says it
|
||||
was retired, never a silent no-op.
|
||||
|
||||
### Requirement 2: HarnessX code, packaging and tests are gone
|
||||
|
||||
**User Story:** As the maintainer, I want no HarnessX code path left to maintain.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. `harness_ext.py` and `hunter_harness.py` are deleted; no module imports `harnessx`; `use_harnessx`
|
||||
parameters, `has_harnessx` checks and `host_under_harnessx` are removed; the deterministic fallbacks become the
|
||||
only code path where a capability is retired.
|
||||
2. The `harnessx` extra, the mypy override for `harnessx.*` and the lock entry are removed; `pip install .` and
|
||||
`pip install '.[all]'` (or whatever extras remain) succeed.
|
||||
3. Tests that only exercise HarnessX are deleted; tests that assert fallback behaviour are kept and renamed to
|
||||
describe the behaviour, not the absence of HarnessX; the module audit is regenerated with no orphan.
|
||||
4. A search for `harnessx` (case-insensitive) in `src/`, `tests/`, `pyproject.toml` and the README returns only
|
||||
the retirement notes of Requirement 4.
|
||||
|
||||
### Requirement 3: Deterministic behaviour is unchanged, and proven so
|
||||
|
||||
**User Story:** As the maintainer, I want evidence, not assurance, that removing HarnessX changed nothing a
|
||||
deterministic run produces.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. Before any deletion, a baseline is recorded without the extra installed: the full host suite, and the outputs
|
||||
of `ousast regress` and one `ousast benchmark` replay on the development pins, stored under
|
||||
`benchmarks/measurements/`.
|
||||
2. After the removal the same commands produce byte-identical outputs (timestamps and durations excluded, and
|
||||
the exclusion list is written down); any difference blocks the merge.
|
||||
3. A config file that still carries a `[harnessx]` section loads with one warning that names the plane, and the
|
||||
section is otherwise ignored.
|
||||
|
||||
### Requirement 4: Documentation and specs tell the new story
|
||||
|
||||
**User Story:** As a reader of the README or a spec, I want to learn that the agentic plane is ax, not HarnessX.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. The README sections "Configuring the HarnessX agentic plane" and "The HarnessX self-improving cycle" are
|
||||
replaced by a short section on the ax plane that points at `ops/ax/README.md`; `docs/examples.md` section 6 and
|
||||
`docs/threat-model.md` (egress and spend caps) describe the plane's egress policy and budgets instead.
|
||||
2. Specs that name HarnessX (`harnessx-self-improving-rulesets`, `learning-harness`, `three-stage-scan`,
|
||||
`openrouter-sast-harness`) and `.kiro/steering/overview.md` get a dated retirement note at the top; their
|
||||
history is not rewritten.
|
||||
3. Maintainer-only Kiro tooling stays out of user docs (standing rule).
|
||||
|
||||
### Requirement 5: The removal lands in one reviewable sequence
|
||||
|
||||
**User Story:** As the maintainer, I want to review the removal in steps that each leave main green.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. Order: baseline (3.1) -> capabilities moved or retired (1) -> code and packaging removed (2) -> equality proof
|
||||
(3.2) -> docs and specs (4). Each step is a commit gated on the full suite's own exit code, ruff and mypy.
|
||||
2. No step deletes code whose replacement (Requirement 1) has not landed and passed its fixture test.
|
||||
|
||||
### Requirement 6: The memory layer carries evolutionary improvement and loop engineering
|
||||
|
||||
**User Story:** As the maintainer, I want the token-saving memory layer to be what the system learns from, so
|
||||
that improvement proposals and the improvement loop come from recorded evidence instead of a HarnessX agent.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. Plane runs persist their memory beyond the run directory in one store keyed by repository and pin: repository
|
||||
facts (`repo-facts`), per-candidate verdicts per pass with the final decision, disputes and tie-breaks, and cost
|
||||
and tool turns per unit; later runs reuse facts for an unchanged pin instead of recomputing them.
|
||||
2. `evolve`'s proposals are generated from that store (for example: families or rules whose candidates are often
|
||||
disputed, missed declared sites, repeated false alerts), deterministically and without a HarnessX agent; they
|
||||
pass through the existing validator and gate unchanged.
|
||||
3. The improvement loop -- run, measure, propose, validate, gate -- is expressible as a plane Run whose tasks bind
|
||||
their own Models and budgets (ai-service-plane Req 7), so each iteration's token spend is attributed per task.
|
||||
4. A proposal records which memory entries it came from, so a later reader can trace every accepted change to the
|
||||
evidence behind it; proposals never train on the population used to qualify them (the train-on-test rule).
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"feature_name": "harnessx-removal",
|
||||
"created_at": "2026-09-30T08:00:00Z",
|
||||
"updated_at": "2026-09-30T16:00:00Z",
|
||||
"language": "en",
|
||||
"phase": "implementation-complete",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
},
|
||||
"design": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
},
|
||||
"tasks": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": true
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
# Implementation Plan
|
||||
|
||||
Order follows the design's commit sequence (Requirement 5): nothing is deleted before its replacement has landed
|
||||
and passed its fixture test. Every task is a commit gated on the full suite's own exit code, `ruff check`,
|
||||
`ruff format --check` and `mypy`; the reconciler stays under 500 lines.
|
||||
|
||||
- [x] 1. Baseline without the extra
|
||||
- Equality script (design §3) with its `find_spec("harnessx")` and freeze checks; re-sync `.venv` without the
|
||||
`harnessx` extra first.
|
||||
- Record `benchmarks/measurements/<date>-harnessx-removal-baseline/`: full suite, `ousast pairs --json`, the
|
||||
`python-vulnerable` quick benchmark, one standard-mode scan, `ousast improve --dry-run`; the exclusion list
|
||||
written next to it.
|
||||
- _Requirements: 3.1, 5.1_
|
||||
|
||||
- [x] 2. Memory store and the `remember` task
|
||||
- `plane/memory.py`: the `MemoryStore` interface with `FileStore` and `S3Store` (object metadata/tags,
|
||||
S3 Select JSON queries probed with a local fallback, versioned provenance, lifecycle, presign stubs;
|
||||
`OUSAST_MEMORY` selects; S3 settings from `.env`; boto3 as an optional extra); JSON rows per
|
||||
repository + pin, facts by content hash, `FileStore` ingest refuses below 1 GiB free,
|
||||
`plane/tasks/remember.py` (model-free), `reconciler.mark_done` under the run lock, seed and ingest wiring in
|
||||
`cli._plane`, generator annotations; fact reuse when repository, pin, candidate set and runner image match.
|
||||
- Fixture tests: rows from a recorded Run, reuse on an unchanged pin, recompute on a changed one, disk refusal.
|
||||
- _Requirements: 6.1, 6.4_
|
||||
|
||||
- [x] 3. Proposals from memory
|
||||
- `improve/memory.py` with rules M1 (demote) and M2 (promote) and the advisory disputed-family signal;
|
||||
`run_round(proposals=...)`; `ousast improve --memory` (opt-in; default output byte-identical); provenance per
|
||||
proposal in the sidecar and journal; the train-on-test guard (qualification population, gated benchmark cases,
|
||||
holdout pairs).
|
||||
- Tests: each rule on crafted rows, the guard drops excluded rows, provenance round-trips, default `improve`
|
||||
unchanged.
|
||||
- _Requirements: 6.2, 6.4_
|
||||
|
||||
- [x] 4. The loop as a plane Run
|
||||
- Tasks `alerts`, `measure`, `propose`, `improve` (no Model, zero budget); `--loop` Run generation; fixed-pin
|
||||
Workspaces; the gate's verdict in `<run>/loop-improve/gate.json`; adoption stays a maintainer commit.
|
||||
- Fixture tests with the fake ax; one live loop Run on the local ax recorded (no model spend beyond the verify
|
||||
passes it reuses from memory).
|
||||
- _Requirements: 6.3_
|
||||
|
||||
- [x] 5. Retire the HarnessX scan paths
|
||||
- `cli.py`, `fusion.py`, `verify_judge.py`, `stage_processors.py` changes; `RetiredConfigError` for
|
||||
`[models] verifier`, `[fusion] panel_model`/`decider_model`; one warning for a `[harnessx]` section (the only
|
||||
tagged retirement text allowed in `src`, enforced by a test); tests renamed to the behaviour they assert.
|
||||
- _Requirements: 1.1, 1.2, 1.3, 3.3_
|
||||
|
||||
- [x] 6. Delete HarnessX code and packaging
|
||||
- Delete `harness_ext.py`, `hunter_harness.py`, `verify_judge.py`'s judge path, the HarnessX-only tests; remove
|
||||
the extra, the mypy override and the `uv.lock` entry; rename the conftest fixture; audit rows and manifest; the
|
||||
reference search test (Req 2.4).
|
||||
- _Requirements: 2.1, 2.2, 2.3, 2.4_
|
||||
|
||||
- [x] 7. Equality proof
|
||||
- Rerun the baseline script; commit the comparison record next to the baseline; any difference outside the
|
||||
exclusion list blocks the merge.
|
||||
- _Requirements: 3.2_
|
||||
|
||||
- [x] 8. Documentation and specs
|
||||
- README ax-plane section pointing at `ops/ax/README.md`; `docs/examples.md` §6 and `docs/threat-model.md`
|
||||
(egress policy, budgets); dated retirement notes in the four specs and `.kiro/steering/overview.md`; release
|
||||
note naming the silent loss of the second hunter for users who set `[models] hunter`.
|
||||
- _Requirements: 4.1, 4.2, 4.3_
|
||||
@@ -1,5 +1,8 @@
|
||||
# Design Document
|
||||
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
|
||||
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
|
||||
|
||||
## Overview
|
||||
|
||||
**Purpose.** This feature splits OpenUltraSAST's single overloaded regex layer — which today is detector, severity authority, and volume knob at once — into three governed owners: **rules detect**, a **central CWE policy decides severity and scope**, and a **single project score** becomes the optimization target. On top of that governance plane it adopts the HarnessX runtime exactly where it pays off — the LLM hunter loop, the LLM-judge verifier, and a meta-harness self-improvement loop that adapts rules and policy gates under hard bounds.
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** retired with HarnessX (see the note below). Its task 6.3 modules `stage_processors.py` and `slot_contract.py` were deleted on 2026-10-02: nothing imported them. Kept as decision history; nothing in src, tests or CI points here.
|
||||
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
|
||||
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
|
||||
|
||||
## Introduction
|
||||
|
||||
OpenUltraSAST today fuses three responsibilities into one regex layer: detection patterns also carry their own severity strings and act as the sole volume control, so a noisy rule can dominate scan output, set its own criticality, and fire forever with no central governance. This feature splits those responsibilities into three governed owners — rules detect, a central CWE policy decides severity and scope, and a single project score becomes the optimization target — and adopts the HarnessX runtime exactly where it pays off: the LLM hunter loop, the LLM-judge verifier, and a meta-harness self-improvement loop that adapts rules and policy gates under hard bounds.
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Brief: ir-consolidation — stop maintaining an IR we did not need, and adopt one we do
|
||||
|
||||
> **Status 2026-10-02:** dormant. Brief only and unapproved. The subtraction it asks for still needs a measured recall comparison before the flat IR goes.
|
||||
|
||||
**Status:** brief only, unapproved.
|
||||
|
||||
## Why now
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
# Brief: a trained decision engine and A/B experiments instead of hand-coded detection
|
||||
|
||||
Maintainer, 2026-09-30: "you need to go to a different approach with a b testing to extrapolate detection rather
|
||||
than hardcoding it and train a decision engine".
|
||||
|
||||
## Why
|
||||
|
||||
Every detection gain so far was a hand edit made after a miss: regex rules, taint vocabulary, guards, prompt
|
||||
lines, a tie-break rule. Each edit was tuned on the cases that exposed it and had to be re-qualified on a fresh
|
||||
population; v1 -> v2 showed engine tuning does not transfer (0/17). The plane changed what is possible: every task
|
||||
records per-candidate outcomes, costs and repository facts into one memory store, and runs are cheap to repeat.
|
||||
|
||||
## Direction
|
||||
|
||||
- Instruments become signal producers: quick rules, the Joern engine, model sink classification, verify passes,
|
||||
tie-break, repo facts (callers, reachability), triage. None decides alone.
|
||||
- A decision engine, trained on labelled outcomes, turns the signals for a candidate into a calibrated probability
|
||||
that it is a real vulnerability of its family; operating points (block, report, drop) are thresholds on it.
|
||||
- Changes to the pipeline (prompts, models, pass counts, rule sets, features) are variants compared by A/B
|
||||
experiments run as plane Runs with pre-registered metrics and confidence intervals, never adopted on a single
|
||||
run's point estimate.
|
||||
- Detection on unseen code is extrapolated from measured learning curves and leave-one-population-out results,
|
||||
reported with uncertainty, and checked once on the frozen v3.
|
||||
|
||||
## Data on hand (2026-09-30)
|
||||
|
||||
- Pair corpus: 317 vulnerable/fixed pairs, 342 labelled findings (`ousast pairs`), with train/holdout splits.
|
||||
- Populations: v1 (spent), v2 validation set (46 candidates, 20 declared sites; plane runs recorded), PHP
|
||||
development corpus (pmpro, v1 PHP, MediaWiki, WP Statistics). v3 (PHP, 16 cases) is being frozen now and stays
|
||||
untouched until the final check.
|
||||
- Memory store rows: facts, per-pass verdicts, disputes, tie-breaks, costs, alerts, coverage.
|
||||
|
||||
## Risks
|
||||
|
||||
- Small labelled data: the engine must be simple and calibrated (regularised linear or shallow trees), with
|
||||
population-level folds; a deep model would memorise.
|
||||
- Leakage: labels and features must never come from the population a number is reported on (the train-on-test
|
||||
rule; see the closed-loop leak).
|
||||
- Label quality: "rejected by the plane" is not "safe"; negatives come from fixed and benign pins and adjudicated
|
||||
findings, not from absence of an alert.
|
||||
|
||||
## Relation to earlier specs
|
||||
|
||||
Replaces the hand-edit loop of `improve` for detection decisions (the memory proposer's M1/M2 status edits become
|
||||
one variant family among others). `learning-harness` and `harnessx-self-improving-rulesets` are retired with
|
||||
HarnessX. `finding-feedback-loop`, `constrained-detector` and `model-grounded-detection` are inputs to read in
|
||||
design, not dependencies.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,162 @@
|
||||
# Requirements Document
|
||||
|
||||
## Introduction
|
||||
|
||||
Detection decisions stop being hand-coded. Every instrument becomes a producer of per-candidate signals, a trained
|
||||
decision engine turns signals into a calibrated probability per candidate and family, pipeline changes are chosen
|
||||
by A/B experiments run on the plane, and expected detection on unseen code is extrapolated with stated uncertainty.
|
||||
Maintainer direction and context: `brief.md`.
|
||||
|
||||
## Boundary Context
|
||||
|
||||
- **In scope**: a feature record per candidate built from every instrument and the memory store; a labelled
|
||||
dataset with provenance; training, calibration and versioning of the decision engine; A/B experiments as plane
|
||||
Runs; learning curves and leave-one-population-out estimates; the engine's use in `ousast scan`, `ousast
|
||||
pre-push` and plane Runs; a final one-time check on frozen population v3.
|
||||
- **Out of scope**: new instruments (they plug in as signal producers later); deep or large models; any change to
|
||||
population v3 or its protocol; publishing results (licences still gate that).
|
||||
- **Adjacent expectations**: instruments keep working standalone; the memory store (harnessx-removal Req 6) is
|
||||
the system of record; the plane attributes every experiment's token spend per task.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement 1: Instruments emit signals, not verdicts
|
||||
|
||||
**User Story:** As the maintainer, I want every detector's output to become evidence for one decision, so that no
|
||||
single hand-written rule or prompt decides whether code is vulnerable.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. For each candidate (repository, pin, file, function, family) the system assembles one feature record from every
|
||||
instrument that ran: quick-rule hits (rule ids, counts), engine findings and questions completed, model sink
|
||||
classification, triage, verify pass verdicts and turns, tie-break, repository facts (callers, entry-point
|
||||
distance), coverage, language and family.
|
||||
2. A missing instrument is recorded as missing, never as a negative signal (coverage `none` stays distinct from
|
||||
"ran and found nothing").
|
||||
3. Feature records are stored in the memory store with the instrument versions and image digests that produced
|
||||
them, so any record can be rebuilt.
|
||||
|
||||
### Requirement 2: Labels come from ground truth with provenance
|
||||
|
||||
**User Story:** As the maintainer, I want the engine to learn only from outcomes we can defend.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. Positive labels: candidates matching a declared site or fix range on a vulnerable pin (populations, pair
|
||||
corpus). Negative labels: the same functions on fixed pins, benign-control pins, and findings adjudicated false
|
||||
by the maintainer or a judge task with recorded evidence.
|
||||
2. A plane verdict alone (agreed, rejected, disputed) is a feature, never a label.
|
||||
3. Every label records its source (population, pair, adjudication), split, and the date it was created; labels from
|
||||
a reserved or frozen population are unusable for training until that population's evaluation is recorded.
|
||||
|
||||
### Requirement 3: An AI classifier with local memory, compiled DSPy-style
|
||||
|
||||
**User Story:** As the maintainer, I want the decision made by an AI classifier that learns from a small, growing
|
||||
memory of labelled cases, because "We will not be able to scale up data" and "we need to have the decision
|
||||
engine on AI classifier not ML. with local memory, dspy style" (maintainer, 2026-10-01).
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. The decision engine is a declared language-model program (DSPy-style signature: candidate code, instrument
|
||||
signals, inferred repository roles in; verdict, family, confidence and rationale out), built in-house on the
|
||||
plane's metered client so every call stays within per-task budgets, attribution and Model routing; no
|
||||
statistical model is trained.
|
||||
2. For each candidate the program retrieves similar labelled cases from the local memory store as in-context
|
||||
examples: nearest neighbours by the repository-agnostic signal profile, re-ranked by code embeddings
|
||||
(OpenRouter, embeddings only); identities are never used for retrieval.
|
||||
3. The program is compiled, not fitted: its instructions and example-selection policy are optimised against a
|
||||
pre-registered metric on training folds (bootstrap few-shot and instruction search); each compiled program is
|
||||
versioned in the memory store with its data snapshot, model, prompts, retrieval settings and metrics, and the
|
||||
same inputs reproduce it.
|
||||
4. Retrieval never crosses the evaluation boundary: when a number is reported for a repository, examples come
|
||||
only from other repositories (and other frameworks in leave-one-framework-out folds); a test proves it.
|
||||
5. The confidence the program reports is calibrated on held-out repositories before operating points are set;
|
||||
where it does not calibrate, only ADVISORY is offered (Req 7.4).
|
||||
6. New labels (adjudications, dismissals in the opt-in adaptation layer) improve decisions by entering memory,
|
||||
without retraining; their effect is measured by A/B experiments like any other change.
|
||||
|
||||
### Requirement 4: Pipeline changes are chosen by A/B experiments
|
||||
|
||||
**User Story:** As the maintainer, I want changes to prompts, models, pass counts, rule sets and features compared
|
||||
by controlled experiments, so that nothing is adopted on one run's number.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. An experiment is declared before it runs: arms (variants as plane Run configurations), unit of randomisation
|
||||
(candidate, file or repository), metric (detection, precision, cost per candidate), minimum detectable effect,
|
||||
sample size or sequential stopping rule, and budget.
|
||||
2. Arms run on the same units where the design allows (paired), with assignment recorded; results and spend go to
|
||||
the memory store per arm.
|
||||
3. A variant is adopted only when the pre-registered test shows a difference with its confidence interval
|
||||
excluding zero, or declared equivalent within a margin for cost-only changes; otherwise the result is recorded
|
||||
as inconclusive.
|
||||
4. The first experiment is the known open question: the known-callers prompt line (17 vs 19 sites flagged in any
|
||||
pass), on the development corpus.
|
||||
|
||||
### Requirement 5: Detection is extrapolated, with uncertainty
|
||||
|
||||
**User Story:** As the maintainer, I want to know how well the system will detect on code nobody has seen, before
|
||||
and without tuning on it.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. Learning curves: engine metrics as a function of memory size (labelled cases available for retrieval) and number of populations, per family,
|
||||
with confidence bands.
|
||||
2. Expected recall and precision on unseen code are reported per family and language as intervals from the
|
||||
population-level folds; a family with too few labels is reported as "insufficient data", never as a number.
|
||||
3. The extrapolation is checked exactly once against frozen population v3 under protocol v3, after the engine and
|
||||
its operating points are fixed; the comparison of predicted and measured intervals is recorded.
|
||||
|
||||
### Requirement 6: The engine is what users get
|
||||
|
||||
**User Story:** As a user scanning a fresh codebase, I want the decision engine's judgement, not a raw list of
|
||||
rule hits.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. `ousast scan`, `ousast pre-push` and plane Runs report findings with the engine's probability, the operating
|
||||
point they passed and the top contributing features.
|
||||
2. Without a trained model available the tools fall back to today's behaviour and say so in the report.
|
||||
3. Hand-written rules remain as signal producers; changing their enabled/shadow status no longer changes what is
|
||||
reported unless the engine's decision changes.
|
||||
|
||||
### Requirement 7: Built for unseen repositories -- a repo safety net
|
||||
|
||||
**User Story:** As the maintainer, I want the engine judged by how it behaves on repositories it never saw, because
|
||||
"the tool will never run against the test cases it's being trained with. it's purpose is a repo safety net."
|
||||
(maintainer, 2026-09-30).
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. Every reported metric comes from folds grouped by repository across all corpora: a repository in an evaluation
|
||||
fold contributes no row to training.
|
||||
2. Features are repository-agnostic by an allow-listed schema (no repository, path, file, function or commit
|
||||
identities), enforced by a test.
|
||||
3. Two operating points serve the safety net: BLOCK, chosen for >= 95% precision on unseen repositories, and
|
||||
ADVISORY, trading precision for recall; both reported with out-of-repository intervals. The decision unit is a
|
||||
candidate in changed code (`pre-push` delta), with whole-repository scans as a secondary mode.
|
||||
4. Calibration is checked on held-out repositories; where it does not hold, only ADVISORY is offered for that
|
||||
family.
|
||||
5. Any adaptation to a user's own repository (facts reuse, their dismissals) is opt-in, separate from the
|
||||
generalisation model, and never contributes to its reported numbers.
|
||||
|
||||
### Requirement 8: No framework or application knowledge as a prerequisite
|
||||
|
||||
**User Story:** As the maintainer, I want the safety net to work on any application in a language, because "if
|
||||
there is hardcoded application specification needed to build trees this is problematic. wordpress isn't the only
|
||||
php application. same for python, js etc" (maintainer, 2026-09-30).
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. Hand-written knowledge is limited to the language level (built-ins, superglobals, process execution, raw
|
||||
database drivers, deserialisers). Roles specific to a framework or application (its request objects, query
|
||||
builders, hooks, sanitisers, guards) are inferred per repository at scan time from its own source and its
|
||||
dependencies, not required from a hand-written table.
|
||||
2. Existing framework-specific entries (WordPress, Django, Flask, Express, Spring and others in `ruleset/semantic/`
|
||||
and `ruleset/<lang>/`) are tagged with their framework and enter the engine only as optional prior features; a
|
||||
prior is kept only if an A/B experiment on held-out repositories shows it helps, and the engine must still work
|
||||
with every prior switched off.
|
||||
3. Evaluation is stratified by framework and includes leave-one-framework-out folds (for example: train without any
|
||||
WordPress repository, evaluate on WordPress), so a number never depends on having seen the target's framework;
|
||||
results are reported per framework and for "framework unseen in training".
|
||||
4. The share of each reported result that depends on framework priors is stated (with priors vs without).
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"feature_name": "learned-decision-engine",
|
||||
"created_at": "2026-09-30T14:00:00Z",
|
||||
"updated_at": "2026-10-01T10:20:00Z",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
},
|
||||
"design": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
},
|
||||
"tasks": {
|
||||
"generated": true,
|
||||
"approved": true
|
||||
}
|
||||
},
|
||||
"ready_for_implementation": true,
|
||||
"note": "2026-10-01: AI-classifier design approved; 6.1-6.5 (no model spend) authorised; paid steps need go-ahead."
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
# Implementation Plan
|
||||
|
||||
Order follows the design's commit sequence. Every task is a commit gated on the full suite's own exit code,
|
||||
`ruff check`, `ruff format --check` and `mypy`. No reported number changes until task 9 adopts a compiled program;
|
||||
without an adopted program every path is today's. Requirement 3 changed on 2026-10-01 (an AI classifier with local
|
||||
memory, compiled DSPy-style in-house); tasks 1-5 are unaffected, tasks 6-10 are re-planned for it. Sub-tasks marked
|
||||
**(budget go-ahead)** spend model money or host hours: each starts only after the maintainer confirms its budget
|
||||
against the provider balance read just before it, and none starts with a ceiling above that balance. No test makes
|
||||
a model or embedding call.
|
||||
|
||||
- [x] 1. Schema and features
|
||||
- `learn/schema.py` (closed, typed allow-list; no identities; missing != zero), `learn/features.py` (host
|
||||
builder), new memory `KINDS`, `plane/tasks/features.py` and the `remember` input; tests enforcing the
|
||||
allow-list and the exclusion of label-carrying fields.
|
||||
- _Requirements: 1.1, 1.2, 1.3, 7.2_
|
||||
|
||||
- [x] 2. Framework tags
|
||||
- `ruleset/frameworks.toml`; tag and split every framework/library entry listed in design section 2; `priors=`
|
||||
in the semantic and quick loaders (default `all`, so today's scan is unchanged -- the golden benchmark test
|
||||
stays byte-identical); a lint test that no untagged framework entry remains.
|
||||
- _Requirements: 8.2, 8.4_
|
||||
|
||||
- [x] 3. Role inference
|
||||
- `learn/roles.py` (deterministic wrapper inference), `plane/tasks/roles.py` (model roles, the `model_sinks.py`
|
||||
approach), the vocabulary overlay for the engine; fixture tests with no model calls.
|
||||
- _Requirements: 8.1_
|
||||
|
||||
- [x] 4. Labels
|
||||
- `learn/sources.toml` (fail-closed allowed sources, v3 excluded unread), `learn/labels.py`, `ousast learn
|
||||
labels`, repository-group normalisation and CVE merge, the `assumed_benign` source mined from non-security
|
||||
commits (separate, never merged); a first label snapshot committed as counts only.
|
||||
- _Requirements: 2.1, 2.2, 2.3, 7.1_
|
||||
|
||||
- [x] 5. Harvest (needs budget go-ahead)
|
||||
- A plane Run of verify a/b on the verify-family pairs and development cases (~$6, ceiling $10), model roles on
|
||||
the candidate files (ceiling $10), engine features for the pair corpus on the host (frozen source, one
|
||||
container at a time, background).
|
||||
- Done 2026-10-01 (`benchmarks/measurements/2026-10-01-decision-engine-harvest/record.json`): verify $2.54 and roles
|
||||
$1.40 by attribution. The engine job finished the same day (682 of 686 pair sides; the 2 family-unknown pairs
|
||||
not run) and the feature records were rebuilt; dev-php pins have no engine run (`missing`).
|
||||
- Advisory-fix pairs harvested 2026-10-02 (`benchmarks/measurements/2026-10-02-decision-engine-harvest-advisory/record.json`):
|
||||
194 sides, verify a/b on the 34 untrusted_destination sides and model roles on all ($0.38 metered, ceiling $5),
|
||||
engine on the host (54 ran, 58 asked no question, 82 no language model); static memory 724 -> 904 examples. The
|
||||
leak audit flags nothing new in the static profile; the plane profile newly flags `verify.flag_b` and `ms.operation`.
|
||||
- _Requirements: 1.1_
|
||||
|
||||
- [ ] 6. Program, local memory and compilation (design sections 4-5)
|
||||
- [x] 6.1 Excerpts and the example store (no model)
|
||||
- `learn/excerpt.py` (function span at the label's pin, bounds 80/4,000 for candidates and 40/2,000 for
|
||||
examples, numbered lines, no path/repository/commit, advisory ids and security wording in comments
|
||||
redacted, delta diff); `learn/examples.py` and `ousast learn memory build` (label + feature record +
|
||||
excerpt -> `example` row; refuses `origin: user` and sources outside `sources.toml`; exits 2 when more than
|
||||
10% of a source has no excerpt); `example` added to `KINDS`; `put_blob`/`get_blob` on both backends.
|
||||
- Tests: `test_learn_excerpt.py`, example-builder refusals, blob round trip on FileStore (S3 gated).
|
||||
- _Requirements: 3.2, 3.6, 7.2_
|
||||
- [x] 6.2 Metered embeddings and the cache
|
||||
- `plane/models/openrouter-embedding.yaml` (`input_per_m: 0.02`), `MeteredEmbeddingClient` in
|
||||
`plane/budget.py`, usage kept by `parse_embedding_response`; cache `embeddings/<model>/<excerpt_sha>.json`;
|
||||
`ousast learn memory embed`; scripted-client tests.
|
||||
- **(budget go-ahead)** embedding the store: ~5.0 M tokens, ~$0.10, ceiling $1 (OpenRouter account).
|
||||
Done 2026-10-01: 616 distinct excerpts of the 724 static examples, 198,819 tokens, 20 calls, $0.0040 metered
|
||||
(`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`).
|
||||
- _Requirements: 3.2_
|
||||
- [x] 6.3 Retrieval and the evaluation boundary (no model)
|
||||
- `learn/retrieve.py`: per-instrument Gower distance with explicit missing, `n1 = 40`, embedding re-rank
|
||||
`r = lam(1 - D) + (1 - lam)cos`, `signals_only` mode, balance rule (`k_ret = 6`), `eligible(example, target,
|
||||
fold)`, near-duplicate drop in evaluation, deployment self-exclusion.
|
||||
- `learn/folds.py`: compile split (25% of groups), outer grouped 5-fold, leave-one-source-out,
|
||||
leave-one-framework-out.
|
||||
- Tests: `test_learn_retrieve.py` including `test_boundary` (every rendered example and demonstration eligible,
|
||||
for all fold kinds; `memory.seed` excludes the case's group), `test_learn_folds.py`.
|
||||
- _Requirements: 3.2, 3.4, 7.1, 8.3_
|
||||
- [x] 6.4 The program (no model in tests)
|
||||
- `learn/program.py`: the signature, prompt rendering (instruction + demonstrations as a byte-identical prefix,
|
||||
then retrieved examples, then the candidate), JSON parsing with one retry -> `unsure, parse_failed`,
|
||||
`Classify(k)` with temperature 0 then 0.7, the score `s` and majority verdict; `responses/<sha>.json` cache
|
||||
and replay; `temperature`/`logprobs` pass-through in `MeteredClient.complete` and `DeepSeekChatClient._call`.
|
||||
- Tests: `test_learn_program.py` (no `EXCLUDED_FIELDS` in prompts, prefix identity, parsing, `unsure` cannot
|
||||
BLOCK, budget ceiling, replay).
|
||||
- _Requirements: 3.1, 3.3_
|
||||
- [x] 6.5 Compile, calibrate, evaluate (no model in tests)
|
||||
- `learn/compile.toml` (pre-registered metric: balanced Brier score; seeds; budget), `learn/compile.py`
|
||||
(bootstrap few-shot on `C_boot`, 6 proposed + baseline instructions scored on `C_val`, 4 demonstration sets,
|
||||
alternates for leave-one-framework-out, artifact `programs/<sha>.json`), `learn/calibrate.py` (cross-fitted
|
||||
Platt, reliability/slope/intercept/ECE, ADVISORY-only downgrade), `learn/evaluate.py` (operating points from
|
||||
the other folds, repository-cluster bootstrap, strata, "insufficient data"), `ousast learn
|
||||
compile|evaluate|curve`.
|
||||
- Tests: `test_learn_compile.py` (`test_reproducible_from_cache` on both backends), `test_learn_calibrate.py`,
|
||||
`test_learn_decide.py`.
|
||||
- _Requirements: 3.3, 3.5, 5.2, 7.3, 7.4_
|
||||
- [x] 6.6 First paid slice **(budget go-ahead: ~$5 expected, ceiling $7)**
|
||||
- Smoke run on 20 candidates (~$0.10): real token counts, prefix-cache hit share, parse rate; it fixes `k`
|
||||
(5 if samples 2..k hit the cache for >= 80% of their prompt tokens, else 1) before anything else is spent.
|
||||
- One compile per profile with data (`static` first; ~$2.2, ceiling $3).
|
||||
- Outer-fold evaluation, priors off, evaluable families only (675 candidates: ~$2.0 at k = 1, ~$3.0 at k = 5),
|
||||
with the 30-candidate canary set; report under `benchmarks/measurements/<date>-decision-engine-v1/` (counts
|
||||
and intervals only, no identities).
|
||||
- Done 2026-10-01 for family **injection only** (the maintainer's scope for this slice; memory holds every family,
|
||||
at most 2 contrast examples per prompt): smoke fixed `k = 5` (repeat hit share 0.96); program `c1237a49...`
|
||||
(proposed instruction 5, balanced Brier 0.128 on 13 `C_val` candidates vs 0.199 baseline); 110 out-of-repository
|
||||
candidates in 29 groups: ADVISORY recall 0.90 [0.82, 0.97], precision 0.54 [0.44, 0.65]; AUC of `s` 0.74 [0.65,
|
||||
0.85]; BLOCK not offered (ECE 0.064 > 0.05, and no fold reached a Wilson lower bound of 0.95); canary agreement
|
||||
0.83; $0.91 metered at list price (provider balance $18.20 -> $17.90). Record:
|
||||
`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`. The other evaluable families
|
||||
are not evaluated yet.
|
||||
- _Requirements: 3.1-3.5, 5.2, 7.3, 7.4_
|
||||
- [ ] 6.7 Second paid slice **(budget go-ahead: ~$7 expected, ceiling $10; needs a top-up)**
|
||||
- Leave-one-source-out (~$1.4-2.1), leave-one-framework-out where a framework reaches 10 groups ($0 today),
|
||||
learning curves over memory size (0/25/50/100%) and number of sources (~$5.2), recorded beside 6.6.
|
||||
- Partial 2026-10-02 (`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json`): after the
|
||||
advisory-fix harvest (memory 724 -> 904), one compile per evaluable family and its outer-fold evaluation for all
|
||||
six (injection, untrusted_destination, path, output_encoding, deserialization, access_control); out-of-repository
|
||||
AUC 0.77-0.96, BLOCK offered for none (calibration holds only for output_encoding, where no fold reaches the
|
||||
Wilson bound); one memory-size point for injection (724 vs 904 examples, paired: AUC 0.76 vs 0.77). $2.88
|
||||
metered (ceiling $8). Not done: leave-one-source-out, the 0/25/50% curve points, the source-count curve.
|
||||
- _Requirements: 5.1, 5.2, 8.3_
|
||||
|
||||
- [ ] 7. Experiments (exp-001 and program variants need budget go-ahead)
|
||||
- `learn/experiments.py`, `register`, `experiment-outcome`, Run annotation checks, `OUSAST_VERIFY_CALLERS`;
|
||||
exp-001 (known-callers line on/off, ~$7.7 expected, ceiling $15) registered, run and recorded -- an
|
||||
inconclusive result is pre-registered as likely.
|
||||
- Program-variant experiments (paired on candidates, repository cluster, arm A replayed from cache at $0): the
|
||||
machinery and its test; the first variant (`k = 5` vs `k = 1`, or the confidence source) **(budget go-ahead:
|
||||
~$0.9-1.35 per arm-B pass of 300 candidates, ceiling $2 each)**.
|
||||
- _Requirements: 3.6, 4.1-4.4_
|
||||
|
||||
- [ ] 8. Integration
|
||||
- `plane/tasks/embed.py` (Model `openrouter-embedding`) and `plane/tasks/decide.py` (Model `deepseek-flash`,
|
||||
metered, exits 0/2/3), the host seed of the adopted program and the case's boundary-filtered memory; the `scan`
|
||||
decision stage and `dropped_by_engine.json`; `pre-push` admission with the engine (structural admission reasons
|
||||
unchanged; concurrency, `undecided: deadline`, a per-push budget); report fields (p, operating point,
|
||||
program id, verdict, rationale, cited lines); the fallback line (no program, no key, budget exhausted,
|
||||
insufficient data, calibration failed); the opt-in adaptation layer with user-local memory.
|
||||
- Tests use scripted clients; a plane fixture Run proves `decide` never receives an example of its own case.
|
||||
- _Requirements: 6.1-6.3, 7.3, 7.5_
|
||||
|
||||
- [ ] 9. Adopt
|
||||
- `ousast learn adopt`; the packaged `ruleset/decision/program-<profile>.json` with only licence-cleared
|
||||
demonstrations (the maintainer decides which licences qualify), re-evaluated as its own variant; the canary
|
||||
drift check before adoption; the priors-on/off experiment per framework with a recorded decision
|
||||
**(budget go-ahead per experiment, ~$1-2 each)**.
|
||||
- _Requirements: 3.3, 8.2_
|
||||
|
||||
- [ ] 10. The one-time v3 check **(budget go-ahead for the engine's v3 spend, ~$0.0045 per candidate)**
|
||||
- Commit `prediction-v3.json` first (program id, memory snapshot digest, model ids, operating points, predicted
|
||||
intervals, spend ceiling from the owner's candidate count); then v3 is run under protocol v3 and
|
||||
`result-v3-engine.json` is committed. Nothing between the two commits may touch the program, memory snapshot,
|
||||
features, priors or thresholds. This spec's code never opens v3 files.
|
||||
- _Requirements: 5.3_
|
||||
@@ -1,5 +1,8 @@
|
||||
# Brief: learning-harness
|
||||
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
|
||||
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
|
||||
|
||||
## Problem
|
||||
|
||||
The tool is a hand-authored static analyzer with a learning loop bolted on the side, and the loop cannot learn. Three deliberate choices cap it: closed vocabularies (sinks, sources, obligation kinds) that grow only by writing a new spec; a corpus that is the only teacher and a shape that is the only lesson; and a ban on retaining anything the model discovers. Results have ceilings by construction, the loop trains on its own test set, and there is no failure analysis feeding the next round. Detection logic is ~16% of the source and the other 84% cannot say whether the 16% works.
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
# Design Document: learning-harness
|
||||
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
|
||||
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
|
||||
|
||||
## Overview
|
||||
|
||||
**Purpose**: Turn OpenUltraSAST from a hand-authored analyzer with a leaking improvement loop into a learning harness: an auto-classifier routes every pair, finding and code region to one of ten verifier-aligned families; one detector configuration per family is evolved one change per round from structured failure facts; a class-aware pair-wise scorer with unscorable denominators and a measured per-family noise floor decides acceptance; only verifier-confirmed claims are reported above suspicion; the split is enforced by construction.
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** retired, superseded by `model-grounded-detection` (spec.json `superseded_by`). 30 of 33 tasks were done; the open three will not be. Kept as decision history; nothing in src, tests or CI points here.
|
||||
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
|
||||
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
|
||||
|
||||
## Introduction
|
||||
|
||||
OpenUltraSAST's detection numbers cannot be trusted and cannot be improved. The static machinery is treated as the detector while the model with tools already outperforms it (12/14 versus 5/14 pair-correct on the vibe-py holdout once scoring is keyed on class and function), the pair scorer keys detection on text tokens rather than on a vulnerability class, the improve loop learns from its own holdout pairs, and no change is attributed to the class it targets, so an improvement in one class silently degrades another. This feature turns the tool into a learning harness: an auto-classifier assigns every labeled pair, finding and code region to one of ten verifier-aligned families; one detector configuration per family is evolved round by round from structured failure facts; a class-aware, pair-wise scorer with unscorable denominators and a measured per-family noise floor decides acceptance; only verifier-confirmed claims are reported above suspicion; and the train/holdout split is enforced by construction. Context, measurements and sources: `brief.md`, `research.md`, `benchmarks/measurements/2026-09-06-*`.
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Requirements Document: model-grounded-detection
|
||||
|
||||
> **Status 2026-10-02:** complete, 16 of 16 tasks (spec.json phase corrected to `implementation-complete`). On 2026-10-02 three of its modules were deleted because nothing imported them: `model/judge.py` (Req 8; `model/pipeline.py` does the judging), `model/calibrate.py` (Req 9; the decision engine's `learn/calibrate.py` is the live calibration) and `model/execution.py`. Deleting `execution.py` closes the Req 10 execution-tier option: a future execution tier is a new design, not this seam.
|
||||
|
||||
## Introduction
|
||||
|
||||
This feature supersedes `learning-harness` and `constrained-detector`. It replaces the assumption that the LLM
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
{
|
||||
"feature_name": "model-grounded-detection",
|
||||
"created_at": "2026-09-08T00:00:00Z",
|
||||
"updated_at": "2026-09-08T10:00:00Z",
|
||||
"updated_at": "2026-10-02T00:00:00Z",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"phase": "implementation-complete",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": true,
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
# Design: OpenRouter SAST Harness
|
||||
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
|
||||
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
|
||||
|
||||
## Architecture Summary
|
||||
|
||||
OpenUltraSAST is a harness-oriented SAST system, not a monolithic scanner. The core runtime composes processors, model roles, retrieval, tools, sandbox policy, traces, verification gates, and reporting into a reproducible scan harness.
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# Requirements: OpenRouter SAST Harness
|
||||
|
||||
> **Status 2026-10-02:** retired (see the note below). Its OpenCode skill `openultrasast-kiro-impl`, which still sent agents to this tasks.md, was deleted. Kept as decision history; nothing in src, tests or CI points here.
|
||||
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
|
||||
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
|
||||
|
||||
## Overview
|
||||
|
||||
OpenUltraSAST shall be an independent OpenCode-driven security harness for finding, validating, reporting, and optionally fixing source-code vulnerabilities at scale. It shall use OpenRouter-selected chat models for reasoning, OpenRouter embeddings for semantic retrieval, Docker for isolated analysis, and an explicit evidence ladder to prevent AI suspicion from being reported as verified security impact.
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"feature_name": "openrouter-sast-harness",
|
||||
"language": "en",
|
||||
"status": "retired",
|
||||
"updated_at": "2026-10-02T00:00:00Z",
|
||||
"retired_note": "2026-09-30: HarnessX was removed by harnessx-removal; this spec had no spec.json before 2026-10-02 and is kept as decision history."
|
||||
}
|
||||
@@ -1,5 +1,7 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** complete, 32 of 32 tasks (spec.json phase corrected to `implementation-complete`).
|
||||
|
||||
## Introduction
|
||||
|
||||
Engineers evolving OpenUltraSAST's overlay cannot trust the pair scoreboard: coverage detections are dropped, vfc labels mostly say `sink = "unknown"`, a quarter of the vfc corpus fails to parse because excerpts start mid-comment, two sast pairs are unlabelable as vendored, and there is no Node/TypeScript/Python web corpus and no agent-authored corpus. This spec makes the scoreboard honest per slice, per mechanism, and per provenance, adds three web and agent-authored slices, and lets the improve loop accept a change only when no provenance profile regresses. It supersedes `real-world-vfc-slice` for the corpus and scorer plane.
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
{
|
||||
"feature_name": "pair-corpus-honesty",
|
||||
"created_at": "2026-09-04T16:12:17Z",
|
||||
"updated_at": "2026-09-05T12:30:00Z",
|
||||
"updated_at": "2026-10-02T00:00:00Z",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"phase": "implementation-complete",
|
||||
"supersedes": [
|
||||
"real-world-vfc-slice"
|
||||
],
|
||||
|
||||
@@ -24,6 +24,68 @@ an irrelevant old defect, or an unactionable witness can still produce noise.
|
||||
- Do not build a second ranker/scope heuristic, an incremental CPG engine, a daemon, an automatic fixer,
|
||||
or a mandatory model service. Do not scan intermediate pushed history in this feature.
|
||||
|
||||
## What this hook is, and is not — 2026-09-21
|
||||
|
||||
A general-purpose pre-commit hook, not an application-specific one. It may be measured on WordPress
|
||||
plugins and Node applications, and it may not become a tool that only works on them. The practical
|
||||
test at review time: would this change still be correct on a repository nobody here has seen? A path,
|
||||
a directory name or an identifier peculiar to one project fails that test, and one was declined on
|
||||
exactly this ground when a plugin's bundled library tree would have fixed a single subject while
|
||||
claiming something false about every other.
|
||||
|
||||
The fact tables are an ontology of libraries, frameworks and language constructs, and
|
||||
`contributor-scan` owns it. Its design records how it may grow and its group 7 makes growth
|
||||
measurable rather than anecdotal. Nearly all of this tool's accuracy lives there: the 2026-09 family
|
||||
census found five of seven families establishing nothing, every one of them for an ontology reason
|
||||
with the flow engine ready and nothing to aim at.
|
||||
|
||||
## Two invariants this project keeps rediscovering
|
||||
|
||||
Added 2026-09-21 after the same two defects appeared eleven times between them. Each instance was
|
||||
found by measurement, fixed in one place, and then found again somewhere else, because the RULE was
|
||||
never written down. Both are cheap to check in review and expensive to find in a benchmark.
|
||||
|
||||
### A degradation is owned by the narrowest scope that caused it
|
||||
|
||||
A gap recorded at repository scope silences questions it has no evidence about. The test for a
|
||||
repository-wide gap is whether it could make a reported answer WRONG everywhere, not whether it cost
|
||||
coverage somewhere.
|
||||
|
||||
| instance | the blast radius | where it was fixed |
|
||||
|---|---|---|
|
||||
| vendor exclusion acting as a global completion veto | every question in the repository | M1b, task 10.6 |
|
||||
| line ambiguity inherited by every question | every question in a file | M1b |
|
||||
| an empty scope blocking another question's comparison | a sibling's answer | task 12.3 |
|
||||
| a language partition boundary demoting every answer | 4,425 PHP regions, by 36 JavaScript ones | task 13.4 |
|
||||
| a partition census gap demoting every answer | 4,425 PHP regions, by four build artefacts | task 13.4 |
|
||||
|
||||
The fourth and fifth cost the declared v0.1 target its entire admissibility: a WordPress plugin is
|
||||
PHP plus admin JavaScript, so the product could detect defects on its own target and never admit
|
||||
one. Neither passing subject could reveal it, because both hold one language.
|
||||
|
||||
**In review:** every new degradation reason must state the scope it reaches — one question, one file,
|
||||
one partition, or the whole read — and a reason that demotes an outcome must justify reaching that
|
||||
far. A gap that costs coverage without threatening correctness is a declared exclusion: it stays
|
||||
published and it does not demote.
|
||||
|
||||
### A token matches the thing it names, not text that contains it
|
||||
|
||||
Every matcher in this project searches text for a declared name, and each time the search was allowed
|
||||
to look somewhere the name does not appear AS the thing, it claimed something else.
|
||||
|
||||
| instance | what it matched instead | where it was fixed |
|
||||
|---|---|---|
|
||||
| `gets` inside `fgets` | the safe API reported as the dangerous one, 26 times | task 5.13 |
|
||||
| `resolve` inside `resolveUrl` | a sanitizer that sanitized nothing | contributor-scan |
|
||||
| `user` inside `users` in a SQL string | a textbook IDOR read as guarded | authorization-obligations |
|
||||
| a sink word inside a comment | a prototype finding on a handler with no such call | task 14.1 |
|
||||
| a dotted setting matched on its last segment | an editor's `setOptions` read as a template engine's | task 14.3 |
|
||||
| a flow query reading the call's receiver | `res.end()`, which takes no data, reported like a write | task 14.3 |
|
||||
|
||||
**In review:** a text match must be bounded AND confined to the part of the node that names the
|
||||
callee. Whole-node text, an argument subtree, a receiver and a trailing path segment are all places
|
||||
the name can appear without being what is called.
|
||||
|
||||
## Boundary Commitments
|
||||
|
||||
### This Spec Owns
|
||||
|
||||
@@ -0,0 +1,330 @@
|
||||
# M1a evidence: the NodeGoat detection with every veto recorded
|
||||
|
||||
Run date 2026-09-16. Milestone **M1a** of [release-milestones.md](release-milestones.md).
|
||||
Instrument `benchmarks/push/m1a_record_vetoes.py`; flag `--experimental-record-vetoes`.
|
||||
Evidence: [summary](../../../benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json)
|
||||
and its 11-file [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json.gz).
|
||||
|
||||
This record diagnoses. It admits no defect, enables no capability, qualifies no language and
|
||||
makes no latency claim. All three artifacts report `finding_status: none` with zero admitted
|
||||
defects; every recorded section carries its experimental label.
|
||||
|
||||
## What was declared before running
|
||||
|
||||
Expected base/head pins, expected operation and expected outcome for all three comparisons
|
||||
were written to `m1a-record.json` before the first scan. The declared target is
|
||||
`app/routes/contributions.js`, `handleContributionsUpdate`, family `injection`, lines 32–34.
|
||||
|
||||
| Case | Base | Head | Declared expectation |
|
||||
|---|---|---|---|
|
||||
| nodegoat-introduce | fixed `0b453d5a` | vulnerable `c5cb68a7` | raw finding at the three eval lines, recorded novelty new |
|
||||
| nodegoat-repair | vulnerable `c5cb68a7` | fixed `0b453d5a` | fixed twin analyzed and quiet; three base-only findings |
|
||||
| nodegoat-benign | vulnerable `c5cb68a7` | benign `f772c59e` | backlog unchanged; never new or worsened |
|
||||
|
||||
Each revision's target file was reopened before the run: 2,485, 2,491 and 2,498 bytes with
|
||||
recorded SHA-256 digests. Each head scan reported a complete 44-file JavaScript census.
|
||||
Installed engine, facts, query, policy, core and semantics provenance matched the frozen
|
||||
values in all three artifacts.
|
||||
|
||||
## Observed outcome
|
||||
|
||||
| Case | Elapsed | Target findings | Target question answered | Deadline reached |
|
||||
|---|---:|---:|---|---|
|
||||
| nodegoat-introduce | 349.8 s | 3 | yes, 9 rows | no |
|
||||
| nodegoat-repair | 153.8 s | 0 | yes, 0 rows | no |
|
||||
| nodegoat-benign | 261.4 s | 3 | yes | no |
|
||||
|
||||
All three ran under an explicitly named 900-second development budget. These durations are
|
||||
development measurements and cannot satisfy or measure the 30-second hook latency gate.
|
||||
|
||||
Verdict recorded by the instrument: raw witness present, fixed twin quiet, unchanged revision
|
||||
not reported as a new regression, recorded evaluation produced for all three. The milestone's
|
||||
**kill criterion did not trigger**: the fixed twin was genuinely analyzed, answering its target
|
||||
question with zero rows and no deadline reason, while the same three findings appear on its
|
||||
base side as base-only findings. Silence from failure was not accepted as a pass.
|
||||
|
||||
## The vetoes that actually fired
|
||||
|
||||
Identical for each of the three `eval` witnesses on the vulnerable revision. Five recognized
|
||||
vetoes fired and were recorded rather than applied:
|
||||
|
||||
| Stage | Veto | Evidence |
|
||||
|---|---|---|
|
||||
| head question | `change_context_incomplete` | the answered target question carried 9 rows and was then demoted |
|
||||
| head scan | `dynamic_external_or_depth_context_unresolved` | on the target's own question identity, not an unrelated one |
|
||||
| head scan | `vendor_semantics_unresolved` | recorded both as a degradation and as a scope boundary |
|
||||
| base scan | `graph_incomplete` | the base question answered with 0 rows and was then demoted |
|
||||
| change context | `ambiguous_line_correspondence` | file-level, on the target file |
|
||||
|
||||
The earlier owner diagnosis named four independent vetoes and expected query-to-operation
|
||||
provenance and question-owned vendor relevance to be the two that mattered. Both fired. The
|
||||
run also confirms the third and fourth, so all four named vetoes are real on this case.
|
||||
|
||||
## What M1a corrected in the diagnosis
|
||||
|
||||
**A fifth veto is the actual remaining blocker.** With all five recognized vetoes lifted, the
|
||||
detection does not become new. Its reason advances from `witness_identity_unresolved` to
|
||||
`operation_correspondence_unresolved`, because the head operation sits on a changed line and
|
||||
therefore has no base line anchor. The comparison can classify an operation only when it maps
|
||||
to a base location.
|
||||
|
||||
The unchanged revision proves this is the mechanism rather than a coincidence. Its comment-only
|
||||
edit leaves lines 32–34 unchanged, the base location resolves to the same lines, and the
|
||||
recorded novelty becomes `unchanged` / `same_supported_mechanism`. Same evidence, same policy,
|
||||
opposite correspondence outcome:
|
||||
|
||||
| Revision | Operation line | Base location | Recorded novelty |
|
||||
|---|---|---|---|
|
||||
| vulnerable | changed | unresolved | unknown, `operation_correspondence_unresolved` |
|
||||
| unchanged | unchanged | resolved to the same line | unchanged, `same_supported_mechanism` |
|
||||
|
||||
**Query identity differs from witness identity, as predicted.** The target is selected through a
|
||||
file-fallback question with `function=None`, while the rows report method `<lambda>2` at lines
|
||||
32–34. The declared benchmark function is `handleContributionsUpdate`. This is recorded, not
|
||||
applied: it is why production reported `witness_identity_unresolved` before any lifting, since
|
||||
an unresolved question yields no comparable operation at all.
|
||||
|
||||
**Admission reasons that no flag can remove.** `capability_unavailable` remains on every
|
||||
candidate because the registry is empty and this flag cannot enable a capability.
|
||||
`dependency_unresolved` remains because unlifted coverage reasons still reach admission.
|
||||
|
||||
## What M1b must repair, ordered by this evidence
|
||||
|
||||
1. **Authentic per-family change context.** `change_context_incomplete` demoted every answered
|
||||
question in all three cases. Dominance and configuration questions need their own context
|
||||
producers describing their actual query scope; taint context must not be copied into them.
|
||||
2. **Question-owned vendor relevance.** `vendor_semantics_unresolved` turned an otherwise
|
||||
complete 44-file first-party census into `graph_incomplete` on every base question. Keep the
|
||||
exclusion physical; establish which question actually requires excluded semantics.
|
||||
3. **Operation correspondence for changed lines.** The blocker M1a surfaced. Bind the required
|
||||
base/head correspondence to the specific operation rather than to a line anchor, with absent,
|
||||
renamed, removed-guard and ambiguous-operation controls. Do not delete the ambiguity check
|
||||
and do not treat lexical alignment as semantic proof.
|
||||
4. **Correspondence ambiguity scoped to its file region.** `ambiguous_line_correspondence` is
|
||||
recorded file-wide and inherited by every question, including uniquely mapped operations.
|
||||
5. **Query-to-operation provenance.** Carry verified provenance from the selected question to the
|
||||
reported operation. Do not inject benchmark function names into regions.
|
||||
|
||||
Vetoes that did not fire on this case stay as they are and their families stay experimental.
|
||||
|
||||
## Limits
|
||||
|
||||
Three comparisons on one inspected teaching repository. NodeGoat is development and transfer
|
||||
data; it cannot qualify production Node support, and Node evidence cannot qualify PHP or Python.
|
||||
The lifted view is a diagnostic reconstruction, not a comparison result: it shows what the
|
||||
existing evidence would support if those vetoes were repaired, and no repair has been made.
|
||||
Runtime, independent qualification and rollout all remain NO-GO.
|
||||
|
||||
## M1b step 1: what the first three repairs changed
|
||||
|
||||
Rerun 2026-09-16 after tasks 10.1–10.3, same instrument, same pins, same budget.
|
||||
Evidence: [summary](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-step1.json)
|
||||
and its [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-step1.json.gz).
|
||||
|
||||
The recorded view now classifies the case correctly. The vulnerable revision reaches
|
||||
`new` / `operation_absent_from_comparable_base`, and the unchanged revision reaches
|
||||
`unchanged` / `same_supported_mechanism` with its base location resolved to the same lines.
|
||||
Before the repairs both stopped at `operation_correspondence_unresolved` and
|
||||
`witness_identity_unresolved`.
|
||||
|
||||
| Revision | Before 10.1–10.3 | After 10.1–10.3 |
|
||||
|---|---|---|
|
||||
| vulnerable | unknown, `operation_correspondence_unresolved` | new, `operation_absent_from_comparable_base` |
|
||||
| unchanged | unknown, `witness_identity_unresolved` | unchanged, `same_supported_mechanism` |
|
||||
|
||||
The rerun also caught a defect in the repair itself: the new novelty reason was missing from
|
||||
the admission allowlist, so admission rejected the candidate as `change_unsupported`. Fixed
|
||||
with a control.
|
||||
|
||||
The production path is unchanged so far, because two vetoes still demote the target
|
||||
questions: `change_context_incomplete` on the head from the question's own
|
||||
`dynamic_external_or_depth_context_unresolved`, and `graph_incomplete` on the base from the
|
||||
vendor exclusion.
|
||||
|
||||
## Why the last two vetoes need new query evidence
|
||||
|
||||
Neither remaining veto can be scoped or relaxed soundly as the plan originally described,
|
||||
and the rerun is what showed it.
|
||||
|
||||
A taint answer reports traced flows. An absent row therefore means either that the operation
|
||||
does not exist in the base, or that it exists and no traced flow reached it.
|
||||
`dynamic_external_or_depth_context_unresolved` exists precisely to stop the second being read
|
||||
as the first. Removing or narrowing it would let an unreached operation look absent, which
|
||||
turns a wrong absence into a wrong new finding: the false-positive direction this project
|
||||
cannot afford.
|
||||
|
||||
At the same time the flag carries no discrimination. It is computed over every in-scope
|
||||
method, and on any real Express application ordinary framework and dependency calls have
|
||||
unresolved destinations, so it is always true and can never be discharged. A veto that is
|
||||
always on and can never be lifted is why the tool reports nothing on real projects.
|
||||
|
||||
The resolution is neither to keep it nor to drop it, but to supply the evidence that makes
|
||||
absence provable independently of reachability: export the operations the query actually
|
||||
enumerated in the question's scope, separately from the flows it traced. Absence is then
|
||||
established from the inventory, while a traced-flow claim still requires complete
|
||||
reachability. Vendor exclusion can then stop acting as a global completion veto for the same
|
||||
reason, since dependency on excluded code becomes visible in the evidence rather than assumed.
|
||||
|
||||
That prerequisite is now task 10.4, ahead of the two vetoes it unblocks. It adds no detector,
|
||||
alias resolver or second source IR.
|
||||
|
||||
## M1b outcome: the same case through the production rules
|
||||
|
||||
Rerun 2026-09-16 after tasks 10.1 to 10.6, same instrument, same pins, same budget.
|
||||
Evidence: [summary](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-production.json)
|
||||
and its [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-production.json.gz).
|
||||
|
||||
The production comparison now classifies all three revisions without any lifting, and its
|
||||
verdicts match the recorded view exactly. That match is the point: the vetoes M1a recorded
|
||||
were repaired rather than bypassed.
|
||||
|
||||
| Revision | Production verdict | Recorded verdict |
|
||||
|---|---|---|
|
||||
| vulnerable | new, `operation_absent_from_comparable_base` | identical |
|
||||
| fixed twin | quiet, analyzed, three base-only findings | identical |
|
||||
| unchanged | unchanged, `same_supported_mechanism` | identical |
|
||||
|
||||
Compared with M1a, where production reported `witness_identity_unresolved` on every finding
|
||||
and every one of the 436 questions was demoted on both sides.
|
||||
|
||||
The head target questions now complete, the base scan completes all 436 questions, and the
|
||||
three `eval` witnesses at contributions.js lines 32 to 34 are attributed to the change that
|
||||
introduced them. Timings were 325.7 s, 158.9 s and 260.3 s under the named 900-second
|
||||
development budget, which measures nothing about the hook gate.
|
||||
|
||||
All three artifacts still report `finding_status: none` with zero admitted defects. Admission
|
||||
rejects on `capability_unavailable`, which is the deliberate eligibility gate, and on
|
||||
`dependency_unresolved`, which is a coarse plumbing defect recorded as its own task: the runner
|
||||
passes every coverage reason as every candidate's dependency gaps rather than the gaps that
|
||||
pertain to that candidate. Neither is a comparison failure.
|
||||
|
||||
### What each repair contributed, measured
|
||||
|
||||
1. **10.1 operation correspondence.** Moved the vulnerable revision from
|
||||
`operation_correspondence_unresolved` to a classifiable operation.
|
||||
2. **10.2 required comparison scope.** Stopped unrelated unsupported families from blocking a
|
||||
complete target comparison.
|
||||
3. **10.3 ambiguity scoping.** Stopped one file's repeated unchanged lines invalidating every
|
||||
question in the repository.
|
||||
4. **10.4 operation inventory.** Made absence provable from enumeration rather than from the
|
||||
absence of a traced flow, which is what let 10.5 and 10.6 be done soundly.
|
||||
5. **10.5 reachability binding.** Let the head questions complete, since bounded reachability
|
||||
cannot invalidate a flow that was traced.
|
||||
6. **10.6 declared exclusion.** Let the base scan's 436 answers stand, since excluding
|
||||
dependencies is a scope choice rather than a failed read.
|
||||
|
||||
A seventh change was needed after the first rerun: the same exclusion and ambiguity distinction
|
||||
was still missing inside the comparison's own degradation and boundary checks, so the verdict
|
||||
stayed at `head_context_incomplete` even once both scans completed. The rerun is what surfaced
|
||||
it, not the unit suite.
|
||||
|
||||
### Limits
|
||||
|
||||
Three comparisons on one inspected teaching repository, under a development budget. NodeGoat is
|
||||
development and transfer data: it cannot qualify production Node support, and Node evidence
|
||||
cannot qualify PHP or Python. No capability is enabled, no defect is admitted, no alert is
|
||||
emitted, and no latency claim is made. Runtime, independent qualification and rollout all
|
||||
remain NO-GO.
|
||||
|
||||
## M1b exit run (tasks 10.1 to 10.8)
|
||||
|
||||
Rerun 2026-09-16 after every repair, same instrument, same pins, same 900-second development
|
||||
budget. Evidence: [summary](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-exit.json)
|
||||
and its [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-exit.json.gz).
|
||||
|
||||
| Revision | Verdict | Admission | Elapsed |
|
||||
|---|---|---|---:|
|
||||
| vulnerable | all three sites new, `operation_absent_from_comparable_base` | `capability_unavailable` only | 326.4 s |
|
||||
| fixed twin | analyzed and quiet, three base-only findings | none to admit | 157.8 s |
|
||||
| unchanged | all three sites `unchanged`, `same_supported_mechanism` | `capability_unavailable`, `unchanged` | 256.8 s |
|
||||
|
||||
The vulnerable revision's only remaining admission reason is the deliberate eligibility gate.
|
||||
`dependency_unresolved` is gone, which verifies task 10.7 on real data: those gaps belonged to
|
||||
unrelated questions.
|
||||
|
||||
Each finding now carries its provenance explicitly: question function `None`, engine method
|
||||
`<lambda>2`, witness `eval(req.body.preTax) -> const preTax = eval(req.body.preTax)` at
|
||||
contributions.js line 32, and likewise for lines 33 and 34. The unchanged revision resolves each
|
||||
base location to the same line, which is why it reads `unchanged` rather than new.
|
||||
|
||||
Source receipts were reopened before the run, and the head census is the complete 44 files.
|
||||
|
||||
### Which vetoes were repaired and which remain
|
||||
|
||||
All five vetoes M1a recorded are repaired, plus two the repairs exposed:
|
||||
|
||||
| Veto | State |
|
||||
|---|---|
|
||||
| `change_context_incomplete` on the target question | repaired, question completes |
|
||||
| `graph_incomplete` on the base from vendor exclusion | repaired, all 436 base questions complete |
|
||||
| `ambiguous_line_correspondence` inherited repository-wide | repaired, decided per operation |
|
||||
| `dynamic_external_or_depth_context_unresolved` as a blanket veto | repaired, bound to reachability claims |
|
||||
| `operation_correspondence_unresolved` on a changed line | repaired via the enumerated inventory |
|
||||
| comparison's own degradation and boundary checks | repaired after the first rerun exposed them |
|
||||
| `dependency_unresolved` on every candidate | repaired, gaps attributed per candidate |
|
||||
|
||||
What remains, recorded rather than repaired, because each is outside this case or a later
|
||||
milestone:
|
||||
|
||||
- 231 of 436 head questions still report `change_context_incomplete`, from
|
||||
`context_projection_unavailable` and `context_scope_empty` on families whose queries export no
|
||||
context. Those are dominance and configuration questions. They did not block this case and
|
||||
their families stay experimental, so per the milestone they stay as they are.
|
||||
- The vendor exclusion and the file's correspondence ambiguity remain reported in coverage. That
|
||||
is intended: they are recorded limits, not repaired absences.
|
||||
|
||||
### What M1b has not yet demonstrated
|
||||
|
||||
The milestone also asks for a developer-readable evaluation finding carrying a supported
|
||||
security consequence and a concrete repair direction. Those two come from an evaluated
|
||||
capability declaration, and the registry is deliberately empty, so nothing renders them today.
|
||||
Writing that text without a declaration is precisely what admission exists to prevent, so it is
|
||||
not something to work around here. The comparison half of M1b is verified; the explanation half
|
||||
waits on a declaration, which is M4's subject.
|
||||
|
||||
### Limits
|
||||
|
||||
Three comparisons on one inspected teaching repository under a development budget. NodeGoat is
|
||||
development and transfer data: it cannot qualify production Node support, and Node evidence
|
||||
cannot qualify PHP or Python. No capability is enabled, no defect is admitted, no alert is
|
||||
emitted and no latency claim is made. Runtime, independent qualification and rollout all remain
|
||||
NO-GO.
|
||||
|
||||
## M1b explanation: the finding as a developer would read it
|
||||
|
||||
Run 2026-09-16 after task 10.10, same pins and budget, with an unreviewed experimental
|
||||
declaration generated by the instrument and bound to the run's own analysis semantics.
|
||||
Evidence: [summary](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-explained.json)
|
||||
and its [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-explained.json.gz).
|
||||
|
||||
Three evaluation findings were rendered on the vulnerable revision, one per eval site. For
|
||||
line 32:
|
||||
|
||||
- **Location** app/routes/contributions.js:32, family injection, novelty new with
|
||||
`operation_absent_from_comparable_base`.
|
||||
- **Witness** `eval(req.body.preTax) -> const preTax = eval(req.body.preTax)`.
|
||||
- **Change attribution** `base:app/routes/contributions.js:32-34`.
|
||||
- **Provenance** `file_scope`: the question named no function, the engine reported `<lambda>2`.
|
||||
- **Consequence** "A request field reaches const preTax = eval(req.body.preTax), where it is
|
||||
evaluated as JavaScript, so an attacker controlling that field can execute arbitrary code in
|
||||
the server process."
|
||||
- **Repair** "Replace the evaluation at const preTax = eval(req.body.preTax) with a numeric
|
||||
conversion such as Number(...) or parseInt(...), then validate the result before use."
|
||||
|
||||
Both sentences come from the declaration's templates through the grounded-template path, not from
|
||||
generated prose. The declaration reports verdict `experimental` and `enabled: false`, the
|
||||
production result still reports `finding_status: none`, and all three artifacts carry zero
|
||||
admitted defects. The rendering path never reaches admission, so no supplied declaration can
|
||||
change that.
|
||||
|
||||
### A defect this surfaced, recorded not repaired
|
||||
|
||||
The operation symbol admission extracts is the text before the first parenthesis, which for
|
||||
`const preTax = eval(req.body.preTax)` is `const preTax = eval`, not `eval`. The demonstration
|
||||
declaration had to enumerate each assignment spelling for its coverage check to pass. A real
|
||||
qualified declaration cannot enumerate every assignment a codebase might write, so admission
|
||||
would reject genuine findings on `operation_semantics_mismatch`. That is recorded as its own task
|
||||
ahead of M4; it is not worked around here.
|
||||
|
||||
M1b is met: the comparison classifies all three revisions through the production rules, and the
|
||||
finding reads as a developer would need it to, with every claim traceable to its evidence.
|
||||
@@ -0,0 +1,288 @@
|
||||
# Release milestones: prove usefulness, then speed and transfer
|
||||
|
||||
Accepted direction: 2026-09-14. The maintainer agreed to the five-step path and asked
|
||||
for it to be recorded in the roadmap as a spec. This amendment defines the order and
|
||||
observable exit criteria for the remaining release work within pre-push-safety-net.
|
||||
It preserves the approved requirements, completed tasks and existing owner boundaries.
|
||||
It does not approve an unreviewed implementation design or enable any capability.
|
||||
|
||||
## Current position
|
||||
|
||||
**Next milestone: M3, with M4 population preparation running in parallel. M1a and M1b are met;
|
||||
M2 is measured NO-GO; none of M1–M5 is verified.** The published `v1.2.0-alpha.1` is an
|
||||
experimental implementation release, not a qualified advisory rollout. See
|
||||
[current status](status.md) for measured evidence and limitations.
|
||||
|
||||
The immediate goal was one real changed-code defect with a defensible explanation and a quiet
|
||||
fixed twin. That is met. Task 8.3 remains blocked.
|
||||
|
||||
## Strategy amendment, 2026-09-21
|
||||
|
||||
Accepted after the family detection census and the PHP correction. Four decisions, in the order
|
||||
they should be executed. They change sequence and declared scope; they change no acceptance
|
||||
threshold and no owner boundary.
|
||||
|
||||
1. **Close M3 at a scale that completes, before anything else.** PHP transfer is recorded as not
|
||||
established, and that record came from two instrument defects rather than from the analyser:
|
||||
a census verdict that checked question completion before findings, and a region budget whose
|
||||
question count cannot fit any ceiling. Bounded to 25 and 100 regions, Paid Memberships Pro
|
||||
completes with no timeout and establishes a true-positive SQL injection of the class its own
|
||||
fact table predicted. Re-run the declared transfer replay at a completing scale and let the
|
||||
result stand either way.
|
||||
|
||||
2. **Reserve and review the untouched M4 population now, in parallel.** M4 already permits
|
||||
population preparation to proceed before the frozen candidate exists, and it is the longest
|
||||
lead-time item on this path: nothing can be qualified without it however good the analyser
|
||||
gets. Starting it after M2 serialises the release for no reason.
|
||||
|
||||
3. **M2 is the only engineering item that remains, and it cannot be scoped away.** A changed-code
|
||||
transaction spends its whole budget in the build with zero query time. Graph reuse across
|
||||
revisions is the accepted direction. Narrowing what a push analyses is explicitly not a way to
|
||||
meet the latency gate, so the work is architectural and should have the effort that family
|
||||
vocabulary work has been absorbing.
|
||||
|
||||
4. **Declare the v1 advisory scope as the families whose claims can be ADMITTED.** Admission
|
||||
requires the entailed rung. Injection, untrusted destination and configuration entail with
|
||||
witnesses; access control does not, because on any application that splits its route from its
|
||||
data access the obligation is carried and a carried obligation corroborates by design. Shipping
|
||||
it would ship silence. Output encoding, path, prototype and deserialization have not earned a
|
||||
place either. M4 explicitly does not require every language or family to become eligible.
|
||||
|
||||
**Amendment to the amendment, same day.** Decision 1's re-run exposed something larger than the
|
||||
runtime limit it was chasing. A repository holding two languages can never complete a question,
|
||||
because a partition boundary is classified as a graph-integrity gap and such a gap demotes every
|
||||
completed outcome; and a completed outcome is what admission requires. NodeGoat completes 299 of
|
||||
299 and VAmPI 35 of 35 because each holds one language, while PMPro completes none of 520 and
|
||||
still establishes 43 findings. The declared v0.1 target is WordPress plugins, and a plugin is PHP
|
||||
plus its admin JavaScript, so on the target this tool was built for it can detect and never admit.
|
||||
Task 13.4 carries it, and it now ranks with M2 rather than behind it: a narrow JavaScript-only
|
||||
release can proceed without it, and the WordPress target cannot proceed at all.
|
||||
|
||||
**The risk this amendment does not remove.** Every number so far comes from two repositories used
|
||||
to build the analyser, and one week of examining them surfaced seven false positives. On untouched
|
||||
code the precision estimate will move, against a bar of 95% with zero false alerts on the fixed and
|
||||
benign sides. Expect the first M4 evaluation to fail and budget for a second. That is the strongest
|
||||
argument for decision 2.
|
||||
|
||||
## M1 — One real security change works end to end
|
||||
|
||||
Status: **NEXT — not verified**. Dependencies: existing immutable replay and scan machinery.
|
||||
M1 is split into M1a (diagnose with vetoes recorded, not applied) and M1b (repair only
|
||||
the vetoes M1a showed to matter). M1 is verified only when M1b passes.
|
||||
|
||||
Why the split: the detector already reports the NodeGoat contribution injection. The
|
||||
2026-09-14 post-census bundle carries the raw `<lambda>2` witness at contributions.js
|
||||
lines 32–34, and the same bundle carries thousands of `context_incomplete`,
|
||||
`graph_incomplete`, `vendor_semantics_unresolved` and `ambiguous_line_correspondence`
|
||||
reasons that discard it before any developer sees it. Four independent admission vetoes
|
||||
are known: no context producer for dominance/config families, vendor exclusion acting as
|
||||
a global completion veto, line ambiguity inherited by every question, and query identity
|
||||
differing from witness identity. Requiring all four repaired before the first visible
|
||||
result is a full production pass, not one detection.
|
||||
|
||||
### M1a — Show the detection with every veto recorded
|
||||
|
||||
Status: **MET 2026-09-16**, inside its one-week box. Evidence:
|
||||
[M1a record](m1a-recorded-vetoes.md) and
|
||||
[measurement](../../../benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json).
|
||||
Raw witnesses present at contributions.js lines 32–34; fixed twin analyzed and quiet with
|
||||
zero rows and no deadline reason; unchanged revision analyzed and recorded `unchanged`. The
|
||||
kill criterion did not trigger. Five vetoes fired and were recorded, and M1a corrected the
|
||||
diagnosis: with all five lifted the detection stops at `operation_correspondence_unresolved`,
|
||||
a fifth veto that was not previously named. No defect was admitted and no capability enabled.
|
||||
|
||||
- Use the pinned NodeGoat contribution injection case, its fixed twin and an unchanged
|
||||
vulnerable revision as development data. Record exact base/head revisions, expected
|
||||
operation and outcome before running. Exercise the shipped replay path, existing ranker
|
||||
and physically vendor-free graph; do not substitute a hand-selected slice or hard-coded
|
||||
target function.
|
||||
- Run all three revisions under an explicit experimental evaluation flag that **records**
|
||||
each admission and comparison veto per finding instead of applying it. The flag is
|
||||
task-local, off by default, cannot enable any hook capability, and every output it
|
||||
produces is labeled experimental. A longer, explicitly named development budget applies;
|
||||
it cannot satisfy the hook's latency gate.
|
||||
- Deliverable: the raw finding on the vulnerable revision with location and witness, the
|
||||
fixed twin genuinely analyzed and quiet, the unchanged revision not reported as a new
|
||||
regression, and for every finding the exact list of vetoes that fired and why. Silence
|
||||
from failure is not a pass; a fixed twin that timed out or was not analyzed fails M1a.
|
||||
|
||||
**Exit evidence:** a replayable three-revision record under the experimental flag with
|
||||
exact revision, question and operation provenance, the per-finding veto list, and readable
|
||||
source and census receipts. **Kill criterion:** if the fixed twin is not quiet under the
|
||||
recorded-veto run, M1 has a detector problem, and M1b does not start until that is
|
||||
diagnosed and recorded.
|
||||
|
||||
### M1b — Repair only the vetoes M1a showed to matter
|
||||
|
||||
Status: **NEXT — not verified**. M1a supplied the veto list: per-family change context,
|
||||
question-owned vendor relevance, operation correspondence for an operation on a changed line,
|
||||
file-wide correspondence ambiguity scope, and query-to-operation provenance. The owner
|
||||
diagnosis expected the first and the last two; operation correspondence is the addition M1a
|
||||
measured and is the blocker that survives every other repair.
|
||||
|
||||
- From the M1a veto list, repair only the context/family/identity contracts this case
|
||||
actually needs. The owner diagnosis expects query-to-operation provenance and question-
|
||||
owned vendor relevance to be the two that fire here; M1a confirms or corrects that.
|
||||
Vetoes that did not fire on this case stay as they are and their families stay
|
||||
experimental. Resolve vendor and correspondence boundaries through evidence of dependency
|
||||
and ownership; unknown relevance remains unresolved.
|
||||
- Rerun the three revisions **without** the recorded-veto flag through the production
|
||||
evidence and comparison rules. Demonstrate a developer-readable evaluation finding with a
|
||||
supported security consequence, exact location, witness, change attribution and concrete
|
||||
repair direction. Verify the fixed twin is analyzed and produces no actionable defect, and
|
||||
the unchanged vulnerable revision is not reported as a new regression.
|
||||
- Keep qualification eligibility disabled. The demonstration uses explicit experimental
|
||||
evaluation output; it cannot advertise an unqualified normal hook alert.
|
||||
|
||||
**Exit evidence:** a replayable vulnerable/fixed/unchanged comparison record through the
|
||||
production rules, exact question and operation provenance, completed required checks,
|
||||
reviewed explanation, readable source and census receipts, task-local regression
|
||||
verification, and a list of which vetoes were repaired and which remain. Raw findings, rank
|
||||
positions, synthetic-only controls and screenshots alone do not satisfy M1b.
|
||||
|
||||
Owners: contributor-scan for query/operation/context evidence; pre-push-safety-net for
|
||||
comparison, admission, the recorded-veto flag and reporting. Requirements: 1.1–1.4,
|
||||
2.1–2.4, 3.1–3.5, 6.3, 7.1–7.3.
|
||||
|
||||
## M2 — The same useful analysis fits the push budget
|
||||
|
||||
Status: **MEASURED AND UNMET, 2026-09-17.** Two attacks on engine startup were measured and
|
||||
neither reaches the gate. A transaction-owned session is implemented and proven equivalent for
|
||||
every shipped query kind, and it amortizes startup only across a long transaction: 326 s to 243 s
|
||||
under a development budget, while under the 30-second deadline its own 9 to 13 second startup
|
||||
means it usually declines to start. Removing JavaScript's redundant second engine start is a real
|
||||
11.0 s against 21.0 s on identical graphs, but the attempt was reverted for an asymmetric census
|
||||
gate that made 12 profile transactions pay two engine starts.
|
||||
|
||||
The profile names the actual cost: a changed-code transaction reaches its deadline inside the
|
||||
build, head build 25.2 s of 30 s with query time 0.0 s. It is building a fresh whole-repository
|
||||
graph for every pushed revision.
|
||||
|
||||
**Direction accepted 2026-09-17: do not rebuild a whole graph per push. Reuse the graph across
|
||||
revisions instead of further optimising the rebuild.** The 30-second and 2-second gates are
|
||||
unchanged; this milestone is not met, and the work is not scheduled here.
|
||||
|
||||
- Use the successful M1 case as a correctness baseline for a transaction-owned Joern
|
||||
session within the existing backend. The measured census probe supports investigating
|
||||
this design; it does not establish full-query equivalence or production readiness.
|
||||
- Verify equivalent taint, dominance and configuration answers under matching semantics,
|
||||
fresh request receipts, isolated graph identities, cache invalidation, crash handling
|
||||
and process-group cancellation. Keep one deadline across the complete transaction.
|
||||
- Rerun the frozen seven-class runtime profile with changed-code cases, cold starts,
|
||||
dependency/configuration edits, growth and multi-ref transactions. Retain all timeouts
|
||||
and unresolved checks. Identical-tip reuse cannot substitute for warm changed-code work.
|
||||
|
||||
**Exit evidence:** the M1 outcome remains correct; representative warm changed-code p95
|
||||
is at most 30 seconds, cancellation/reporting at most two seconds, and supported-check
|
||||
completion at least 95%, under the predeclared profile. A fast census alone does not pass.
|
||||
|
||||
Owner: contributor-scan runtime, with pre-push-safety-net deadline/cache integration.
|
||||
Requirements: 4.1–4.4, 6.2–6.5, 7.3.
|
||||
|
||||
## M3 — The shared design transfers across languages
|
||||
|
||||
Status: **PARTIAL, 2026-09-17. Not met.** Python is established and PHP is blocked by runtime.
|
||||
|
||||
Python: all three declared VAmPI targets detected at their declared lines with their questions
|
||||
answered, 35 of 35 questions complete on every case, nothing reported new on identical revisions,
|
||||
zero admitted defects. That includes the SQL injection this spec names as previously unresolved,
|
||||
and both broken-object-level-authorization targets, which is the first time access control has
|
||||
answered on Python at all.
|
||||
|
||||
PHP: not established, and the blocker is now named. All PMPro comparisons exhausted every budget
|
||||
tried, up to a 4500-second deadline with a 900-second query ceiling, and produced no artifact, so
|
||||
there is no target evidence. Splitting a failed batch into parts did not rescue it.
|
||||
|
||||
A probe narrowed the cause: the evidence pass answers all 4,425 PMPro regions in one batch in
|
||||
29.7 seconds with a 681 KB response, and its cost is fixed rather than per request. Neither
|
||||
request count nor response size is anywhere near a limit. What fails is the execution pass, which
|
||||
runs `reachableByFlows` per candidate. The pins, receipts and declared target are correct, so the
|
||||
question is being asked properly and the dataflow work is what cannot finish.
|
||||
|
||||
The exit asks for Node, PHP and Python together. Two of three are in hand.
|
||||
|
||||
- Repeat the useful vulnerable/fixed comparisons with PMPro/PHP and VAmPI/Python,
|
||||
including the previously unresolved VAmPI SQLI query/path evidence. Preserve explicit
|
||||
limits for authorization and other families whose evidence is still unsupported.
|
||||
- Keep framework knowledge in frontends/adapters and versioned facts. Equivalent normalized
|
||||
evidence must retain the same core ranking, scope and admission behavior.
|
||||
- Record core and adapter changes separately. Any PMPro-driven core change must be frozen
|
||||
and replayed on Node/Express before claiming transfer; the initial Node-first repair
|
||||
does not remove that existing requirement.
|
||||
|
||||
**Exit evidence:** reviewed development/regression results for Node, PHP and Python,
|
||||
including positive, fixed and unchanged controls, exact provenance, coverage and runtime.
|
||||
Do not label these repeatedly inspected projects independent qualification. C/C++ bounds
|
||||
remain a separate unsupported abstraction; adding a new engine is not a prerequisite here.
|
||||
|
||||
Owners: contributor-scan shared evidence and framework adapters; pre-push-safety-net evaluation.
|
||||
Requirements: 6.2–6.4, 8.1–8.4.
|
||||
|
||||
## M4 — Independent examples establish useful detection and low noise
|
||||
|
||||
Status: **PENDING M3**. Population preparation may proceed earlier, but evaluation follows
|
||||
the frozen candidate. Reserve untouched cases before examining their scan results.
|
||||
|
||||
- Freeze reviewed vulnerable, fixed and benign changes from projects not used to develop
|
||||
or tune the particular language/framework/family. Ghost remains reserved until its
|
||||
actual reviewed population exists. Node evidence cannot qualify PHP or Python.
|
||||
- Measure actual actionable alerts, supported recall, fixed-side silence, benign interruptions,
|
||||
completed coverage and runtime together, with counts and uncertainty. Keep unlocated
|
||||
targets, unanswered queries and unsupported cases visible in their declared populations.
|
||||
- Apply the existing acceptance profile: at least 95% observed actionable precision,
|
||||
at least 90% supported recall, at least 95% supported-check completion, the 30s/2s runtime
|
||||
gates, nonzero reviewed emitted alerts and positive controls, and zero known fixed/benign
|
||||
false alerts. Silence on everything fails. A core retune requires new untouched evaluation.
|
||||
|
||||
**Exit evidence:** reproducible, reviewed per-capability qualification with every mandatory
|
||||
gate passing together. Report failures honestly and leave those capabilities experimental;
|
||||
do not require every possible language or family to become eligible for a narrow release.
|
||||
|
||||
Owner: pre-push-safety-net task 8.3, consuming verified contributor-scan semantics.
|
||||
Requirements: 6.1–6.5, 8.1–8.4 and the accepted experimental acceptance profile.
|
||||
|
||||
## M5 — Enable a narrow advisory rollout
|
||||
|
||||
Status: **PENDING M4**.
|
||||
|
||||
- Generate eligibility bound to the exact passing runtime, query, fact, ranking and policy
|
||||
identities. Enable only independently qualified capabilities; stale or missing evidence
|
||||
enables none. Advisory retains the same evidence bar as blocking.
|
||||
- Revalidate the packaged vulnerable/fixed/benign push experience, compact reporting,
|
||||
truthful incomplete coverage, local no-model operation and preservation/removal of hooks.
|
||||
- Publish the supported scope, limitations and evidence with the advisory release.
|
||||
Expand capabilities only when they earn qualification. Blocking deployment is a later,
|
||||
explicit decision, even though opt-in blocking plumbing already exists.
|
||||
|
||||
**Exit evidence:** passing packaged acceptance validation and exact eligibility for the
|
||||
declared advisory scope, with the release decision recorded. Only then mark task 8.3
|
||||
and release readiness complete if their mandatory criteria are satisfied.
|
||||
|
||||
Owner: pre-push-safety-net admission, packaging and release validation.
|
||||
Requirements: 3.1–3.5, 5.1–5.5, 6.5, 7.1–7.4.
|
||||
|
||||
## Execution and reporting contract
|
||||
|
||||
Use M1a → M1b → M2 → M3 → M4 → M5 as the critical path. Preparation may overlap, but later
|
||||
milestones cannot substitute for missing earlier evidence. Necessary runtime repairs
|
||||
that make M1 executable are allowed; a standalone speed benchmark cannot close M1.
|
||||
Keep ranker-owned scope, vendor exclusion, language-agnostic core behavior and all
|
||||
acceptance thresholds unchanged. No new source IR, second detector, differential graph
|
||||
mutation, learned ranker or unrelated backlog is introduced by this amendment.
|
||||
|
||||
Before production edits, make the required owner design/task amendment concrete and
|
||||
apply the existing Kiro approval, review and verification workflow. Prior authorizations
|
||||
remain valid within their scope; this milestone record does not reset them.
|
||||
|
||||
Progress updates shall lead with the current milestone, observed developer outcome,
|
||||
remaining blocker and next concrete action. Report implementation task counts separately.
|
||||
Change a milestone to verified only with linked evidence; do not infer progress from
|
||||
elapsed effort or a completed experimental release tag.
|
||||
|
||||
Immediate next action (M3, task 13.3): re-run the declared PHP transfer replay bounded to a
|
||||
region budget whose questions complete, using the corrected census verdict, and record the
|
||||
outcome either way. Then task 13.4, PHP question completion, because admission reads outcomes
|
||||
and M4 measures completion at 95%. Task 15.1 (M4 population reservation) runs in parallel and
|
||||
blocks on nothing. Do not begin another family vocabulary campaign: the census has said which
|
||||
families can speak, and the constraint has moved to runtime and to evidence from code this
|
||||
project has not seen.
|
||||
@@ -2,6 +2,11 @@
|
||||
|
||||
Status: approved for task generation on 2026-09-12 following the maintainer's "ok, next kiro phase". The acceptance profile defines evaluation targets, not measured achievements.
|
||||
|
||||
Release sequencing amendment accepted 2026-09-14: [release milestones](release-milestones.md)
|
||||
defines M1 usefulness → M2 speed → M3 shared-language transfer → M4 independent qualification
|
||||
→ M5 narrow advisory rollout, with observable exit evidence. The requirements and numerical
|
||||
acceptance thresholds below remain unchanged.
|
||||
|
||||
## Introduction and Boundary
|
||||
|
||||
OpenUltraSAST helps developers catch concrete security regressions in the resulting commits they
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"feature_name": "pre-push-safety-net",
|
||||
"created_at": "2026-09-12T15:35:00Z",
|
||||
"updated_at": "2026-09-14T06:10:56.975614+00:00",
|
||||
"updated_at": "2026-09-16T17:15:00+00:00",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"approvals": {
|
||||
@@ -39,17 +39,17 @@
|
||||
{
|
||||
"at": "2026-09-12T16:20:40.814596Z",
|
||||
"source": "approve.",
|
||||
"action": "Accepted the generated task plan and authorized starting group 1 (1.1\u20131.4). Implementation started; tasks require independent review and fresh verification before completion."
|
||||
"action": "Accepted the generated task plan and authorized starting group 1 (1.1–1.4). Implementation started; tasks require independent review and fresh verification before completion."
|
||||
},
|
||||
{
|
||||
"at": "2026-09-12T18:32:19.033933Z",
|
||||
"source": "ok, next kiro phase, approved; correct. we can continue with the spec",
|
||||
"action": "Authorized group 2 (2.1\u20132.3), resumed after local Claude Mem repair. Independent review and fresh verification remain required."
|
||||
"action": "Authorized group 2 (2.1–2.3), resumed after local Claude Mem repair. Independent review and fresh verification remain required."
|
||||
},
|
||||
{
|
||||
"at": "2026-09-13T07:48:02.155687Z",
|
||||
"source": "continue with the task",
|
||||
"action": "Authorized continuing group 3 (3.1\u20133.4) after the group 3 next-step summary. Independent task review and fresh verification remain required."
|
||||
"action": "Authorized continuing group 3 (3.1–3.4) after the group 3 next-step summary. Independent task review and fresh verification remain required."
|
||||
},
|
||||
{
|
||||
"at": "2026-09-13T09:17:01.788332+00:00",
|
||||
@@ -59,7 +59,7 @@
|
||||
{
|
||||
"at": "2026-09-13T10:26:47.307154+00:00",
|
||||
"source": "move on to group 4 as a whole",
|
||||
"action": "Authorized all of group 4 (4.1\u20134.3), sequential implementation, independent task review and fresh verification. Groups 5\u20138 remain pending."
|
||||
"action": "Authorized all of group 4 (4.1–4.3), sequential implementation, independent task review and fresh verification. Groups 5–8 remain pending."
|
||||
},
|
||||
{
|
||||
"at": "2026-09-13T12:40:57.961577+00:00",
|
||||
@@ -69,17 +69,22 @@
|
||||
{
|
||||
"at": "2026-09-13T13:58:52.336506+00:00",
|
||||
"source": "continue with group 6 as a whole. do we need to amend the spec to get a go for the rollout",
|
||||
"action": "Authorized group 6 (6.1\u20136.4). Clarified complete preparation/discovery reuse after measured group 5 bottleneck; rollout gates remain unchanged. Manual Kiro review fallback remains necessary after a fresh agent dispatch hit the thread limit."
|
||||
"action": "Authorized group 6 (6.1–6.4). Clarified complete preparation/discovery reuse after measured group 5 bottleneck; rollout gates remain unchanged. Manual Kiro review fallback remains necessary after a fresh agent dispatch hit the thread limit."
|
||||
},
|
||||
{
|
||||
"at": "2026-09-13T15:45:15.902603+00:00",
|
||||
"source": "ok, continue with group 7 as a whole",
|
||||
"action": "Authorized tasks 7.1\u20137.3; manual Kiro fallback after fresh dispatch hit thread limit. Group 8 and rollout remain pending."
|
||||
"action": "Authorized tasks 7.1–7.3; manual Kiro fallback after fresh dispatch hit thread limit. Group 8 and rollout remain pending."
|
||||
},
|
||||
{
|
||||
"at": "2026-09-13T16:41:21.889949+00:00",
|
||||
"source": "move on to group 8",
|
||||
"action": "Authorized tasks 8.1\u20138.4 and their qualification work; preserve joint gates and record NO-GO wherever prerequisites or measurements do not pass. Fresh dispatch hit thread limit; manual Kiro fallback."
|
||||
"action": "Authorized tasks 8.1–8.4 and their qualification work; preserve joint gates and record NO-GO wherever prerequisites or measurements do not pass. Fresh dispatch hit thread limit; manual Kiro fallback."
|
||||
},
|
||||
{
|
||||
"at": "2026-09-16T13:30:00+00:00",
|
||||
"source": "add the M1a task to tasks.md and start it; continue with the plan",
|
||||
"action": "Added group 9 (M1a) to the approved task plan as the bounded owner amendment the milestone requires, implemented the task-local recorded-veto flag (9.1) and ran the three NodeGoat revisions (9.2). Diagnosis only: no capability enabled, no gate changed, rollout remains NO-GO."
|
||||
}
|
||||
],
|
||||
"task_review": {
|
||||
@@ -118,7 +123,18 @@
|
||||
"7.3",
|
||||
"8.1",
|
||||
"8.2",
|
||||
"8.4"
|
||||
"8.4",
|
||||
"9.1",
|
||||
"9.2",
|
||||
"10.1",
|
||||
"10.2",
|
||||
"10.3",
|
||||
"10.4",
|
||||
"10.5",
|
||||
"10.6",
|
||||
"10.7",
|
||||
"10.8",
|
||||
"10.9"
|
||||
],
|
||||
"remaining_tasks": 1,
|
||||
"verification_record": "implementation-group-8.md",
|
||||
@@ -132,11 +148,11 @@
|
||||
"implementation-group-7.md",
|
||||
"implementation-group-8.md"
|
||||
],
|
||||
"next_group": 8,
|
||||
"next_group": 10,
|
||||
"blocked_tasks": [
|
||||
"8.3"
|
||||
],
|
||||
"feasibility": "NO-GO: all 21 frozen NodeGoat transactions timed out; 0/24 declared target checks completed. Cancellation/reporting remained below 2 seconds. Priming did not qualify graph reuse; named census and engine lifecycle work routed to contributor-scan.",
|
||||
"feasibility": "NO-GO: post-census profile measured all 21 cases; 18 timed out, three identical-tip replays completed in 18.521–18.874s with valid reuse; 0/24 target checks completed. Maximum cancellation/report overrun 0.874s. See 2026-09-14-nodegoat-post-census-feasibility.json.",
|
||||
"qualification": "NO-GO: eligibility implementation and evidence delivered; required independent populations and verified VAmPI transitive witness remain missing. Zero capabilities enabled.",
|
||||
"validation": {
|
||||
"decision": "NO-GO",
|
||||
@@ -149,6 +165,35 @@
|
||||
"status": "VERIFIED",
|
||||
"record": "../contributor-scan/census-repair-verification.md",
|
||||
"effect": "Node 44/44 input census restored; partial/missing census remains explicitly unresolved. Existing eligibility stale; runtime/context/independent qualification remain NO-GO."
|
||||
},
|
||||
"remediation_research": "../contributor-scan/release-remediation-research.md",
|
||||
"m1a": {
|
||||
"status": "MET",
|
||||
"date": "2026-09-16",
|
||||
"record": "m1a-recorded-vetoes.md",
|
||||
"measurement": "../../../benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json",
|
||||
"kill_criterion_triggered": false,
|
||||
"result": "Raw <lambda>2 witnesses present at contributions.js 32-34; fixed twin analyzed and quiet (target question answered with zero rows, no deadline reason); unchanged revision recorded 'unchanged'. Five vetoes fired and were recorded. With all five lifted the detection stops at operation_correspondence_unresolved, a fifth veto not previously named. No defect admitted, no capability enabled."
|
||||
},
|
||||
"m1b": {
|
||||
"status": "IN_PROGRESS",
|
||||
"verified": [
|
||||
"10.1",
|
||||
"10.2",
|
||||
"10.3",
|
||||
"10.4",
|
||||
"10.5",
|
||||
"10.6",
|
||||
"10.7",
|
||||
"10.8",
|
||||
"10.9"
|
||||
],
|
||||
"remaining": [
|
||||
"10.10"
|
||||
],
|
||||
"measurement": "../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-exit.json",
|
||||
"result": "Exit run recorded. All three pinned NodeGoat revisions classify through the production rules: vulnerable three sites new with operation_absent_from_comparable_base and admission rejecting on capability_unavailable alone; fixed twin analyzed and quiet with three base-only findings; unchanged recorded unchanged with base locations resolved. Every veto M1a listed is repaired, plus two the repairs exposed. Zero admitted defects.",
|
||||
"remaining_gap": "The supported security consequence and concrete repair direction are not demonstrated. Both are owned by an evaluated capability declaration and the registry is deliberately empty. Task 10.10 renders them through an explicitly experimental declaration with eligibility still disabled; inventing the text is what admission exists to prevent."
|
||||
}
|
||||
},
|
||||
"release": {
|
||||
@@ -157,5 +202,26 @@
|
||||
"channel": "alpha",
|
||||
"rollout_decision": "NO-GO",
|
||||
"status_record": "status.md"
|
||||
},
|
||||
"release_milestones": {
|
||||
"record": "release-milestones.md",
|
||||
"direction_approved": true,
|
||||
"approval_source": "I agree. please note this into the raodmap as a spec",
|
||||
"current": "M1b",
|
||||
"verified": [
|
||||
"M1a"
|
||||
],
|
||||
"sequence": [
|
||||
"M1a: NodeGoat detection with admission vetoes recorded, not applied (MET 2026-09-16)",
|
||||
"M1b: repair only the vetoes M1a showed to matter; rerun through production rules",
|
||||
"M2: equivalent bounded runtime",
|
||||
"M3: shared-language transfer",
|
||||
"M4: independent qualification",
|
||||
"M5: narrow advisory rollout"
|
||||
],
|
||||
"implementation_design_approval": "Existing approvals preserved; this records milestone order and exit criteria, not blanket approval of future design changes.",
|
||||
"m1_split_accepted": "2026-09-14: maintainer asked to split M1 into M1a/M1b after review",
|
||||
"m1a_outcome": "Met inside the one-week box. Corrected the owner diagnosis: all four named vetoes fire, and operation correspondence for an operation on a changed line is the blocker that survives every other repair.",
|
||||
"m1b_position": "Comparison half verified and recorded on NodeGoat; the consequence and repair rendering (10.10) remains before M1b is verified."
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,10 +1,70 @@
|
||||
# Pre-push safety net status — v1.2.0-alpha.1
|
||||
|
||||
Updated 2026-09-14. Phase: **implementation**. Requirements, design and tasks are
|
||||
approved. **26/27 executable tasks complete; task 8.3 remains blocked.**
|
||||
Updated 2026-09-16. Phase: **implementation**. Requirements, design and tasks are
|
||||
approved. **28/29 executable tasks complete; task 8.3 remains blocked.**
|
||||
Rollout: **NO-GO**. The alpha tag publishes experimental implementation and evidence;
|
||||
it does not approve normal hook capabilities or production enforcement.
|
||||
|
||||
## Next release milestone
|
||||
|
||||
**M1a is met and the M1b comparison now works through the production rules.** The other four
|
||||
milestones remain unverified, and rollout remains NO-GO.
|
||||
|
||||
Every veto M1a recorded has been repaired. On 2026-09-16 the production comparison classified
|
||||
the pinned NodeGoat revisions without any lifting: the vulnerable revision is `new` with the
|
||||
three `eval` witnesses attributed to the change that introduced them, the fixed twin is
|
||||
analyzed and quiet, and the unchanged revision is `unchanged`. Production and the recorded view
|
||||
agree exactly, which is how we know the vetoes were repaired rather than bypassed. No capability
|
||||
is enabled, no defect is admitted and no alert is emitted. See
|
||||
[M1a and M1b evidence](m1a-recorded-vetoes.md).
|
||||
|
||||
The exit run is recorded. Every veto M1a listed is repaired, plus two the repairs exposed: the
|
||||
comparison's own degradation and boundary checks, and the runner handing every candidate the
|
||||
whole transaction's gaps. On the vulnerable revision admission now rejects on
|
||||
`capability_unavailable` alone.
|
||||
|
||||
One half of M1b remains. The milestone also asks for a developer-readable finding carrying a
|
||||
supported security consequence and a concrete repair direction. Both are owned by an evaluated
|
||||
capability declaration and the registry is deliberately empty, so nothing renders them. Writing
|
||||
that text without a declaration is what admission exists to prevent, so it is not worked around.
|
||||
Task 10.10 renders it through an explicitly experimental declaration with eligibility still
|
||||
disabled.
|
||||
|
||||
M1a ran on 2026-09-16 within its one-week box. The raw `<lambda>2` witnesses are present at
|
||||
contributions.js lines 32–34, the fixed twin was genuinely analyzed and quiet, and the
|
||||
unchanged revision was not reported as a new regression, so the kill criterion did not
|
||||
trigger. Five vetoes fired and were recorded; with all five lifted the detection still stops
|
||||
at `operation_correspondence_unresolved`, because a newly introduced operation sits on a
|
||||
changed line and has no base anchor. That fifth veto was not in the earlier diagnosis and is
|
||||
now M1b's central repair. See [M1a evidence](m1a-recorded-vetoes.md).
|
||||
|
||||
The 28/29 task count is implementation progress, not proximity to rollout. Follow the
|
||||
accepted [release milestone spec](release-milestones.md): usefulness → speed → shared
|
||||
PHP/Python transfer → independent qualification → narrow advisory rollout.
|
||||
|
||||
## Security capability, 2026-09-17
|
||||
|
||||
Group 12 closed three defects that limited what the tool can establish, each measured on the
|
||||
pinned NodeGoat revisions with detections unchanged throughout.
|
||||
|
||||
- **Access control could not answer anywhere.** Dominance and configuration were never asked for
|
||||
change context, because the evidence pass dispatches taint only. Each family now describes its
|
||||
own scope. Head completion rose from 205 to 298 of 436, `context_projection_unavailable` fell to
|
||||
zero, and access control went from 0 to 52 completed questions.
|
||||
- **Admission rejected genuine findings.** The operation symbol was the text before the first
|
||||
parenthesis, so `const preTax = eval(...)` was the operation `const preTax = eval`, which no
|
||||
qualified declaration could ever name. It is now the trailing qualified callee, with receiver
|
||||
spelling preserved.
|
||||
- **Unresolved questions said why, and then there were none.** The remaining 138 each named a
|
||||
function its file does not define. A route-registration module registers handlers other modules
|
||||
define, and regions were scoped to methods those files do not hold. The entry record keeps the
|
||||
handler name, because the authorization question is cross-file by nature, and the region is now
|
||||
scoped to the file instead. **Every question the scan asks now completes on both revisions,
|
||||
299 of 299**, with detections unchanged.
|
||||
|
||||
Evidence: [group 12 final](../../../benchmarks/measurements/2026-09-17-group-12-final.json) and
|
||||
[region attribution](../../../benchmarks/measurements/2026-09-17-region-attribution.json).
|
||||
|
||||
## Verified
|
||||
|
||||
Immutable pushed snapshots, shared multi-ref deadlines, ranker-owned scope, physical
|
||||
@@ -21,13 +81,16 @@ gates and a packaged native PHP/JavaScript partition smoke. See
|
||||
|
||||
## Rollout blockers
|
||||
|
||||
1. **Useful checks within the deadline are unproven.** The last full representative
|
||||
runtime profile timed out on all 21 Node transactions, completing 0/24 declared target
|
||||
checks. Cancellation/reporting stayed below the two-second allowance. The subsequent
|
||||
census repair is verified, but the full changed-code runtime/coverage profile has not
|
||||
passed under its new identities. Amortize engine startup/loading work and rerun the
|
||||
frozen seven-class profile; require warm p95 <=30s and >=95% completed supported checks.
|
||||
2. **Target and change-context evidence remains incomplete.** VAmPI preserves raw findings
|
||||
1. **Useful checks within the deadline are unproven.** The fresh post-census profile
|
||||
completed all 21 measurements: all 18 changed/cold/growth/multi-ref transactions timed
|
||||
out; the three identical-tip replays took 18.521–18.874s with valid graph/query reuse.
|
||||
Still 0/24 declared target checks completed. Maximum observed cancellation/report
|
||||
overrun was 0.874s, below the two-second allowance. After M1 establishes the useful
|
||||
comparison, amortize engine startup/loading and rerun the frozen seven-class profile; require warm p95 <=30s and >=95% completed supported checks.
|
||||
2. **Target and change-context evidence remains incomplete.** M1a measured exactly which
|
||||
contracts fail on the Node case: per-family change context, question-owned vendor
|
||||
relevance, operation correspondence for changed lines, file-wide correspondence ambiguity
|
||||
and query-to-operation provenance. VAmPI preserves raw findings
|
||||
at all three known sites, but the exact SQLI function question/transitive witness and
|
||||
supported authorization context are unresolved. Raw findings or rank positions cannot
|
||||
establish an actionable new regression. Repair this in the shared scan boundary and
|
||||
@@ -60,5 +123,135 @@ quality evaluation and packaged hook validation. Keep task 8.3 unchecked and the
|
||||
in implementation until those mandatory results are established.
|
||||
|
||||
Evidence: [group 8 validation](validation-group-8.md),
|
||||
[runtime scorecard](../../../benchmarks/measurements/2026-09-13-nodegoat-feasibility.json),
|
||||
[post-census runtime scorecard](../../../benchmarks/measurements/2026-09-14-nodegoat-post-census-feasibility.json),
|
||||
[census repair](../../../benchmarks/measurements/2026-09-14-node-census-repair.json).
|
||||
|
||||
The owner investigation now distinguishes missing family context producers, globally
|
||||
propagated vendor/correspondence gaps, and query-to-operation identity. See
|
||||
[remediation research](../contributor-scan/release-remediation-research.md). These are
|
||||
next implementation prerequisites, not completed fixes or changed acceptance gates.
|
||||
|
||||
An isolated Joern session probe matched the full Node census with four warm requests in
|
||||
0.221–0.467s and verified cancellation/reaping in 0.043s. It identifies a promising engine
|
||||
lifecycle change; full-query equivalence and changed-code performance remain unverified.
|
||||
|
||||
## Family detection census, 2026-09-19
|
||||
|
||||
Four of seven declared families establish findings on the two development subjects:
|
||||
injection, access_control, config_secrets and untrusted_destination. Output encoding, path,
|
||||
prototype and deserialization have established nothing anywhere. Every question completed on
|
||||
both subjects, 299 of 299 on NodeGoat and 35 of 35 on VAmPI, so the silences are answers
|
||||
rather than failures — but three of them are diagnosed gaps rather than clean code:
|
||||
NodeGoat's documented SSRF is invisible because `needle.get` is not in the sink vocabulary,
|
||||
its XSS is invisible because output encoding declares only DOM sinks, and its two
|
||||
access-control defects raise no obligation because the modelled data operations live in DAO
|
||||
files while the family's scope is the handler's own file.
|
||||
|
||||
The census also found and fixed one false positive: NodeGoat's only prototype report was
|
||||
the word `set` in a code comment, matched because the sink text clause searched a whole
|
||||
assignment node. Nothing about this changes the release gates. No capability is qualified by
|
||||
these subjects.
|
||||
|
||||
Two of the three gaps are closed. The untrusted-destination vocabulary now holds the client
|
||||
surface a Node server actually calls, and NodeGoat's SSRF is reported at
|
||||
`app/routes/research.js:16` with a witness naming the call. Output encoding now asks about the
|
||||
server half of its own family and establishes a finding at `app/routes/research.js:25`, and it
|
||||
states in its declaration that template-mediated escaping is not something it can decide. Five
|
||||
of seven families now establish findings.
|
||||
|
||||
Access control was the third gap and its structural half is closed. The obligation now follows
|
||||
the one named call a handler writes into its data-access module, attributed by the graph's own
|
||||
provenance rather than by name, and located at the call a contributor changed. On NodeGoat it is
|
||||
raised at six handlers where it was raised nowhere, and three reach a verdict. All three are
|
||||
corroborated and none is reported without a judge, which is the honest rung: entailing a carried
|
||||
obligation reported a signup handler that is public by design, because its authenticated
|
||||
siblings share the same data module. VAmPI is unchanged.
|
||||
|
||||
The commonest collection read now carries the obligation too, so NodeGoat's documented insecure
|
||||
direct object reference raises one where it never had. `find` had been left out because the name
|
||||
is shared with array searches and jQuery selectors, so it is claimed only in the argument shape
|
||||
of a query document. Measured on a plugin that ships jQuery, all 88 of its `find` calls stay
|
||||
unclaimed. Seven NodeGoat handlers now reach a verdict, all corroborated; the census is
|
||||
unchanged, because the corroborated band needs a judge it does not configure.
|
||||
|
||||
What a session read discharges is now decided by where its value goes. A destructuring spells
|
||||
`session.userId` away, so the arbiter walks back from the names the query is given; two NodeGoat
|
||||
handlers that were correct code stop being asked about, while the memo page that reads every memo
|
||||
and merely renders the session's user stays reported. The sibling scope was decided by
|
||||
measurement and left at the file: on this subject the file scope entails two handlers, both
|
||||
public routes and both wrong, and an application-wide scope entails four true defects and the
|
||||
same two wrong ones. Neither scope can tell a public route from a protected one, so the carried
|
||||
rung stays capped and that question stays with the judge.
|
||||
|
||||
Configuration reports every setting it decides, not the first one in the file. NodeGoat's
|
||||
server.js now carries both its permissive session and its disabled template auto-escaping, each
|
||||
with its own site; the escaping defect had been detected on every run since task 14.3 and
|
||||
reported on none, because an earlier setting in the same function won the single verdict.
|
||||
|
||||
The PHP record is corrected. The malformed-CPG bug this project reported upstream, a `global`
|
||||
inside a closure giving a node two AST parents, was fixed in Joern 4.0.625 and the image has
|
||||
pinned that version throughout; it is verified today on the real trigger file, on 4.0.625 and
|
||||
4.0.631 alike. What remains under task 13.2 is runtime alone: a 2,526-question batch over a
|
||||
673-file plugin does not finish in thirty minutes, and a six-fold larger ceiling only made the
|
||||
failure take longer. The engine pin stays where it is, because the newer release measures the
|
||||
same here.
|
||||
|
||||
That runtime reading was itself incomplete, and the correction matters more than the reading.
|
||||
PHP is not silent: at 25 and 100 regions PMPro completes with no timeout at all and establishes
|
||||
19 and 43 findings, including a true-positive SQL injection at
|
||||
`adminpages/discountcodes.php:749` of exactly the class the PHP fact table predicted. It looked
|
||||
silent because the census harness ordered its verdict on question completion before findings,
|
||||
and because only the 500-region runs had been examined. The harness is fixed. What remains is a
|
||||
scale limit of roughly one to two seconds per question, and one real gap that is bookkeeping
|
||||
rather than detection: no PHP question is ever marked completed, even in the runs that find
|
||||
things.
|
||||
|
||||
Closing the second gap surfaced four defects, three of which were false positives it would
|
||||
otherwise have shipped: a permissive-value set shared across settings whose booleans have
|
||||
opposite polarity, a dotted setting matched on its trailing segment alone, a flow query asked of
|
||||
a call's receiver as well as its arguments, and a file's settings ordered by the spelling of
|
||||
their line numbers. Template auto-escaping disabled at `server.js:135` is detected and still not
|
||||
reported, because the arbiter returns one verdict per question and that file has an earlier
|
||||
permissive setting; task 14.6 carries it.
|
||||
|
||||
Evidence: [family detection census](../../../benchmarks/measurements/2026-09-19-family-detection-census.json),
|
||||
[untrusted destination vocabulary](../../../benchmarks/measurements/2026-09-19-untrusted-destination-vocabulary.json),
|
||||
[output encoding family](../../../benchmarks/measurements/2026-09-19-output-encoding-family.json),
|
||||
[access control across the module split](../../../benchmarks/measurements/2026-09-20-access-control-module-split.json),
|
||||
[collection read obligation](../../../benchmarks/measurements/2026-09-20-collection-read-obligation.json),
|
||||
[sibling scope and identity](../../../benchmarks/measurements/2026-09-20-sibling-scope-and-identity.json),
|
||||
[every permissive setting](../../../benchmarks/measurements/2026-09-20-every-permissive-setting.json),
|
||||
[PHP runtime after the graph fix](../../../benchmarks/measurements/2026-09-20-php-runtime-after-graph-fix.json),
|
||||
[PHP detection corrected](../../../benchmarks/measurements/2026-09-20-php-detection-corrected.json).
|
||||
|
||||
## Strategy, 2026-09-21
|
||||
|
||||
Sequence amended after the family census and the PHP correction: close M3 at a completing scale
|
||||
first (task 13.3), fix PHP question completion because admission reads outcomes (13.4), reserve
|
||||
the untouched M4 population in parallel starting now (15.1), and give the remaining engineering to
|
||||
M2 graph reuse, which cannot be scoped away because meeting the latency gate by analysing less is
|
||||
explicitly not the same result. The v1 advisory scope is declared as the families whose claims can
|
||||
be admitted at the entailed rung (15.2): injection, untrusted destination and configuration.
|
||||
Access control is excluded because a carried obligation corroborates by design and would ship as
|
||||
silence.
|
||||
|
||||
No acceptance threshold and no owner boundary changes. The risk that remains is that every number
|
||||
so far comes from subjects used to build the analyser, and one week of examining two of them
|
||||
surfaced seven false positives; the first independent evaluation should be expected to fail.
|
||||
|
||||
## PHP reaches its declared target, 2026-09-22
|
||||
|
||||
The M3 PHP target is detected: `classes/class.memberorder.php:936:getMemberOrderByCode`, a request
|
||||
value concatenated into SQL with no prepare. It had never been reached, and nothing about detection
|
||||
changed to reach it. The population became affordable.
|
||||
|
||||
Four repairs did that, and three of them are the same rule the design now carries. Tier-zero pruning
|
||||
was switched off for a whole scan by any degradation, and a multi-language repository always carries
|
||||
one. Batches were sized by request count when the cost is per matched sink, so a family whose sinks
|
||||
are `echo` and `print` was priced like one whose sinks are rare. A per-query timeout cancelled the
|
||||
entire scan, abandoning work with half an hour of budget left. And nothing was held back for
|
||||
arbitration, so a scan could spend everything on queries and report none of what they answered.
|
||||
|
||||
On the 500-region population that previously completed zero questions at every budget tried: 271 of
|
||||
406 completed, 213 findings, 2,950 seconds. The three declared comparisons are running at that
|
||||
budget; detection is not transfer, and only the comparison decides M3.
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
# Strategy review after the independent evaluations (2026-09-28)
|
||||
|
||||
Status: for maintainer decision. Nothing below changes the approved specs until a direction is chosen.
|
||||
|
||||
## 1. What was measured
|
||||
|
||||
Two populations of real vulnerabilities that no rule had seen, each with its fix and a benign control, scored by
|
||||
protocols committed before the first scan (`benchmarks/independent/`).
|
||||
|
||||
| | v1 (11 cases, 2026-09-24) | v2 (17 cases, 2026-09-28) |
|
||||
|---|---|---|
|
||||
| Analyzer | 13180a1 | a75b9f6 (v1 plus every retune learned from v1) |
|
||||
| Recall | 1 / 11 | **0 / 17** (Wilson 95% 0–0.18) |
|
||||
| Sampled precision | 1 / 12 | **0 / 17** (0–0.18) |
|
||||
| Question completion | near zero | **12%** (13,206 / 108,374) |
|
||||
| Benign false alerts | — | 3 |
|
||||
| Run-to-run stability | not measured | 2 of 100 findings differ |
|
||||
|
||||
Between v1 and v2 the analyzer received every fix v1 suggested: file-owned graph gaps, `.gitignore` semantics,
|
||||
Node `vm` sinks, Django sources, quoted-only SQL escapes, fixed-origin redirects, guards and allowlists. On the
|
||||
v1 cases those retunes raised detection from 1 to 4 of 11. On untouched cases they transferred to nothing.
|
||||
|
||||
## 2. Why, per class of cause (observed in v2, from source)
|
||||
|
||||
1. **Entry points are framework-shaped, and our sources are token lists.** Request data arrives as
|
||||
`filter_input(INPUT_GET, …)` and the WordPress REST `$request` (GoPay, Directorist), plugin helpers such as
|
||||
`get_url_var()` (Groundhogg), FastAPI and NestJS typed or decorated handler parameters (qwed, ReactPress,
|
||||
LMDeploy, ContextForge), and Koa's `ctx.request` (Budibase). A vocabulary grows one framework at a time, and
|
||||
each new population brings frameworks it lacks.
|
||||
2. **Flows cross module boundaries the frontends do not resolve.** JavaScript `require`d modules (FUXA's route
|
||||
to its storage module), dependency-injected receivers (YesWiki, v1), registry dispatch (OpenCVE, v1), parser
|
||||
combinators (alerta, v1).
|
||||
3. **Sinks are matched by name, so precision fails where names collide.** Every sampled v2 finding was false:
|
||||
a regex `.match()`/`.exec()` read as injection, `header()` with an integer cast, parameterised ORM statements,
|
||||
client-SDK code, a process's own `argv`, a generated database name.
|
||||
4. **Scale.** Large repositories complete a few percent of their questions inside a 40-minute budget (pgAdmin
|
||||
2%, ContextForge 6%, Budibase 1%); one repository crashes the taint batch on every scan (LMDeploy).
|
||||
|
||||
Classes 1–3 are breadth problems: each fix is correct and local, and the next repository needs another. That is
|
||||
the pattern both evaluations show, and the reason more patching on the same design is unlikely to reach M4.
|
||||
|
||||
## 3. What we know about the alternative
|
||||
|
||||
The tool hunter (a model with `read_file` / `grep_repo` / `find_refs`) was measured once, on 2026-09-06, on the
|
||||
vibe-py pair holdout: **12 of 14 detected, Youden +0.857, no leaks**, against the overlay's 5 of 14 on the same
|
||||
pairs (class-aware scoring; `benchmarks/measurements/2026-09-06-vibe-py-holdout-hunter-*.json`). Caveats: one
|
||||
language, curated excerpts, a pair corpus rather than whole repositories, and never run on an independent
|
||||
population. Classes 1–3 above are exactly where a model reading the code has an advantage over token lists;
|
||||
class 4 (cost, scope) is where it has a disadvantage.
|
||||
|
||||
## 4. Options
|
||||
|
||||
**A. Continue engine-first.** Model framework entry points, fix the sink-collision classes, raise completion,
|
||||
then qualify on a v3 population. Every item is sound; the evidence (v1 → v2) says the breadth does not converge
|
||||
at the pace one population teaches. Highest effort, weakest evidence.
|
||||
|
||||
**B. Model proposes, engine proves.** The hunter supplies recall; the CPG, where it can reach, supplies the
|
||||
witness and admission evidence; findings the engine cannot confirm are reported as unconfirmed rather than
|
||||
dropped. This inverts `propose-adjudicate-prove`'s current out-of-scope line ("making an LLM the adjudicator")
|
||||
only partly: the model proposes, it does not adjudicate. Needs a cost and scope design (which files a pre-push
|
||||
scan hands the model) and a run of the **unchanged** hunter on an untouched population before any tuning.
|
||||
|
||||
**C. Narrow the product.** Qualify one ecosystem first (for example WordPress plugins, the declared v0.1 target),
|
||||
where a framework model is finite, and ship only that capability (M5 already allows per-capability enablement).
|
||||
Compatible with A or B.
|
||||
|
||||
## 5. Recommendation
|
||||
|
||||
Run the decisive, cheap experiment before choosing: **the unchanged tool hunter on the v2 pins**, scored with
|
||||
protocol v2's matching, within a stated model-cost budget. v2 is spent for the engine, but the hunter was never
|
||||
tuned on it; the rule is that no hunter prompt, tool or configuration may change before that measurement is
|
||||
recorded, and the scope handed to it must be decided and written down first (it must not use the fix diff or the
|
||||
declared sites, which would leak the answer).
|
||||
|
||||
- If the hunter reaches useful recall on v2 with few fixed-side and benign alerts, re-plan M2–M4 around B,
|
||||
narrowed by C.
|
||||
- If it does not, pursue A narrowed by C, starting with the WordPress entry-point model and the sink-collision
|
||||
precision classes, and qualify on a v3 population.
|
||||
|
||||
## 6. Decisions needed
|
||||
|
||||
1. Direction: run the hunter experiment first (recommended), or choose A, B or C now.
|
||||
2. For the experiment: the scope handed to the model per pin, the model, and the cost ceiling.
|
||||
3. Whether the roadmap's M1b/M2 work (rendering, runtime) pauses until the direction is chosen.
|
||||
|
||||
|
||||
## 7. Update 2026-09-29: what the experiment showed
|
||||
|
||||
Recorded in `benchmarks/independent/results-v2-model-pipeline.json` (exploratory; v2 is spent for tuning).
|
||||
|
||||
**Instrument first.** Four defects had made every model-backed result since 2026-09-10 unmeasured: the default
|
||||
model id was the product name and the API rejected it (cf57995); the hunter reached the embeddings provider, not
|
||||
the chat provider (cf57995); the final-answer turn used the provider's JSON mode and the answers were lost
|
||||
(3ffb8b4); one grep match in a minified bundle overflowed the model's context (8794884). The "hunter 1/17" number
|
||||
in section 5's experiment was produced through the broken answer turn and is void.
|
||||
|
||||
**Scope selection, source only.** The declared vulnerable function is among the candidates for: the ranker's
|
||||
top-20 entry points 1/17; the shipped sink vocabulary 2/17; a model reading every product file 15/17 ($14). The
|
||||
vocabulary route (option A) is confirmed not to converge on this population.
|
||||
|
||||
**Verification.** Per-file triage, one 6-step tool hunt per file, two independent passes, report only agreement:
|
||||
- validation set: 16 of 20 declared sites agreed, 12 of 15 cases; the 26 pgAdmin candidates that flickered
|
||||
between single runs reduce to 9 stable, 4 disputed, 13 quiet;
|
||||
- full run, 7 of 17 cases before the account emptied ($23): detected 4/7 (pgAdmin, qwed, LMDeploy,
|
||||
ReactPress); ContextForge and FUXA found in one pass of two; winml missed because a CORS literal is triaged
|
||||
"constant" -- the config family needs its own question, not the reachability one;
|
||||
- cost: $0.009-0.022 per candidate for two passes (old single-pass design: $0.017); a full two-pass run of the
|
||||
population is roughly $45-50.
|
||||
|
||||
**Precision is the open problem.** 196 agreed findings across 7 cases (pgAdmin alone 126 of 875 verified). Five
|
||||
pgAdmin findings read against source: three true (privileged; raw `{{ data.* }}` in SQL templates, unescaped
|
||||
stored `schema_res`), two false with rationales that contradict their own verdict (an ORM call the model itself
|
||||
calls parameterized; a pickle from the signed session). Agreement cannot catch a mistake both passes make. The
|
||||
proposed next filter is a no-tools judge over each agreed finding's rationale and code (~$0.002 per finding),
|
||||
which was not run. Two-pass agreement is itself not stable between runs (ContextForge, FUXA).
|
||||
|
||||
**Standing.** Recall: the model route reaches 12/15 candidate-site cases where the engine reached 0/17 and the
|
||||
vocabulary 2/17; that decides between options A and B in the model's favour for recall. Precision, fixed-pin
|
||||
behaviour and cost at product scale are unmeasured. No gate is met. Any qualification needs population v3 and
|
||||
a pre-registered protocol that also states the judge stage, the pass rule and the config-family question.
|
||||
@@ -1,12 +1,16 @@
|
||||
# Implementation Plan: pre-push-safety-net
|
||||
|
||||
Status: task plan approved by the maintainer on 2026-09-12. Groups 1–6 (tasks 1.1–6.4)
|
||||
are implemented, reviewed and verified (20/27 subtasks). Tasks 4.3 and groups 5–6 used the
|
||||
explicit Kiro manual review fallback after fresh agent dispatch reached its thread limit.
|
||||
Group 5's six cold PMPro/Node timeouts remain historical NO-GO evidence. Group 6 verifies
|
||||
compatible reuse with equal evidence/admission in a packaged authored JavaScript control;
|
||||
its identical-revision run completed in 2.00s. This is not representative warm p95 or
|
||||
independent capability qualification. Groups 7–8 remain pending; rollout remains NO-GO.
|
||||
Status: the existing task plan remains approved. Groups 1–7 and tasks 8.1, 8.2 and 8.4
|
||||
are verified (28/29 executable subtasks, including group 9); task 8.3 and rollout remain NO-GO. Later work
|
||||
used the documented manual review fallback where agent dispatch reached its limit.
|
||||
|
||||
The accepted [release milestone spec](release-milestones.md) now orders remaining work:
|
||||
M1a diagnose → M1b repair → M2 speed → M3 shared-language transfer → M4 independent
|
||||
qualification → M5 narrow advisory rollout. All milestones remain unverified.
|
||||
Milestones are release outcomes, not checked implementation tasks. Group 9 below is the
|
||||
bounded owner amendment M1a requires; it adds the task-local recorded-veto evaluation and
|
||||
its three-revision development run. It enables no capability and changes no gate.
|
||||
Preserve all completed task evidence below.
|
||||
|
||||
## Execution contract
|
||||
|
||||
@@ -243,9 +247,383 @@ independent capability qualification. Groups 7–8 remain pending; rollout remai
|
||||
- _Requirements: 3.1, 3.4, 5.1, 5.2, 5.3, 5.4, 5.5, 6.5, 7.1, 7.3, 7.4, 8.3, 8.4_
|
||||
|
||||
|
||||
- [x] 9. Diagnose the first useful detection with vetoes recorded (release milestone M1a)
|
||||
- [x] 9.1 Add the task-local recorded-veto evaluation flag
|
||||
- Add an explicit experimental evaluation path that **records** each comparison and admission veto per raw finding instead of applying it. Keep it task-local, off by default, and available only on explicit `--base`/`--head` replay; refuse it on the Git stdin path so it can never become a hook capability.
|
||||
- Lift only recognized completeness vetoes over a copy of the evidence: answered-then-demoted question outcomes, the vendor exclusion completion veto, and the diagnosed context/correspondence boundary kinds. Never lift failures: deadline exhaustion, unanswered questions, unreadable input, empty graphs and unparsed files stay applied, because silence from failure is not a quiet fixed twin.
|
||||
- Retain the production novelty, reason and admission reasons beside the recorded ones for every finding, plus the query-to-witness identity difference, base-only findings and readable census receipts. Every payload carries an explicit experimental label; the recorded path admits no defect and enables no capability.
|
||||
- Done when the flag is off by default, produces a labeled artifact section and terminal line under explicit replay, is refused in Git stdin mode, and unit controls prove recognized vetoes are recorded while failures and unrecognized boundaries remain applied.
|
||||
- _Boundary: Delta and Actionability Policy, Push runner, Result and Hook Adapter, CLI integration_
|
||||
- _Requirements: 3.1, 3.4, 5.5, 6.3, 7.3_
|
||||
- _Verified 2026-09-16:_ 1,330 tests passed, nine skipped; global Ruff/format and touched mypy passed. Flag off by default, refused in Git stdin mode, labeled artifact section and terminal line under explicit replay. Controls prove recognized vetoes are recorded while deadline exhaustion, unanswered questions, empty graphs and unrecognized boundaries stay applied, and that the recorded path emits no defect even when a lifted candidate would otherwise admit. Regenerated the committed module audit for the added module.
|
||||
- [x] 9.2 Run the three NodeGoat revisions and produce the per-finding veto list
|
||||
- Record exact base/head revisions, expected operation and expected outcome before running. Exercise the shipped replay path, existing ranker and physically vendor-free graph; do not substitute a hand-selected slice or hard-coded target function.
|
||||
- Replay the pinned vulnerable, fixed and unchanged revisions under the flag with an explicitly named development budget. That budget cannot satisfy or measure the hook latency gate.
|
||||
- Deliverable: the raw finding on the vulnerable revision with location and witness, the fixed twin genuinely analyzed and quiet, the unchanged revision not reported as a new regression, and for every finding the exact list of vetoes that fired and why, with source and census receipts.
|
||||
- Done when one replayable three-revision record states each expectation against its observed outcome and names the vetoes that actually fired. A fixed twin that timed out or was not analyzed fails M1a and blocks M1b until diagnosed.
|
||||
- _Boundary: Evaluation harness, full-run integration_
|
||||
- _Depends: 9.1_
|
||||
- _Requirements: 1.1, 2.1, 2.4, 3.4, 6.3, 7.1, 7.3_
|
||||
- _Verified 2026-09-16:_ all three pinned revisions replayed through the production CLI under a named 900-second development budget; expectations frozen on disk first. Raw `<lambda>2` witnesses present at lines 32–34; fixed twin analyzed and quiet (target question answered with zero rows, no deadline reason, three base-only findings); unchanged revision analyzed and recorded `unchanged`. Five vetoes fired and were recorded; the remaining blocker is `operation_correspondence_unresolved` for an operation on a changed line. Zero admitted defects. Evidence: [M1a record](m1a-recorded-vetoes.md) and `benchmarks/measurements/2026-09-16-nodegoat-m1a-recorded-vetoes.json`. Diagnosis only; no capability enabled and no latency claim.
|
||||
|
||||
- [x] 10. Repair only the vetoes M1a recorded (release milestone M1b)
|
||||
- [x] 10.1 Establish operation correspondence for an operation on a changed line
|
||||
- M1a measured this as the blocker that survives every other repair: a newly introduced operation sits on a changed line, has no base line anchor, and therefore cannot be classified even when every completeness veto is lifted.
|
||||
- Bind the required base/head correspondence to the specific operation identity rather than to a line anchor. A complete base answer for the counterpart question that contains no operation with the same operation and source establishes absence; a matching operation elsewhere in that answer is movement, not novelty.
|
||||
- Keep the existing conservative rules: an incomplete or failed base stays unknown, benign movement and renames stay unchanged, and lexical alignment is never semantic proof.
|
||||
- Add absent, renamed, removed-guard, moved-operation and repeated-identical-operation controls. Done when a newly introduced operation on a changed line is classified new, while movement, renames and an incomplete base remain unchanged or unknown.
|
||||
- _Boundary: Delta and Actionability Policy_
|
||||
- _Requirements: 2.4, 3.1, 3.4_
|
||||
- _Verified 2026-09-16:_ 1,337 tests passed, nine skipped; Ruff/format/mypy passed. Correspondence now reports `anchored`, `ambiguous`, `changed_line`, `unmapped` or `unchanged_path`, and only `changed_line` supports an introduced operation. Movement keys on the operation spelling, which the existing benign-rename control proved necessary. One superseded control was updated with its reason; movement, ambiguity, unmapped operations, discharged or unentailed claims and an incomplete base all remain unknown.
|
||||
- [x] 10.2 Establish the required comparison scope explicitly
|
||||
- Comparison currently requires every selected base question to complete with no base degradation and no non-tier-zero deferred question, so one unsupported family anywhere prevents an otherwise complete target comparison. The owner research names this and requires the amendment to state the required scope rather than relax the bar.
|
||||
- Define the comparison's required scope as the counterpart question plus the questions its evidence actually depends on, and require completeness over exactly that scope. Retain every other gap in aggregate coverage; a question outside the required scope stays reported and unresolved, never silently discharged.
|
||||
- A missing raw finding in an incomplete base still cannot establish novelty. Unknown dependency remains inside the required scope.
|
||||
- Done when an unrelated unsupported family no longer prevents a complete target comparison, while an incomplete counterpart, an incomplete dependency of it, or unknown dependency ownership still yields unknown.
|
||||
- _Boundary: Delta and Actionability Policy_
|
||||
- _Depends: 10.1_
|
||||
- _Requirements: 2.2, 2.4, 3.4, 6.4_
|
||||
- _Verified 2026-09-16:_ 1,342 tests passed, nine skipped; Ruff/format/mypy passed. Completeness is now demanded over the counterpart plus its recorded dependencies. Controls prove an unrelated unsupported family no longer blocks a complete target comparison, while an incomplete counterpart, an incomplete recorded dependency, an unknown-ownership boundary and a truncated base scan all still yield unknown. Every other gap stays in aggregate coverage.
|
||||
- [x] 10.3 Scope correspondence ambiguity to the operations it actually affects
|
||||
- `ambiguous_line_correspondence` is recorded file-wide and inherited by every question, so repeated unchanged lines invalidate distinct, uniquely mapped operations elsewhere.
|
||||
- Retain ambiguous mappings as unavailable and record which regions are ambiguous, so only an operation whose required correspondence needs an ambiguous line is vetoed. Do not delete the ambiguity check.
|
||||
- Done when a uniquely mapped operation is comparable in a file that also contains repeated unchanged lines, while an operation needing an ambiguous line stays unresolved.
|
||||
- _Boundary: Snapshot Adapter, Delta and Actionability Policy_
|
||||
- _Depends: 10.2_
|
||||
- _Requirements: 2.4, 3.4_
|
||||
- _Verified 2026-09-16:_ 1,347 tests passed, nine skipped; Ruff/format/mypy passed. Ambiguity is no longer inherited as a context gap by every question, and no longer blocks the transaction comparison; it is decided per operation by the correspondence classifier. Controls prove a uniquely mapped or newly introduced operation stays comparable in a file with repeated unchanged lines, an operation needing an ambiguous anchor stays unresolved, and any other transaction gap still blocks. The mapping is still withheld and still reported in aggregate coverage.
|
||||
- [x] 10.4 Export the enumerated operation inventory so absence is provable
|
||||
- **Prerequisite discovered 2026-09-16, not in the original plan.** The 10.1-10.3 rerun showed the remaining two vetoes cannot be scoped or relaxed soundly as written. A taint answer reports flows, so an absent row means either that the operation does not exist in the base or that no traced flow reached it. `dynamic_external_or_depth_context_unresolved` exists precisely to stop the second being read as the first, and on a real Express application it is always true, because ordinary framework and dependency calls have unresolved destinations. It therefore carries no discrimination and can never be discharged, while removing it would make an unreached operation look absent.
|
||||
- Export the operations the query actually enumerated in the question's scope, separately from the flows it traced, so absence of an operation is established by the inventory rather than by the absence of a flow. Sanitizer and source facts are unchanged; this adds no detector, alias resolver or second IR.
|
||||
- Done when a base answer distinguishes "this operation does not exist here" from "no traced flow reached it", with an unenumerated scope remaining explicitly unresolved.
|
||||
- _Boundary: contributor-scan query/evidence contract, Delta and Actionability Policy_
|
||||
- _Depends: 10.1_
|
||||
- _Requirements: 2.1, 2.2, 3.4, 8.1, 8.2_
|
||||
- _Verified 2026-09-16:_ 1,354 tests passed, nine skipped; Ruff/format/mypy passed. The taint query exports one row per enumerated operation plus a completion marker, which is what makes an empty inventory meaningful. Comparison now separates three outcomes: operation present in the base without a traced flow, operation absent from an enumerated base, and a base scope never enumerated, which stays explicitly unresolved along with any malformed inventory. The arbiter skips every row carrying a kind, so an inventory row cannot become a flow with an empty source. Real-engine smoke on an authored two-revision JavaScript control confirmed emission, the zero-operation marker and unchanged answer integrity: `benchmarks/measurements/2026-09-16-operation-inventory-smoke.json`. Pre-inventory fixtures were given an enumerated base, since an unenumerated one can no longer establish absence.
|
||||
- [x] 10.5 Make the external and dynamic context boundary question-owned
|
||||
- M1a recorded `dynamic_external_or_depth_context_unresolved` on the target's own question. The flag is computed over every in-scope method, so one unresolved call anywhere vetoes every question.
|
||||
- With 10.4 supplying inventory evidence, bind the boundary to what it actually limits: reachability, not enumeration. A question establishing absence from the inventory no longer needs complete reachability; a question asserting a traced flow still does. Unknown ownership remains a blocker; absent evidence of independence is not proof of independence.
|
||||
- Done when an absence established from the inventory survives an unrelated unresolved destination, while a reachability claim under a bounded scope stays unresolved.
|
||||
- _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
|
||||
- _Depends: 10.4_
|
||||
- _Requirements: 2.1, 2.2, 8.1, 8.2_
|
||||
- _Verified 2026-09-16:_ 1,362 tests passed, nine skipped; Ruff/format/mypy passed. The boundary is now bound to what it limits. At scan level it is recorded against its owning question without demoting that question's answer, while an unavailable projection, an empty scope and an unreadable extent still demote. In comparison it no longer blocks the head's own traced flow, and no longer blocks an absence read off a structural enumeration, but still blocks every claim resting on a flow not having been traced. The query declares its enumeration basis, because a scope followed through the call graph can be shrunk by the same unresolved destination; such an inventory cannot establish absence while reachability is bounded. Controls cover both directions and both bases; every other gap kind still blocks.
|
||||
- [x] 10.6 Stop vendor exclusion acting as a global completion veto
|
||||
- `vendor_semantics_unresolved` degrades every otherwise complete outcome to `graph_incomplete`, including on a complete 44-file first-party census with successful query receipts.
|
||||
- Keep the exclusion physical and keep the coverage note. Let the inventory and reachability evidence from 10.4 and 10.5 carry whether a specific answer needed excluded semantics; a recorded dependency on an excluded path still blocks that question.
|
||||
- Done when an unrelated vendor exclusion no longer invalidates a first-party answer, while a question with evidence of dependency on excluded code remains unresolved and the exclusion stays out of targets and graphs.
|
||||
- _Boundary: contributor-scan graph/scope evidence, Cache and Engine Boundary_
|
||||
- _Depends: 10.5_
|
||||
- _Requirements: 2.2, 2.3, 8.2, 8.4_
|
||||
- _Verified 2026-09-16:_ 1,365 tests passed, nine skipped; Ruff/format/mypy passed. Graph boundaries are split into integrity failures, which still demote a completed answer, and declared exclusions, which do not. Whether a particular answer needed the excluded semantics is carried by that question's own unresolved call destinations from 10.5, not assumed for every answer. Controls exercise the real partition logic rather than an injected reason: a vendored directory leaves the answer completed with the exclusion still reported, an unresolved symlink still demotes, and the two together still demote. The exclusion stays physical and out of targets and graphs. The NodeGoat rerun showed the same distinction was still missing inside comparison, where any degradation and any boundary blocked: comparison now filters declared exclusions and correspondence ambiguity on both sides, keeping integrity degradations and every other boundary blocking.
|
||||
- [x] 10.7 Bind candidate dependency gaps to the candidate
|
||||
- **Discovered 2026-09-16 in the M1b production rerun.** The runner passes every coverage reason as every candidate's `dependency_gaps`, so admission always answers `dependency_unresolved`. That is not "an unresolved dependency that undermines the claim"; it is every gap anywhere in the transaction.
|
||||
- Supply the gaps that pertain to this candidate's required comparison scope, the same scope 10.2 established. Unknown ownership still counts against the candidate.
|
||||
- Not an M1b exit blocker, since eligibility stays disabled and no alert is emitted either way. It would prevent admission from ever passing after qualification, so it belongs before M4.
|
||||
- Done when an unrelated transaction gap no longer appears as this candidate's unresolved dependency, while a gap in its required scope still does.
|
||||
- _Boundary: Push runner, Delta and Actionability Policy_
|
||||
- _Depends: 10.2_
|
||||
- _Requirements: 3.1, 3.3, 3.4_
|
||||
- _Verified 2026-09-16:_ 1,371 tests passed, nine skipped; Ruff/format/mypy passed. Gaps are now attributed: a boundary carrying a question id counts only when that question is in this candidate's required scope, an unattributed gap always counts because unknown ownership is not evidence of independence, and integrity degradations count. A declared exclusion, correspondence ambiguity and bounded reachability do not, because each was already weighed against this specific claim. The runner no longer hands every candidate the whole transaction's reason list, which had included other candidates' comparison outcomes.
|
||||
- [x] 10.8 Carry query-to-operation provenance into the reported evidence
|
||||
- The target is selected through a file-fallback question with `function=None` while rows report method `<lambda>2`. M1a showed this did not block the comparison once the question completed, so it is provenance quality rather than the blocker.
|
||||
- Record verified query-to-operation provenance on the evidence; do not inject benchmark function names into regions or count coincident sites.
|
||||
- Done when a reported operation names the question that produced it and the method the engine reported, with unresolved provenance explicit.
|
||||
- _Boundary: Delta and Actionability Policy, contributor-scan normalization_
|
||||
- _Requirements: 3.1, 3.4, 6.3_
|
||||
- _Verified 2026-09-16:_ 1,371 tests passed, nine skipped; Ruff/format/mypy passed. Evidence now carries the method the engine reported beside the question that produced it, so the link is inspectable rather than inferred or dug out of a detail blob. Provenance is recorded as `exact`, `file_scope` when the question named no function and the engine supplied one, `unresolved` when the two disagree, and `method_unreported` when the engine named none. Controls cover all four. No region is renamed, no benchmark function name is injected and no coincident site is counted; the state is recorded, not applied.
|
||||
- [x] 10.9 Rerun the three revisions through the production rules
|
||||
- Replay the same pinned vulnerable, fixed and unchanged revisions **without** the recorded-veto flag under the named development budget. Demonstrate a developer-readable evaluation finding with a supported consequence, exact location, witness, change attribution and repair direction.
|
||||
- Verify the fixed twin is analyzed and produces no actionable defect and the unchanged revision is not reported as a new regression. Keep qualification eligibility disabled; experimental evaluation output cannot advertise an unqualified hook alert.
|
||||
- Done when one replayable record states which vetoes were repaired and which remain, with completed required checks and readable source and census receipts.
|
||||
- _Boundary: Evaluation harness, full-run integration_
|
||||
- _Depends: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8_
|
||||
- _Requirements: 1.1, 2.1, 2.4, 3.1, 3.4, 6.3, 7.1, 7.3_
|
||||
- _Verified 2026-09-16 with one named exception:_ all three revisions replayed through the production rules with exact pins and reopened source receipts. Vulnerable: three sites new with `operation_absent_from_comparable_base`, admission rejecting on `capability_unavailable` alone. Fixed twin: analyzed and quiet with three base-only findings. Unchanged: three sites `unchanged` with base locations resolved to the same lines. Every finding carries its question, the engine-reported method and its witness; the head census is the complete 44 files; all 436 base questions complete. Zero admitted defects. The repaired and remaining veto lists are in [the M1b record](m1a-recorded-vetoes.md). **Exception:** the supported security consequence and concrete repair direction are not demonstrated, because both are owned by an evaluated capability declaration and the registry is deliberately empty. M1b's comparison half is verified; its explanation half waits on M4.
|
||||
|
||||
- [x] 10.10 Render the experimental evaluation finding with its consequence and repair
|
||||
- **The remaining half of M1b's exit, named by the 10.9 run.** The comparison is verified, but the milestone also asks for a developer-readable finding carrying a supported security consequence and a concrete repair direction. Both come from an evaluated capability declaration and the registry is deliberately empty, so nothing renders them.
|
||||
- Generate an explicitly unreviewed, experimental declaration for this capability with the existing registry tool, and render the finding through the same grounded-template path the admitted case uses. Keep eligibility disabled: an experimental declaration is not enabled and not PASS, so admission must still reject it and no normal hook alert may be advertised.
|
||||
- Do not write consequence or repair text that no declaration supports. Refusing generic advice without a supported context and operation is the rule admission exists to enforce, and the experimental path must not become a way around it.
|
||||
- Done when the experimental output shows family, exact location, witness, change attribution, consequence and repair direction for the vulnerable revision, is labeled experimental throughout, and the production result still reports no actionable defect with zero admitted defects.
|
||||
- _Boundary: Result and Hook Adapter, Delta and Actionability Policy, Evaluation harness_
|
||||
- _Depends: 10.9_
|
||||
- _Requirements: 3.1, 3.2, 3.3, 5.1, 5.5, 6.3_
|
||||
- _Verified 2026-09-16:_ 1,386 tests passed, nine skipped; Ruff/format/mypy passed. Three evaluation findings rendered on the vulnerable revision with exact location, witness, change attribution, provenance, consequence and repair, all from the declaration's templates through the grounded-template path. The declaration reports verdict experimental and enabled false, the production result still reports no actionable defect, and all three artifacts carry zero admitted defects. Controls prove an ungrounded template is refused, an uncovered operation symbol is stated, no matching declaration leaves both sentences absent with the reason, and a declaration claiming PASS still admits nothing because the rendering path is outside admission. Evidence: `benchmarks/measurements/2026-09-16-nodegoat-m1b-explained.json`.
|
||||
|
||||
|
||||
- [x] 11. Fit the same useful analysis into the push budget (release milestone M2)
|
||||
- [x] 11.1 Give the backend a transaction-owned engine session
|
||||
- The measured cost is JVM startup, not analysis. In the M1b runs each disposable `joern --script` invocation cost 13 to 23 seconds while the work itself took under a second, and the 2026-09-14 lifecycle probe measured one session at 12.8 s to start, 3.6 s to load a graph and 0.22 to 0.47 s per warm request.
|
||||
- Own the session inside the existing backend for the length of one transaction: loopback only on an ephemeral port with no published port, a private working directory, one shared deadline, serialized requests, explicit graph identity and epoch, and kill-and-reap of the whole process group on cancellation, failure or exit.
|
||||
- An HTTP status is not an answer contract. Require a per-request receipt written to a fresh private file and verified before the answer is accepted, the way the probe's repaired instrument did after its first attempt read an empty stdout as a census.
|
||||
- Poison the session on any crash or malformed response so a failed session can never answer a later request, and fall back to the disposable path rather than reporting a zero.
|
||||
- Done when a controlled hanging in-session request is cancelled inside the allowance with no surviving process group, a crashed session answers nothing afterwards, and the disposable path still works unchanged.
|
||||
- _Boundary: contributor-scan runtime, Cache and Engine Boundary_
|
||||
- _Requirements: 4.1, 4.2, 7.3_
|
||||
- _Verified 2026-09-16:_ 1,391 tests passed, nine skipped; Ruff/format/mypy passed. Sixteen session tests drive a real fake-server process: receipt verified before an answer is accepted, a mismatched receipt or crash poisons the session so nothing answers afterwards, an empty body with its receipt is a valid empty answer, a hanging request is cancelled and the process group is gone inside the allowance, and a launch failure is recorded rather than raised. The probe took four iterations, each failing loudly: `runScript` is absent from the server REPL; the server answers success even for a compile error, so the receipt is the only proof; a shipped `@main` script cannot be nested in a block, so definitions go to the top level unwrapped; and the REPL shadows `Console`, so capture must qualify `scala.Console.withOut`. Evidence: `benchmarks/measurements/2026-09-16-session-script-execution.json`.
|
||||
- [x] 11.2 Prove every shipped query kind answers identically through both transports
|
||||
- Run taint, dominance, configuration and census over the same graph through the session and through the disposable path and require identical payloads, including empty answers and the distinction between an empty answer and an unavailable one.
|
||||
- Cover a malformed or absent response, a switched graph, a second graph loaded in the same session, and a request arriving after cancellation. Equivalence on a census alone does not establish equivalence for security queries.
|
||||
- Done when every shipped query kind matches byte for byte on a real graph, and every failure shape is reported as unavailable rather than empty.
|
||||
- _Boundary: contributor-scan runtime, engine integration tests_
|
||||
- _Depends: 11.1_
|
||||
- _Requirements: 4.1, 4.2, 6.4, 7.3_
|
||||
- _Verified 2026-09-16:_ taint, dominance and configuration all return byte-identical payloads through both transports on a real NodeGoat graph, with parameters built by each arbiter's own request function through the driver's family lookup. Failure shapes are correct: an empty answer stays present and distinguishable from an unavailable one, an unknown query and an absent graph are both unavailable, and two graphs in one session each answer about the graph the request names. Amortized over five taint requests the session took 22.48 s against 71.96 s disposable, all answered and identical; a single request per session is its worst case and is recorded as such. Evidence: `benchmarks/measurements/2026-09-16-session-transport-equivalence.json`.
|
||||
- [ ] 11.3 Route the batch path through the session behind an explicit setting
|
||||
- Consume the session from the existing batch and census paths, keeping the disposable path as the default and as the fallback. No capability, ranking decision or evidence semantics changes; only the transport does.
|
||||
- Account for session startup, load, requests and teardown inside the one shared deadline, and record them as separate stage costs so the next decision is made on measurements rather than estimates.
|
||||
- Done when a replay produces the same evidence and admission through either transport under a sufficient budget, with per-stage costs recorded for both.
|
||||
- _Boundary: Cache and Engine Boundary, Push runner_
|
||||
- _Depends: 11.2_
|
||||
- _Requirements: 4.1, 4.3, 6.4, 7.3_
|
||||
- [x] 11.4 Remove the startup cost from the critical path where the measurement says it pays
|
||||
- With 11.3's stage costs in hand, decide what else to change. The candidates the M1b runs point at are the overlay pass, 15.8 s in a separate invocation, and session startup, 12.8 s, which may overlap the frontend build rather than follow it.
|
||||
- Change only what the measurement implicates, and keep each change's correctness evidence separate from its timing evidence. Do not substitute a faster census for a completed security check.
|
||||
- Done when the recorded stage costs after the change are compared against the same profile before it, with any regression in completed checks reported rather than traded away.
|
||||
- _Boundary: contributor-scan runtime_
|
||||
- _Depends: 11.3_
|
||||
- _Requirements: 4.1, 4.2, 6.2, 6.5_
|
||||
- _Verified 2026-09-17:_ 1,399 tests passed, nine skipped; Ruff/format/mypy passed. The overlay, the last fixed cost bypassing the session, now runs in-session and the saved graph is verified rather than assumed. The first session-enabled replay then showed correct detections and no speedup, and a targeted probe disproved the sequence hypothesis before finding the real defect: a build runs on a copy of the backend, so the session it started was never closed, leaking a JVM holding the configured heap, twice per transaction, enough to exhaust the container. With that fixed the session stays alive for the whole transaction and no `session_query_failed` appears. Measured on the same pins and budget, the vulnerable revision went from 326.4 s to 243.3 s, head query time from 63.6 s to 32.0 s and base from 85.1 s to 51.2 s, with dominance and configuration each falling from about 14 s to about 4 s. Detections unchanged. Evidence: `benchmarks/measurements/2026-09-17-nodegoat-session-transport.json`.
|
||||
- [x] 11.5 Rerun the frozen seven-class runtime profile and decide
|
||||
- Rerun the unchanged seven-class profile: identical-tip, function edit, dependency edit, configuration edit, cold, growth and multi-ref, three samples each, retaining every timeout and unresolved check. Identical-tip reuse cannot substitute for warm changed-code work.
|
||||
- Verify the M1 outcome still holds on the same three revisions through the production rules, so speed is not bought with a lost detection.
|
||||
- Apply the existing gates without alteration: representative warm changed-code p95 at most 30 s, cancellation and reporting at most 2 s, supported-check completion at least 95%. A failed gate stays NO-GO and names the dominant remaining cost.
|
||||
- Done when one versioned scorecard reports p50 and p95 with sample counts beside the M1 correctness outcome, and states GO or NO-GO against the unchanged gates.
|
||||
- _Boundary: Evaluation harness, runtime integration_
|
||||
- _Depends: 11.4_
|
||||
- _Requirements: 4.1, 4.2, 6.2, 6.3, 6.5, 7.3_
|
||||
- _Measured 2026-09-17, verdict NO-GO:_ the unchanged seven-class profile, three samples each, session enabled. Every changed-code workload still times out at the 30-second deadline, and identical-tip got worse, p50 rising from 18.5 s to 24.2 s. The cause is recorded in the artifacts rather than inferred: 13 transactions report `session_unavailable:session_startup_timeout`, so under the real deadline the session usually never starts and everything falls back. Startup is 9 to 13 seconds, about a third of the whole budget, spent before any analysis. Where a session did start, on cached-graph transactions with little remaining work, it cost more than it saved. Correctness is unchanged: the same three revisions through the production rules with the session on reproduce the M1 verdict. Evidence: `benchmarks/measurements/2026-09-17-session-runtime-profile.json`. **Dominant remaining cost: engine startup itself, which a transaction-owned session relocates but cannot remove.**
|
||||
|
||||
- [x] 12. Security capability work found during M1a and M1b
|
||||
- Group 11 established that M2 needs graph reuse rather than more runtime tuning, and that decision is recorded there. This group returns to what the tool can actually establish about a codebase, which is where the remaining release risk sits: two of the three shipped families cannot complete their change context at all, and admission rejects genuine findings on a mis-extracted operation symbol.
|
||||
- [x] 12.1 Extract the operation symbol from the operation, not from the statement
|
||||
- **Found 2026-09-16 in the M1b explanation run.** Admission takes the text before the first parenthesis as the operation symbol, so `const preTax = eval(req.body.preTax)` yields `const preTax = eval`. The demonstration declaration had to enumerate each assignment spelling for its coverage check to pass.
|
||||
- A qualified declaration cannot enumerate every assignment a codebase might write, so admission would reject genuine findings on `operation_semantics_mismatch`. Take the symbol from the called operation the engine reported, preserving receiver spelling, and leave an unresolved or dynamic spelling ineligible as it is today.
|
||||
- Done when the symbol for an assigned call is the call's own name, a receiver-qualified call keeps its receiver, a dynamic spelling stays ineligible, and a declaration naming the plain symbol covers the assigned form.
|
||||
- _Boundary: Delta and Actionability Policy_
|
||||
- _Requirements: 3.1, 3.2, 3.4_
|
||||
- _Verified 2026-09-17:_ 1,403 tests passed, nine skipped; Ruff/format/mypy passed. The symbol is now the trailing qualified callee rather than everything before the first parenthesis, so `const preTax = eval(...)` yields `eval` and a declaration naming the call covers the assigned form. Receiver spelling is preserved, so `$wpdb->prepare` and `prepare` remain different operations. A spelling that is not a qualified name, such as a dynamic `handlers[name](x)`, yields the statement text, matches no declaration and stays ineligible exactly as before. The recorded-veto report reads the same helper rather than repeating the extraction.
|
||||
- [x] 12.2 Supply per-family context producers for dominance and configuration
|
||||
- The M1b exit run left 231 of 436 head questions reporting `change_context_incomplete` from `context_projection_unavailable` and `context_scope_empty`, because those families' queries export no context. They did not block the Node injection case, so M1b correctly left them alone, but their families stay experimental until they can answer.
|
||||
- Give dominance and configuration their own context producers describing the actual scope of the query they accompany. Do not copy taint context into them: dominance evaluates sibling operations across a file and configuration has different scope and obligations.
|
||||
- Done when a dominance and a configuration question complete their change context on a real repository, with missing extents and unsupported reasoning still explicit.
|
||||
- _Boundary: contributor-scan query/context evidence_
|
||||
- _Requirements: 2.1, 2.2, 8.1, 8.2_
|
||||
- _Verified 2026-09-17:_ 1,405 tests passed, nine skipped; Ruff/format/mypy passed. The gap was larger than a missing query block: the evidence pass dispatches taint only, because only taint has an evidence vector, so these families were never asked for context at all. Each query now describes its own scope, dominance the file's methods labelled operation, guard or sibling because its claim is about a file contradicting itself, and configuration the file's methods labelled setting or sibling. Neither copies taint's reachability projection, which would describe work they never do, and both report an empty scope explicitly. A new context pass asks each family with the flag set; a family whose query answers nothing keeps no context and stays unresolved, which a control asserts so no family can complete on another's context. Confirmed on a real NodeGoat graph: both produce a context summary and seven scoped methods. Evidence: `benchmarks/measurements/2026-09-17-per-family-context.json`.
|
||||
|
||||
- [x] 11.6 Decide what to do about engine startup, which the session cannot remove
|
||||
- **Named by the 11.5 measurement.** A transaction-owned session amortizes engine startup across the invocations inside one transaction, and a warm changed-code push does not have enough of them: startup is 9 to 13 seconds of a 30-second budget, so the session either fails to start or costs more than it saves. Group 11 as designed cannot reach the gate.
|
||||
- The options are a decision for the maintainer, not an implementation detail, because two of them change the product's shape. A session shared across transactions is a resident local service, which the accepted milestone spec explicitly does not imply. A smaller budget for what a push analyses is a scope change, not a speed change. A faster cold path would mean attacking the frontend and overlay costs directly, which the session does not touch.
|
||||
- Record the measured cost of each candidate before choosing, and keep the 30-second and 2-second gates unchanged; a gate met by analysing less is not the same result.
|
||||
- Done when one record states the candidates, their measured or estimated costs, their effect on the product's shape, and the maintainer's choice.
|
||||
- _Boundary: contributor-scan runtime, release direction_
|
||||
- _Depends: 11.5_
|
||||
- _Requirements: 4.1, 4.2, 6.5_
|
||||
- _Decided 2026-09-17 by the maintainer: do not rebuild a whole graph per push._ Both startup attacks were measured and neither reaches the gate. The session helps a long transaction, 326 s to 243 s, and cannot help a 30-second one: startup is 9 to 13 seconds of the budget, so it usually declines. Removing JavaScript's second engine start is real, 11.0 s against 21.0 s on identical graphs, but its census gate was asymmetric and it was reverted after costing two engine starts on 12 profile transactions. What the profile names is not startup at all: a changed-code transaction reaches its deadline during the build, head build 25.2 s of 30 s with query time 0.0 s. The remaining cost is building a fresh whole-repository graph for every pushed revision. **Direction: reuse the graph across revisions instead of rebuilding it, rather than further optimising the rebuild.** Not scheduled here; M2 stays unmet and the 30-second and 2-second gates stay unchanged.
|
||||
|
||||
- [x] 12.3 Resolve function-scoped questions whose function the query cannot locate
|
||||
- **Named by the 12.2 measurement.** With per-family context in place, head completion rose from 205 to 298 of 436 and `context_projection_unavailable` fell to zero. Every remaining unresolved question reports `context_scope_empty`, and they sit in three files: `app/routes/index.js` (114), `app/routes/tutorial.js` (12) and `server.js` (12). None is access control.
|
||||
- These are function-scoped questions whose named function the query cannot locate in the graph, so the scope it describes is honestly empty. It is the same query-to-operation identity problem M1a recorded at the witness end, appearing at the scope end.
|
||||
- Carry verified provenance from the region's declared function to the method the engine actually holds, and leave an unresolvable name explicit. Do not widen a question to the whole file to make it complete, and do not inject a declared name into a region.
|
||||
- Done when a question whose function the graph holds completes, a question whose function it does not hold stays unresolved with the name recorded, and no question completes by widening its scope.
|
||||
- _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
|
||||
- _Depends: 12.2_
|
||||
- _Requirements: 2.1, 2.2, 6.3, 8.1_
|
||||
- _Verified 2026-09-17:_ 1,406 tests passed, nine skipped; Ruff/format/mypy passed. The cause is recorded rather than guessed: `app/routes/index.js` requires seven handler modules and defines none of them, so a question naming `handleContributionsUpdate` there names a function that file genuinely does not hold and the empty scope is correct. Taint and configuration now separate the two causes, reporting `context_function_unresolved` with the name when a question names a function the file does not define, and `context_scope_empty` only when the file has no methods at all. The driver no longer adds its generic empty-scope gap on top of a query that already explained itself, and the new gap is question-owned so it cannot block another question's comparison. Completion is unchanged by design: a question whose function the graph does not hold stays unresolved, and nothing completes by widening its scope to the whole file.
|
||||
|
||||
- [x] 12.4 Attribute a region to the file that defines its handler
|
||||
- **The region-attribution half of 12.3.** A route-registration module registers handlers other modules define. `app/routes/index.js` requires seven handler modules and defines none, yet regions there were scoped to `handleContributionsUpdate` and its siblings, so the graph rightly held no such method and every question in those regions was unresolvable: 138 across six families.
|
||||
- The entry record still names the handler, because the authorization question is which middleware guards which handler and that is cross-file by nature. It now also records whether the file defines what it names, and a region built from an entry that does not is scoped to the file rather than to a method the file lacks.
|
||||
- _Verified 2026-09-17:_ 1,407 tests passed, nine skipped; Ruff/format/mypy passed. A first attempt changed the mapper to stop naming cross-file handlers and was reverted: an existing control proved the name carries the authorization attribution, so the fix belongs in region construction, not in the mapper. Controls assert the registering file asks about itself with no function-scoped region, while the defining module keeps its function-scoped region.
|
||||
- _Boundary: Ranker Scope Contract, contributor-scan entry mapping_
|
||||
- _Depends: 12.3_
|
||||
- _Requirements: 2.1, 2.2, 8.1_
|
||||
|
||||
|
||||
- [ ] 13. Show the shared design transfers across languages (release milestone M3)
|
||||
- Status 2026-09-17: Python established, PHP blocked by runtime on a large repository. M3 is not met. Its exit asks for Node, PHP and Python together, and two of the three are in hand.
|
||||
- [x] 13.1 Replay the declared Python family through the shared core
|
||||
- Generalize the replay instrument so a case family is an argument rather than a hard-coded target: a harness that only knows one language cannot say whether the design transfers. Targets, pins and expectations come from the committed inputs.
|
||||
- VAmPI is a regression snapshot with the same revision on both sides, so it can establish detection and completion but never novelty. The mode asserts that too: with nothing changed, a target reported new would be a comparison defect rather than a finding.
|
||||
- _Verified 2026-09-17:_ all three declared targets detected at their declared lines with their questions answered, 35 of 35 questions complete on every case, nothing reported new, zero admitted defects. The SQL injection at `models/user_model.py:73` is the evidence the milestone names as previously unresolved. Two of the three are access control, the family that could not complete a question anywhere before group 12, so this is the first time broken object-level authorization has been asked on Python. Evidence: `benchmarks/measurements/2026-09-17-m3-python-transfer.json`.
|
||||
- _Limits:_ identical revisions establish detection, not novelty; VAmPI is inspected teaching data and cannot qualify production Python support.
|
||||
- _Boundary: Evaluation harness, full-run integration_
|
||||
- _Requirements: 6.2, 6.3, 8.1, 8.2_
|
||||
- [ ] 13.2 Replay the declared PHP pair through the shared core
|
||||
- PMPro 2.9.7 against 2.9.8 with its upstream prepare fix, plus a comment-only benign variant. A different language, a different mechanism and a fix this project did not author.
|
||||
- The first attempt selected one case of three, because the manifest labels a case by what it is while the case id says what the comparison does; the harness now maps label to role explicitly. That run also exhausted a 900-second budget on the single case it ran.
|
||||
- Done when all three comparisons complete with their declared target evidence recorded, or the budget they need is stated as a finding rather than reported as a pass.
|
||||
- _Measured 2026-09-17, PHP transfer NOT established._ All three comparisons exhausted a 3000-second deadline with a 1200-second query ceiling and produced no artifact, so there is no target evidence at all. The engine reports `cpg batch taint failed` twice per case on the 1,274-file PMPro tree: first an exception whose trace runs through ujson array transformation, then a timeout on the retry. Budgets tried: 900 s, 1200 s and 3000 s deadlines against 300 s and 1200 s query ceilings. Pins, source receipts and the declared target are all correct, so the question was asked properly and the engine could not answer it. This is a runtime blocker on a large repository, not evidence about whether the design transfers to PHP. Evidence: `benchmarks/measurements/2026-09-17-m3-php-transfer.json`.
|
||||
- _Two harness defects found and fixed on the way, each of which had made an earlier run look like a transfer failure:_ the manifest labels a case by what it is while the id says what the comparison does, and filtering on one while keying expectations by the other ran a single case; and the per-query ceiling is taken as the minimum with the remaining transaction time, so raising only the deadline could never have helped.
|
||||
- _Retried 2026-09-18 with batch splitting, still not established._ A batch that fails whole is now retried in eight parts, which a probe made look promising: the evidence pass answers all 4,425 PMPro regions in one batch in 29.7 s with a 681 KB response, and cost is fixed rather than per request, so neither count nor size is near a limit. That probe sent `evidenceOnly=true` and therefore measured the structural pass, not the failing one; recorded because those numbers would otherwise read as evidence that PMPro's taint work is cheap. With a 4500-second deadline and a 900-second query ceiling the run was stopped after 26 minutes with no case completed. Splitting also makes an unrecoverable batch slower to abandon, since each part burns its own share of the ceiling.
|
||||
- _Correction 2026-09-20: the malformed-CPG bug is fixed and was never what these runs were hitting._ joernio/joern#6281, this project's own report, is a PHP `global` inside a closure producing a node with two AST parents, which made the dataflow overlay throw and cost the WHOLE graph. It was filed against 4.0.623 and fixed by #6270 in **v4.0.625**, which is the version the image has pinned since before any measurement here. Verified today on the real trigger: PMPro's `includes/fields.php`, the file the original bisection found, builds and applies the default overlays cleanly on 4.0.625 and on 4.0.631. So what remains under this task is a RUNTIME failure and must not be re-diagnosed as the graph bug.
|
||||
- _Quantified 2026-09-20, and it is a runtime failure._ A taint batch of 2,526 questions over the 673-file checkout does not complete in thirty minutes. Raising the query ceiling six-fold, from 300 to 1,800 seconds, changed nothing but how long the failure took: query time went from 363 s to 1,874 s and zero questions completed in both. The eight-way split retry did not rescue it, finishing its parts in about three minutes against a deadline with 3,300 seconds still on it, which is worth its own look. Joern 4.0.625 and 4.0.631 are indistinguishable here, 627.7 s against 619.1 s. The engine pin stays at 4.0.625: the newer release was built and measured and is equivalent on this subject, so there is no benefit to weigh against re-verifying every JavaScript and Python measurement. Evidence: `benchmarks/measurements/2026-09-20-php-runtime-after-graph-fix.json`.
|
||||
- _Corrected 2026-09-20: PHP is not silent, and this project's own instrument is why it looked silent._ The census harness ordered its verdict completion-first, so a family with eighteen findings and no completed question reported "asked but no question completed" — and only the 500-region runs, the ones that time out, had been looked at. Fixed: findings are checked first, because a question-outcome count is bookkeeping and cannot hide the thing being censused. NodeGoat and VAmPI were never affected, since they complete every question.
|
||||
- _What PHP actually detects on PMPro._ At 25 regions, 19 findings in seven minutes with no timeout; at 100 regions, 43 findings in twelve minutes with no timeout. Among them a true positive at `adminpages/discountcodes.php:749`: `sanitize_text_field($_REQUEST["s"])` reaching `$wpdb->get_results($sqlQuery, OBJECT)` through `$sqlQuery .= "WHERE code LIKE '%$s%' "` with no `prepare`. That is the class of CVE-2023-23488, which `php.toml`'s own comment names as the reason those WordPress sanitizers are excluded — the prediction and the measurement agree.
|
||||
- _And a measured precision cost beside it._ Output encoding reports eighteen findings at 25 regions and the three sampled are all properly escaped, two with `esc_attr` and one coerced with `intval`. Those escapers are deliberately absent from a sanitizer list that is flat across families, because none of them stops SQL injection; keeping the injection finding above costs this family its precision, and that is the documented tension's price.
|
||||
- _The scale limit, quantified._ Roughly one to two seconds per question. 25 and 100 regions complete cleanly; 500 regions asks 2,526 questions, needs more than the 1,800-second ceiling and is killed. Nothing pathological, and the retry's fast failure is still unexplained.
|
||||
- _What is NOT the problem, each measured rather than assumed:_ the malformed-CPG bug (fixed in 4.0.625, verified on the real trigger file); the dataflow overlay (42 s on the full 637-file graph); the engine version (4.0.625 and 4.0.631 indistinguishable); and the bundled Stripe, Braintree and Twocheckout SDKs under `includes/lib`, which the vendored list does not cover — that was the hypothesis and the overlay costs 42 s with them and 37 s without, so it is refuted.
|
||||
- **The remaining real gap is bookkeeping, not detection:** `questions_completed` is zero in every PHP run including the ones that find things, with `cross_partition_semantics_unresolved` and `partition_file_census_incomplete` present throughout. Findings and question outcomes disagree in public, and the admission policy reads the outcomes. Group 12 raised completion for JavaScript and Python and PHP was never measured. Evidence: `benchmarks/measurements/2026-09-20-php-detection-corrected.json`.
|
||||
- **IDENTIFIED 2026-09-22.** The cost is the number of CALL NODES a family's sink names match in the repository. Not the request count, not the declared sink count, not any request parameter. `reachableByFlows` is asked per matched sink, and PHP's output-encoding sinks are `echo` and `print`, which a WordPress plugin writes thousands of times.
|
||||
- _Three controlled sweeps on one graph, built once._ Batch size is exonerated: 400 requests answer in 170 s and cost per request FALLS with size, because roughly 70 s is fixed JVM start and graph load. Request parameters are exonerated: at 200 requests the full production shape, call depth 3 with parameter sources, costs 221 s against a 150 s baseline. The family is the variable: at the same 100 requests, SQL injection costs 147 s with six declared sinks, path costs 194 s with sixteen, and output encoding costs 704 s with seven.
|
||||
- _And the driver named its own expensive step, once it was asked to._ `taint: 2435 request(s) in 45.5s, answered` for the evidence pass, then `taint: 315 request(s) in 1200.1s, unanswered` for arbitration across all families, then a split abandoned for want of time.
|
||||
- _Why it survived since September:_ every reproduction outside the driver carried the SQL sinks, because that is the family the declared target belongs to. The one family that is cheap on this subject was the one being re-measured, and nine hypotheses were eliminated that way.
|
||||
- _Why splitting never rescued it:_ a failed batch is split into eight parts BY REQUEST COUNT, and each part carries the same everywhere-sinks, so the whole is no cheaper. Splitting by request count cannot help when the cost is per matched sink. Evidence: `benchmarks/measurements/2026-09-22-php-query-cost-identified.json`.
|
||||
- **Blocker identified; the fix is contributor-scan 7.7.** Six hypotheses were measured and eliminated: batch size (17,700 requests answer in 44 s with a 2.6 MB response), dataflow cost (50 execution requests in 24 s), call depth 3 with parameter sources (24.7 s for 20), argument length (a 16 KB hook table against a 128 KB cap), hook callback resolution (1 s difference with 233 hooks), and build or sharding (79 s, no shards). Preparation measures 273.5 s. Every reproduction of the failing 22,125-request batch outside the push path answers in under a minute, and the difference from the production invocation is not known. The exception has never been read, because the backend logged a stack trace's tail and the corrected extractor reports its head, which on this engine is JVM startup noise. Evidence: `benchmarks/measurements/2026-09-19-php-blocker-open.json`.
|
||||
- _Boundary: Evaluation harness, full-run integration_
|
||||
- _Depends: 13.1_
|
||||
- _Requirements: 6.2, 6.3, 8.1, 8.2_
|
||||
|
||||
- [ ] 14. Ask every declared family whether it establishes anything (precondition for M4)
|
||||
- Seven families are declared and two were proven end to end. A declared family that has never been asked whether it finds anything is advertised coverage, and M4 cannot qualify a capability that establishes nothing.
|
||||
- [x] 14.1 Census the seven declared families on NodeGoat and VAmPI
|
||||
- Run production discovery and scan over each whole checkout and report, per family, the regions that offered it, the questions asked, the questions completed, and what was found with the witness and rung. No comparison, no novelty, no admission: detection only.
|
||||
- A family that answered and found nothing must be reported differently from one that was never asked, and neither may read as clean.
|
||||
- _Verified 2026-09-19:_ 1,411 tests passed, ten skipped; the six Joern-gated query tests pass in the image. Four families establish findings across the two subjects: injection (three `eval` flows on NodeGoat, one SQL flow on VAmPI), access_control (two sibling contradictions on VAmPI), config_secrets (one on each) and untrusted_destination (one open redirect on NodeGoat). All 299 NodeGoat and 35 VAmPI questions completed. Evidence: `benchmarks/measurements/2026-09-19-family-detection-census.json`.
|
||||
- _The census found one false positive and it is fixed._ NodeGoat's only prototype report was the word `set` inside the comment `// set these up in case we have an error case`: the matched node is the `<operator>.assignment` holding the whole 1,000-character signup handler, and its text was searched entire. `sinkMatches` now confines its text clause to the callee and excludes operator nodes, which is the fourth instance of this bug class after `fgets`/`gets`, `resolve`/`resolveUrl` and `user`/`users`. Measured on NodeGoat's non-vendor calls: injection sinks 28 to 3, prototype 13 to 2, untrusted_destination 26 to 7, with no true positive lost. `req.query` had been an injection sink, because `query` is word-bounded inside it.
|
||||
- _Boundary: contributor-scan detection, Evaluation harness_
|
||||
- _Requirements: 6.2, 6.3, 8.1_
|
||||
- [x] 14.2 Give untrusted destination the sinks a Node server actually calls
|
||||
- **Named by the 14.1 census.** NodeGoat's documented SSRF is `needle.get(req.query.url + req.query.symbol)` at `app/routes/research.js:16`, and `needle.get` is absent from a vocabulary holding axios, fetch, got, `http.get` and request. The family works and found the open redirect in the same repository; it could not ask about the request.
|
||||
- This is a vocabulary gap, so fix it as data rather than as a query change, and state which client libraries the declared set now covers.
|
||||
- Done when the research handler's request is asked about, with a witness naming the call, and no benign client call is reported.
|
||||
- _Verified 2026-09-19:_ 1,411 tests passed, eleven skipped; seven Joern-gated query tests pass in the image; Ruff/format/mypy passed. One data change, no query change. The declared set now covers axios, got, needle, request, superagent, unirest, undici and fetch as bare module tokens, plus `http.get`, `http.request`, `https.get` and `https.request` dotted. Bare is the documented call form of each module and also covers its verb methods, because a token matches word-bounded against the callee and `axios` is bounded inside `axios.get`; `axios.get` and `axios.post` were therefore dropped as subsumed. `http` and `https` stay dotted because their namespace also holds `createServer`, and a bare token would have claimed every server a repository starts.
|
||||
- The research handler's request is now reported at `app/routes/research.js:16`, model_entailed, witness `req.query.url + req.query.symbol -> needle.get(url, ...)`. This family goes from one finding to two on NodeGoat and every other family is unchanged, with 299 of 299 questions still completing. Measured on that graph, `needle` matches exactly one call and the eleven other clients match none, in first-party or vendored code, so NodeGoat shows the gap closing and cannot show those clients' noise.
|
||||
- A control asserts both halves rather than only the recall half: a handler calling `axios.get` with a literal and `needle.get` with `req.query.url` reports the tainted client alone, and the control reads the shipped vocabulary rather than a copy so removing the client fails it. The CWE-601 redirect entry is untouched; a Location-header redirect is value-shaped and stays outside what this family asks. Evidence: `benchmarks/measurements/2026-09-19-untrusted-destination-vocabulary.json`.
|
||||
- _Boundary: shipped semantic facts_
|
||||
- _Depends: 14.1_
|
||||
- _Requirements: 6.2, 8.1_
|
||||
- [x] 14.3 Decide what output encoding claims for server-rendered JavaScript
|
||||
- **Named by the 14.1 census.** All six declared JavaScript sinks are DOM sinks. NodeGoat's XSS is server-side, rendered into Swig templates and written with `res.write`, so the 44 questions this family completed could not have found it whatever the code did.
|
||||
- Either the family gains server-side sinks and template-rendering sinks, or it declares itself client-side only. Both are honest; silence while declaring XSS coverage is not.
|
||||
- Done when one record states the decision, and the family either asks about `res.write` and template rendering or says in its declaration what it does not cover.
|
||||
- _Verified 2026-09-19:_ 1,415 tests passed, thirteen skipped; nine Joern-gated query tests pass in the image; Ruff/format/mypy passed. **Decision: the family asks about a value reaching a rendering call and states in its own declaration that it cannot decide template-mediated escaping.** Whether `res.render` is safe depends on the engine's autoescape setting and on the template's markup, neither of which is at the call, so `res.render` is not a sink and the family's `limits` now says so. The setting is claimed where its evidence is instead, as a configuration defect.
|
||||
- It now establishes a finding: `app/routes/research.js:25`, model_entailed, `req.query.url + req.query.symbol -> res.write(body)`. The handler fetches a URL the caller chooses and writes the response body into a text/html response, so the document served from this origin is the attacker's. The record states how coarse the step is: the path from `url` to `body` runs through the client's callback parameter, which the graph treats as reachable from the call's arguments, so the conclusion rests on the call's semantics rather than on what the graph established. Five of seven families now establish findings. Evidence: `benchmarks/measurements/2026-09-19-output-encoding-family.json`.
|
||||
- _Four defects found on the way, three of them false positives this change would otherwise have shipped._ The permissive-value set was shared across every setting while a boolean's polarity belongs to the setting, so `swig.setDefaults({autoescape: true})` would have been reported as the defect on the line that repairs it; `ConfigSpec` now carries per-setting values and Jinja2's `Environment` was moved out of Python's shared list for the same latent reason. `config.sc` matched a dotted setting on its trailing segment alone, so `marked.setOptions` would have claimed an editor's `setOptions`. `taint.sc` asked its flow query of every argument including the receiver, so `res.end()` — which takes no data — was reported exactly like the `res.write(body)` two lines above it. And the arbiter sorted a file's settings by line number as text, so which setting a file reported depended on the spelling of its line numbers.
|
||||
- _Boundary: shipped semantic facts, family taxonomy_
|
||||
- _Depends: 14.1_
|
||||
- _Requirements: 6.2, 8.1_
|
||||
- [x] 14.6 Report every permissive setting in a file, not the first one
|
||||
- **Named by the 14.3 measurement.** The configuration arbiter returns one verdict per question. NodeGoat's `server.js` holds a permissive session at line 78 and template auto-escaping disabled at line 135 in the same function, so the escaping defect is detected — measured model_entailed against the real query and the real arbiter — and never reported. A contributor is shown one of two defects and told nothing about the other.
|
||||
- This is the arbiter's contract, not a data gap: `verdict()` returns `Verdict | None` and the scan collects one per question. Changing it touches every family that returns a verdict, so it is scheduled rather than folded into a vocabulary change.
|
||||
- Done when a file with two permissive settings reports both, with their own sites and witnesses, and no family reports a defect twice.
|
||||
- _Verified 2026-09-20:_ 1,417 tests passed, seventeen skipped; thirteen Joern-gated query tests pass in the image; Ruff/format/mypy passed. Configuration now returns one verdict per setting whose value the model could read. The pipeline already emitted one finding per entailed answer, so only this arbiter was still folding them to one. NodeGoat's `server.js` reports both defects with their own sites, `server.js:78` for the permissive session and `server.js:135` for the disabled template auto-escaping, which is the defect task 14.3 could detect and never report. Its applied fix at line 126 stays silent. VAmPI is unchanged and no other family moved. Evidence: `benchmarks/measurements/2026-09-20-every-permissive-setting.json`.
|
||||
- _Three boundaries drawn rather than assumed._ A setting whose value is COMPUTED stays one verdict per region: that band is a question for the judge, and returning every undecided setting would multiply model calls for claims the graph did not establish. A call carrying two permissive literals is one finding, because `cors({origin: true, credentials: true})` is one misconfiguration. And config verdicts now carry a `path:line:function` location, without which the plural answer collapsed straight back to one in the deduplicator, which is the same gap dominance carried until its findings stopped arriving with no line.
|
||||
- _Dominance is left answering once per region, and that is settled rather than pending._ Its claim is that a region contradicts itself, so the region is the unit and a second verdict would restate the same inconsistency. The pipeline says so where it used to call this a known gap.
|
||||
- _Boundary: contributor-scan arbitration, Rung ladder_
|
||||
- _Depends: 14.3_
|
||||
- _Requirements: 6.3, 8.1_
|
||||
- [x] 14.4 Raise the access-control obligation where the application splits route from query
|
||||
- **Named by the 14.1 census and measured with the real dominance query.** Every modelled data operation in NodeGoat lives in `app/data/*-dao.js`; none lives in a route handler file. Operations are scoped to the region's own file and access_control regions are the handler files, so no obligation is raised where the question is asked, and inside the DAO files every operation is equally unguarded so the sibling rung has nothing to contradict. The family answers correctly on VAmPI, which keeps handler and query in one file.
|
||||
- The file scope was chosen deliberately, because unscoped rows attributed a model's unguarded query to a handler of the same name. Widening it is not the fix; carrying the obligation along the call the handler actually makes might be.
|
||||
- NodeGoat's two documented defects are also outside what the model can claim: an authenticated handler reading `userId` from `req.params` (`app/routes/allocations.js:17`), and a commented-out `isAdmin` in the route table (`app/routes/index.js:58`). Record which of the two this task is for, and do not silently widen the claim to the other.
|
||||
- Done when a route-and-DAO split raises the obligation at the handler with a witness naming both sites, and a same-name model function is still not attributed to a handler.
|
||||
- _Verified 2026-09-20:_ 1,415 tests passed, fifteen skipped; eleven Joern-gated query tests pass in the image; Ruff/format/mypy passed. **This task is for the first defect, the route-and-DAO split itself.** The obligation now follows the one named call the handler writes, out of the region's file and no further. Attribution is by the graph's own provenance, not by name: jssrc2cpg leaves the callee an external stub, and that stub's fullName records the file and member where the implementation was written, so a module the handler never imports is never consulted. A row is attributed to the region's DECLARED function, which is what lets a question about `displayAllocations` match a graph that holds a `<lambda>`. A carried row is also checked for a guard at the CALL, because that is where a handler establishes who is asking, and the method holding the carried operation joins the question's context scope so the query's own module cannot change unattributed.
|
||||
- Measured on NodeGoat: 15 handlers asked, obligations raised at zero before and at six now, three of which reach a verdict. All three are corroborated and none is reported without a judge. VAmPI is unchanged at two entailed findings, because it keeps handler and query in one file and carries nothing. Evidence: `benchmarks/measurements/2026-09-20-access-control-module-split.json`.
|
||||
- _A carried obligation corroborates, it does not entail, and that was measured rather than assumed._ Entailing it reported `handleSignup`, which looks a user up by name to check the name is free during registration and is public by design, because two authenticated siblings in the same file pass `req.session.userId` into the same module. The entailed rung's justification is that one FILE contradicts itself, same file and same trust context; carried, the compared operations live in a data-access module that handlers of different trust contexts share. The corroborated band already asks the right residual: does this operation require a check, or is the route legitimately public?
|
||||
- _What this does not reach, recorded rather than implied:_ NodeGoat's documented IDOR at `app/routes/allocations.js:17` still raises nothing, because its module reads with `allocationsCol.find(...)` and `find` is not a modelled operation (task 14.8). The commented-out `isAdmin` at `app/routes/index.js:58` is not claimable by this abstraction at all, since the evidence that `/benefits` needs a stronger guard than its siblings exists only in a comment.
|
||||
- _Boundary: contributor-scan dominance, Ranker Scope Contract_
|
||||
- _Depends: 14.1_
|
||||
- _Requirements: 6.2, 6.3, 8.1_
|
||||
- [x] 14.7 Decide the sibling scope, and see identity that a destructuring spelled away
|
||||
- **Named by the 14.4 measurement.** Two of NodeGoat's three corroborated claims are correct code: `const { userId } = req.session` followed by `dao.getByUserId(userId, ...)` is identity-constrained, and the destructuring erases the `session.userId` spelling the token list holds, so the guard is invisible at the call. A judge would be asked about handlers that are not defects.
|
||||
- The contradiction that would make NodeGoat's IDOR entailed is cross-file: `allocations` passes a URL parameter while `contributions` and `profile` pass session identity. The sibling set is one file, and widening it is a decision, not an implementation detail: measured, a route file mixes public with authenticated handlers, so application-wide siblings would contradict every login and signup handler there is.
|
||||
- Record the measured cost of each candidate before choosing, and do not widen the scope to make a finding appear.
|
||||
- Done when one record states the candidates and the choice, a destructured identity read discharges the obligation it actually discharges, and no public route is reported.
|
||||
- _Verified 2026-09-20:_ 1,415 tests passed, sixteen skipped; thirteen Joern-gated query tests pass in the image; Ruff/format/mypy passed. A destructured identity read now discharges what it reaches, through a bounded three-hop walk back over the method's own assignments from the names the operation is GIVEN. `req.session` joins the identity sources and is only safe because of that requirement: what discharges an object-level obligation is that the value PASSED to the operation comes from the authenticated context, not that a session is mentioned nearby. On NodeGoat `displayContributions` and `displayProfile` go silent, which is correct code no longer put to a judge, and VAmPI is unchanged.
|
||||
- _Two exclusions the measurement forced, each caught by a page whose whole purpose is the leak it demonstrates._ Taking identifiers from the whole argument subtree credited NodeGoat's memo listing with a discharge, because `memosDAO.getAllMemos((err, docs) => ... userId ...)` reads every memo and only RENDERS the session's user; a callback is not a constraint, so method references are excluded. Then the session read still dominated the query and the dominating-guard clause credited that instead; an assignment is a read of the authenticated context rather than a check applied to it, so operator calls are excluded from that clause, which is the same exclusion the sink matcher needed. The leak stays reported.
|
||||
- _The sibling scope is decided by measurement, and the choice is to keep the file._ On NodeGoat the region's own file entails two handlers, both public routes and both false, and no true defect at all. Every handler in the application entails six: four true, and the same two public routes. Both scopes carry the same false positives, because neither can tell a public route from a protected one — which is a semantic question and already the corroborated band's residual. With the carried rung capped at corroborated the scope changes no verdict, only the witness, so widening it would buy a repository-wide walk per question and no precision. What would change the decision is a discriminator for a public route; with one, the wider scope is worth four true entailments on this subject. Evidence: `benchmarks/measurements/2026-09-20-sibling-scope-and-identity.json`.
|
||||
- _Boundary: contributor-scan dominance, shipped obligation facts_
|
||||
- _Depends: 14.4_
|
||||
- _Requirements: 6.2, 6.3, 8.1_
|
||||
- [x] 14.8 Model the collection read an application actually writes
|
||||
- **Named by the 14.4 measurement.** NodeGoat's documented insecure direct object reference reads with `allocationsCol.find(searchCriteria())`, and `find` is absent from an operation vocabulary holding `findOne`, `findAll` and `findById`. It is the commonest read there is, so the family's obligation is missing at exactly the shape it exists for.
|
||||
- It is absent for a reason: `Array.prototype.find` is everywhere in JavaScript and jQuery's `.find` is everywhere in a vendored bundle. A collection read takes a query document where those take a closure or a selector, so the discriminator is the argument's shape rather than the name.
|
||||
- Done when a collection read raises the obligation, an array search and a selector do not, and the false-positive count is measured on a repository with vendored bundles rather than argued from the shape alone.
|
||||
- _Verified 2026-09-20:_ 1,415 tests passed, sixteen skipped; twelve Joern-gated query tests pass in the image; Ruff/format/mypy passed. The obligation is claimed where the first argument has the shape of a query document, which the operation fact states as `query_document_arg` and the arbiter decides: no arguments at all, an object literal, or a builder call. Not claimed: a method reference, a literal, an operator call, and a bare identifier.
|
||||
- _The false-positive count is measured, on wp-statistics, a WordPress plugin that ships jQuery._ Its assets hold 88 calls named `find` — 70 jQuery selectors, 13 array predicates and 5 selectors built at run time by a template literal or a concatenation, and no collection read at all. Zero became an obligated operation. The zero is a measured one and not an empty graph: the same run's census reports 60 files and 4,552 methods, and the files holding those 88 calls are in it.
|
||||
- On NodeGoat all three `find` calls are collection reads and all three are claimed, which raises the obligation at four more handlers including `app/routes/allocations.js:displayAllocations` — the documented insecure direct object reference, which had raised nothing at any point in the pipeline before. Seven of fifteen handlers now reach a verdict, all corroborated, none reported without a judge, and the census is unchanged on both subjects. Evidence: `benchmarks/measurements/2026-09-20-collection-read-obligation.json`.
|
||||
- _A bare identifier is deliberately not claimed._ `col.find(query)` and `arr.find(pred)` are the same shape and neither repository measured holds one, so this is a stated false negative rather than an unmeasured false-positive risk.
|
||||
- _Boundary: shipped obligation facts, contributor-scan dominance_
|
||||
- _Depends: 14.4_
|
||||
- _Requirements: 6.2, 8.1_
|
||||
- [x] 14.5 Ask the prototype and deserialization families somewhere they can answer
|
||||
- **Named by the 14.1 census.** Prototype pollution has been asked on one repository and its only report was a false positive; deserialization was never asked at all, because VAmPI holds no such sink and NodeGoat is not a language where the family is declared. Neither has established anything anywhere.
|
||||
- Find or record a subject where the family has a sink, and report what it does. A negative on a subject that contains the mechanism is a result; silence on a subject that does not contain it is not.
|
||||
- _Verified 2026-09-20:_ 1,417 tests passed, eighteen skipped; Ruff/format/mypy passed. **Neither family has a subject that can answer, and each for its own reason, which is the result rather than a failure to produce one.**
|
||||
- _Deserialization: the mechanism exists on exactly one subject and that subject cannot be queried._ Across four WordPress plugins and 1,164 PHP files there is one deserialization site, `maybe_unserialize` at `includes/fields.php:1111` in Paid Memberships Pro. The vocabulary held only the bare `unserialize`, which never reached it and correctly so: the matcher is word-bounded and the character before the token inside `maybe_unserialize` is a word character. That spelling is now declared, so the family has a sink to ask about — and the census asked 487 deserialization questions on that plugin and completed none, because no query over that repository completes at all. That is task 13.2's runtime failure, now quantified there, not a statement about this family.
|
||||
- _Prototype pollution: no available subject contains the mechanism._ The only first-party matches on the Node subject are `app.set("view engine", "html")` and its sibling, Express settings whose arguments are literals, and the other JavaScript available is minified browser bundles with no request source. So the family's ability to answer is established on a control instead: a merge of the request body into an object is reported, and the Express setting beside it is not. Declared and never demonstrated has become demonstrated on a control and never on a real subject, which is a different and honest claim.
|
||||
- _Boundary: Evaluation harness_
|
||||
- _Depends: 14.1_
|
||||
- _Requirements: 6.2, 8.1_
|
||||
|
||||
|
||||
- [ ] 13.3 Re-run the declared PHP transfer at a scale whose questions complete
|
||||
- **Named by the 2026-09-20 correction.** The not-established record came from two instrument defects, not from the analyser: the census verdict checked question completion before findings, and a 500-region budget asks 2,526 questions, which no ceiling this project sets can hold. Bounded to 25 and 100 regions, PMPro completes with no timeout and establishes a true-positive SQL injection at `adminpages/discountcodes.php:749`.
|
||||
- Replay the three declared comparisons with the corrected harness at a region budget that completes, and record the declared target's evidence or its absence. State the budget as a limit of the run, not as a property of the repository: a bounded population establishes transfer, never coverage.
|
||||
- Done when all three comparisons complete with their declared target evidence recorded, or the budget they need is stated as a finding.
|
||||
- _Attempted 2026-09-21, and the attempt named its own blocker._ At 25 regions the first case now runs to completion and writes a full artifact, in 1,706 s, where at 100 it had exhausted a 2,400 s deadline with nothing. But 25 regions of 4,461 is 0.6% of the population, and the declared target at `classes/class.memberorder.php:936` was not among them: zero findings at the target, because the question was never asked. A bounded population establishes transfer only if it CONTAINS the target, and this one cannot.
|
||||
- **So this task depends on contributor-scan 7.3, not on a larger budget.** The population that holds the target is affordable only if arbitration stops being paid for questions whose evidence says there is nothing to find. Raising the budget is the thing already measured to fail; making most of the budget unnecessary is the thing that has not been tried.
|
||||
- _Two further observations from that run, recorded because they cost nothing to keep._ The head scan cost 1,178 s against the base's 219 s, which is worth explaining before M2 designs around either number. And every coverage reason was `ambiguous_line_correspondence`, one per file — so in the PUSH path, unlike the census, correspondence is what stands between this subject and a completed question. The partition work removed one blocker and revealed the next.
|
||||
- _Sized and attempted 2026-09-22._ One whole-repository scan at 500 regions with a 3,600 s deadline puts the declared target in scope and finds it: 271 of 406 questions completed, 213 findings, 2,950 s. The three-case replay at that budget then exhausted a 7,200 s case deadline with no artifact, and the failure is mine to own twice over: the container mounts `src` live and I edited the backend heavily during the run, including a version that could hang on a timed-out pipe; and the push's EVIDENCE pass is one unportioned 22,125-request batch protected only by the ceiling, which it exceeded under the CPU and memory contention my concurrent test runs created. Neither is the algorithm. The evidence pass is now portioned, streaming keeps what a killed portion finished, and the rerun uses a frozen export of the committed source with per-invocation logging so the record names its own expensive step.
|
||||
- _Attempted again 2026-09-23 on frozen `cda679c`, after contributor-scan 7.9, and it failed at a new wall._ Case 1 (introduce) spent its whole 7,200 s in the HEAD side's evidence pass and saved no artifact; cases 2 and 3 were stopped on the same path. The evidence portions took 13, 31 and 51 minutes each. A push asks evidence of the WHOLE population with change context, 22,125 requests, where the census asks about 2,400 without it.
|
||||
- _Diagnosed by probe, not by timestamp._ On one PMPro graph from the production request builder, context costs ~20% more engine time per request -- not the wall. The wall is OUTPUT: each request itemised every method its scope reached, up to 1,845 rows; 200 requests into the bundled Braintree SDK returned 52,831 rows, 9.4 MB, against 211 without context. `affected_context` only ever attaches a location in a CHANGED file and discards every other row after checking it, so nearly all of it was shipped to be thrown away. Fixed in `3c195c9`: the engine itemises the changed files and summarises the rest into one row carrying what the check reads. On PMPro the same 2,440 relationships and zero gaps are decided from both forms, 52,831 rows down to 3,241 with the most-referenced file as the change. Probe: `benchmarks/push/context_cost_probe.py`. The replay is re-running on frozen `3c195c9`.
|
||||
- _Corrected the same day: that was real but it was not the wall._ The re-run on `3c195c9` showed the same 13 → 31 → 51-minute portions. Sampled in flight, no engine was running for minutes at a time while the `pre-push` Python process sat at 83% CPU. The cause was the reuse cache: every publication listed and stat'ed the WHOLE cache to total its size, and a push publishes one answer per question, so the pass was quadratic. Measured on the live cache, one scan took 0.68 s at 12,238 entries, ~70 minutes of scanning by then and ~2.3 hours to finish the pass. The growing portion times were the quadratic, not the engine. Fixed in `7cf256b`: the scan is remembered, keyed by the cache root's modification stamp, so another writer still forces a rescan and the size limit stays exact. The probe did not catch it because it called the engine directly, without the cache -- the instrument lesson again: a probe that skips a layer cannot clear that layer. The output fix stays: it is still 16× less data per push. The replay is re-running on frozen `7cf256b`.
|
||||
- _The loop, named and broken, 2026-09-23._ Walls kept appearing one per two-hour run -- the cache rescan, then the harness losing a killed case's output (`982cb14`), then a linear-list relationship check that also overran the deadline by >60 s (`e7e2941`) -- because every one was Python bookkeeping between engine calls, every one grew with the repository, and the only full-scale instrument was a Joern replay. Two instruments replace that: an in-process sampling profiler (`OUSAST_SAMPLE_PROFILE`, on for every replay case) and `benchmarks/push/python_overhead.py`, which runs the real `replay()` on the real PMPro pins with an instant stub engine answering real recorded rows. Its first pass found the remaining two walls at once: discovery re-parsed the ruleset per region (89,127 TOML parses, 339 s → 7 s, output digest identical) and change attribution decoded every line anchor per row (78% of a stubbed push; now indexed). The whole stubbed PMPro push now costs 58.5 s of Python; per-entry cache fsync is gone too (`b4b9308`, `6b85a60`).
|
||||
- _The budget, stated correctly._ The push's head scan can select 2,500 questions (500 regions × 5 families) where change context leaves them incomplete, because tier-zero pruning never prunes an incomplete question -- ~6 hours of engine at ~9 s per PHP question. That does NOT make a 7,200 s case infeasible: contributor-scan 7.7–7.9 portion the arbitration phase in ranked order and hold time back for arbitration, so a case answers what fits and reports the rest as `query_budget_reserved`. What defeated every earlier case was the phases BEFORE portioning -- discovery, the cache, change attribution -- each spending the deadline before arbitration began. With those fixed (preparation 290 s → 19.5 s in the real run), the three cases re-run at 7,200 s on frozen `6b85a60`; completion is expected to be partial and stated, and the declared target's file ranks 19th of 86.
|
||||
- _Boundary: Evaluation harness, full-run integration_
|
||||
- _Depends: 13.2, contributor-scan 7.3, contributor-scan 7.8_
|
||||
- _Requirements: 6.2, 6.3, 8.1, 8.2_
|
||||
- [ ] 13.4 Let a question inside one partition complete, whatever else the repository holds
|
||||
- **Diagnosed 2026-09-21, and PHP was never the variable.** A repository holding TWO languages can never complete a question, so it can never admit a finding. `partitions.py` raises `cross_partition_semantics_unresolved` whenever more than one language partition exists; that reason is a member of `GRAPH_INTEGRITY_GAPS`; and a graph-integrity gap demotes every completed outcome in the scan.
|
||||
- The evidence is one column wide: NodeGoat is JavaScript alone and completes 299 of 299, VAmPI is Python alone and completes 35 of 35, PMPro is PHP with 36 regions of bundled JavaScript and completes 0 of 127 and 0 of 520 while establishing 19 and 43 findings. NodeGoat's own `vendor_semantics_unresolved` is a DECLARED exclusion rather than an integrity gap, which is why it does not demote.
|
||||
- **This is larger than the PHP runtime limit and it lands on the declared v0.1 target.** A WordPress plugin is PHP plus its admin JavaScript, so every one of them is multi-language, so on the target this tool was built for it can detect defects and never admit one. Neither subject that currently passes could reveal it.
|
||||
- Recording the boundary is right: a flow from PHP into JavaScript is semantics the partitioned graph cannot follow. Demoting every question is not, including one wholly inside a single PHP file. Task 12.3 already fixed this bug class one level down, making an empty-scope gap question-owned "so it cannot block another question's comparison"; the same principle applies to a partition boundary.
|
||||
- Do not reclassify the reason as a declared exclusion. That would claim cross-language flows are out of scope when they are unresolved, which is the opposite of what this project's degradation vocabulary exists to say.
|
||||
- Done when a question whose scope lies inside one partition completes, a question whose scope could cross a partition boundary stays unresolved with the boundary named, and no question completes by widening its scope. Evidence: `benchmarks/measurements/2026-09-21-two-languages-defeat-completion.json`.
|
||||
- _Both halves implemented 2026-09-21._ A language boundary is now a declared limit: it can hide a finding and never invent one, so it stays published and stops demoting. And an integrity gap raised by a partition census now carries the partition it came from and reaches only that partition's questions, because a missing JavaScript file cannot make a PHP answer wrong. Two controls exercise the real boundaries rather than injecting a reason, and both were confirmed to FAIL with the classification restored. The rule behind both is now written in the design as an invariant, with all five instances listed, because this is the fifth time it has been found and fixed one instance at a time.
|
||||
- _Verified on the subject 2026-09-21:_ PMPro completes **120 of 127** questions where it completed none, with its 19 findings unchanged. The seven still unresolved are the JavaScript partition's own questions, which is correct: the four missing files are its files, so its answers are the ones that could be wrong. 1,419 tests pass, eighteen skipped; Ruff/format/mypy pass.
|
||||
- _Three causes, not one, and the third was hidden by the record rather than by the code._ The same census gap is raised in TWO passes, arbitration and evidence, and carrying the partition in one but not the other left an unscoped copy that kept demoting everything, so the fix looked applied and was not. It was found by making the census print each outcome's reason instead of only its status, after which three separate causes stopped looking identical.
|
||||
- _Admission is scoped too._ `_blocking_degradations` already exempted declared exclusions; it now also ignores a gap another language partition raised, so a JavaScript build artefact cannot block a PHP finding. A gap naming no partition still reaches everything, and so does every gap when the claim's own language is unknown, because unknown scope is not evidence of independence. Evidence: `benchmarks/measurements/2026-09-21-partition-gap-blast-radius.json`.
|
||||
- _Boundary: contributor-scan query/context evidence, Ranker Scope Contract_
|
||||
- _Depends: 13.3_
|
||||
- _Requirements: 2.1, 2.2, 6.3, 8.1_
|
||||
|
||||
- [ ] 15. Prepare the independent population and declare the shippable scope (M4 preparation)
|
||||
- Runs in parallel with M2 and M3. M4 already permits population preparation before the frozen candidate exists, and it is the longest lead-time item on the path: nothing can be qualified without it however good the analyser gets.
|
||||
- [ ] 15.1 Reserve and review an untouched evaluation population
|
||||
- Freeze reviewed vulnerable, fixed and benign changes from projects not used to develop or tune this analyser. NodeGoat, VAmPI, PMPro, WP Statistics, MW WP Form and libpng are all development subjects and none of them may qualify anything.
|
||||
- Reserve the cases untouched: do not examine their scan results before the population is frozen, and record the reservation so a later run cannot be mistaken for a blind one.
|
||||
- **The population MUST hold a multi-language repository.** Task 13.4 found that a second language in a repository cost every question its completion, and so cost every finding its admission — on the declared v0.1 target, which is PHP plus admin JavaScript. Neither subject that passes today could reveal it, because both hold exactly one language. A population of single-language repositories would qualify a capability that fails on the product's own target.
|
||||
- Done when a frozen population exists with its provenance and review recorded, at least one member holds more than one language, and nothing in it has been scanned.
|
||||
- _Reserved 2026-09-24, not yet frozen._ Eleven cases selected by the maintainer from fifteen researched by advisory and fix diff alone (no clone, no scan), none among the 262 repositories this project's benchmarks reference, plus the earlier Ghost reservation: 4 PHP with JavaScript (Ultimate Member and LearnPress WordPress plugins, phpMyFAQ, YesWiki; all SQL injection), 4 Node/TypeScript (Signal K SSRF, Saltcorn SQLi, mongo-express code injection, Mockoon permissive CORS), 3 Python (alerta SQLi, wger open redirect, OpenCVE SSRF). Record: `benchmarks/independent/population-v1.toml`; research and rejected alternates: `research-2026-09-24.md`. A guard test fails if any reserved repository is referenced elsewhere in the tree. Remaining to FREEZE: per case, verify the pins and the sink by reading source, and choose a benign change from the same history without scanning; the OpenCVE licence must be checked before any result is published.
|
||||
- _Frozen 2026-09-24._ `benchmarks/independent/freeze.py` verified every case with git and file reads only -- no analyzer, query or tooling: both pins present and ordered, the sink text present at the vulnerable pin (LearnPress's fix is source-side and recorded as `fix_site`; Mockoon's sink is recorded as `sink_evidence`), the fix touching the sink or its declared fix site, and SHA-256 of the sink file at both pins. Benign controls were proposed mechanically and REVIEWED by diff: the keyword filter alone let through an access-restriction, a filter check, API-token scopes, a login change and a TLS-verification option, all rejected; Ultimate Member's and alerta's controls were replaced by hand (a coding-standards reformat; a dependency-plumbing change with both sides still vulnerable). Record: `freeze-v1.json`. The guard test now requires a benign pair for every case of a frozen population. Nothing in it has been scanned; the first scan is the evaluation.
|
||||
- _Boundary: Evaluation harness, corpus provenance_
|
||||
- _Requirements: 8.1, 8.2_
|
||||
- [ ] 15.2 Declare the v1 advisory scope as the families whose claims can be admitted
|
||||
- **Admission requires the entailed rung.** Injection, untrusted destination and configuration entail with witnesses on the measured subjects. Access control does not: on an application that splits its route from its data access the obligation is carried, and a carried obligation corroborates by design, so shipping the family would ship silence. Output encoding, path, prototype and deserialization have not earned a place.
|
||||
- Record the scope, the reason each excluded family is excluded, and what a contributor is therefore NOT told. M4 does not require every language or family to become eligible, and a published limitation is the honest form of a narrow release.
|
||||
- Done when one record states the declared scope and every exclusion's reason, and the packaged advisory output names the limits it ships with.
|
||||
- _Boundary: pre-push-safety-net admission, release validation_
|
||||
- _Depends: 15.1_
|
||||
- _Requirements: 3.1, 5.1, 5.6, 7.1_
|
||||
|
||||
|
||||
- [ ] 16. Make a harness report the cause of its own failure (robustness)
|
||||
- **Named by a week in which three separate causes looked identical.** The census reported 19 findings and 0 completed questions side by side for days without complaint; a stale image measured the engine a bug had been diagnosed against; and a partition fix looked applied while an unscoped copy of the same gap kept demoting everything. Each was found by reading code, and each would have been visible in a record that carried its own causes.
|
||||
- Every harness result should carry the counts that disambiguate its failure modes, the degradations whole rather than by reason, and the engine and ruleset identities actually used. Two of these were added reactively during that week and should be the standard shape rather than a reaction.
|
||||
- Add a self-consistency assertion to each harness. Findings above zero with completion at zero is a contradiction, and an instrument that says so loudly is the difference between an hour and a week.
|
||||
- Done when each harness record carries its provenance and per-cause counts, and a deliberately contradictory result is reported as a contradiction rather than as a number.
|
||||
- _Boundary: Evaluation harness_
|
||||
- _Requirements: 8.1, 8.2_
|
||||
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Task5.2 measured PMPro timing out after both snapshots were materialized but before change context/graph analysis. Group6 must account for compatible snapshot/discovery reuse as well as graph reuse; identical-tip hits cannot substitute for the later representative changed-code latency gate. No new detector, scope heuristic or incremental CPG mutation is implied.
|
||||
- Cancellation must cover semantic planning and reporting, not only engine subprocesses: `_collect` and `_scope_work` now consume the shared deadline, and replay instantiates the budget-aware backend once. Preserve the 400-question failure-census and real Node artifact-retention regressions.
|
||||
|
||||
- Group 9 (M1a) corrected the owner diagnosis: all four named vetoes fire on the NodeGoat case, and a fifth, `operation_correspondence_unresolved`, is the actual remaining blocker because a newly introduced operation sits on a changed line and has no base anchor. The unchanged revision resolves its base location on the same lines and is classified `unchanged`, which isolates correspondence as the mechanism. M1b repairs context producers, question-owned vendor relevance, operation correspondence, file-wide ambiguity scope and query-to-operation provenance; vetoes that did not fire stay unchanged.
|
||||
|
||||
- Group 6: complete discovery/graph/query reuse is explicit via `--cache-dir`; source/declaration edits invalidate affected units. Installed frontend/adaptation bytes now bind graph and admission provenance. A valid cached census cannot qualify a fresh answer that omitted its own census. Identical-revision timing is not the changed-code rollout gate.
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** complete, 22 of 22 tasks (spec.json phase corrected to `implementation-complete`).
|
||||
|
||||
## Introduction
|
||||
|
||||
OpenUltraSAST users scan **working trees they are writing**: mixed files, incomplete edits, generated code, framework wrappers, and no labels. Today the harness still *decides* with pattern inventory. The complexity map ranks; the sandbox can run; neither adjudicates “this sink is fed by a source” versus “this sink is hardcoded, sanitized, or unreadable.”
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
{
|
||||
"feature_name": "propose-adjudicate-prove",
|
||||
"created_at": "2026-09-04T10:58:28Z",
|
||||
"updated_at": "2026-09-04T12:00:00Z",
|
||||
"updated_at": "2026-10-02T00:00:00Z",
|
||||
"language": "en",
|
||||
"phase": "implementation",
|
||||
"phase": "implementation-complete",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": true,
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** dormant. 23 tasks generated, nothing approved; adopting Joern (`cpg/`, `model/taint.py`) delivered most of what it asked for. Reopen it only for a reachability gap Joern's taint does not cover.
|
||||
|
||||
## Introduction
|
||||
|
||||
OpenUltraSAST cannot yet steer effort where risk is reduced because it has no model of how an entry point reaches a sink. Reachability today means "inside a handler," taint is one hop inside one file, the LLM hunter cannot create a sandbox candidate, and the improve loop cannot add the facts that would raise recall. This spec adds a call graph and composed taint summaries over the existing `FunctionIR` records, attaches path-based reachability to findings and ranking, puts the hunter on paths with one checkable question per gap, turns model answers into gated facts, and ranks fix points by reachable risk removed. Concolic execution is out of scope; the sandbox remains the only place target code runs.
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** superseded by `pair-corpus-honesty` (spec.json `superseded_by`); its 12 tasks were done and the vfc pairs live on in that spec's corpus. Kept as decision history; nothing in src, tests or CI points here.
|
||||
|
||||
## Introduction
|
||||
|
||||
AppSec engineers cannot tell whether OpenUltraSAST improved on real product code. The labeled pair slices today are in-tree fixtures (`local`), small GitHub/SVEN functions (`github`), and textbook OWASP/Juliet files (`sast`). Operators care whether the harness fires on a real buggy snapshot of OpenSSL, Firefox, or Chromium and stays quiet after the patch. This spec adds a reviewed, isolated-function **vfc** slice drawn from public vulnerability-fixing commits in those projects, scored as an honesty dashboard (not a merge gate), so the tool can be evolved against labeled product code instead of cheat-sheet recall.
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
# Brief: three-stage-scan
|
||||
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
|
||||
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
|
||||
|
||||
## Problem
|
||||
|
||||
AppSec engineers and DevOps teams running OpenUltraSAST in CI get a regex static scan scored against cheat-sheet fixtures. They cannot tell what actually exists in a real repository, where complexity concentrates, how to aim tests, or whether a finding is worth a CI failure. The previous HarnessX spec added governance around those regexes (CWE policy, score, shadow status) without making the hunter evolutionary or the evaluation realistic.
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
# Design Document
|
||||
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
|
||||
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
|
||||
|
||||
## Overview
|
||||
|
||||
**Purpose.** This feature turns an OpenUltraSAST scan into three operator questions: what exists (static inventory), where problems are most likely and how to aim tests (complexity map), and whether a finding is worth fixing (isolated regression). It is a follow-up to the HarnessX governance spec, not a replacement of CWE policy, scoring, or rules-as-data.
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** retired (see the note below); all 31 tasks were done before HarnessX was removed. Kept as decision history; nothing in src, tests or CI points here.
|
||||
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax
|
||||
> (`ai-service-plane`), and evolutionary proposals come from the plane memory. History below is unchanged.
|
||||
|
||||
## Introduction
|
||||
|
||||
OpenUltraSAST today answers one question well: which regexes fired. It does not answer the three questions a rugged DevOps CI workflow actually asks: what exists in this repository, where problems are most likely and how to aim tests, and whether a finding is worth fixing. This feature turns the scan into those three stages. Stage 1 is a static inventory of what exists. Stage 2 is a complexity map of likely problem areas plus test-tuning hints. Stage 3 is isolated regression that proves triggerability before CI treats a finding as blocking.
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Requirements Document
|
||||
|
||||
> **Status 2026-10-02:** complete, 13 of 13 tasks (spec.json phase corrected from `tasks-generated` to `implementation-complete`).
|
||||
|
||||
## Introduction
|
||||
|
||||
AppSec engineers scanning mixed working trees need overlay adjudication on Python, JavaScript, C/C++, and Java, not only on files a Python stdlib parse can read. Today a `standard` or `deep` scan leaves JS/C/Java inventory as `language_unsupported` even when a host parser CLI is present, so those proposals never promote or demote. This spec makes an **optional semantic extra** the way to turn on a real structured parse for those languages, while `quick` stays usable with no extra packages and a missing extra stays fail-closed.
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
{
|
||||
"feature_name": "tree-sitter-overlay-extra",
|
||||
"created_at": "2026-09-04T14:00:00Z",
|
||||
"updated_at": "2026-09-04T15:00:00Z",
|
||||
"updated_at": "2026-10-02T00:00:00Z",
|
||||
"language": "en",
|
||||
"phase": "tasks-generated",
|
||||
"phase": "implementation-complete",
|
||||
"approvals": {
|
||||
"requirements": {
|
||||
"generated": true,
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Brief: zero-with-a-reason — a finding count is meaningless without a denominator
|
||||
|
||||
> **Status 2026-10-02:** dormant. Brief only and unapproved; not touched since 2026-09-10.
|
||||
|
||||
**Status:** brief only, unapproved.
|
||||
|
||||
## Why now
|
||||
|
||||
@@ -1,6 +1,37 @@
|
||||
# Overview — what this is, and what is actually true of it
|
||||
|
||||
**Current direction, 2026-09-12:** an actionable pre-push security safety net for AI-accelerated
|
||||
## Current state, 2026-10-02
|
||||
|
||||
Everything below this section is dated history; where it disagrees with this section, this section wins.
|
||||
User-facing documentation: `README.md`, `docs/architecture.md`, `docs/evaluation.md`, `docs/decision-engine.md`,
|
||||
`ops/ax/README.md`.
|
||||
|
||||
- **Scanning.** `ousast scan --mode quick|standard|deep`. Quick rules for C/C++, Python, JavaScript/TypeScript,
|
||||
Java/Groovy and PHP (PHP numbers are in-sample only: `benchmarks/measurements/2026-09-30-php-quick-rules/`).
|
||||
Standard adds the MAP stage with the Joern model layer (shipped in the Docker image); deep adds sandboxed
|
||||
regression snippets when Docker is available. Framework knowledge is tagged as optional priors
|
||||
(`ruleset/frameworks.toml`, `priors=`). `ousast pre-push` compares head against base; the capability registry
|
||||
is empty, rollout NO-GO.
|
||||
- **Agentic plane.** google/ax over Agent Substrate on kind is the only executor of model work over whole
|
||||
repositories (`ousast plane run|status|doctor|workspaces|remember|alerts-engine|harvest`). Credentials travel
|
||||
only in the start request through `atenet-router`; egress is per task; budgets are per task; rows go to the
|
||||
memory store (`OUSAST_MEMORY`: FileStore, or S3Store on an S3-compatible bucket; RustFS is the tested server).
|
||||
- **Removed.** HarnessX (retired 2026-09-30): its config keys fail, a `[harnessx]` section warns. The LLM judge
|
||||
of the verifier and the LLM fusion panels are gone; fusion is deterministic.
|
||||
- **Improvement.** `ousast improve --memory` proposes rule-status edits from plane memory through the unchanged
|
||||
validator and gate; the loop also runs as a plane Run.
|
||||
- **Decision engine** (`learned-decision-engine`, implementation, not adopted). First slice, injection only:
|
||||
ADVISORY recall 0.90, precision 0.54, BLOCK not offered
|
||||
(`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`).
|
||||
- **Evaluation.** Populations v1 and v2 are spent and failed every M4 gate (`benchmarks/independent/results-v1.json`,
|
||||
`results-v2.json`); v3 (PHP) is frozen and untouched for the one-time final check. The plane increment met its
|
||||
gate on the second measurement with a tie-break chosen after the first (`benchmarks/independent/plane-increment-2.json`).
|
||||
- **Providers.** DeepSeek for LLM calls, OpenRouter only for OpenAI embeddings; keys in `.env`, which never
|
||||
overrides an exported shell variable.
|
||||
|
||||
## History
|
||||
|
||||
**Direction, 2026-09-12:** an actionable pre-push security safety net for AI-accelerated
|
||||
development across supported ecosystems. See `safety-net.md` and the approved `pre-push-safety-net`
|
||||
spec. The ranker remains the scope mechanism; third-party vendor code stays outside targets and
|
||||
graphs. Advisory must meet the same actionability bar as blocking.
|
||||
@@ -163,15 +194,26 @@ function-level pairs could not see any of them, and a repository showed all nine
|
||||
|
||||
## Where the specs stand
|
||||
|
||||
Counted 2026-10-02 from each spec's `tasks.md` checkboxes and `spec.json`.
|
||||
|
||||
| spec | phase | state |
|
||||
|---|---|---|
|
||||
| `contributor-scan` | implementation | 14 done, 15 open — **the active one** |
|
||||
| `learned-decision-engine` | implementation | 11 of 17 tasks done |
|
||||
| `pre-push-safety-net` | implementation | 67 of 79 tasks done |
|
||||
| `contributor-scan` | implementation | 27 of 51 tasks done |
|
||||
| `ai-service-plane` | implementation | 11 of 11 tasks done; the ax plane is the agentic path |
|
||||
| `harnessx-removal` | complete | 8 of 8; HarnessX removed 2026-09-30 |
|
||||
| `finding-feedback-loop` | design | requirements approved, design awaiting approval |
|
||||
| `model-grounded-detection` | complete | 16/16; built the arbiter this all rests on |
|
||||
| `authorization-obligations` | tasks approved | 24 open — superseded in practice by the dominance arbiter |
|
||||
| `constrained-detector` | tasks | 4 of 20, tasks unapproved |
|
||||
| `reachability-flow-model` | tasks generated | 23 open, nothing approved — largely delivered by adopting Joern |
|
||||
| `learning-harness` | implementation | 30 done, 3 open |
|
||||
| six others | complete | `pair-corpus-honesty`, `corpus-seeded-mechanisms`, `propose-adjudicate-prove`, `three-stage-scan`, `real-world-vfc-slice`, `harnessx-self-improving-rulesets`, `tree-sitter-overlay-extra` |
|
||||
| `model-grounded-detection` | complete | 16 of 16; built the arbiter this all rests on |
|
||||
| `authorization-obligations` | dormant | tasks approved, 0 of 24 started; superseded in practice by the dominance arbiter |
|
||||
| `reachability-flow-model` | dormant | 23 tasks generated, nothing approved; largely delivered by adopting Joern |
|
||||
| `flow-aware-ranking`, `zero-with-a-reason`, `ir-consolidation` | dormant | brief only, unapproved |
|
||||
| four others | complete | `pair-corpus-honesty`, `corpus-seeded-mechanisms`, `propose-adjudicate-prove`, `tree-sitter-overlay-extra` |
|
||||
| six others | retired or superseded | `openrouter-sast-harness`, `three-stage-scan`, `harnessx-self-improving-rulesets`, `learning-harness` (retired with HarnessX), `constrained-detector`, `real-world-vfc-slice` (superseded) |
|
||||
|
||||
97 source modules, 90 load-bearing and 7 standalone capabilities; 814 tests.
|
||||
> **Retired 2026-09-30:** HarnessX was removed by `harnessx-removal`; the agentic plane is ax (`ai-service-plane`),
|
||||
> and evolutionary proposals come from the plane memory. The HarnessX specs in the table above are history.
|
||||
|
||||
As of 2026-09-09: 97 source modules, 90 load-bearing and 7 standalone capabilities; 814 tests.
|
||||
150 source modules, 145 load-bearing and 5 standalone capabilities, 0 orphaned
|
||||
(`benchmarks/measurements/2026-09-08-module-audit.json`, regenerated 2026-10-02); 1,964 tests collected.
|
||||
|
||||
+72
-67
@@ -1,91 +1,96 @@
|
||||
# Roadmap
|
||||
|
||||
## Current direction and next review — 2026-09-12
|
||||
## Status, 2026-10-02
|
||||
|
||||
The product is an **actionable pre-push security safety net for AI-accelerated development**, across
|
||||
supported languages/frameworks. WordPress is one regression workload. Advisory must be useful and
|
||||
low noise; moving speculative SAST output into a nonblocking hook does not meet the goal. See
|
||||
`safety-net.md` for current steering. The earlier narrative below is retained as dated history and
|
||||
does not override this direction or the latest measurements.
|
||||
The milestone table below dates from 2026-09-14. Since then (`.kiro/specs/pre-push-safety-net/release-milestones.md`):
|
||||
M1a and M1b are met, M2 is measured NO-GO, and M3 is next, with M4 population preparation in parallel; none of
|
||||
M1-M5 is verified and rollout remains NO-GO. Populations v1 and v2 were evaluated and failed every M4 gate
|
||||
(`benchmarks/independent/results-v1.json`, `results-v2.json`); v3 (PHP) is frozen for the one-time final check.
|
||||
|
||||
### Approach decision
|
||||
Work since then moved to: the ax plane as the only agentic executor (`ai-service-plane`, HarnessX retired
|
||||
2026-09-30 by `harnessx-removal`), and the learned decision engine (`learned-decision-engine`, task 6 in
|
||||
progress, not adopted). See `overview.md` for the current-state summary.
|
||||
|
||||
Keep the existing ranker as the scope mechanism. Supply it with pushed-change and affected
|
||||
first-party context. Third-party vendor code remains outside both targets and graphs. Reuse
|
||||
compatible graph/results artifacts and apply a total deadline; treat unknown external semantics
|
||||
as a coverage boundary. Require change attribution, witness and repair direction before a finding
|
||||
earns normal hook visibility. A second heuristic scope selector and an assumed incremental Joern
|
||||
engine are out of scope.
|
||||
## Release path as accepted 2026-09-14
|
||||
|
||||
### Current evidence and limitations
|
||||
Build an actionable pre-push security safety net for AI-accelerated development.
|
||||
The maintainer accepted the following order: **prove one useful detection, make it
|
||||
fast, prove cross-language transfer, qualify independently, then enable narrow advisory
|
||||
rollout**. The executable outcome contract lives in the existing
|
||||
[pre-push-safety-net release milestone spec](../specs/pre-push-safety-net/release-milestones.md).
|
||||
|
||||
- Whole-PMPro analysis has completed with its known CVE; the earlier "does not finish" diagnosis
|
||||
is superseded. The recorded 72-minute scan is not a pre-push performance result.
|
||||
- Historical Joern 4.0.625 + layout position budgets are PMPro 82, WP Statistics 91, MW WP Form 54.
|
||||
`<50` remains unmet. VAmPI's old 16 omitted SQLI; the corrected instrument retains the unmatched
|
||||
target and reports no full-target budget or detection recall without the required evidence.
|
||||
- Equal-score tie-breaking alone cannot bring PMPro or WP Statistics under 50. The existing
|
||||
ranker is useful, but its measured ordering is not enabled by the ordinary CLI caller.
|
||||
- Docker/Compose/ops now default to 4.0.625; task 1.1 passed local PHP/JavaScript image smoke and independent review. No host deployment or hook latency claim follows from that runtime evidence.
|
||||
- `contributor-scan` has 25/41 checked tasks, including the approved and verified frontend-retention prerequisite 2.18.
|
||||
`flow-aware-ranking` is still an unapproved brief despite implementation history.
|
||||
**We are at M1b. M1a is met; no other release milestone is verified yet.** The published
|
||||
`v1.2.0-alpha.1` and 28/29 completed implementation tasks describe delivered machinery,
|
||||
not a demonstrated useful safety net. Task 8.3 and rollout remain NO-GO.
|
||||
|
||||
| Order | Milestone | Observable exit | Current state |
|
||||
|---|---|---|---|
|
||||
| M1a | Show the NodeGoat detection with every admission veto recorded, not applied | Raw finding with witness on the vulnerable revision; analyzed fixed twin and unchanged control quiet; per-finding veto list. One-week time box | MET 2026-09-16 |
|
||||
| M1b | Repair only the vetoes M1a showed to matter | Supported evaluation finding through production rules, exact change/base evidence and repair direction; fixed twin and unchanged control stay quiet | IN PROGRESS: comparison half verified on all three revisions; the consequence and repair rendering (10.10) remains |
|
||||
| M2 | Make that same analysis fast | Equivalent evidence; representative warm changed-code p95 <=30s, cancellation/report <=2s, >=95% supported completion | Pending M1 |
|
||||
| M3 | Prove the shared design transfers | PMPro/PHP and VAmPI/Python vulnerable/fixed evidence; unchanged core rules; Node replay after any PMPro core change | Pending M2 |
|
||||
| M4 | Qualify on independent cases | Reviewed untouched vulnerable/fixed/benign populations pass joint quality, coverage and latency gates per capability | Pending M3 |
|
||||
| M5 | Enable a narrow advisory rollout | Exact passing eligibility and packaged acceptance; only qualified capabilities enabled | Pending M4 |
|
||||
|
||||
### Immediate next action
|
||||
|
||||
Finish M1b by rendering the experimental evaluation finding with its consequence and repair
|
||||
direction (10.10), through an explicitly unreviewed declaration and with eligibility still
|
||||
disabled. The comparison half is verified: the production rules classify the vulnerable, fixed
|
||||
and unchanged NodeGoat revisions correctly, and admission rejects the vulnerable one on the
|
||||
eligibility gate alone. What is missing is the explanation, which no declaration currently
|
||||
supports, and inventing it is exactly what admission exists to prevent. Then M2 runtime.
|
||||
Do not start another broad speed campaign before this outcome exists. See the
|
||||
[M1a evidence](../specs/pre-push-safety-net/m1a-recorded-vetoes.md) and the
|
||||
[owner diagnosis](../specs/contributor-scan/release-remediation-research.md).
|
||||
|
||||
### Approach and boundaries
|
||||
|
||||
The existing ranker owns scope. Vendor code stays out of targets and graphs. Framework
|
||||
facts and frontends supply evidence to a language-agnostic core. Unknown dependencies
|
||||
remain explicit; removing global completeness vetoes requires question-owned evidence,
|
||||
not dropped safety checks. Advisory and blocking share the same actionability bar.
|
||||
|
||||
Graph persistence now works: identical-tip replays take 18.5–18.9s. All 18 other measured
|
||||
transactions still time out and 0/24 target checks complete. A local Joern session census
|
||||
probe is promising, but full-query equivalence and hook performance are unverified.
|
||||
These findings inform M2; they do not complete M1. No second IR/engine, new heuristic
|
||||
scope selector, differential graph mutation or unrelated backlog is part of this path.
|
||||
|
||||
## Existing Spec Updates
|
||||
|
||||
- [ ] `contributor-scan` — review the draft driver/backend integration amendment; reconcile packaging
|
||||
pins, family-specific sanitizer task 5.6, exact scope reporting and repository baseline honesty.
|
||||
Preserve previously approved work; the new amendment is not implicitly approved.
|
||||
- [ ] `flow-aware-ranking` — review the amended experiment contract: ranker owns scope, vendor
|
||||
boundary preserved, missing targets stay unresolved, actual detection and cost accompany rank,
|
||||
untouched evaluation and exploration precede learned-ranking claims.
|
||||
- [ ] `zero-with-a-reason` — reconcile the broader funnel proposal with the hook's consumed coverage
|
||||
contract; it is adjacent work, not a requirement to finish every diagnostics mechanism first.
|
||||
- [ ] pre-push-safety-net — Execute the accepted M1–M5 release milestone amendment.
|
||||
Owns immutable replay, comparison, admission, evaluation and reporting. Preserve the
|
||||
existing completed tasks; task 8.3 remains blocked until qualification passes.
|
||||
- [ ] contributor-scan — Supply the required query/operation/context fixes for M1,
|
||||
followed by equivalent bounded session execution for M2 and shared transfer evidence.
|
||||
Frontend retention 2.18 and named census repair 2.19 are already verified.
|
||||
- [ ] flow-aware-ranking — Preserve ranker-owned scope and normalized cross-language
|
||||
behavior. New ranking research is not a prerequisite for the first useful comparison.
|
||||
- [ ] zero-with-a-reason — Adjacent diagnostics work; finishing its whole backlog is
|
||||
not required before M1.
|
||||
|
||||
## Specs (dependency order)
|
||||
|
||||
- [ ] pre-push-safety-net — Immutable pushed tips, existing-ranker integration, compatible artifact
|
||||
reuse, total deadline, change attribution, actionability admission and compact hook behavior.
|
||||
Dependencies: contributor-scan. Requirements/design approved 2026-09-12; tasks generated
|
||||
(8 groups, 27 executable subtasks), independently reviewed and approved; groups 1–7 are implemented and verified (23/27 tasks), including the resolved frontend-retention prerequisite.
|
||||
- [ ] pre-push-safety-net — Existing approved implementation, dependent on contributor-scan.
|
||||
Groups 1–7 and tasks 8.1, 8.2, 8.4 are verified; 26/27 executable tasks complete.
|
||||
Remaining release work follows [M1–M5](../specs/pre-push-safety-net/release-milestones.md).
|
||||
This amendment creates no duplicate implementation spec and resets no prior approvals.
|
||||
|
||||
### Next concrete step
|
||||
### Progress reporting
|
||||
|
||||
Groups 1–5 now connect immutable replay to the existing ranker, detector, comparison,
|
||||
admission and compact artifacts. The first frozen PMPro/Node experiment is **NO-GO**:
|
||||
all six cold security/fixed/benign cases timed out without completed target checks.
|
||||
After repairing post-deadline semantic planning and a repeated availability probe,
|
||||
every final CLI run saved its artifact and returned in 30.55–30.64 seconds. PMPro spent
|
||||
its deadline in preparation/discovery after materializing both trees; Node reached the
|
||||
driver but did not complete checks. The exact 11-case population remains unresolved.
|
||||
No normal capability is enabled and these six cold samples establish no warm p95 claim.
|
||||
|
||||
Group 6 now provides complete preparation/discovery, graph and query reuse through explicit
|
||||
`--cache-dir`, bounded publication and isolated graph leases. A packaged authored JavaScript
|
||||
control preserved the same evidence/admission as an uncached run; its identical-revision
|
||||
replay took 2.00s. Source/declaration changes invalidate affected units. This correctness
|
||||
control does not establish representative changed-code p95, independent alert precision or
|
||||
eligibility. Rollout remains NO-GO and normal capabilities remain disabled.
|
||||
|
||||
Group 7 now provides the real multi-ref push interface, opt-in hook integration and bounded
|
||||
optional selection among already-admitted witnesses. Group 8 is next and
|
||||
must still pass the joint gates on declared populations. Keep the existing thresholds and
|
||||
profile preparation, graph construction and queries separately when investigating misses;
|
||||
do not substitute identical-tip hits for warm changed-code performance.
|
||||
|
||||
Tasks 4.3 and groups 5–6 used the documented Kiro manual review fallback after the agent
|
||||
thread limit; this is not independent review. See `pre-push-safety-net/implementation-group-5.md`
|
||||
and `benchmarks/measurements/2026-09-13-first-replay-experiment.json` for provenance,
|
||||
initial instrument failures, their repairs and the newly frozen final trial. The NodeGoat
|
||||
teaching workload did not cause a framework/fact adaptation or ranking retune. Ghost remains
|
||||
untouched; independent PHP/Python eligibility populations and C/C++ property support remain
|
||||
separate gaps. Automatic dominance-only context projection is still an upstream limit.
|
||||
Lead with the current milestone, the demonstrated developer outcome, the blocker and
|
||||
the next action. Keep implementation counts separate from release readiness. Preserve
|
||||
prior evidence and review limits in the [implementation records](../specs/pre-push-safety-net/status.md).
|
||||
The earlier PHP-first and runtime-first narrative below is historical and does not
|
||||
supersede this accepted sequence. Blocking deployment follows a later explicit decision.
|
||||
|
||||
## Historical roadmap narrative — September 9–11
|
||||
|
||||
The sections below preserve prior plans and measurements. Statements about the current bottleneck,
|
||||
PHP coverage, rollout order or feature readiness are superseded by the current section above.
|
||||
|
||||
**Rewritten 2026-09-09.** The previous version (kept as `roadmap-2026-09-04.md`) described a pre-Joern world —
|
||||
**Rewritten 2026-09-09.** The previous version (kept as `.kiro/archive/steering/roadmap-2026-09-04.md`, outside steering) described a pre-Joern world —
|
||||
"adjudication is still Python-`ast` plus a tree-sitter CLI stub", "Joern-as-required deferred". Joern was
|
||||
adopted, the model layer was built, wired and measured, and two repositories have been scanned. See
|
||||
`overview.md` for what is true today; this is what happens next and in what order.
|
||||
|
||||
@@ -3,6 +3,19 @@
|
||||
Updated 2026-09-12 from the maintainer's explicit direction. This direction supersedes the
|
||||
WordPress-first release framing in older steering; historical measurements remain historical.
|
||||
|
||||
Release order accepted 2026-09-14: first prove one useful Node changed-code comparison
|
||||
with an analyzed quiet fixed twin, then make the same analysis fast, verify PHP/Python
|
||||
transfer, qualify independently, and enable only passing advisory capabilities. See
|
||||
[the milestone spec](../specs/pre-push-safety-net/release-milestones.md). M1a is met
|
||||
(2026-09-16: witness shown, fixed twin analyzed and quiet, five vetoes recorded), and the M1b
|
||||
comparison now classifies all three NodeGoat revisions through the production rules with no
|
||||
lifting. M1a added a fifth veto the earlier diagnosis missed, operation correspondence for an
|
||||
operation on a changed line, and M1b added a prerequisite it did not anticipate: absence must
|
||||
be provable from an enumerated operation inventory, not from the absence of a traced flow.
|
||||
Task counts and an experimental alpha tag do not establish release readiness. PMPro
|
||||
remains a core regression/development case; any PMPro core retune still requires frozen
|
||||
Node transfer evidence. This sequence supersedes older runtime-first next-step wording.
|
||||
|
||||
## Purpose
|
||||
|
||||
AI accelerates code production. OpenUltraSAST should help developers catch concrete security
|
||||
@@ -120,3 +133,12 @@ questions. Native PHP/JavaScript partition smoke and 1321 tests pass. Runtime/co
|
||||
identity changed, so prior eligibility is stale and remains disabled. This repairs one
|
||||
shared prerequisite; pre-push 8.3, runtime amortization, VAmPI context/transitive
|
||||
evidence and independent qualification remain open. No ranker retune or rollout GO.
|
||||
|
||||
The 2026-09-14 post-census profile now verifies identical-tip reuse (18.521–18.874s,
|
||||
two graph hits and 1,512 query hits per repeat), while all 18 other transactions time
|
||||
out and 0/24 target checks complete. The separate local-session census experiment is
|
||||
promising but does not qualify security queries or hook latency. The contributor-scan
|
||||
`release-remediation-research.md` records missing family context producers, global
|
||||
vendor/correspondence completeness vetoes, and query-to-operation identity as the next
|
||||
shared-contract work. Resolving a boundary requires evidence of its relevance and scope;
|
||||
do not simply drop completeness checks to increase admissions. Rollout remains NO-GO.
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
---
|
||||
name: openultrasast-kiro-impl
|
||||
description: OpenUltraSAST Kiro implementation steering. Use when continuing `.kiro/specs/openrouter-sast-harness/tasks.md`, choosing the next phase, updating checkboxes, or keeping implementation aligned with the spec.
|
||||
---
|
||||
|
||||
# OpenUltraSAST Kiro Implementation
|
||||
|
||||
Use this skill when implementing the OpenUltraSAST Kiro spec from opencode.
|
||||
|
||||
## Rules
|
||||
|
||||
1. Read `.kiro/specs/openrouter-sast-harness/tasks.md` before choosing work.
|
||||
2. Implement the next unchecked task slice unless the user names a different phase.
|
||||
3. Keep each slice independently verifiable with `uv run pytest`.
|
||||
4. Update task checkboxes only after implementation and tests pass.
|
||||
5. Prefer minimal harness-compatible primitives over large framework imports.
|
||||
6. Keep OpenUltraSAST independent of Clearwing; use Clearwing only as an oracle for design patterns.
|
||||
7. Use project-local opencode skills to steer scan, triage, and fix workflows.
|
||||
|
||||
## Current Phase Order
|
||||
|
||||
- Phase 5: Harness runtime events, processors, contracts, and traces.
|
||||
- Phase 5A: Static mapping disciplines for SARIF, Semgrep, CodeQL, differential review, and sharp edges.
|
||||
- Phase 6: Evidence ladder verifier.
|
||||
- Phase 7 and 7A: Docker sandbox and dynamic evidence.
|
||||
- Phase 8 and later: Hunter pool, SARIF, deep mode, patch oracle, MCP, fusion, skill index, hardening.
|
||||
@@ -1,159 +1,78 @@
|
||||
# OpenUltraSAST
|
||||
|
||||
**v1.2.0-alpha.1:** experimental pre-push safety net. Integration and census repairs are
|
||||
verified; useful hook coverage, latency and independent capability qualification remain
|
||||
**NO-GO**. No real hook capability is enabled. See [release notes](RELEASE_NOTES.md) and
|
||||
[spec status](.kiro/specs/pre-push-safety-net/status.md).
|
||||
**v2.0.0 (Python package `2.0.0`), state as of 2026-10-02.** An experimental static
|
||||
security analyser. The pre-push safety net is **NO-GO** for rollout: no hook capability is
|
||||
enabled and no independent population has passed the qualification gates (see
|
||||
[docs/evaluation.md](docs/evaluation.md)). See the [release notes](RELEASE_NOTES.md).
|
||||
|
||||
OpenUltraSAST is a command-line security analysis harness built around one
|
||||
goal: eliminating false positives by turning a suspicion into an
|
||||
evidence-backed finding, and learning from the claims that were rejected. A
|
||||
scan combines static rules, a semantic overlay, optional model-driven hunting
|
||||
and Docker-isolated proof, and every claim carries the evidence that earned its
|
||||
place.
|
||||
OpenUltraSAST separates what it can prove from what it merely suspects, and says which is which.
|
||||
A scan combines language-scoped pattern rules, a semantic overlay, a Joern-based taint engine
|
||||
and optional model calls; every finding carries the evidence that earned its place, and a scan
|
||||
that could not look at something records that as a degradation instead of reporting a clean
|
||||
result.
|
||||
|
||||
The project began as an editor-agent skill and is now a CLI first. Agent
|
||||
integrations remain, but nothing requires one: `ousast` is the interface, and
|
||||
the model-driven parts are optional extras that degrade visibly when absent.
|
||||
|
||||
> **Maturity legend:** ✅ implemented and tested · 🟡 primitive exists, not yet
|
||||
> auto-wired into the scan loop · 🧭 designed, on the roadmap. The current
|
||||
> verified gate is `usable_harness_mvp`.
|
||||
`ousast` is the interface. Model-driven work over whole repositories runs on a separate
|
||||
agentic plane (google/ax), and a learned decision engine that weighs all the signals is in
|
||||
development; neither is needed for a scan.
|
||||
|
||||
## Install and quickstart
|
||||
|
||||
```bash
|
||||
uv sync --group dev
|
||||
|
||||
# Scan a local repository (quick mode = deterministic, no model calls)
|
||||
# Deterministic scan, no model calls
|
||||
uv run ousast scan /path/to/repo --mode quick
|
||||
|
||||
# Build an embedding-ready chunk index
|
||||
uv run ousast index /path/to/repo
|
||||
|
||||
# Score the detector against a benchmark manifest
|
||||
uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
|
||||
|
||||
# Differential pair eval: fire on vuln, stay silent on the fix (local + real GitHub VFCs)
|
||||
uv run ousast pairs
|
||||
uv run ousast pairs --slice sast # OWASP Benchmark Java/Python + Juliet
|
||||
uv run ousast pairs --slice vibe-py --pointers # nightly: also harvest non-vendored pointer pairs into ~/.cache
|
||||
uv run ousast mechanisms export --slice vibe-py # seed mechanism candidates from trusted pairs (offline); `ousast improve` admits them under the holdout gate
|
||||
uv run ousast pairs --slice vibe-py --loo # leave-one-out recall of the corpus as a teacher
|
||||
uv run python -m openultrasast.pair_gate
|
||||
|
||||
# Run the bounded self-improvement loop over the ruleset (benchmark-driven)
|
||||
uv run ousast improve benchmarks/manifests/python-vulnerable.toml --dry-run
|
||||
# CI: exit 1 on any evidence-verified finding
|
||||
uv run ousast scan . --mode quick --fail-on verified
|
||||
```
|
||||
|
||||
Every run writes auditable artifacts under `.openultrasast/runs/<scan-id>/`
|
||||
(`manifest.json`, `findings.json`, `verification.json`, `report.md`,
|
||||
`report.sarif`, `trace/events.jsonl`, ranking/preprocess/mapping JSON).
|
||||
`--fail-on findings|verified` gives CI-friendly exit codes.
|
||||
The core install depends only on PyYAML. Optional extras: `semantic` (tree-sitter grammars
|
||||
for the overlay) and `s3` (the plane's memory store on an S3-compatible bucket, boto3).
|
||||
|
||||
## How a scan works
|
||||
For `standard` and `deep` scans, use the Docker image, which ships Joern (with `php-cli` for
|
||||
the PHP frontend) next to the tool. Source the shell wrapper once and `ousast` runs in the
|
||||
container with your directory mounted read-only:
|
||||
|
||||
The CLI is thin; the `HarnessRuntime` owns the lifecycle and emits an event
|
||||
trace for every stage (`task_start … stage_start/stage_end … task_end`).
|
||||
|
||||
```
|
||||
repo ──▶ preprocess (language, LOC, tags, fuzz entry points)
|
||||
──▶ static mapping (Semgrep/CodeQL SARIF ingest, normalized hints)
|
||||
──▶ entry-point + reachability mapping (routes, CLI, parsers, contracts)
|
||||
──▶ rank (surface·0.5 + influence·0.2 + reachability·0.3)
|
||||
──▶ findings ─ quick: language-scoped pattern rules
|
||||
│ └ standard: tiered hunter pool (retrieval + skill snippets)
|
||||
──▶ independent verifier (evidence ladder enforced)
|
||||
──▶ report.md + report.sarif + manifest.json (shared finding IDs)
|
||||
```bash
|
||||
source /path/to/OpenUltraSAST/ops/shell/ousast.sh # PowerShell: ops/shell/ousast.ps1
|
||||
ousast scan . --mode standard
|
||||
```
|
||||
|
||||
- **quick** ✅: deterministic, language-scoped pattern rules + reachability +
|
||||
verification. No model calls, fully reproducible.
|
||||
- **standard** ✅ scheduling / 🧭 LLM hunters: runs the tiered hunter pool
|
||||
(A/B/C budgets, retrieval context, selected Trail of Bits skill snippets,
|
||||
JSONL trajectories) and verification; language-aware LLM hunters are the next
|
||||
upgrade (today standard reuses the deterministic findings per target).
|
||||
- **deep** 🧭: sandboxed build/fuzz/dynamic reproduction (not yet implemented).
|
||||
Details, including a pinned host install of Joern, are in [ops/README.md](ops/README.md).
|
||||
|
||||
## Detection coverage
|
||||
Every run writes auditable artifacts under `<target>/.openultrasast/runs/<scan-id>/`:
|
||||
`manifest.json`, `findings.json`, `verification.json`, `score.json`, `report.md`,
|
||||
`report.sarif` and `trace/events.jsonl`, plus the stage artifacts of the mode.
|
||||
|
||||
Language-scoped rule families (rules only run against matching languages, which
|
||||
removes cross-language false positives):
|
||||
## Scan modes
|
||||
|
||||
| Mode | What runs | Needs |
|
||||
| --- | --- | --- |
|
||||
| `quick` | Language-scoped pattern rules, entry-point reachability hints, ranking, verification, scoring. Deterministic and reproducible. | nothing |
|
||||
| `standard` | `quick` plus the MAP stage: complexity map, semantic overlay, authorization obligations, and the model layer, which builds one Joern code property graph per repository and decides taint, guard-dominance and configuration questions on it. An LLM is asked only the residual question the graph cannot settle (a `suspicion`); `[models] hunter` adds the tool hunter. | Joern for the model layer (else a `cpg_unavailable` degradation); a provider key for the LLM parts (else they are skipped and recorded) |
|
||||
| `deep` | The MAP stage of `standard` plus REGRESS: promoted candidates are loaded by a small snippet inside a Docker sandbox (no network, read-only source, non-root, memory/pid/time limits) and get a `triggerable` / `not_triggerable` / ... verdict. | a working `docker` (else a `sandbox_unavailable` degradation) |
|
||||
|
||||
`--fail-on never|findings|verified|worth-fixing` sets the exit code: `findings` fails on any
|
||||
reported finding, `verified` on any evidence-verified one, `worth-fixing` on a deep-mode
|
||||
verdict that is worth fixing. `--config openultrasast.toml` loads settings.
|
||||
|
||||
## Detection coverage (quick rules)
|
||||
|
||||
Rules run only against files of their language. The bundled ruleset lives in
|
||||
`src/openultrasast/ruleset/<language>/rules.toml`:
|
||||
|
||||
| Language | Classes (CWE) |
|
||||
| --- | --- |
|
||||
| C / C++ | buffer overflow (120), format string (134), command injection (78) |
|
||||
| Python | command injection (78), code injection (95), deserialization (502), SQLi (89), path traversal (22), SSRF (918), weak hash (327), SSTI (94), insecure default (489) |
|
||||
| JavaScript / TS | command injection (78), code injection (95), reflected/DOM XSS (79), SQLi (89), path traversal (22), SSRF (918), weak hash (327), deserialization (502) |
|
||||
| Java + Groovy templates | command injection (78), SQLi via concatenation (89), weak hash (327), deserialization (502), unescaped template XSS (79) |
|
||||
| C / C++ | stack buffer overflow (121), format string (134), command injection (78) |
|
||||
| Python | command injection (78), code injection (95), SSTI (94), deserialization (502), SQLi (89), path traversal (22), SSRF (918), weak hash (327), insecure default (489) |
|
||||
| JavaScript / TS | command injection (78), code injection (95), XSS (79), SQLi (89), path traversal (22), SSRF (918), weak hash (327), deserialization (502) |
|
||||
| Java + Groovy templates | command injection (78), SQLi (89), weak hash (327), deserialization (502), unescaped template XSS (79) |
|
||||
| PHP | SQLi (89), command injection (78), file inclusion (98), path (22), open redirect (601), SSRF (918); code eval (95), unserialize (502) and echo XSS (79) ship as `shadow` rules |
|
||||
|
||||
Cheat-sheet fixtures live under `benchmarks/fixtures/` and feed the 90/10 smoke
|
||||
gate. **Efficiency** is measured separately on isolated vuln-vs-fixed pairs
|
||||
(`ousast pairs`, `python -m openultrasast.pair_gate`): the harness must fire on
|
||||
the vulnerable snapshot and stay silent after the patch. Local fixture
|
||||
counterparts are a CI gate; vendored GitHub VFCs (SVEN, CWE-Bench-Java, NVD
|
||||
patterns) are the honesty dashboard and the miss/fp signals for `ousast improve`.
|
||||
Dataset pointers (CVEfixes, DiverseVul, PrimeVul, MegaVul, MoreFixes, Vul4J,
|
||||
GitHub Advisory Database, OSV) are in `benchmarks/pairs/datasets.toml`.
|
||||
|
||||
## Project scores and central CWE policy
|
||||
|
||||
Severity used to be whatever string a rule set on itself. It is now governed centrally: one CWE, one severity, decided by policy — not by the rule that fired.
|
||||
|
||||
**Central CWE policy** ✅ (`policy/verycode.py`): a vendored `CWE_Score.tsv` (verycode-policies, ~167 CWEs) is the single source of truth. Each CWE carries a flaw category, a `severity` (0-5), and `static` / `dynamic` scope flags. `load_policy()` reads it positionally (the upstream file ships CRLF and a trailing space in the `Flaw Severity ` header), and `resolve_severity()` resolves a finding's severity *exclusively* from policy, keyed on CWE — any legacy rule-local severity is discarded. CWEs that are not `static` resolve to `0` and are report-only, never scored.
|
||||
|
||||
**Fail-loud startup** ✅: `assert_rules_resolve(PATTERN_RULES, policy)` runs as the `policy_check` stage right after `policy_load`. If any *enabled* rule names a CWE the policy does not govern, it raises `PolicyError` and the scan aborts before doing work — you cannot ship a rule whose severity nobody decided.
|
||||
|
||||
**0-100 project score** ✅ (`scoring/project_score.py`): each finding's penalty is `severity weight × reachability multiplier`, and the score is an exponential decay of the total (`100 · e^(−total/k)`, `k=60`).
|
||||
|
||||
| Severity | Weight (`SEV_WEIGHT`) | | Reachability | Multiplier (`REACH_MULT`) |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 5 | 50 | | `reachable` | 1.0 |
|
||||
| 4 | 25 | | `inferred-file-surface` | 0.6 |
|
||||
| 3 | 10 | | `unknown` | 0.4 |
|
||||
| 2 / 1 / 0 | 2 / 1 / 0 | | | |
|
||||
|
||||
No penalty → 100; one reachable severity-5 finding → ~43. The **reachability multiplier is the false-positive calibration knob**: a confirmed FP lowers a finding's effective reachability instead of deleting the rule, so the score moves without losing the detection.
|
||||
|
||||
**Two-condition gate** ✅: a finding that is both severity-5 *and* `reachable` **always** fails the gate. The `score < min_score` threshold (default `min_score=80`) only fails when `blocking` is enabled — so scoring is **advisory-first by default** and turns into a CI gate when you opt in.
|
||||
|
||||
**Artifacts** ✅: the `score` stage writes `score.json` (project score, `max_severity`, `penalty_total`, `by_category`, `out_of_scope_dynamic_only`, `unmapped_cwe`, gate verdict) and merges the same block into `manifest.json`. Scoring is zero-dependency (stdlib only). One mapping detail: verycode has no CWE-120 (generic buffer copy), so the C/C++ memory-unsafe rules listed under [Detection coverage](#detection-coverage) carry **CWE-121** (Stack-Based Buffer Overflow, severity 5), which the policy does govern.
|
||||
|
||||
🧭 This is the first implemented slice of the self-improving ruleset work: a central, policy-governed severity model and a project score. Rules-as-data and the full self-improvement loop over rulesets are later phases.
|
||||
|
||||
## Running the benchmarks
|
||||
|
||||
Benchmarks wrap the same scan pipeline and score it against ground-truth
|
||||
manifests in `benchmarks/manifests/`. Each manifest lists the *known*
|
||||
vulnerabilities (CWE, file, line, sink, evidence) plus optional external
|
||||
baselines; fixtures live in `benchmarks/fixtures/` and include safe-API files so
|
||||
precision is measured honestly.
|
||||
|
||||
```bash
|
||||
# One language
|
||||
uv run ousast benchmark benchmarks/manifests/cpp-damn-vulnerable.toml --mode quick
|
||||
|
||||
# All in-scope corpora
|
||||
for m in c-cpp-smoke cpp-damn-vulnerable \
|
||||
python-web-smoke python-vulnerable \
|
||||
javascript-node-web-smoke javascript-vulnerable \
|
||||
java-web-smoke java-spring-boot-vulnerable; do
|
||||
uv run ousast benchmark "benchmarks/manifests/$m.toml" --mode quick
|
||||
done
|
||||
```
|
||||
|
||||
Each run writes `.openultrasast/benchmarks/<id>/`:
|
||||
`benchmark_result.json` (recall, precision, misses, false positives, runtime),
|
||||
`calibration_records.json` (every miss as a next-improvement candidate), and
|
||||
`external_baseline_deltas.json` (tool-vs-tool comparison).
|
||||
|
||||
### Latest results (`--mode quick`)
|
||||
|
||||
The corpora are modeled on Damn Vulnerable C/C++, PyGoat/DVPWA and NodeGoat,
|
||||
plus the real [`kiview/damn-vulnerable-spring-boot-app`](https://github.com/kiview/damn-vulnerable-spring-boot-app)
|
||||
vendored verbatim (MIT). Three vulnerabilities are deliberately left in the
|
||||
ground truth that pattern matching *cannot* reach (integer overflow, second-order
|
||||
SQLi, prototype pollution) so recall is not self-fulfilling.
|
||||
The detection gate (`uv run python -m openultrasast.gate`, also `tests/test_detection_benchmarks.py`)
|
||||
enforces the project goal of **at least 90% recall and under 10% false positives** per language
|
||||
on the bundled corpora in `benchmarks/manifests/`. Its output on this tree:
|
||||
|
||||
| Language | Recall | False-positive rate |
|
||||
| --- | --- | --- |
|
||||
@@ -163,255 +82,156 @@ SQLi, prototype pollution) so recall is not self-fulfilling.
|
||||
| Java | 100% (4/4) | 0.0% |
|
||||
| **Overall** | **93.6% (44/47)** | **0.0%** |
|
||||
|
||||
The project goal is **≥90% recall and <10% false positives** per language. The
|
||||
gate lives in `tests/test_detection_benchmarks.py`, so a rule change that drops
|
||||
recall or raises false positives fails CI.
|
||||
These corpora are cheat-sheet fixtures modelled on deliberately vulnerable applications. They
|
||||
show that a rule change did not break a known detection; they are not a recall estimate on
|
||||
real code. Real-code results are in [docs/evaluation.md](docs/evaluation.md).
|
||||
|
||||
## Agent integrations, fusion, and ultra workflows
|
||||
**PHP is not in the gate, and its numbers are in-sample only.** The PHP rules were written and
|
||||
tightened while looking at the development corpus they were measured on
|
||||
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`): 13 of 14 expected
|
||||
findings on the `php-vulnerable` fixture with no false positive, 2 false positives on
|
||||
`php-benign`, and per-rule precision lower bounds from 0.03 to 1.0 on the real pins. Treat
|
||||
them as development numbers, not as a holdout result.
|
||||
|
||||
The `ousast` CLI is the interface. An agent that can run shell commands needs
|
||||
nothing else:
|
||||
**Framework knowledge is optional.** Rule and taint-fact entries that know a framework or
|
||||
library (WordPress, Flask, Django, Express, Spring, ...) carry a `framework` or `library` tag
|
||||
naming a row of `src/openultrasast/ruleset/frameworks.toml`. The loaders take `priors=`
|
||||
(`"all"`, the default and today's behaviour; `"off"`, language-level knowledge only; or a set
|
||||
of ids), so their contribution can be measured and switched off.
|
||||
|
||||
## Pre-push checks (experimental)
|
||||
|
||||
`ousast pre-push` analyses the commits a `git push` would publish, or an explicit
|
||||
`--base`/`--head` pair, and compares the head against its base so that only new or worsened
|
||||
defects are candidates. `--mode advisory` (the default) never blocks; `--mode blocking` and
|
||||
`--incomplete-coverage block` opt into enforcement. The artifact must be written outside the
|
||||
analysed repository.
|
||||
|
||||
```bash
|
||||
uv run ousast scan . --mode quick --fail-on verified
|
||||
uv run ousast pre-push /path/to/repo --base BASE --head HEAD \
|
||||
--artifact /outside/the/repo/result.json
|
||||
```
|
||||
|
||||
For [OpenCode](https://opencode.ai) there are optional project skills that
|
||||
describe the workflows above, so the agent drives the same CLI
|
||||
(`.opencode/skills/`):
|
||||
The default capability registry is empty, so no normal alert is emitted today; every
|
||||
candidate stays diagnostic in the artifact. Installation into a repository's hooks
|
||||
(`ops/install-pre-push`), caching and the optional witness model are documented in
|
||||
[ops/README.md](ops/README.md).
|
||||
|
||||
| Skill | Purpose |
|
||||
| --- | --- |
|
||||
| `openultrasast-scan` ✅ | run/plan `ousast scan`, indexing, evidence-gated analysis |
|
||||
| `openultrasast-triage` ✅ | false-positive elimination, ranking calibration, verifier adjudication |
|
||||
| `openultrasast-fix-audit` ✅ | OpenUltraCode fix lifecycle and adversarial fix review |
|
||||
## Model providers and keys
|
||||
|
||||
### MCP server (narrow surface) ✅
|
||||
Keys live in `.env` in the working directory (gitignored, never committed):
|
||||
|
||||
`ousast mcp` runs a zero-dependency MCP server over stdio (newline-delimited
|
||||
JSON-RPC) exposing only stable, project-level operations — **never** arbitrary
|
||||
shell, Docker, or internal hunter tools. Point an MCP client (OpenCode, Claude
|
||||
Desktop, an IDE) at it:
|
||||
```bash
|
||||
DEEPSEEK_API_KEY=... # every LLM call: model layer, tool hunter, plane tasks
|
||||
OPENROUTER_API_KEY=... # only OpenAI embeddings (openai/text-embedding-3-small)
|
||||
```
|
||||
|
||||
The project uses DeepSeek (`deepseek-flash`) for LLM calls and OpenRouter only for embeddings.
|
||||
Set `DEEPSEEK_API_KEY`: without it, chat calls fall back to OpenRouter when that key is set.
|
||||
|
||||
**`.env` never overrides a variable already exported in your shell.** A stale
|
||||
`export DEEPSEEK_API_KEY=...` in your profile wins over the `.env` file silently; `unset` it
|
||||
when a key change in `.env` seems to have no effect.
|
||||
|
||||
## Benchmarks and pair evaluation
|
||||
|
||||
```bash
|
||||
# Score the scan against a ground-truth manifest
|
||||
uv run ousast benchmark benchmarks/manifests/python-vulnerable.toml --mode quick
|
||||
|
||||
# Vulnerable-vs-fixed pairs: fire on the vulnerable side, stay silent on the fix
|
||||
uv run ousast pairs
|
||||
uv run ousast pairs --slice sast # OWASP Benchmark + Juliet
|
||||
uv run ousast pairs --slice vibe-py --pointers # also fetch non-vendored pointer pairs (network)
|
||||
uv run python -m openultrasast.pair_gate # CI: local pairs must all pass
|
||||
```
|
||||
|
||||
Each benchmark run writes `.openultrasast/benchmarks/<id>/` (`benchmark_result.json`,
|
||||
`calibration_records.json` with every miss, `external_baseline_deltas.json`). The pair
|
||||
catalogs live in `benchmarks/pairs/`.
|
||||
|
||||
## Self-improvement
|
||||
|
||||
`ousast improve` runs a bounded loop over the governed ruleset: benchmark, propose edits to
|
||||
rule status and score constants only, validate, re-benchmark, and accept a round only if
|
||||
recall stays at least 90%, false positives under 10%, the project score does not regress and
|
||||
no matched finding is lost. Accepted rounds land in
|
||||
`<target>/.openultrasast/calibration/rule_policy.json`, which the next scan of that target loads.
|
||||
|
||||
```bash
|
||||
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run
|
||||
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run --memory
|
||||
```
|
||||
|
||||
With `--memory`, proposals also come from the rows plane runs stored (see below), through the
|
||||
same validator and gate; rows from holdout pairs, from the gated manifest's own cases and from
|
||||
any `--qualify-population` are dropped first. Details:
|
||||
[docs/architecture.md](docs/architecture.md#self-improvement).
|
||||
|
||||
## The agentic plane (maintainers)
|
||||
|
||||
Model work over whole repositories runs as a Run manifest on [google/ax](ops/ax/README.md)
|
||||
over Agent Substrate (a kind cluster on the maintainer's host), one isolated actor per task.
|
||||
Each task binds its own Model and its own `usd`/`calls` budget, egress is deny-by-default per
|
||||
task, and the provider key travels only in the task's start request.
|
||||
|
||||
```bash
|
||||
uv run ousast plane doctor # kind, Agent Substrate, ax controller, runner image
|
||||
uv run ousast plane run plane/runs/validation-46.yaml # a rerun skips tasks already done
|
||||
uv run ousast plane status validation-46 # per-task status and token attribution
|
||||
uv run ousast plane remember validation-46 # ingest the run's rows into the memory store
|
||||
```
|
||||
|
||||
`ousast plane workspaces`, `alerts-engine` and `harvest` generate Runs and inputs. Bring-up,
|
||||
the task catalogue, the memory store (`OUSAST_MEMORY`) and the checklist for moving the plane
|
||||
to another Kubernetes cluster are in [ops/ax/README.md](ops/ax/README.md).
|
||||
|
||||
> HarnessX, the earlier optional agentic extra, was retired 2026-09-30, and a leftover
|
||||
> `[harnessx]` section (retired 2026-09-30) is ignored with one warning. `[models] verifier`,
|
||||
> `[fusion] panel_model` and `[fusion] decider_model` fail with a message naming the plane
|
||||
> replacement. The LLM judge and the LLM fusion panels were retired with it; fusion is
|
||||
> deterministic. See [RELEASE_NOTES.md](RELEASE_NOTES.md).
|
||||
|
||||
## Decision engine (in development, not adopted)
|
||||
|
||||
A learned decision engine is being built to replace hand-tuned detection edits: each
|
||||
instrument (quick rules, the Joern engine, model sink classification, verify passes, repository
|
||||
facts) produces signals, never verdicts, and a compiled AI classifier with local memory turns
|
||||
them into a calibrated probability per candidate with BLOCK and ADVISORY operating points.
|
||||
No scan, pre-push or report uses it yet. Status and the first measured slice:
|
||||
[docs/decision-engine.md](docs/decision-engine.md).
|
||||
|
||||
## Agent integrations
|
||||
|
||||
An agent that can run shell commands needs nothing but the CLI. For
|
||||
[OpenCode](https://opencode.ai) there are optional project skills in `.opencode/skills/`:
|
||||
`openultrasast-scan`, `openultrasast-triage` and `openultrasast-fix-audit`.
|
||||
|
||||
`ousast mcp` runs a narrow MCP server over stdio (newline-delimited JSON-RPC) with ten tools:
|
||||
`openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`, `artifacts`,
|
||||
`benchmark`, `explain`, `propose_patch` (degrades visibly) and `export_report`. No tool runs a
|
||||
shell, Docker or a free-form command.
|
||||
|
||||
```jsonc
|
||||
// e.g. an MCP client config entry
|
||||
{ "command": "uv", "args": ["run", "ousast", "mcp"] }
|
||||
```
|
||||
|
||||
Tools: `openultrasast.scan`, `status`, `findings`, `get_finding`, `evidence`,
|
||||
`artifacts`, `benchmark`, `explain`, `propose_patch` (degrades visibly — the patch
|
||||
oracle is a later phase), and `export_report`. The `scan`/`benchmark` tools run the
|
||||
bounded analysis pipeline; no tool accepts a free-form command. A typical flow:
|
||||
`scan {path}` → returns a `run_dir` → `findings {run_dir}` → `explain {run_dir, finding_id}`.
|
||||
> The `kiro-*` skills, `AGENTS.md` and `.kiro/` are the maintainer's development tooling and
|
||||
> can be ignored by users.
|
||||
|
||||
**Ultra workflows (OpenUltraCode discipline):** fixes are never a one-shot patch
|
||||
prompt. `openultrasast-fix-audit` runs the bounded lifecycle
|
||||
`intake → plan → minimal patch → adversarial review → reconcile → fresh
|
||||
verification → ready`, and a patch can only reach `patch_validated` after
|
||||
sandboxed validation passes with no accepted blocking findings.
|
||||
## Further reading
|
||||
|
||||
**Fusion (deepening) ✅:** when a finding needs more reasoning than the normal
|
||||
ranker → hunter → verifier → mapping loop provides (critical/high severity,
|
||||
verifier disagreement, conflicting static vs semantic evidence, risky fixes, or
|
||||
an explicit high-assurance request), fusion runs two independent panels that
|
||||
steel-man the vulnerability and false-positive cases, vote, and a decider issues
|
||||
the disposition (accepted / rejected / mitigated / deferred / blocked) with votes,
|
||||
model IDs, and degradations disclosed. It runs automatically on triggered findings
|
||||
in `--mode standard`, writing `fusion.json` and a `fusion` block in the manifest.
|
||||
Deterministic by default (`fusion.py`); set `[fusion] panel_model` to route the
|
||||
panels through the configured provider:
|
||||
Documentation site with flow diagrams (`docs/`, `mkdocs.yml`): `uv sync --extra docs && uv run mkdocs serve`.
|
||||
|
||||
```toml
|
||||
[fusion]
|
||||
enabled = true # default; runs on triggered findings in standard mode
|
||||
panel_model = "gpt-4o" # optional: LLM panels via the [harnessx] provider
|
||||
decider_model = "gpt-4o" # optional: disclosed in the decision's model IDs
|
||||
high_assurance = false # true forces fusion on every finding
|
||||
```
|
||||
|
||||
> The `kiro-*` skills, `AGENTS.md` and the `.kiro/` directory are the
|
||||
> maintainer's spec-driven development tooling. They are not part of using
|
||||
> OpenUltraSAST and can be ignored by users.
|
||||
|
||||
## Evidence ladder: how a false positive is eliminated
|
||||
|
||||
Evidence level is a state machine, not a label a model may assign. The verifier
|
||||
runs on independent context (it never sees hunter reasoning) and enforces:
|
||||
|
||||
```
|
||||
suspicion ─▶ static_corroboration ─▶ crash_reproduced ─▶ root_cause_explained
|
||||
─▶ exploit_demonstrated ─▶ patch_validated
|
||||
```
|
||||
|
||||
A finding is rejected or held back unless the *artifact* for the next level
|
||||
actually exists:
|
||||
|
||||
- below `static_corroboration` → `NEEDS_EVIDENCE` (a model/heuristic suspicion is
|
||||
never reported as verified);
|
||||
- `static_corroboration` but **reachability unknown** → `REJECTED` with a
|
||||
tie-breaker demanding call-graph / route / CLI / parser / dynamic evidence
|
||||
(this is what kills "pattern matched but nothing attacker-controlled reaches
|
||||
it" findings);
|
||||
- `static_corroboration` **and** function-level reachability → `ACCEPTED`.
|
||||
|
||||
When a finding is still rejected after review, `openultrasast-triage` records a
|
||||
**scoped false-positive learning** (`calibration.py`): a reason from a fixed
|
||||
taxonomy (`unreachable_path`, `missing_attacker_control`, `sanitizer_disproved`,
|
||||
`static_rule_mismatch`, `incorrect_model_assumption`, `duplicate`,
|
||||
`insufficient_impact`, `unsupported`, `contradicted`, `unverified`), evidence,
|
||||
and a **scope**. Scoping keeps the learning narrow: a rejected `syscall_entry`
|
||||
claim in `auth/` demotes only `auth/…`, never the whole vulnerability class. This
|
||||
is covered by `tests/test_calibration.py::test_scoped_false_positive_demotion_does_not_suppress_class_globally`.
|
||||
Demotion (with an audit trail) is preferred over deletion, so an analyst can
|
||||
always see *why* something was downgraded.
|
||||
|
||||
## How a false negative is eliminated
|
||||
|
||||
Misses are first-class signals, surfaced by the benchmark layer rather than
|
||||
hidden. Every expected vulnerability that no finding matched becomes a
|
||||
`BenchmarkCalibrationRecord` in `calibration_records.json`, e.g. the integer
|
||||
overflow the regex engine cannot see:
|
||||
|
||||
```json
|
||||
{
|
||||
"cwe": "CWE-190",
|
||||
"vulnerability_class": "integer overflow",
|
||||
"path": "src/buffer.cpp",
|
||||
"failed_stage": "benchmark_ground_truth_matching",
|
||||
"next_improvement_candidate": "rules_or_language_hunter",
|
||||
"reason": "no OpenUltraSAST finding matched the expected benchmark vulnerability"
|
||||
}
|
||||
```
|
||||
|
||||
The `next_improvement_candidate` routes the gap to the stage that should close
|
||||
it: a static rule, SARIF source, entry-point mapping, retrieval package, hunter
|
||||
prompt, dynamic reproducer, or skill route. Once a miss is
|
||||
addressed, it stays closed because the recall/precision gate runs in CI.
|
||||
`external_baseline_deltas.json` additionally shows where another tool found a
|
||||
vulnerability OpenUltraSAST missed (or vice-versa).
|
||||
|
||||
## Configuring the HarnessX agentic plane
|
||||
|
||||
The LLM hunter pool and the llm-judge verifier are optional. The core install is
|
||||
zero-dependency and `--mode quick` never calls a model. To enable the agentic
|
||||
plane, install the extra, name the models, and pick a provider:
|
||||
|
||||
```bash
|
||||
uv sync --extra harnessx # SHA-pinned optional dependency
|
||||
export OPENAI_API_KEY=sk-... # or ANTHROPIC_API_KEY, matching the provider
|
||||
uv run ousast scan /path/to/code --mode standard --config openultrasast.toml
|
||||
```
|
||||
|
||||
```toml
|
||||
# openultrasast.toml
|
||||
[models]
|
||||
hunter = "gpt-4o" # LLM hunter pool (empty -> deterministic hunter)
|
||||
verifier = "gpt-4o" # llm-judge verifier (empty -> structural verifier)
|
||||
|
||||
[harnessx]
|
||||
provider = "openai" # "anthropic" (default) | "openai" | "litellm"
|
||||
max_cost_usd = 2.0 # per-task spend cap
|
||||
token_threshold = 120000 # per-task token budget
|
||||
```
|
||||
|
||||
The provider selects the HarnessX backend for **both** the hunter and the
|
||||
verifier; each reads its own standard key env var (`openai` → `OPENAI_API_KEY`,
|
||||
`anthropic` → `ANTHROPIC_API_KEY`). The HarnessX path activates only when the
|
||||
extra is installed, a model is configured, and the mode is `standard` — all three
|
||||
gated through one capability seam (`harness_ext.has_harnessx`). If any condition
|
||||
fails, the scan transparently falls back to the deterministic hunter / structural
|
||||
verifier and records a `degradations` entry in `manifest.json`, so a run is never
|
||||
silently downgraded:
|
||||
|
||||
```bash
|
||||
jq '.degradations' "$run/manifest.json" # null = HarnessX ran; entries = fell back
|
||||
```
|
||||
|
||||
## The HarnessX self-improving cycle
|
||||
|
||||
Each scan is a composed harness of typed processors with read/write **state
|
||||
contracts**. Strict mode fails the scan on a violation; warn mode records a
|
||||
degradation. The loop now runs inside the pipeline without an agent: every scan
|
||||
persists verifier rejections as scoped learnings, and the next scan loads them
|
||||
and demotes those scopes before findings are produced.
|
||||
|
||||
```
|
||||
┌──────────── automatic ledger (no agent in the loop) ───────────┐
|
||||
▼ │
|
||||
scan ──▶ verify ──▶ non-accepted findings ──▶ scoped learnings ─────────┤
|
||||
│ (.openultrasast/calibration/ │ loaded by
|
||||
│ false_positive_learnings.json) │ the next
|
||||
next scan ──▶ rank ──▶ calibrate (demote rejected scopes) ──▶ findings ──┘ scan
|
||||
```
|
||||
|
||||
What runs automatically each scan (✅, `calibration.py` + `cli.py`):
|
||||
|
||||
1. **`calibrate` stage** (after `rank`, before findings) loads the persistent
|
||||
ledger and demotes the priority of every scope that previously produced a
|
||||
rejected finding (`calibrate_rankings`), writing `applied_calibrations.json`.
|
||||
2. **`record_calibration` stage** (after `verify`) turns this run's non-accepted
|
||||
verifier outcomes into scoped `FalsePositiveLearning` records
|
||||
(`learnings_from_verifications`) and merges them into the ledger, de-duplicated
|
||||
by finding ID so a repeat rejection does not compound without bound.
|
||||
|
||||
Demotion is **scoped and reversible**: a rejection in `lib.py` demotes only that
|
||||
scope; an accepted (reachable) finding in `app.py` is never touched; and because
|
||||
priority is demoted rather than the finding deleted, the audit trail
|
||||
(`applied_calibrations.json`, the ledger) shows exactly why a surface lost
|
||||
attention. Covered by `tests/test_pipeline_calibration.py`.
|
||||
|
||||
### Rule-level loop: `ousast improve`
|
||||
|
||||
The ranking loop above adjusts *attention*. A second, bounded loop adjusts the
|
||||
**governed ruleset itself** from benchmark feedback — and writes its result to the
|
||||
same loop-owned ledger a scan reads, so the loop closes end-to-end:
|
||||
|
||||
```bash
|
||||
# Preview what the loop would change (writes nothing)
|
||||
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml --dry-run
|
||||
|
||||
# Apply: accepted rounds land in <target>/.openultrasast/calibration/rule_policy.json,
|
||||
# which the next `ousast scan`/`benchmark` of that target loads automatically.
|
||||
uv run ousast improve benchmarks/manifests/java-spring-boot-vulnerable.toml
|
||||
```
|
||||
|
||||
Each round (`improve/evolve.py`): benchmark with the current ledger → per-rule
|
||||
signals (`rule_signals.json`) → propose **bounded** edits → validate → replay
|
||||
smoke → re-benchmark → a **hard acceptance gate** → accept or revert
|
||||
byte-for-byte. The loop may only pull two levers (rule status, score constants);
|
||||
it can never edit a rule's pattern text or the authoritative 0–5 CWE severity, and
|
||||
must stage `enabled → shadow` before `disabled`. A round is accepted only if
|
||||
**recall ≥ 90% AND FP < 10% AND the project score did not regress AND no matched
|
||||
finding was lost** — otherwise the ledger is never written. Auto-shadowing removes
|
||||
a precision-dragger's false-positive cost without deleting it (recall is
|
||||
preserved); persistent misses are *nominated* through the signal file, leaving
|
||||
pattern authoring to a human pull request. The project score is the optimization
|
||||
reward; the recall/FP gate is a separate hard constraint that is never folded into
|
||||
it — and the same gate runs in CI (`python -m openultrasast.gate`), so a loop
|
||||
result that breached it could never merge. Covered by `tests/test_improve.py` and
|
||||
`tests/test_cli_improve.py`.
|
||||
|
||||
This is the deterministic substrate of the self-improvement loop. When the
|
||||
optional `openultrasast[harnessx]` extra is present, the HarnessX `MetaAgent`
|
||||
becomes the richer *proposer* that plugs into this same validated, gated
|
||||
machinery — the safety contract (bounded levers, replay/novelty/journal gates,
|
||||
hard acceptance gate, byte-for-byte revert) is identical either way.
|
||||
|
||||
Beyond these loops, the `openultrasast-triage` skill lets an agent adjust
|
||||
prompt constraints, retrieval filters, skill routing and benchmark-miss triage.
|
||||
Those `RankingCalibration` fields are already produced and will be consumed
|
||||
directly once the per-family detectors land.
|
||||
|
||||
## Security & hardening
|
||||
|
||||
OpenUltraSAST treats scanned code as **untrusted input and never executes it** (quick
|
||||
and standard modes). Scan artifacts are scrubbed of credentials before they are written
|
||||
(`[hardening] redact_secrets`, on by default), agentic spend and output size are bounded
|
||||
(`[harnessx] max_cost_usd`/`token_threshold`, `[hardening] max_findings`), provider calls
|
||||
retry transient failures with backoff, and missing capabilities degrade visibly in the
|
||||
manifest rather than silently. See [docs/threat-model.md](docs/threat-model.md) for the
|
||||
full trust boundaries and sandbox limits, and [docs/examples.md](docs/examples.md) for
|
||||
end-to-end walkthroughs.
|
||||
- [docs/architecture.md](docs/architecture.md): the scan pipeline, evidence ladder, CWE policy
|
||||
and project score, and the self-improving loops.
|
||||
- [docs/evaluation.md](docs/evaluation.md): independent populations, the qualification gates,
|
||||
pair corpora and the plane increment results.
|
||||
- [docs/decision-engine.md](docs/decision-engine.md): the learned decision engine.
|
||||
- [docs/threat-model.md](docs/threat-model.md): trust boundaries, sandbox and hardening.
|
||||
- [docs/token-ergonomics.md](docs/token-ergonomics.md): model spend at runtime (budgets, reuse,
|
||||
measured costs) and in coding sessions.
|
||||
- [docs/examples.md](docs/examples.md): end-to-end walkthroughs.
|
||||
|
||||
## Development
|
||||
|
||||
@@ -421,6 +241,10 @@ uv run ruff check . # lint
|
||||
uv run ruff format --check . # format
|
||||
uv run mypy src/openultrasast # types
|
||||
uv run python -m openultrasast.gate # standalone detection gate
|
||||
uv run python -m openultrasast.pair_gate # local vuln-vs-fix efficiency + GitHub honesty
|
||||
uv run python dagger/ci.py # full containerized CI pipeline
|
||||
uv run python -m openultrasast.pair_gate # local vuln-vs-fix pairs
|
||||
uv run python dagger/ci.py # containerized CI pipeline
|
||||
```
|
||||
|
||||
Maintainer commands not covered above: `ousast repos` (pinned known-vulnerable checkouts),
|
||||
`ousast model candidates` (what the candidate enumerator can reach) and `ousast learn`
|
||||
(the decision engine's data; see [docs/decision-engine.md](docs/decision-engine.md)).
|
||||
|
||||
@@ -1,3 +1,109 @@
|
||||
# v2.0.0 (2026-10-02)
|
||||
|
||||
A new major release: the agentic plane runs on google/ax, HarnessX is gone, detection decisions move
|
||||
toward a trained-by-memory AI classifier, and the plane memory lives in an S3-compatible store.
|
||||
|
||||
- **Agentic plane on ax** (`ousast plane run|status|doctor|workspaces|remember|alerts-engine|harvest`): tasks
|
||||
run in gVisor sandboxes on Agent Substrate; one Model and one budget per task; token attribution per task
|
||||
and model; credentials only in the start request; per-task egress policies. First increment measured:
|
||||
16/20 declared sites agreed at $0.0205 per candidate (`benchmarks/independent/plane-increment-2.json`).
|
||||
- **HarnessX removed.** Retired config keys fail with a reason; `[harnessx]` warns. Deterministic outputs
|
||||
proven byte-identical (`benchmarks/measurements/2026-09-30-harnessx-removal-equality/`).
|
||||
- **Memory store** (`OUSAST_MEMORY`): local files or any S3-compatible store with versioning, lifecycle,
|
||||
tags and S3 Select; RustFS is the tested server (`docs/rustfs.md`). The store verifies its bucket at
|
||||
startup and never falls back; every row carries its kind's queryable fields. New extra `s3` (boto3).
|
||||
- **Decision engine (in development, not adopted):** an in-house DSPy-style AI classifier with local
|
||||
memory; labels from ground truth; repository-grouped evaluation. Slice 2 on six families: out-of-repo
|
||||
AUC 0.77-0.96 (within-pair 0.72-0.93); BLOCK not offered anywhere; see
|
||||
`benchmarks/measurements/2026-10-02-decision-engine-slice-2/record.json` and `docs/memory-and-detection.md`.
|
||||
- **Improvement from memory:** `ousast improve --memory`; the loop as a plane Run.
|
||||
- **PHP quick-mode rules** (in-sample measurement only) beside the Joern engine; framework knowledge
|
||||
tagged as optional priors (`ruleset/frameworks.toml`).
|
||||
- **Populations:** v3 (PHP, 15 cases) frozen and untouched as the one-time final check; 97 advisory-fix
|
||||
pairs added for negatives; licences of published cases verified (v1's OpenCVE case is BUSL-1.1).
|
||||
- **Removed legacy:** dead modules (`model.judge`, `model.execution`, `model.calibrate`,
|
||||
`stage_processors`, `slot_contract`); unread config keys `[dynamic]`, `[evidence]`, `[variants]`,
|
||||
`[models] patcher` retired (`resolved_config.json` loses those sections); reference-only scripts under
|
||||
`benchmarks/archive/`; the module audit now classifies by real imports.
|
||||
- **Docs:** MkDocs site (`mkdocs serve`), including token ergonomics and the RustFS setup.
|
||||
|
||||
Detailed notes for this release follow.
|
||||
|
||||
## PHP quick-mode rules (2026-09-30)
|
||||
|
||||
`ruleset/php/rules.toml` adds 18 PHP rules (SQL injection, command injection, file inclusion,
|
||||
path, open redirect, SSRF; code eval, unserialize and echo XSS as `shadow`). They were written
|
||||
on the development corpus, so their numbers are in-sample only
|
||||
(`benchmarks/measurements/2026-09-30-php-quick-rules/measurement.json`); PHP is not part of
|
||||
the detection gate. `ousast plane alerts-engine` produces a Run's `alerts` for PHP from the
|
||||
Joern engine image on the host.
|
||||
|
||||
## Framework knowledge as optional priors (2026-09-30)
|
||||
|
||||
Rule and taint-fact entries that know a framework or library carry a `framework`/`library`
|
||||
tag naming a row of `ruleset/frameworks.toml`; the loaders take `priors` (`"all"` by default,
|
||||
unchanged behaviour; `"off"`; or a set of ids). Scan output is unchanged.
|
||||
|
||||
## Decision engine, in development (2026-10-01)
|
||||
|
||||
The learned decision engine (signals from every instrument, labels from ground truth, an AI
|
||||
classifier with local memory compiled in house) is in development and **not adopted**: no scan,
|
||||
pre-push check or report uses it. New maintainer commands: `ousast learn labels|memory|compile|
|
||||
evaluate|curve|audit-leaks` and `ousast plane harvest`; new plane tasks `features` and `roles`;
|
||||
the `advisory-fixes` pair source (97 advisory fix commits). First slice (injection, 110
|
||||
out-of-repository candidates): ADVISORY recall 0.90, precision 0.54, BLOCK not offered
|
||||
(`benchmarks/measurements/2026-10-01-decision-engine-injection-slice/record.json`;
|
||||
[docs/decision-engine.md](docs/decision-engine.md)).
|
||||
|
||||
## Documentation (2026-10-02)
|
||||
|
||||
The README is reduced to user-facing material; scan internals moved to
|
||||
[docs/architecture.md](docs/architecture.md) and measured results to
|
||||
[docs/evaluation.md](docs/evaluation.md). Removed references to commands that no longer exist
|
||||
(`ousast mechanisms export`, `ousast pairs --loo`) and corrected the description of `deep`
|
||||
mode, which runs sandboxed regression snippets when Docker is available.
|
||||
## Unread config keys retired (2026-10-02)
|
||||
These keys loaded but nothing ever read them. They now fail with `RetiredConfigError` (the
|
||||
CLI prints it and exits 2), naming why. Remove them from your config:
|
||||
- `[models] patcher`: no patching stage calls a model.
|
||||
- `[dynamic] enabled`, `[dynamic] network_scope`: no dynamic or deep tier exists yet.
|
||||
- `[evidence] minimum_report_verified`, `minimum_exploit`, `minimum_patch`: evidence tiers
|
||||
come from the program model's ladder, not from configuration.
|
||||
- `[variants] enabled`, `[variants] max_mechanisms`: the variant search they configured was
|
||||
deleted earlier.
|
||||
**Output shape.** `resolved_config.json` no longer has the `dynamic`, `evidence` and
|
||||
`variants` sections or `models.patcher`. `harness.json` keeps its shape:
|
||||
`model_roles.patcher` is always `null`, like `model_roles.verifier`.
|
||||
|
||||
## HarnessX removed; the ax plane is the agentic path (2026-09-30)
|
||||
|
||||
HarnessX, the optional agentic extra (`openultrasast[harnessx]`), is removed with its code,
|
||||
packaging extra and lock entries (retired 2026-09-30). Agentic work runs on the ax plane:
|
||||
`ousast plane run|status|doctor|remember`, with one Model and one `usd`/`calls` budget per
|
||||
task, deny-by-default egress per task, and the provider key sent only in the task's start
|
||||
request. See [ops/ax/README.md](ops/ax/README.md). Deterministic outputs (pair replay,
|
||||
quick benchmark, standard scan, `improve --dry-run`) are byte-identical before and after.
|
||||
|
||||
Retired config keys (retired 2026-09-30):
|
||||
|
||||
- `[models] verifier` fails with `RetiredConfigError` (the CLI prints it and exits 2): LLM
|
||||
verification runs on the plane (`verify` + `agree`). Remove the key.
|
||||
- `[fusion] panel_model` and `[fusion] decider_model` fail the same way: fusion is
|
||||
deterministic; independent LLM agreement is the plane's `agree` task. Remove the keys.
|
||||
- A `[harnessx]` section (retired 2026-09-30) loads with one warning naming the plane and is
|
||||
otherwise ignored.
|
||||
|
||||
**Silent change for `[models] hunter`.** If you set `[models] hunter` and had the extra
|
||||
installed, a standard scan ran two LLM hunters: the MAP-stage tool hunter and the
|
||||
HarnessX hunter pool (findings with ids `hx-hunter:<path>:<line>`). The key is still
|
||||
valid and still enables the tool hunter, so nothing fails and nothing warns, but the
|
||||
second hunter's findings no longer appear. Repository-wide LLM hunting runs on the plane
|
||||
(`verify` passes and `agree`).
|
||||
|
||||
The LLM judge, the LLM fusion panels and the in-loop hosting of deterministic stages are
|
||||
retired without a local replacement. `ousast improve --memory` now proposes rule-status
|
||||
edits from the plane memory, through the unchanged validator and gate.
|
||||
|
||||
# v1.2.0-alpha.1
|
||||
|
||||
Experimental pre-push security safety net for AI-accelerated development. Python package
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# Archived benchmark scripts
|
||||
|
||||
Reference-only scripts, moved here with `git mv` on 2026-10-02 (legacy cleanup) so their history
|
||||
follows them. Nothing in `src/` imports them and no current workflow runs them. They stay because
|
||||
committed records cite them and their numbers: to re-run a record, check out the commit the record
|
||||
names rather than running these copies against today's package.
|
||||
|
||||
The old paths still appear inside committed records and in some docstrings. [paths.md](paths.md)
|
||||
maps each old path to the new one. The records themselves are not rewritten.
|
||||
|
||||
| script | what it was | records that cite it | last recorded run |
|
||||
|---|---|---|---|
|
||||
| `verify_batched.py` | Stage-2 verifier, second design: triage per file, verify per file, report what two passes agree on. Replaced by the plane's `verify` + `agree` tasks. | `independent/results-v2-model-pipeline.json`, `independent/verifier-batched-check-2026-09-29.json`, `independent/plane-increment-2.json` (parity baseline) | `072a102` (named in `results-v2-model-pipeline.json`) |
|
||||
| `score_batched.py` | Exploratory scorer for the batched verifier: sites anchored to the candidate, matched by `evaluate.matches_v2`. | `independent/verifier-batched-check-2026-09-29.json`, `independent/plane-increment-2.json`; `tests/test_plane_scoring_parity.py` still loads it as the reference | added in `6acd354` (2026-09-29); the parity test replays it on hosts with the recorded scans |
|
||||
| `verify_sinks.py` | The first stage-2 verifier, one candidate at a time. `verify_batched.py` imports `ACCOUNT_ERRORS` and `_Recording` from it. | `independent/verifier-check-2026-09-29.json` | the known-answer check after `041d871` |
|
||||
| `model_sinks.py` | Stage-1 source-only model classification of sinks. Moved into the package as `plane/tasks/roles.py`. | `independent/selection-v2-model-sinks.json`, `measurements/2026-10-01-decision-engine-harvest/record.json`, both `2026-10-01-decision-engine-leak-audit` records | `05d5699` (named in `selection-v2-model-sinks.json`) |
|
||||
| `sink_candidates.py` | Diagnostic: vocabulary-declared sink candidates per case. Its declaration patterns live on in `plane/tasks/repo_facts.py`. | `independent/results-v2-model-pipeline.json` ("vocabulary sink-first: 2/17") | no commit recorded |
|
||||
| `hunt.py` | The v2 tool-hunter run over the population (protocol `hunter-protocol-v2.md`, including its `FAMILY_WORDS` table). Replaced by the plane's `verify` / `agree` tasks. | `independent/hunter-protocol-v2.md`, `independent/results-v2-hunter.json` (a label source in `learn/sources.toml`) | `6bfb05f`, the commit that introduced the protocol, which pins the runner |
|
||||
| `harnessx_removal_equality.py` | One-off equality proof for `harnessx-removal` (record and compare deterministic outputs). The spec is closed. `tests/test_harnessx_removal_equality.py` still checks that the comparison detects differences. | `measurements/2026-09-30-harnessx-removal-baseline/`, `measurements/2026-09-30-harnessx-removal-equality/` | baseline `ca5caf7`, candidate `b9afb66` |
|
||||
|
||||
The six `independent` scripts still read the population, `evaluate.py` and the results from
|
||||
`benchmarks/independent/`. Their `HERE` points there, so outputs land where they always did.
|
||||
`evaluate.py` and `freeze.py` are current and were not moved.
|
||||
|
||||
`verify_sinks.py --help` fails with an argparse `%` formatting error. That bug was already there
|
||||
before the move. The script still imports cleanly.
|
||||
@@ -0,0 +1,396 @@
|
||||
"""Equality proof for the HarnessX removal (spec harnessx-removal, design section 3, Requirement 3).
|
||||
|
||||
``record`` exports a frozen source tree, runs the deterministic commands with no model reachable and the
|
||||
``harnessx`` extra absent, and stores their normalized outputs. ``compare`` diffs two such records and exits
|
||||
non-zero on the first difference outside the exclusion list, naming the file and key.
|
||||
|
||||
Usage:
|
||||
python benchmarks/archive/harnessx_removal_equality.py record <out-dir> [--tree-ish HEAD] [--workdir DIR]
|
||||
python benchmarks/archive/harnessx_removal_equality.py compare <baseline-dir> <candidate-dir> \\
|
||||
[--suite-map map.json] [--record comparison.json]
|
||||
|
||||
The instrument checks come first and fail loudly: ``harnessx`` must not be importable, ``tree_sitter`` must be,
|
||||
every child must import ``openultrasast`` from the export, every input manifest must be non-empty, every
|
||||
command must exit 0, and every command must produce non-zero counts (pairs, TPs, expected findings, suite tests).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import xml.etree.ElementTree as ET
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2] # archived under benchmarks/archive/
|
||||
|
||||
# Design section 3 "Comparison": timestamps, durations and run identifiers are excluded; nothing else is.
|
||||
EXCLUDED_KEYS = frozenset({"runtime_seconds", "scan_id", "timestamp", "started", "finished", "created_at", "generated_at"})
|
||||
EXCLUDED_SUFFIXES = ("_seconds", "_ms")
|
||||
RUN_ID = re.compile(r"\d{8}T\d{6}Z-[0-9a-f]{8}")
|
||||
EXCLUSIONS = [
|
||||
f"JSON keys: {', '.join(sorted(EXCLUDED_KEYS))}",
|
||||
f"JSON keys ending in: {', '.join(EXCLUDED_SUFFIXES)}",
|
||||
"run directory names / run ids (<timestamp>-<8 hex>, benchmark.py create_benchmark_run, run.py create_scan_run) -> <RUN_ID>",
|
||||
"trace/events.jsonl as a whole (timings); never copied",
|
||||
"the export's absolute root path -> <ROOT>; the children's TMPDIR -> <TMP>",
|
||||
"suite: only the outcome per test id is kept (junit durations dropped)",
|
||||
"env.json is a provenance record (commit, freeze hash, instrument checks), not compared",
|
||||
]
|
||||
|
||||
INPUTS = (
|
||||
"benchmarks/pairs/catalog.toml",
|
||||
"benchmarks/manifests/python-vulnerable.toml",
|
||||
"benchmarks/manifests/java-spring-boot-vulnerable.toml",
|
||||
)
|
||||
SCAN_KEEP = ("findings.json", "verification.json", "fusion.json", "report.sarif", "harness.json", "manifest.json")
|
||||
BENCH_KEEP = ("benchmark_result.json", "calibration_records.json", "external_baseline_deltas.json")
|
||||
|
||||
|
||||
class InstrumentError(SystemExit):
|
||||
def __init__(self, message: str) -> None:
|
||||
super().__init__(f"instrument check failed: {message}")
|
||||
|
||||
|
||||
def normalize(value: Any, root: str, tmp: str) -> Any:
|
||||
"""Drop excluded keys recursively and scrub the export root, TMPDIR and run ids from strings."""
|
||||
if isinstance(value, dict):
|
||||
return {k: normalize(v, root, tmp) for k, v in value.items() if not _excluded(k)}
|
||||
if isinstance(value, list):
|
||||
return [normalize(v, root, tmp) for v in value]
|
||||
if isinstance(value, str):
|
||||
return scrub(value, root, tmp)
|
||||
return value
|
||||
|
||||
|
||||
def _excluded(key: str) -> bool:
|
||||
return key in EXCLUDED_KEYS or key.endswith(EXCLUDED_SUFFIXES)
|
||||
|
||||
|
||||
def scrub(text: str, root: str, tmp: str) -> str:
|
||||
text = text.replace(tmp, "<TMP>").replace(root, "<ROOT>")
|
||||
return RUN_ID.sub("<RUN_ID>", text)
|
||||
|
||||
|
||||
def dump(value: Any) -> str:
|
||||
return json.dumps(value, indent=1, sort_keys=True) + "\n"
|
||||
|
||||
|
||||
def _child_env(root: Path, tmp: Path) -> dict[str, str]:
|
||||
env = {k: v for k, v in os.environ.items() if not k.endswith("_API_KEY") and k not in {"PYTHONPATH", "VIRTUAL_ENV"}}
|
||||
env.update(OPENULTRASAST_SKIP_DOTENV="1", PYTHONPATH=str(root / "src"), TMPDIR=str(tmp), PYTHONHASHSEED="0")
|
||||
return env
|
||||
|
||||
|
||||
def _run(cmd: list[str], root: Path, env: dict[str, str], label: str) -> str:
|
||||
proc = subprocess.run(cmd, cwd=root, env=env, capture_output=True, text=True, check=False)
|
||||
if proc.returncode != 0:
|
||||
tail = (proc.stdout + proc.stderr)[-2000:]
|
||||
raise InstrumentError(f"{label} exited {proc.returncode}: {' '.join(cmd)}\n{tail}")
|
||||
return proc.stdout
|
||||
|
||||
|
||||
def _field(stdout: str, name: str) -> str:
|
||||
for line in stdout.splitlines():
|
||||
if line.startswith(f"{name}="):
|
||||
return line.partition("=")[2]
|
||||
raise InstrumentError(f"no {name}= line in output")
|
||||
|
||||
|
||||
def _freeze_hash() -> str:
|
||||
uv = shutil.which("uv")
|
||||
if uv is None:
|
||||
raise InstrumentError("uv not found; cannot hash the environment")
|
||||
out = subprocess.run([uv, "pip", "freeze", "--python", sys.executable], capture_output=True, text=True, check=True).stdout
|
||||
return hashlib.sha256(out.encode()).hexdigest()
|
||||
|
||||
|
||||
def _suite(xml_path: Path) -> dict[str, str]:
|
||||
outcomes: dict[str, str] = {}
|
||||
for case in ET.parse(xml_path).getroot().iter("testcase"):
|
||||
test_id = f"{case.get('classname')}::{case.get('name')}"
|
||||
kinds = {child.tag for child in case}
|
||||
outcome = next((k for k in ("error", "failure", "skipped") if k in kinds), "passed")
|
||||
outcomes[test_id] = outcome
|
||||
return dict(sorted(outcomes.items()))
|
||||
|
||||
|
||||
def record(out: Path, tree_ish: str, workdir: Path) -> None:
|
||||
if importlib.util.find_spec("harnessx") is not None:
|
||||
raise InstrumentError("harnessx is importable; re-sync the venv without the extra")
|
||||
if importlib.util.find_spec("tree_sitter") is None:
|
||||
raise InstrumentError("tree_sitter is not importable; the semantic extra is missing")
|
||||
workdir.mkdir(parents=True, exist_ok=True)
|
||||
root = Path(tempfile.mkdtemp(prefix="export-", dir=workdir)).resolve()
|
||||
tmp = Path(tempfile.mkdtemp(prefix="tmp-", dir=workdir)).resolve()
|
||||
archive = subprocess.run(["git", "-C", str(REPO), "archive", tree_ish], capture_output=True, check=True).stdout
|
||||
subprocess.run(["tar", "-x", "-C", str(root)], input=archive, check=True)
|
||||
env = _child_env(root, tmp)
|
||||
rs, ts = str(root), str(tmp)
|
||||
py = sys.executable
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
checks: dict[str, Any] = {}
|
||||
|
||||
module = _run([py, "-c", "import openultrasast; print(openultrasast.__file__)"], root, env, "import probe").strip()
|
||||
if not module.startswith(rs):
|
||||
raise InstrumentError(f"openultrasast imported from {module}, not the export {rs}")
|
||||
checks["openultrasast_module"] = scrub(module, rs, ts)
|
||||
inputs = {name: (root / name).stat().st_size for name in INPUTS}
|
||||
if not all(inputs.values()):
|
||||
raise InstrumentError(f"empty input manifest: {inputs}")
|
||||
print(f"export={root} module={module} inputs={inputs}", flush=True)
|
||||
|
||||
# 1. Full suite, outcome per test id.
|
||||
junit = tmp / "suite.xml"
|
||||
_run([py, "-m", "pytest", "-q", "-p", "no:cacheprovider", f"--junitxml={junit}"], root, env, "pytest")
|
||||
suite = _suite(junit)
|
||||
counts = {k: sum(1 for v in suite.values() if v == k) for k in ("passed", "skipped", "failure", "error")}
|
||||
if counts["passed"] == 0:
|
||||
raise InstrumentError("suite recorded no passing tests")
|
||||
(out / "suite.json").write_text(dump(suite))
|
||||
checks["suite"] = counts
|
||||
print(f"suite={counts}", flush=True)
|
||||
|
||||
# 2. Pair replay (offline, vendored pairs only).
|
||||
pairs = normalize(json.loads(_run([py, "-m", "openultrasast.cli", "pairs", "--json"], root, env, "pairs")), rs, ts)
|
||||
checks["pairs"] = _pair_counts(pairs)
|
||||
(out / "pairs.json").write_text(dump(pairs))
|
||||
print(f"pairs={checks['pairs']}", flush=True)
|
||||
|
||||
# 3. Quick benchmark replay.
|
||||
empty = tmp / "empty.toml"
|
||||
empty.write_text("")
|
||||
cmd = [
|
||||
py,
|
||||
"-m",
|
||||
"openultrasast.cli",
|
||||
"benchmark",
|
||||
"benchmarks/manifests/python-vulnerable.toml",
|
||||
"--mode",
|
||||
"quick",
|
||||
"--config",
|
||||
str(empty),
|
||||
]
|
||||
stdout = _run(cmd, root, env, "benchmark")
|
||||
expected = int(_field(stdout, "benchmark_expected"))
|
||||
if expected <= 0:
|
||||
raise InstrumentError("benchmark expected no findings")
|
||||
checks["benchmark"] = {k: int(_field(stdout, f"benchmark_{k}")) for k in ("expected", "matched", "missed")}
|
||||
checks["benchmark"]["findings"] = int(_field(stdout, "findings"))
|
||||
_keep(Path(_field(stdout, "benchmark_run_dir")), BENCH_KEEP, out / "benchmark", rs, ts, checks, "benchmark_files")
|
||||
(out / "benchmark" / "stdout.txt").write_text(scrub(stdout, rs, ts))
|
||||
print(f"benchmark={checks['benchmark']}", flush=True)
|
||||
|
||||
# 4. One standard-mode scan (verify + fusion, no model configured).
|
||||
cmd = [py, "-m", "openultrasast.cli", "scan", "benchmarks/fixtures/python-vulnerable", "--mode", "standard", "--config", str(empty)]
|
||||
stdout = _run(cmd, root, env, "scan")
|
||||
checks["scan"] = {"findings": int(_field(stdout, "findings")), "file_targets": int(_field(stdout, "file_targets"))}
|
||||
if checks["scan"]["file_targets"] <= 0:
|
||||
raise InstrumentError("standard scan saw no file targets")
|
||||
_keep(Path(_field(stdout, "run_dir")), SCAN_KEEP, out / "scan", rs, ts, checks, "scan_files")
|
||||
(out / "scan" / "stdout.txt").write_text(scrub(stdout, rs, ts))
|
||||
print(f"scan={checks['scan']}", flush=True)
|
||||
|
||||
# 5. improve --dry-run; the shim keeps the dry-run journal the CLI writes to a throwaway directory.
|
||||
shim_out = tmp / "improve"
|
||||
cmd = [py, str(Path(__file__).resolve()), "_improve", str(shim_out), "benchmarks/manifests/java-spring-boot-vulnerable.toml"]
|
||||
stdout = _run(cmd, root, env, "improve")
|
||||
(out / "improve").mkdir(exist_ok=True)
|
||||
(out / "improve" / "stdout.txt").write_text(scrub(stdout, rs, ts))
|
||||
# A round with no proposals writes no journal (evolve.py returns before append_round); its absence is
|
||||
# recorded and compared like any file, so a journal appearing after the removal is a difference.
|
||||
rounds = [line for line in stdout.splitlines() if line.startswith("round ")]
|
||||
if not rounds or "rounds=" not in stdout:
|
||||
raise InstrumentError("improve --dry-run printed no round")
|
||||
journal = shim_out / "improve_journal.json"
|
||||
checks["improve"] = {"rounds": len(rounds), "journal_bytes": journal.stat().st_size if journal.is_file() else None}
|
||||
if journal.is_file():
|
||||
journal_data = normalize(json.loads(journal.read_text()), rs, ts)
|
||||
(out / "improve" / "improve_journal.json").write_text(dump(journal_data))
|
||||
print(f"improve={checks['improve']}", flush=True)
|
||||
|
||||
head = subprocess.run(["git", "-C", str(REPO), "rev-parse", tree_ish], capture_output=True, text=True, check=True).stdout.strip()
|
||||
env_record = {
|
||||
"tree_ish": head,
|
||||
"freeze_sha256": _freeze_hash(),
|
||||
"harnessx_importable": False,
|
||||
"tree_sitter": True,
|
||||
"python": sys.version.split()[0],
|
||||
"child_env": {"OPENULTRASAST_SKIP_DOTENV": "1", "PYTHONHASHSEED": "0", "api_keys": "every *_API_KEY unset", "config": "empty"},
|
||||
"inputs": inputs,
|
||||
"instrument_checks": checks,
|
||||
"exclusions": EXCLUSIONS,
|
||||
}
|
||||
(out / "env.json").write_text(dump(env_record))
|
||||
(out / "README").write_text("Excluded from comparison: " + "; ".join(EXCLUSIONS) + "\n")
|
||||
shutil.rmtree(root, ignore_errors=True)
|
||||
shutil.rmtree(tmp, ignore_errors=True)
|
||||
print(f"recorded {out}")
|
||||
|
||||
|
||||
def _pair_counts(pairs: Any) -> dict[str, int]:
|
||||
overall = pairs.get("overall", {}) if isinstance(pairs, dict) else {}
|
||||
counts = {k: int(overall.get(k, 0)) for k in ("pairs", "detected_vuln", "pair_correct", "labeled_expected", "labeled_matched")}
|
||||
counts["outcomes"] = len(pairs.get("outcomes", [])) if isinstance(pairs, dict) else 0
|
||||
if counts["pairs"] <= 0 or counts["detected_vuln"] <= 0 or counts["outcomes"] <= 0:
|
||||
raise InstrumentError(f"pair replay saw {counts}")
|
||||
return counts
|
||||
|
||||
|
||||
def _keep(run_dir: Path, names: tuple[str, ...], dest: Path, rs: str, ts: str, checks: dict[str, Any], key: str) -> None:
|
||||
if not run_dir.is_dir():
|
||||
raise InstrumentError(f"run directory missing: {run_dir}")
|
||||
dest.mkdir(parents=True, exist_ok=True)
|
||||
present = {}
|
||||
for name in names:
|
||||
source = run_dir / name
|
||||
if source.is_file():
|
||||
present[name] = source.stat().st_size
|
||||
(dest / name).write_text(dump(normalize(json.loads(source.read_text()), rs, ts)))
|
||||
checks[key] = present
|
||||
|
||||
|
||||
def _improve_shim(out: Path, manifest: str) -> int:
|
||||
"""Run ``ousast improve --dry-run --no-pair-gate`` and copy the throwaway journal out before cleanup."""
|
||||
from openultrasast import cli
|
||||
|
||||
original = tempfile.TemporaryDirectory
|
||||
|
||||
class Keeping(original): # type: ignore[misc,valid-type]
|
||||
def cleanup(self) -> None:
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
for name in ("improve_journal.json",):
|
||||
if (Path(self.name) / name).is_file():
|
||||
shutil.copy2(Path(self.name) / name, out / name)
|
||||
super().cleanup()
|
||||
|
||||
def __exit__(self, *exc: object) -> None:
|
||||
self.cleanup()
|
||||
|
||||
cli.tempfile.TemporaryDirectory = Keeping # type: ignore[misc]
|
||||
try:
|
||||
return int(cli.main(["improve", manifest, "--dry-run", "--no-pair-gate"]) or 0)
|
||||
finally:
|
||||
cli.tempfile.TemporaryDirectory = original # type: ignore[misc]
|
||||
|
||||
|
||||
def compare(baseline: Path, candidate: Path, suite_map: dict[str, Any]) -> list[str]:
|
||||
"""Return every difference outside the exclusion list; empty means equal."""
|
||||
diffs: list[str] = []
|
||||
diffs += _compare_suite(
|
||||
json.loads((baseline / "suite.json").read_text()), json.loads((candidate / "suite.json").read_text()), suite_map
|
||||
)
|
||||
files = sorted(
|
||||
{p.relative_to(baseline).as_posix() for p in baseline.rglob("*") if p.is_file()}
|
||||
| {p.relative_to(candidate).as_posix() for p in candidate.rglob("*") if p.is_file()}
|
||||
)
|
||||
for name in files:
|
||||
if name in {"suite.json", "env.json", "README", "comparison.json", "determinism.json"}:
|
||||
continue
|
||||
a, b = baseline / name, candidate / name
|
||||
if not a.is_file() or not b.is_file():
|
||||
diffs.append(f"{name}: present only in {'candidate' if b.is_file() else 'baseline'}")
|
||||
continue
|
||||
if a.read_bytes() == b.read_bytes():
|
||||
continue
|
||||
if name.endswith((".json", ".sarif")):
|
||||
diffs.append(f"{name}: {_first_json_diff(json.loads(a.read_text()), json.loads(b.read_text()), '$')}")
|
||||
else:
|
||||
diffs.append(f"{name}: first differing line {_first_line_diff(a.read_text(), b.read_text())}")
|
||||
return diffs
|
||||
|
||||
|
||||
def _compare_suite(a: dict[str, str], b: dict[str, str], suite_map: dict[str, Any]) -> list[str]:
|
||||
deleted = set(suite_map.get("deleted", ()))
|
||||
renamed: dict[str, str] = dict(suite_map.get("renamed", {}))
|
||||
added = set(suite_map.get("added", ())) | set(renamed.values())
|
||||
diffs = []
|
||||
for test_id, outcome in a.items():
|
||||
if test_id in deleted:
|
||||
continue
|
||||
target = renamed.get(test_id, test_id)
|
||||
if target not in b:
|
||||
diffs.append(f"suite.json: {test_id} missing from candidate")
|
||||
elif b[target] != outcome:
|
||||
diffs.append(f"suite.json: {test_id} {outcome} -> {b[target]}")
|
||||
diffs += [f"suite.json: {t} new in candidate" for t in b if t not in a and t not in added]
|
||||
return diffs
|
||||
|
||||
|
||||
def _first_json_diff(a: Any, b: Any, path: str) -> str:
|
||||
if type(a) is not type(b):
|
||||
return f"{path}: type {type(a).__name__} -> {type(b).__name__}"
|
||||
if isinstance(a, dict):
|
||||
for key in sorted(set(a) | set(b)):
|
||||
if key not in a or key not in b:
|
||||
return f"{path}.{key}: present only in {'candidate' if key in b else 'baseline'}"
|
||||
if a[key] != b[key]:
|
||||
return _first_json_diff(a[key], b[key], f"{path}.{key}")
|
||||
if isinstance(a, list):
|
||||
if len(a) != len(b):
|
||||
return f"{path}: length {len(a)} -> {len(b)}"
|
||||
for i, (x, y) in enumerate(zip(a, b, strict=True)):
|
||||
if x != y:
|
||||
return _first_json_diff(x, y, f"{path}[{i}]")
|
||||
return f"{path}: {json.dumps(a)[:120]} -> {json.dumps(b)[:120]}"
|
||||
|
||||
|
||||
def _first_line_diff(a: str, b: str) -> str:
|
||||
la, lb = a.splitlines(), b.splitlines()
|
||||
for i, (x, y) in enumerate(zip(la, lb, strict=False), 1):
|
||||
if x != y:
|
||||
return f"{i}: {x[:120]!r} -> {y[:120]!r}"
|
||||
return f"line count {len(la)} -> {len(lb)}"
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
rec = sub.add_parser("record")
|
||||
rec.add_argument("out", type=Path)
|
||||
rec.add_argument("--tree-ish", default="HEAD", help="git tree-ish to export (a commit, or `git write-tree` for the index)")
|
||||
rec.add_argument("--workdir", type=Path, default=Path(tempfile.gettempdir()) / "hxr-equality")
|
||||
cmp_ = sub.add_parser("compare")
|
||||
cmp_.add_argument("baseline", type=Path)
|
||||
cmp_.add_argument("candidate", type=Path)
|
||||
cmp_.add_argument("--suite-map", type=Path, help='JSON {"deleted": [...], "renamed": {old: new}, "added": [...]}')
|
||||
cmp_.add_argument("--record", type=Path, help="write the comparison result here as JSON")
|
||||
shim = sub.add_parser("_improve")
|
||||
shim.add_argument("out", type=Path)
|
||||
shim.add_argument("manifest")
|
||||
args = parser.parse_args(argv)
|
||||
if args.command == "record":
|
||||
record(args.out.resolve(), args.tree_ish, args.workdir.resolve())
|
||||
return 0
|
||||
if args.command == "_improve":
|
||||
return _improve_shim(args.out, args.manifest)
|
||||
suite_map = json.loads(args.suite_map.read_text()) if args.suite_map else {}
|
||||
diffs = compare(args.baseline, args.candidate, suite_map)
|
||||
if args.record:
|
||||
args.record.write_text(
|
||||
dump(
|
||||
{
|
||||
"baseline": str(args.baseline),
|
||||
"candidate": str(args.candidate),
|
||||
"equal": not diffs,
|
||||
"differences": diffs,
|
||||
"exclusions": EXCLUSIONS,
|
||||
}
|
||||
)
|
||||
)
|
||||
for line in diffs[:20]:
|
||||
print(line)
|
||||
print(f"equal={not diffs} differences={len(diffs)}")
|
||||
return 1 if diffs else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,192 @@
|
||||
"""Run the UNCHANGED tool hunter over an independent population's pins, scoreable by the evaluation protocol.
|
||||
|
||||
The hunter (`tool_hunter.run_tool_hunter`, default prompt, default tools, default step budget) is handed the
|
||||
regions the ranker selects for the case's family -- the same discovery and ranking the static scan starts from,
|
||||
top `--max-regions` by rank, in groups of `--per-hunt` -- and nothing else: not the fix diff, not the advisory,
|
||||
not the declared sites. Each pin's result is written in the scan-result shape `evaluate.py score` reads, into
|
||||
`~/ousast-results/independent-<vN>-hunter/scans/`, so the same pre-registered matching scores it.
|
||||
|
||||
Model calls go to the resolved hunter client (DeepSeek when its key is set); the run stops, and says so, when
|
||||
the recorded spend reaches `--budget-usd`.
|
||||
|
||||
Usage:
|
||||
python benchmarks/archive/hunt.py --population population-v2.toml [--only ID,...] [--pins a,b]
|
||||
python benchmarks/archive/hunt.py --population population-v2.toml --score
|
||||
python benchmarks/archive/hunt.py --pilot-root <checkout> --family injection # a development case
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import dataclasses
|
||||
import json
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
|
||||
HERE = Path(__file__).resolve().parents[1] / "independent"
|
||||
sys.path.insert(0, str(HERE))
|
||||
|
||||
import evaluate # noqa: E402
|
||||
|
||||
from openultrasast.mapping import analyze_entry_points # noqa: E402
|
||||
from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL # noqa: E402
|
||||
from openultrasast.model.regions import regions_for # noqa: E402
|
||||
from openultrasast.model.shipped import declared_sources # noqa: E402
|
||||
from openultrasast.preprocess import build_file_target, enumerate_source_files # noqa: E402
|
||||
|
||||
# The model names a family in its own words; the protocol scores three. Fixed before any v2 pin is hunted.
|
||||
FAMILY_WORDS = {
|
||||
"untrusted_destination": ("ssrf", "server-side request", "open redirect", "redirect", "url fetch", "request forgery"),
|
||||
"config_secrets": ("cors", "debug", "secret", "hardcoded", "hard-coded", "credential", "misconfig", "configuration", "tls", "certificate"),
|
||||
"injection": ("inject", "sql", "sqli", "nosql", "command", "rce", "code execution", "eval", "template", "ssti", "xss", "deserializ", "ldap", "xpath"),
|
||||
}
|
||||
|
||||
|
||||
def family_of(raw: str, title: str) -> str:
|
||||
text = f"{raw} {title}".lower()
|
||||
for family, words in FAMILY_WORDS.items():
|
||||
if any(word in text for word in words):
|
||||
return family
|
||||
return "other"
|
||||
|
||||
|
||||
def hotspots_for(root: Path, family: str, max_regions: int) -> list:
|
||||
from openultrasast.complexity.map import Hotspot
|
||||
|
||||
files = enumerate_source_files(root)
|
||||
if not files or not sum(p.stat().st_size for p in files if p.is_file()):
|
||||
raise SystemExit(f"read nothing under {root}")
|
||||
targets = [build_file_target(root, p) for p in files]
|
||||
regions = [
|
||||
dataclasses.replace(r, families=tuple(f for f in r.families if f == family))
|
||||
for r in regions_for(analyze_entry_points(root, targets), targets, shipped=declared_sources(root))
|
||||
]
|
||||
ranked = sorted((r for r in regions if r.families), key=lambda r: (-r.rank, r.path, r.function or ""))[:max_regions]
|
||||
return [
|
||||
Hotspot(
|
||||
path=r.path,
|
||||
function_name=r.function or None,
|
||||
score=float(r.rank),
|
||||
band="ranker",
|
||||
signals={},
|
||||
rationale=f"ranked region offering {family}",
|
||||
test_hint=None,
|
||||
inventory_finding_ids=(),
|
||||
)
|
||||
for r in ranked
|
||||
]
|
||||
|
||||
|
||||
def _spent(client: object) -> float:
|
||||
cost = getattr(client, "cost_usd", None)
|
||||
return float(cost()) if callable(cost) else 0.0
|
||||
|
||||
|
||||
def hunt(root: Path, family: str, args: argparse.Namespace, spent: list[float]) -> dict:
|
||||
from openultrasast import tool_hunter
|
||||
|
||||
client = tool_hunter.resolve_hunter_client()
|
||||
if client is None:
|
||||
raise SystemExit("no hunter client: set DEEPSEEK_API_KEY")
|
||||
started = time.monotonic()
|
||||
hotspots = hotspots_for(root, family, args.max_regions)
|
||||
groups = [hotspots[i : i + args.per_hunt] for i in range(0, len(hotspots), args.per_hunt)]
|
||||
findings, done, errors = [], 0, []
|
||||
for group in groups:
|
||||
if sum(spent) + _spent(client) >= args.budget_usd:
|
||||
errors.append("budget_exhausted")
|
||||
break
|
||||
try:
|
||||
found = tool_hunter.run_tool_hunter(root, group, client=client, model=args.model, max_steps=tool_hunter.DEFAULT_MAX_STEPS)
|
||||
done += 1
|
||||
except Exception as error: # noqa: BLE001 -- recorded per hunt; a pin with no completed hunt is an instrument failure
|
||||
errors.append(f"{type(error).__name__}: {str(error)[:200]}")
|
||||
continue
|
||||
for f in found:
|
||||
raw = next((t.split(":", 1)[1] for t in f.tags if t.startswith("family:")), "")
|
||||
findings.append(
|
||||
{
|
||||
"site": f"{f.path}:{f.line or 0}:{f.function_name or ''}",
|
||||
"family": family_of(raw, f.title),
|
||||
"model_family": raw,
|
||||
"title": f.title,
|
||||
"witness": f.rationale[:400],
|
||||
}
|
||||
)
|
||||
cost = _spent(client)
|
||||
spent.append(cost)
|
||||
return {
|
||||
"root": str(root),
|
||||
"families": [family],
|
||||
"model": args.model,
|
||||
"seconds": round(time.monotonic() - started, 1),
|
||||
"questions": len(groups),
|
||||
"completed": done,
|
||||
"completed_regions": sorted({f"{h.path}:{h.function_name or ''}" for g in groups[:done] for h in g}),
|
||||
"hunt_errors": errors,
|
||||
"cost_usd": round(cost, 4),
|
||||
"findings": sorted(findings, key=lambda f: f["site"]),
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--population", type=Path, default=evaluate.POPULATION)
|
||||
parser.add_argument("--only", default="")
|
||||
parser.add_argument("--pins", default="")
|
||||
parser.add_argument("--pilot-root", type=Path)
|
||||
parser.add_argument("--family", default="injection")
|
||||
parser.add_argument("--model", default=DEFAULT_DETECTOR_MODEL)
|
||||
parser.add_argument("--max-regions", type=int, default=20)
|
||||
parser.add_argument("--per-hunt", type=int, default=5)
|
||||
parser.add_argument("--budget-usd", type=float, default=10.0)
|
||||
parser.add_argument("--score", action="store_true", help="score the hunter's results with the population's protocol")
|
||||
args = parser.parse_args()
|
||||
from openultrasast.config import load_config
|
||||
|
||||
load_config() # loads the project's .env: DeepSeek is the chat provider, OpenRouter only embeddings
|
||||
spent: list[float] = []
|
||||
if args.pilot_root:
|
||||
result = hunt(args.pilot_root, args.family, args, spent)
|
||||
print(json.dumps({k: v for k, v in result.items() if k != "findings"} | {"findings": result["findings"]}, indent=1))
|
||||
return 0
|
||||
evaluate.use(args.population)
|
||||
evaluate.RESULTS = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-hunter")
|
||||
if args.score:
|
||||
return evaluate.score_v2()
|
||||
only = {c.strip() for c in args.only.split(",") if c.strip()}
|
||||
wanted = {p.strip() for p in args.pins.split(",") if p.strip()}
|
||||
for case in evaluate.cases():
|
||||
if only and case["id"] not in only:
|
||||
continue
|
||||
for label, commit in evaluate.pins(case).items():
|
||||
if wanted and label not in wanted:
|
||||
continue
|
||||
out = evaluate.RESULTS / "scans" / f"{case['id']}--{label}.json"
|
||||
if out.is_file():
|
||||
continue
|
||||
if sum(spent) >= args.budget_usd:
|
||||
print(json.dumps({"stopped": "budget", "spent_usd": round(sum(spent), 4)}), flush=True)
|
||||
return 1
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
checkout = evaluate.export(case, label, commit)
|
||||
try:
|
||||
result = hunt(checkout, case["family"], args, spent)
|
||||
finally:
|
||||
shutil.rmtree(checkout, ignore_errors=True)
|
||||
if not result["completed"]:
|
||||
result["instrument_error"] = "no hunt completed: " + "; ".join(result["hunt_errors"][:3])
|
||||
result["root"] = "/case"
|
||||
out.write_text(json.dumps(result, indent=2) + "\n")
|
||||
print(
|
||||
json.dumps({"case": case["id"], "pin": label, "findings": len(result["findings"]), "hunts": f"{result['completed']}/{result['questions']}", "usd": result["cost_usd"], "total_usd": round(sum(spent), 4)}),
|
||||
flush=True,
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,175 @@
|
||||
"""Diagnostic: does a MODEL, reading source, find the operations the sink vocabulary misses?
|
||||
|
||||
Sink-first selection with the shipped vocabulary put the vulnerable function in scope in 2 of 17 v2 cases: the
|
||||
real sinks (wp_redirect, download_url, spawn, ky, Session.get, parse_expr, Jinja render, Mongo find, JCR
|
||||
createQuery, ...) are not in any token list, and each population brings new ones. This asks the model instead,
|
||||
with no vocabulary: per product source file, which functions perform the family's dangerous operation on data
|
||||
that may not be constant. Candidates are scored exactly as `sink_candidates.py` scores the vocabulary.
|
||||
|
||||
Source only. Files: the case's languages, product code (tests, docs, examples, vendored trees, minified bundles
|
||||
and files over 200 kB excluded), sent in chunks of `--chunk-lines` numbered lines. A chunk whose call fails is
|
||||
counted as UNCLASSIFIED, never as "no sinks". Hard spend ceiling `--budget-usd`; DeepSeek (`deepseek-flash`).
|
||||
|
||||
v2 is spent for tuning; this informs the selection design and qualifies nothing.
|
||||
|
||||
Usage: python benchmarks/archive/model_sinks.py [--population population-v2.toml] [--only ID,...]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import json
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
|
||||
HERE = Path(__file__).resolve().parents[1] / "independent"
|
||||
sys.path.insert(0, str(HERE))
|
||||
|
||||
import evaluate # noqa: E402
|
||||
|
||||
from openultrasast.config import load_config # noqa: E402
|
||||
from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL # noqa: E402
|
||||
from openultrasast.preprocess import detect_language, enumerate_source_files # noqa: E402
|
||||
|
||||
NOT_PRODUCT = re.compile(r"(^|/)(tests?|__tests__|spec|specs|docs?|examples?|fixtures?|e2e|cypress|storybook)/|\.(test|spec|stories)\.[jt]sx?$|_test\.py$|(^|/)test_[^/]*\.py$|\.min\.js$|(^|/)dist/|(^|/)build/")
|
||||
OPERATIONS = {
|
||||
"injection": (
|
||||
"executes a database query (SQL, NoSQL, JCR/graph query), an operating-system command or shell, code or "
|
||||
"expression evaluation (eval, exec, dynamic parsing into executable form), server-side template rendering "
|
||||
"from a template string, or deserialization"
|
||||
),
|
||||
"untrusted_destination": "makes an outbound HTTP/network request, downloads a URL, or redirects the user to a URL",
|
||||
"config_secrets": (
|
||||
"sets security-relevant configuration: CORS policy, debug mode, TLS/certificate verification, cookie or "
|
||||
"session security, or embeds a secret, password or API key"
|
||||
),
|
||||
}
|
||||
PROMPT = (
|
||||
"You are auditing source code. List every function in the code below that {operation}, where the query, "
|
||||
"command, expression, template, URL or setting is not a compile-time constant. Answer with JSON only: "
|
||||
'{{"functions": [{{"function": "<name, or <global> for top-level code>", "line": <line number>, '
|
||||
'"operation": "<the call>"}}]}}. Return {{"functions": []}} if there is none.\n\nFile: {path}\n\n{code}'
|
||||
)
|
||||
|
||||
|
||||
def product_files(root: Path, languages: set[str]) -> list[Path]:
|
||||
files = []
|
||||
for path in enumerate_source_files(root):
|
||||
relative = path.relative_to(root).as_posix()
|
||||
if detect_language(path) in languages and not NOT_PRODUCT.search(relative) and path.stat().st_size <= 200_000:
|
||||
files.append(path)
|
||||
return files
|
||||
|
||||
|
||||
def chunks(text: str, size: int) -> list[str]:
|
||||
lines = text.splitlines()
|
||||
return ["\n".join(f"{n + 1:5d}| {line}" for n, line in enumerate(lines[i : i + size], start=i)) for i in range(0, max(len(lines), 1), size)]
|
||||
|
||||
|
||||
class Meter:
|
||||
def __init__(self, ceiling: float) -> None:
|
||||
self.ceiling, self.clients, self.lock = ceiling, [], threading.Lock()
|
||||
|
||||
def add(self, client: object) -> None:
|
||||
with self.lock:
|
||||
self.clients.append(client)
|
||||
|
||||
def spent(self) -> float:
|
||||
with self.lock:
|
||||
return sum(float(c.cost_usd()) for c in self.clients)
|
||||
|
||||
|
||||
def classify(root: Path, path: Path, family: str, args: argparse.Namespace, meter: Meter, local: threading.local) -> tuple[list[tuple[str, str, int]], int, int]:
|
||||
from openultrasast import tool_hunter
|
||||
|
||||
if not hasattr(local, "client"):
|
||||
local.client = tool_hunter.resolve_hunter_client()
|
||||
meter.add(local.client)
|
||||
relative = path.relative_to(root).as_posix()
|
||||
found, failed, asked = [], 0, 0
|
||||
for code in chunks(path.read_text(errors="ignore"), args.chunk_lines):
|
||||
if meter.spent() >= args.budget_usd:
|
||||
return found, failed + 1, asked
|
||||
asked += 1
|
||||
prompt = PROMPT.format(operation=OPERATIONS[family], path=relative, code=code)
|
||||
try:
|
||||
response = local.client.complete(model=args.model, messages=[{"role": "user", "content": prompt}], tools=[], timeout_seconds=90, json_object=True)
|
||||
items = json.loads(response.content or "{}").get("functions", [])
|
||||
except Exception: # noqa: BLE001 -- counted as unclassified, never as "no sinks"
|
||||
failed += 1
|
||||
continue
|
||||
for item in items if isinstance(items, list) else []:
|
||||
if isinstance(item, dict) and isinstance(item.get("function"), str):
|
||||
line = item.get("line") if isinstance(item.get("line"), int) else 0
|
||||
found.append((relative, item["function"].strip() or "<global>", line))
|
||||
return found, failed, asked
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
|
||||
parser.add_argument("--only", default="")
|
||||
parser.add_argument("--model", default=DEFAULT_DETECTOR_MODEL)
|
||||
parser.add_argument("--chunk-lines", type=int, default=400)
|
||||
parser.add_argument("--workers", type=int, default=8)
|
||||
parser.add_argument("--budget-usd", type=float, default=8.0)
|
||||
args = parser.parse_args()
|
||||
load_config() # the project's .env: DeepSeek is the chat provider
|
||||
evaluate.use(args.population)
|
||||
evaluate.RESULTS = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-modelsinks")
|
||||
evaluate.RESULTS.mkdir(parents=True, exist_ok=True)
|
||||
only = {c.strip() for c in args.only.split(",") if c.strip()}
|
||||
meter, local, rows = Meter(args.budget_usd), threading.local(), []
|
||||
for case in evaluate.cases():
|
||||
if only and case["id"] not in only:
|
||||
continue
|
||||
if meter.spent() >= args.budget_usd:
|
||||
print(json.dumps({"stopped": "budget", "spent_usd": round(meter.spent(), 4)}), flush=True)
|
||||
break
|
||||
started, before = time.monotonic(), meter.spent()
|
||||
checkout = evaluate.export(case, "vulnerable", case["vulnerable"])
|
||||
try:
|
||||
files = product_files(checkout, {case["language"], *case.get("also", [])})
|
||||
found, failed, asked = [], 0, 0
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=args.workers) as pool:
|
||||
for f, fail, ask in pool.map(lambda p: classify(checkout, p, case["family"], args, meter, local), files):
|
||||
found += f
|
||||
failed += fail
|
||||
asked += ask
|
||||
finally:
|
||||
shutil.rmtree(checkout, ignore_errors=True)
|
||||
functions = {(p, fn) for p, fn, _ in found}
|
||||
changed = set(evaluate.hunks(case, "old"))
|
||||
sites = {tuple(site.split("::", 1)) for site in case.get("sites", [])}
|
||||
row = {
|
||||
"id": case["id"], "family": case["family"], "files": len(files), "chunks": asked, "unclassified_chunks": failed,
|
||||
"candidate_functions": len(functions), "candidate_files": len({p for p, _ in functions}),
|
||||
"declared_site_in_candidates": sorted(f"{p}::{fn}" for p, fn in functions if (p, fn) in sites),
|
||||
"fix_or_site_file_in_candidates": sorted({p for p, _ in functions if p in changed or p in {s[0] for s in sites}})[:5],
|
||||
"usd": round(meter.spent() - before, 4), "seconds": round(time.monotonic() - started, 1),
|
||||
} # fmt: skip
|
||||
(evaluate.RESULTS / f"{case['id']}.json").write_text(json.dumps({**row, "candidates": sorted(found)}, indent=1) + "\n")
|
||||
rows.append(row)
|
||||
print(json.dumps(row), flush=True)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"cases": len(rows),
|
||||
"declared_site_in_candidates": sum(bool(r["declared_site_in_candidates"]) for r in rows),
|
||||
"fix_or_site_file_in_candidates": sum(bool(r["fix_or_site_file_in_candidates"]) for r in rows),
|
||||
"spent_usd": round(meter.spent(), 4),
|
||||
}
|
||||
),
|
||||
flush=True,
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,24 @@
|
||||
# Old path to new path (2026-10-02)
|
||||
|
||||
Committed records, specs and some docstrings name these scripts by their old path. Those files are
|
||||
not rewritten. Read the old path as the new one:
|
||||
|
||||
| old path | new path |
|
||||
|---|---|
|
||||
| `benchmarks/independent/verify_batched.py` | `benchmarks/archive/verify_batched.py` |
|
||||
| `benchmarks/independent/score_batched.py` | `benchmarks/archive/score_batched.py` |
|
||||
| `benchmarks/independent/verify_sinks.py` | `benchmarks/archive/verify_sinks.py` |
|
||||
| `benchmarks/independent/model_sinks.py` | `benchmarks/archive/model_sinks.py` |
|
||||
| `benchmarks/independent/sink_candidates.py` | `benchmarks/archive/sink_candidates.py` |
|
||||
| `benchmarks/independent/hunt.py` | `benchmarks/archive/hunt.py` |
|
||||
| `benchmarks/harnessx_removal_equality.py` | `benchmarks/archive/harnessx_removal_equality.py` |
|
||||
|
||||
Files that still use the old path:
|
||||
|
||||
- Records: `independent/hunter-protocol-v2.md`, `independent/results-v2-model-pipeline.json`,
|
||||
`independent/selection-v2-model-sinks.json`, `independent/verifier-batched-check-2026-09-29.json`,
|
||||
`measurements/2026-09-30-harnessx-removal-equality/README`
|
||||
- Specs: `.kiro/specs/ai-service-plane/`, `.kiro/specs/harnessx-removal/design.md`,
|
||||
`.kiro/specs/learned-decision-engine/`
|
||||
- Docstrings: `src/openultrasast/plane/generate.py`, `plane/tasks/{verify,agree,roles,repo_facts}.py`.
|
||||
These are left for the owners of those packages.
|
||||
@@ -0,0 +1,95 @@
|
||||
"""Reference-only since 2026-09-30: the ai-service-plane tasks `plane/tasks/verify.py` and `agree.py`, run on google/ax
|
||||
through `ousast plane run plane/runs/validation-46.yaml`, replace this script (benchmarks/independent/
|
||||
plane-increment-2.json). It stays as the reference behaviour the plane was measured against.
|
||||
|
||||
Exploratory score of a batched-verifier run against the population's cases (protocol v2 matching).
|
||||
|
||||
The batched verifier writes one vulnerable-pin result per case (and a fixed-pin recheck with --fixed), not the
|
||||
six pins protocol v2 scores, so this applies the same matching rule (`evaluate.matches_v2`) to what exists:
|
||||
detected when an AGREED finding matches; unstable when only a DISPUTED one does; fixed-side alert when an agreed
|
||||
finding at the fixed pin matches the fix's new side. The precision sample is every third agreed vulnerable-pin
|
||||
finding (all if 15 or fewer), to be adjudicated by hand. v2 is spent for tuning: this qualifies nothing.
|
||||
|
||||
Usage: python benchmarks/archive/score_batched.py [--results-suffix batched] [--population population-v2.toml]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
|
||||
HERE = Path(__file__).resolve().parents[1] / "independent"
|
||||
sys.path.insert(0, str(HERE))
|
||||
|
||||
import evaluate # noqa: E402
|
||||
|
||||
|
||||
def anchored(result: dict) -> dict:
|
||||
"""Sites anchored to the candidate asked about (older records used the model's reported location)."""
|
||||
for key in ("findings", "disputed"):
|
||||
for f in result.get(key, []):
|
||||
path, _, function = f["candidate"].partition("::")
|
||||
site_path, _, rest = f["site"].partition(":")
|
||||
line = rest.split(":")[0] if site_path == path else "0"
|
||||
f["site"] = f"{path}:{line}:{function}"
|
||||
return result
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
|
||||
parser.add_argument("--results-suffix", default="batched")
|
||||
args = parser.parse_args()
|
||||
evaluate.use(args.population)
|
||||
results = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-" + args.results_suffix)
|
||||
scans = results / "scans"
|
||||
per_case, pool = [], []
|
||||
for case in evaluate.cases():
|
||||
path = scans / f"{case['id']}--vulnerable_a.json"
|
||||
if not path.is_file():
|
||||
per_case.append({"id": case["id"], "status": "incomplete"})
|
||||
continue
|
||||
vulnerable = anchored(json.loads(path.read_text()))
|
||||
old, new = evaluate.hunks(case, "old"), evaluate.hunks(case, "new")
|
||||
agreed = [f["site"] for f in vulnerable["findings"] if evaluate.matches_v2(f, case, old)]
|
||||
disputed = [f["site"] for f in vulnerable.get("disputed", []) if evaluate.matches_v2(f, case, old)]
|
||||
fixed_path = scans / f"{case['id']}--fixed_a.json"
|
||||
fixed = anchored(json.loads(fixed_path.read_text())) if fixed_path.is_file() else None
|
||||
fixed_alerts = [f["site"] for f in fixed["findings"] if evaluate.matches_v2(f, case, new)] if fixed else None
|
||||
per_case.append(
|
||||
{
|
||||
"id": case["id"], "family": case["family"], "detected": bool(agreed), "detected_sites": agreed,
|
||||
"unstable": bool(disputed) and not agreed, "unstable_sites": disputed,
|
||||
"agreed_findings": len(vulnerable["findings"]), "disputed_findings": len(vulnerable.get("disputed", [])),
|
||||
"triaged_out": vulnerable.get("triaged_out"), "candidates": vulnerable.get("questions"),
|
||||
"fixed_side_alerts": fixed_alerts, "fixed_agreed_total": len(fixed["findings"]) if fixed else None,
|
||||
"removed_by_fix": fixed.get("removed_by_fix") if fixed else None, "usd": vulnerable.get("usd"), "usage": vulnerable.get("usage"),
|
||||
}
|
||||
) # fmt: skip
|
||||
pool += [{"case": case["id"], **f} for f in vulnerable["findings"]]
|
||||
pool.sort(key=lambda f: (f["case"], f["site"]))
|
||||
sample = pool if len(pool) <= 15 else pool[0::3]
|
||||
scored = [c for c in per_case if c.get("status") != "incomplete"]
|
||||
detected = sum(c["detected"] for c in scored)
|
||||
with_fixed = [c for c in scored if c["fixed_side_alerts"] is not None]
|
||||
result = {
|
||||
"protocol": "protocol-v2.md matching, exploratory (batched verifier, vulnerable + fixed pins only)",
|
||||
"cases_scored": len(scored), "cases_incomplete": [c["id"] for c in per_case if c.get("status") == "incomplete"],
|
||||
"recall": [detected, len(scored)], "recall_wilson95": evaluate.wilson(detected, len(scored)),
|
||||
"unstable_detections": [c["id"] for c in scored if c["unstable"]],
|
||||
"fixed_side_alerts": sum(len(c["fixed_side_alerts"]) for c in with_fixed) if with_fixed else None,
|
||||
"fixed_pins_rechecked": len(with_fixed),
|
||||
"pooled_agreed_findings": len(pool), "precision_sample_size": len(sample),
|
||||
"usd": round(sum(float(c.get("usd") or 0) for c in scored), 2),
|
||||
"cases": per_case, "precision_sample": sample,
|
||||
} # fmt: skip
|
||||
(results / "score.json").write_text(json.dumps(result, indent=2) + "\n")
|
||||
print(json.dumps({k: v for k, v in result.items() if k not in ("cases", "precision_sample")}, indent=1))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,122 @@
|
||||
"""Diagnostic: would SINK-FIRST selection, from source alone, put the vulnerable function in scope?
|
||||
|
||||
The v2 hunter experiment showed scope selection is the bottleneck: the ranker's top 20 entry-point regions held
|
||||
the vulnerable file in 1 of 17 cases. This asks the opposite direction without any engine or model: at each
|
||||
case's vulnerable pin, every call matching the family's shipped sink vocabulary (`taint_specs`) in product
|
||||
source is a candidate, attributed to its enclosing function by a per-language declaration scan. It reports how
|
||||
many candidates a repository yields (the budget a hunter would need) and whether a declared site function, or
|
||||
at least a file the fix changes, is among them.
|
||||
|
||||
Source only: the checkout is the pin's tree; git is used to export it and, for the score, to read the fix's
|
||||
changed files -- never to choose scope.
|
||||
|
||||
v2 is spent for tuning; this informs the selection design and qualifies nothing.
|
||||
|
||||
Usage: python benchmarks/archive/sink_candidates.py [--population population-v2.toml] [--only ID,...]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
|
||||
HERE = Path(__file__).resolve().parents[1] / "independent"
|
||||
sys.path.insert(0, str(HERE))
|
||||
|
||||
import evaluate # noqa: E402
|
||||
|
||||
from openultrasast.model.specs import taint_specs # noqa: E402
|
||||
from openultrasast.preprocess import detect_language, enumerate_source_files # noqa: E402
|
||||
|
||||
DECLARATION = {
|
||||
"python": re.compile(r"^\s*(?:async\s+)?def\s+(\w+)"),
|
||||
"php": re.compile(r"^\s*(?:(?:public|private|protected|static|final|abstract)\s+)*function\s+&?(\w+)"),
|
||||
"javascript": re.compile(r"^\s*(?:export\s+)?(?:async\s+)?function\s*\*?\s*(\w+)|^\s*(?:(?:public|private|protected|static|async)\s+)*(\w+)\s*\([^;]*\)\s*(?::[^{=]+)?\{\s*$|^\s*(?:export\s+)?(?:const|let|var)\s+(\w+)\s*=\s*(?:async\s*)?(?:function|\()"),
|
||||
}
|
||||
DECLARATION["typescript"] = DECLARATION["javascript"]
|
||||
KEYWORDS = {"if", "for", "while", "switch", "catch", "return", "function", "else"}
|
||||
|
||||
|
||||
def token(name: str) -> re.Pattern[str]:
|
||||
before = r"(?<![A-Za-z0-9_$])" if name[:1].isalnum() or name[:1] in "_$" else ""
|
||||
return re.compile(before + re.escape(name) + r"\s*\(")
|
||||
|
||||
|
||||
def enclosing(lines: list[str], index: int, language: str) -> str:
|
||||
pattern = DECLARATION.get(language)
|
||||
if pattern is None:
|
||||
return ""
|
||||
for back in range(index, -1, -1):
|
||||
match = pattern.match(lines[back])
|
||||
if match:
|
||||
name = next((g for g in match.groups() if g), "")
|
||||
if name and name not in KEYWORDS:
|
||||
return name
|
||||
return "<global>"
|
||||
|
||||
|
||||
def candidates(root: Path, family: str) -> dict[tuple[str, str], list[int]]:
|
||||
found: dict[tuple[str, str], list[int]] = {}
|
||||
patterns: dict[str, list[re.Pattern[str]]] = {}
|
||||
for path in enumerate_source_files(root):
|
||||
language = detect_language(path)
|
||||
if language not in patterns:
|
||||
spec = taint_specs(language=language).get(family) if language != "unknown" else None
|
||||
patterns[language] = [token(n) for n in spec.sinks] if spec else []
|
||||
if not patterns[language]:
|
||||
continue
|
||||
try:
|
||||
lines = path.read_text(errors="ignore").splitlines()
|
||||
except OSError:
|
||||
continue
|
||||
relative = path.relative_to(root).as_posix()
|
||||
for index, line in enumerate(lines):
|
||||
stripped = line.strip()
|
||||
if stripped.startswith(("//", "#", "*", "/*")):
|
||||
continue
|
||||
if any(p.search(line) for p in patterns[language]):
|
||||
found.setdefault((relative, enclosing(lines, index, language)), []).append(index + 1)
|
||||
return found
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
|
||||
parser.add_argument("--only", default="")
|
||||
args = parser.parse_args()
|
||||
evaluate.use(args.population)
|
||||
evaluate.RESULTS = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-sinkfirst")
|
||||
only = {c.strip() for c in args.only.split(",") if c.strip()}
|
||||
rows = []
|
||||
for case in evaluate.cases():
|
||||
if only and case["id"] not in only:
|
||||
continue
|
||||
checkout = evaluate.export(case, "vulnerable", case["vulnerable"])
|
||||
try:
|
||||
found = candidates(checkout, case["family"])
|
||||
finally:
|
||||
shutil.rmtree(checkout, ignore_errors=True)
|
||||
changed = set(evaluate.hunks(case, "old"))
|
||||
sites = {tuple(site.split("::", 1)) for site in case.get("sites", [])}
|
||||
site_hit = sorted(f"{p}::{f}" for p, f in found if (p, f) in sites)
|
||||
file_hit = sorted({p for p, _ in found if p in changed or p in {s[0] for s in sites}})
|
||||
row = {
|
||||
"id": case["id"], "family": case["family"], "candidate_functions": len(found),
|
||||
"candidate_files": len({p for p, _ in found}), "declared_site_in_candidates": site_hit,
|
||||
"fix_or_site_file_in_candidates": file_hit[:5],
|
||||
} # fmt: skip
|
||||
rows.append(row)
|
||||
print(json.dumps(row), flush=True)
|
||||
sites_in = sum(bool(r["declared_site_in_candidates"]) for r in rows)
|
||||
files_in = sum(bool(r["fix_or_site_file_in_candidates"]) for r in rows)
|
||||
print(json.dumps({"cases": len(rows), "declared_site_in_candidates": sites_in, "fix_or_site_file_in_candidates": files_in}))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,291 @@
|
||||
"""Reference-only since 2026-09-30: the ai-service-plane tasks `plane/tasks/verify.py` and `agree.py`, run on google/ax
|
||||
through `ousast plane run plane/runs/validation-46.yaml`, replace this script (benchmarks/independent/
|
||||
plane-increment-2.json). It stays as the reference behaviour the plane was measured against.
|
||||
|
||||
Stage 2, second design: triage per file, verify per file, report only what two passes agree on.
|
||||
|
||||
The first verifier (`verify_sinks.py`) asked one 6-step tool hunt per candidate function. Measured on pgAdmin
|
||||
(verifier-check-2026-09-29.json): $0.014-0.020 per candidate, and on 180 candidates verified twice the two runs
|
||||
flagged 15 and 13 with only 2 in common -- single-pass verdicts on borderline candidates flip. This design
|
||||
changes three things, each checked against the 15 declared sites and those 26 flickering candidates:
|
||||
|
||||
1. TRIAGE, one no-tools call per file: the file's relevant chunks and its candidate functions, asking where the
|
||||
operation's data ORIGINATES. `request`, `parameter` and `stored` go on to verification; `internal` and
|
||||
`constant` do not; anything unclassified goes on (a failed call is never a "safe" verdict).
|
||||
2. BATCHED verification, one 6-step tool hunt per file for up to `--per-hunt` surviving candidates, with one
|
||||
verdict per listed operation. The file read and the caller searches are shared instead of repeated ~5 times.
|
||||
3. AGREEMENT: `--passes` independent hunts per file (fresh conversations); a candidate is reported only when
|
||||
every pass flags it. Findings from single passes are kept in the record as `disputed`.
|
||||
|
||||
Source only; the stage-1 candidates come from `model_sinks.py`. Results in `~/ousast-results/independent-<vN>-
|
||||
<suffix>/scans/` in the shape `evaluate.py` scores; a per-case `.jsonl` makes a stopped run resumable per file.
|
||||
EXPLORATORY: v2 is spent for tuning; this qualifies nothing.
|
||||
|
||||
Usage:
|
||||
python benchmarks/archive/verify_batched.py [--only ID,...] [--subset sets.json] [--passes 2] [--budget-usd 40]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import json
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
|
||||
HERE = Path(__file__).resolve().parents[1] / "independent"
|
||||
sys.path.insert(0, str(HERE))
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent)) # archived siblings: model_sinks, verify_sinks
|
||||
|
||||
import evaluate # noqa: E402
|
||||
from model_sinks import OPERATIONS, chunks # noqa: E402
|
||||
from verify_sinks import ACCOUNT_ERRORS, _Recording # noqa: E402
|
||||
|
||||
from openultrasast.config import load_config # noqa: E402
|
||||
from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL # noqa: E402
|
||||
|
||||
VERIFY = {"request", "parameter", "stored"}
|
||||
TRIAGE = (
|
||||
"Below is a source file with numbered lines, and a list of functions in it that each {operation}. For EACH "
|
||||
"listed function, say where the data used by that operation ORIGINATES, tracing within this file only:\n"
|
||||
"- request: an HTTP request (parameters, body, headers, cookies, uploads), a webhook or API payload;\n"
|
||||
"- parameter: a parameter of the function whose callers are not in this file;\n"
|
||||
"- stored: a database row, file or setting written earlier, possibly from external input;\n"
|
||||
"- internal: values the program computes from constants, the operator's environment, CLI arguments or "
|
||||
"trusted framework state;\n"
|
||||
"- constant: a literal.\n"
|
||||
'Answer with JSON only: {{"functions": [{{"function": "<name exactly as listed>", "data_origin": "<one of the five>"}}]}}.'
|
||||
"\n\nFunctions: {functions}\n\nFile: {path}\n\n{code}"
|
||||
)
|
||||
SYSTEM = (
|
||||
"You are a security reviewer with repository tools (read_file, grep_repo, find_refs). You are given several "
|
||||
"operations in ONE file, each named by its function. For EACH operation decide whether data an attacker "
|
||||
"controls -- HTTP request parameters, body, headers, cookies, uploaded or imported content, webhook or API "
|
||||
"payloads, values stored earlier from such input -- can reach it without a sufficient guard (validation or "
|
||||
"allowlisting, escaping that fits the exact context, parameterization, or a fixed destination). Use the tools "
|
||||
"to follow callers and data. Report an operation only if you can state the concrete path from the source to "
|
||||
"it. Answer with a JSON array: one object per vulnerable operation with path, line, function_name (exactly as "
|
||||
"listed), title and rationale (the source, each hop, and why no guard applies); [] if none is vulnerable or "
|
||||
"you cannot establish a path."
|
||||
)
|
||||
|
||||
_CLIENT = None
|
||||
|
||||
|
||||
def _client():
|
||||
global _CLIENT
|
||||
if _CLIENT is None:
|
||||
from openultrasast import tool_hunter
|
||||
|
||||
load_config()
|
||||
_CLIENT = tool_hunter.resolve_hunter_client()
|
||||
return _CLIENT
|
||||
|
||||
|
||||
def _relevant_code(text: str, lines: list[int], chunk_lines: int, max_chunks: int = 6) -> str:
|
||||
parts = chunks(text, chunk_lines)
|
||||
if not lines or any(line <= 0 for line in lines) or len(parts) <= max_chunks:
|
||||
return "\n".join(parts[:max_chunks])
|
||||
wanted = sorted({min((line - 1) // chunk_lines, len(parts) - 1) for line in lines})
|
||||
return "\n".join(parts[i] for i in wanted[:max_chunks])
|
||||
|
||||
|
||||
def triage(root: Path, path: str, candidates: list[list], family: str, model: str, chunk_lines: int) -> dict[str, str]:
|
||||
"""Function name -> data_origin for this file's candidates; a name missing from the answer is unclassified."""
|
||||
client = _client()
|
||||
try:
|
||||
text = (root / path).read_text(errors="ignore")
|
||||
except OSError:
|
||||
return {}
|
||||
names = sorted({fn for _, fn, _ in candidates})
|
||||
prompt = TRIAGE.format(operation=OPERATIONS[family], functions=json.dumps(names), path=path, code=_relevant_code(text, [ln for _, _, ln in candidates], chunk_lines))
|
||||
response = client.complete(model=model, messages=[{"role": "user", "content": prompt}], tools=[], timeout_seconds=90, json_object=True)
|
||||
try:
|
||||
items = json.loads(response.content or "{}").get("functions", [])
|
||||
except (json.JSONDecodeError, AttributeError):
|
||||
return {}
|
||||
return {str(i.get("function")): str(i.get("data_origin", "")).strip().lower() for i in items if isinstance(i, dict) and i.get("function") in names}
|
||||
|
||||
|
||||
def hunt(root: Path, path: str, group: list[list], family: str, model: str, steps: int) -> tuple[list[dict], dict]:
|
||||
from openultrasast import tool_hunter
|
||||
from openultrasast.complexity.map import Hotspot
|
||||
|
||||
recording = _Recording(_client())
|
||||
listing = "\n".join(f"- function `{fn}` around line {ln}" for _, fn, ln in group)
|
||||
prompt = (
|
||||
f"Operations to judge, all in `{path}`, each of which {OPERATIONS[family]}:\n{listing}\n"
|
||||
"The file is above. For each, trace where the operation's data comes from and whether it is guarded."
|
||||
)
|
||||
spots = [Hotspot(path=path, function_name=fn, score=1.0, band="candidate", signals={}, rationale="model-classified sink", test_hint=None, inventory_finding_ids=()) for _, fn, _ in group]
|
||||
found = tool_hunter.run_tool_hunter(root, spots, client=recording, model=model, max_steps=steps, system_prompt=SYSTEM, user_prompt=prompt, context_files=[path], tags=("sink-verifier",))
|
||||
names = {fn for _, fn, _ in group}
|
||||
rows = []
|
||||
for f in found:
|
||||
fn = f.function_name or ""
|
||||
if fn not in names: # attribute by the nearest listed line when the model renamed the function
|
||||
fn = min(group, key=lambda c: abs((c[2] or 0) - (f.line or 0)))[1]
|
||||
# The verdict is about the candidate operation, so the site is the candidate's file and function; the
|
||||
# model's own location is kept beside it (it often names the file where the input enters instead).
|
||||
line = f.line if f.path == path else next(c[2] for c in group if c[1] == fn)
|
||||
rows.append({"candidate": f"{path}::{fn}", "site": f"{path}:{line or 0}:{fn}", "reported_at": f"{f.path}:{f.line or 0}", "family": family, "title": f.title, "witness": f.rationale[:600]})
|
||||
return rows, recording.usage()
|
||||
|
||||
|
||||
def verify_file(root: str, path: str, candidates: list[list], family: str, args_: dict) -> dict:
|
||||
"""One file, in a worker process: triage, then `passes` batched hunts over the survivors."""
|
||||
client = _client()
|
||||
before = float(client.cost_usd())
|
||||
record: dict = {"path": path, "candidates": candidates, "triage": {}, "passes": [], "usage": {}}
|
||||
try:
|
||||
if not args_["no_triage"]:
|
||||
record["triage"] = triage(Path(root), path, candidates, family, args_["model"], args_["chunk_lines"])
|
||||
kept = [c for c in candidates if record["triage"].get(c[1], "unclassified") in VERIFY | {"unclassified"} or record["triage"].get(c[1]) not in {"internal", "constant"}]
|
||||
record["kept"] = [c[1] for c in kept]
|
||||
usage: dict[str, int] = {}
|
||||
for _pass in range(args_["passes"]):
|
||||
flagged: list[dict] = []
|
||||
for i in range(0, len(kept), args_["per_hunt"]):
|
||||
rows, used = hunt(Path(root), path, kept[i : i + args_["per_hunt"]], family, args_["model"], args_["max_steps"])
|
||||
flagged += rows
|
||||
usage = {k: usage.get(k, 0) + used.get(k, 0) for k in set(usage) | set(used)}
|
||||
record["passes"].append(flagged)
|
||||
record["usage"] = usage
|
||||
except Exception as error: # noqa: BLE001 -- recorded; an unverified file is not a clean one
|
||||
record["unverified"] = f"{type(error).__name__}: {str(error)[:160]}"
|
||||
record["usd"] = round(float(client.cost_usd()) - before, 4)
|
||||
return record
|
||||
|
||||
|
||||
def agreement(record: dict) -> tuple[list[dict], list[dict]]:
|
||||
"""(agreed, disputed): flagged in every pass vs in some."""
|
||||
if not record.get("passes"):
|
||||
return [], []
|
||||
by_pass = [{r["candidate"]: r for r in flagged} for flagged in record["passes"]]
|
||||
every = set.intersection(*(set(b) for b in by_pass))
|
||||
some = set.union(*(set(b) for b in by_pass))
|
||||
first = by_pass[0]
|
||||
pick = lambda c: next(b[c] for b in by_pass if c in b) # noqa: E731
|
||||
return [dict(first[c], passes=len(by_pass)) for c in sorted(every)], [dict(pick(c), passes=sum(c in b for b in by_pass)) for c in sorted(some - every)]
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
|
||||
parser.add_argument("--only", default="")
|
||||
parser.add_argument("--subset", type=Path, help='JSON {case_id: [[path, function], ...]}: verify only these candidates')
|
||||
parser.add_argument("--model", default=DEFAULT_DETECTOR_MODEL)
|
||||
parser.add_argument("--max-steps", type=int, default=6)
|
||||
parser.add_argument("--per-hunt", type=int, default=6)
|
||||
parser.add_argument("--passes", type=int, default=2)
|
||||
parser.add_argument("--chunk-lines", type=int, default=400)
|
||||
parser.add_argument("--no-triage", action="store_true")
|
||||
parser.add_argument("--workers", type=int, default=4)
|
||||
parser.add_argument("--budget-usd", type=float, default=40.0)
|
||||
parser.add_argument("--results-suffix", default="batched")
|
||||
parser.add_argument("--fixed", action="store_true", help="recheck the AGREED candidates of each case at its fixed pin (no triage)")
|
||||
args = parser.parse_args()
|
||||
load_config()
|
||||
evaluate.use(args.population)
|
||||
candidates_dir = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-modelsinks")
|
||||
evaluate.RESULTS = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-" + args.results_suffix)
|
||||
out_dir = evaluate.RESULTS / "scans"
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
only = {c.strip() for c in args.only.split(",") if c.strip()}
|
||||
subset = json.loads(args.subset.read_text()) if args.subset else None
|
||||
settings = {k: getattr(args, k) for k in ("model", "max_steps", "per_hunt", "passes", "chunk_lines", "no_triage")}
|
||||
spent = 0.0
|
||||
for case in evaluate.cases():
|
||||
if only and case["id"] not in only or (subset is not None and case["id"] not in subset):
|
||||
continue
|
||||
label = "fixed_a" if args.fixed else "vulnerable_a"
|
||||
out = out_dir / f"{case['id']}--{label}.json"
|
||||
if out.is_file():
|
||||
continue
|
||||
log = out_dir / f"{case['id']}--{label}.jsonl"
|
||||
done = {r["path"]: r for r in (json.loads(row) for row in log.read_text().splitlines()) if "unverified" not in r} if log.is_file() else {}
|
||||
spent += sum(float(r.get("usd", 0.0)) for r in done.values())
|
||||
record = json.loads((candidates_dir / f"{case['id']}.json").read_text())
|
||||
unique = list({(p, fn): [p, fn, ln] for p, fn, ln in sorted(record["candidates"])}.values())
|
||||
if subset is not None:
|
||||
wanted = {tuple(x) for x in subset[case["id"]]}
|
||||
unique = [c for c in unique if (c[0], c[1]) in wanted]
|
||||
if args.fixed:
|
||||
# The agreed findings at the vulnerable pin, by (path, function), asked again at the fixed pin. Line
|
||||
# numbers are the vulnerable pin's; the file itself is in the prompt. A function the fix removed
|
||||
# cannot be asked and counts as removed.
|
||||
vulnerable_out = out_dir / f"{case['id']}--vulnerable_a.json"
|
||||
if not vulnerable_out.is_file():
|
||||
continue
|
||||
agreed = {tuple(f["candidate"].split("::", 1)) for f in json.loads(vulnerable_out.read_text())["findings"]}
|
||||
unique = [c for c in unique if (c[0], c[1]) in agreed]
|
||||
settings = dict(settings, no_triage=True)
|
||||
by_file: dict[str, list[list]] = {}
|
||||
for c in unique:
|
||||
by_file.setdefault(c[0], []).append(c)
|
||||
pending = {p: cs for p, cs in by_file.items() if p not in done}
|
||||
started = time.monotonic()
|
||||
checkout = evaluate.export(case, label, case["fixed" if args.fixed else "vulnerable"])
|
||||
if args.fixed:
|
||||
present = [c for c in unique if (checkout / c[0]).is_file()]
|
||||
removed = [f"{c[0]}::{c[1]}" for c in unique if (checkout / c[0]).is_file() is False]
|
||||
unique, by_file = present, {}
|
||||
for c in unique:
|
||||
by_file.setdefault(c[0], []).append(c)
|
||||
pending = {p: cs for p, cs in by_file.items() if p not in done}
|
||||
account_error = ""
|
||||
try:
|
||||
with concurrent.futures.ProcessPoolExecutor(max_workers=args.workers) as pool, log.open("a") as sink:
|
||||
futures = {pool.submit(verify_file, str(checkout), p, cs, case["family"], settings): p for p, cs in pending.items()}
|
||||
for future in concurrent.futures.as_completed(futures):
|
||||
if future.cancelled():
|
||||
continue
|
||||
rec = future.result()
|
||||
if any(code in str(rec.get("unverified", "")) for code in ACCOUNT_ERRORS):
|
||||
account_error = str(rec["unverified"])[:200]
|
||||
for other in futures:
|
||||
other.cancel()
|
||||
continue
|
||||
spent += float(rec.get("usd", 0.0))
|
||||
sink.write(json.dumps(rec) + "\n")
|
||||
sink.flush()
|
||||
if "unverified" not in rec:
|
||||
done[rec["path"]] = rec
|
||||
if spent >= args.budget_usd:
|
||||
for other in futures:
|
||||
other.cancel()
|
||||
finally:
|
||||
shutil.rmtree(checkout, ignore_errors=True)
|
||||
agreed, disputed = [], []
|
||||
for rec in done.values():
|
||||
a, d = agreement(rec)
|
||||
agreed += a
|
||||
disputed += d
|
||||
triaged_out = sum(len(r["candidates"]) - len(r.get("kept", [])) for r in done.values())
|
||||
usage = {k: sum(int(r.get("usage", {}).get(k, 0)) for r in done.values()) for k in ("prompt_tokens", "prompt_cache_hit_tokens", "completion_tokens", "calls")}
|
||||
complete = len(done) >= len(by_file) and not account_error
|
||||
result = {
|
||||
"root": "/case", "families": [case["family"]], "questions": len(unique), "completed": sum(len(r["candidates"]) for r in done.values()),
|
||||
"completed_regions": sorted(f"{c[0]}:{c[1]}" for r in done.values() for c in r["candidates"]),
|
||||
"triaged_out": triaged_out, "passes": args.passes, "usage": usage, "usd": round(sum(float(r.get("usd", 0.0)) for r in done.values()), 4),
|
||||
"seconds": round(time.monotonic() - started, 1), "findings": sorted(agreed, key=lambda f: f["site"]), "disputed": sorted(disputed, key=lambda f: f["site"]),
|
||||
} # fmt: skip
|
||||
if args.fixed:
|
||||
result["removed_by_fix"] = removed
|
||||
if complete:
|
||||
out.write_text(json.dumps(result, indent=1) + "\n")
|
||||
print(json.dumps({"case": case["id"], "files": f"{len(done)}/{len(by_file)}", "candidates": len(unique), "triaged_out": triaged_out, "agreed": len(agreed), "disputed": len(disputed), "usd": result["usd"], "total_usd": round(spent, 4), "complete": complete}), flush=True)
|
||||
if account_error:
|
||||
print(json.dumps({"stopped": "account", "error": account_error}), flush=True)
|
||||
return 2
|
||||
if spent >= args.budget_usd:
|
||||
print(json.dumps({"stopped": "budget", "spent_usd": round(spent, 4)}), flush=True)
|
||||
break
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,235 @@
|
||||
"""Stage 2 of source-only, model-driven detection: verify each model-classified sink candidate.
|
||||
|
||||
`model_sinks.py` put the vulnerable function among the candidates in 15 of 17 v2 cases, among ~4,800 candidates.
|
||||
This asks, per candidate, one bounded question with repository tools (the existing tool hunter with its
|
||||
per-family overrides): can attacker-controlled data reach THIS operation without a sufficient guard? A candidate
|
||||
is reported only when the model states a concrete source-to-sink path and names no sufficient guard.
|
||||
|
||||
Scope: the candidates `model_sinks.py` recorded for each case's vulnerable pin
|
||||
(~/ousast-results/independent-v2-modelsinks/<case>.json). One run per pin; the fixed pin is rechecked only for
|
||||
detected sites (`--fixed`). Findings are written in the scan-result shape `evaluate.py` reads. Hard spend
|
||||
ceiling. EXPLORATORY: v2 is spent for tuning; this qualifies nothing.
|
||||
|
||||
Usage: python benchmarks/archive/verify_sinks.py [--only ID,...] [--budget-usd 40] [--fixed]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import json
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
# Archived 2026-10-02 from benchmarks/independent/: HERE stays that directory (population, evaluate.py, results).
|
||||
HERE = Path(__file__).resolve().parents[1] / "independent"
|
||||
sys.path.insert(0, str(HERE))
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent)) # archived siblings: model_sinks, verify_sinks
|
||||
|
||||
import evaluate # noqa: E402
|
||||
from model_sinks import OPERATIONS # noqa: E402
|
||||
|
||||
from openultrasast.config import load_config # noqa: E402
|
||||
from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL # noqa: E402
|
||||
|
||||
# The provider refusing the ACCOUNT (no credit, bad key): every later call fails the same way.
|
||||
ACCOUNT_ERRORS = ("HTTP Error 402", "HTTP Error 401", "Insufficient Balance")
|
||||
|
||||
SYSTEM = (
|
||||
"You are a security reviewer with repository tools (read_file, grep_repo, find_refs). You are given ONE "
|
||||
"operation in ONE function. Decide whether data an attacker controls -- HTTP request parameters, body, "
|
||||
"headers, cookies, uploaded or imported content, webhook or API payloads, values stored earlier from such "
|
||||
"input -- can reach that operation without a sufficient guard (validation or allowlisting, escaping that fits "
|
||||
"the exact context, parameterization, or a fixed destination). Use the tools to follow callers and data. "
|
||||
"Report it only if you can state the concrete path from the source to the operation. Answer with a JSON array: "
|
||||
"one object with path, line, function_name, title, rationale (the source, each hop, and why no guard applies) "
|
||||
"if it is vulnerable; [] if it is not, or if you cannot establish the path."
|
||||
)
|
||||
|
||||
|
||||
_CLIENT = None
|
||||
|
||||
|
||||
def _client():
|
||||
global _CLIENT
|
||||
if _CLIENT is None:
|
||||
from openultrasast import tool_hunter
|
||||
|
||||
load_config()
|
||||
_CLIENT = tool_hunter.resolve_hunter_client()
|
||||
return _CLIENT
|
||||
|
||||
|
||||
class _Recording:
|
||||
"""Wraps the chat client for one candidate: counts investigation turns and slices token usage."""
|
||||
|
||||
def __init__(self, client) -> None:
|
||||
self.client, self.turns_with_tools, self.first_usage = client, 0, len(getattr(client, "usage", []))
|
||||
|
||||
def complete(self, **kw):
|
||||
response = self.client.complete(**kw)
|
||||
if response.tool_calls:
|
||||
self.turns_with_tools += 1
|
||||
return response
|
||||
|
||||
def usage(self) -> dict[str, int]:
|
||||
rows = getattr(self.client, "usage", [])[self.first_usage :]
|
||||
keys = ("prompt_tokens", "prompt_cache_hit_tokens", "completion_tokens")
|
||||
return {k: sum(int(r.get(k) or 0) for r in rows) for k in keys} | {"calls": len(rows)}
|
||||
|
||||
|
||||
def _hunt(root: str, spot, prompt: str, path: str, model: str, steps: int):
|
||||
from openultrasast import tool_hunter
|
||||
|
||||
recording = _Recording(_client())
|
||||
found = tool_hunter.run_tool_hunter(
|
||||
Path(root), [spot], client=recording, model=model, max_steps=steps,
|
||||
system_prompt=SYSTEM, user_prompt=prompt, context_files=[path], tags=("sink-verifier",),
|
||||
) # fmt: skip
|
||||
return found, recording
|
||||
|
||||
|
||||
def verify(root: str, candidate: list, family: str, model: str, max_steps: int, escalate_steps: int = 0) -> dict:
|
||||
"""One candidate, in a worker process: CPU-bound repository tools do not serialise behind one interpreter.
|
||||
|
||||
Adaptive budget: the hunt runs at `max_steps`; if it spent EVERY step on tool calls and reported nothing, the
|
||||
model was still investigating when the budget ended, and the hunt is repeated once at `escalate_steps`. A hunt
|
||||
that concluded early (fewer tool turns than steps) is final at the small budget. Measured on the seven declared
|
||||
v2 sites: 3 steps found 5, 8 steps found 6; most candidates conclude in 3.
|
||||
"""
|
||||
from openultrasast.complexity.map import Hotspot
|
||||
|
||||
client = _client()
|
||||
before = float(client.cost_usd())
|
||||
path, function, line = candidate
|
||||
spot = Hotspot(path=path, function_name=function, score=1.0, band="candidate", signals={}, rationale="model-classified sink", test_hint=None, inventory_finding_ids=())
|
||||
prompt = (
|
||||
f"Operation to judge: in `{path}`, function `{function}`, around line {line}, code that {OPERATIONS[family]}. "
|
||||
"The file is above. Trace where the operation's data comes from and whether it is guarded."
|
||||
)
|
||||
usage: dict[str, int] = {}
|
||||
escalated = False
|
||||
try:
|
||||
found, recording = _hunt(root, spot, prompt, path, model, max_steps)
|
||||
usage = recording.usage()
|
||||
if not found and escalate_steps > max_steps and recording.turns_with_tools >= max_steps:
|
||||
escalated = True
|
||||
found, recording = _hunt(root, spot, prompt, path, model, escalate_steps)
|
||||
second = recording.usage()
|
||||
usage = {k: usage.get(k, 0) + second.get(k, 0) for k in set(usage) | set(second)}
|
||||
except Exception as error: # noqa: BLE001 -- recorded; an unverified candidate is not a clean one
|
||||
return {"candidate": candidate, "unverified": f"{type(error).__name__}: {str(error)[:120]}", "usd": float(client.cost_usd()) - before}
|
||||
findings = [
|
||||
{"site": f"{f.path}:{f.line or line}:{f.function_name or function}", "family": family, "title": f.title, "witness": f.rationale[:600], "candidate": f"{path}::{function}"}
|
||||
for f in found
|
||||
]
|
||||
return {"candidate": candidate, "findings": findings, "usd": float(client.cost_usd()) - before, "escalated": escalated, "usage": usage}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--population", type=Path, default=HERE / "population-v2.toml")
|
||||
parser.add_argument("--only", default="")
|
||||
parser.add_argument("--model", default=DEFAULT_DETECTOR_MODEL)
|
||||
parser.add_argument("--max-steps", type=int, default=6)
|
||||
parser.add_argument("--escalate-steps", type=int, default=0, help="rerun a hunt that used every step without a verdict; 0 disables (measured: 88% escalate, +45% cost)")
|
||||
parser.add_argument("--workers", type=int, default=4)
|
||||
parser.add_argument("--budget-usd", type=float, default=40.0)
|
||||
parser.add_argument("--fixed", action="store_true", help="recheck detected sites at the fixed pin")
|
||||
parser.add_argument("--results-suffix", default="verified", help="results go to independent-<vN>-<suffix>/")
|
||||
parser.add_argument("--declared-sites-only", action="store_true", help="verify only the case's declared sites (a known-answer check)")
|
||||
parser.add_argument("--candidates-from", type=Path, help="verify exactly the candidates recorded in this per-case .jsonl")
|
||||
args = parser.parse_args()
|
||||
load_config()
|
||||
evaluate.use(args.population)
|
||||
candidates_dir = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-modelsinks")
|
||||
# Checkouts too, not only results: two runs exporting the same pin into one path deleted each other's tree.
|
||||
evaluate.RESULTS = evaluate.RESULTS.with_name(evaluate.RESULTS.name + "-" + args.results_suffix)
|
||||
out_dir = evaluate.RESULTS / "scans"
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
only = {c.strip() for c in args.only.split(",") if c.strip()}
|
||||
spent = 0.0
|
||||
for case in evaluate.cases():
|
||||
if only and case["id"] not in only:
|
||||
continue
|
||||
label = "fixed_a" if args.fixed else "vulnerable_a"
|
||||
out = out_dir / f"{case['id']}--{label}.json"
|
||||
if out.is_file():
|
||||
continue
|
||||
log = out_dir / f"{case['id']}--{label}.jsonl"
|
||||
done = [json.loads(row) for row in log.read_text().splitlines()] if log.is_file() else []
|
||||
spent += sum(float(d.get("usd", 0.0)) for d in done)
|
||||
record = json.loads((candidates_dir / f"{case['id']}.json").read_text())
|
||||
todo = list({(p, fn): (p, fn, ln) for p, fn, ln in sorted(record["candidates"])}.values()) # one question per function
|
||||
if args.declared_sites_only:
|
||||
sites = {tuple(site.split("::", 1)) for site in case.get("sites", [])}
|
||||
todo = [c for c in todo if (c[0], c[1]) in sites]
|
||||
if args.candidates_from:
|
||||
wanted = {tuple(json.loads(row)["candidate"][:2]) for row in args.candidates_from.read_text().splitlines() if row.strip()}
|
||||
todo = [c for c in todo if (c[0], c[1]) in wanted]
|
||||
if args.fixed:
|
||||
vulnerable = json.loads((out_dir / f"{case['id']}--vulnerable_a.json").read_text())
|
||||
detected = {f["candidate"] for f in vulnerable["findings"]}
|
||||
todo = [c for c in todo if f"{c[0]}::{c[1]}" in detected]
|
||||
# Only a VERIFIED candidate is done. An unverified one -- a failed call -- is asked again on resume; counting
|
||||
# it as done wrote sixteen cases as complete with zero calls made when the account ran out of credit.
|
||||
done = [d for d in done if "findings" in d]
|
||||
finished = {tuple(d["candidate"][:2]) for d in done}
|
||||
pending = [c for c in todo if (c[0], c[1]) not in finished]
|
||||
started = time.monotonic()
|
||||
checkout = evaluate.export(case, label, case["fixed" if args.fixed else "vulnerable"])
|
||||
stopped, account_error = False, ""
|
||||
try:
|
||||
with concurrent.futures.ProcessPoolExecutor(max_workers=args.workers) as pool, log.open("a") as sink:
|
||||
futures = []
|
||||
for candidate in pending:
|
||||
futures.append(pool.submit(verify, str(checkout), list(candidate), case["family"], args.model, args.max_steps, args.escalate_steps))
|
||||
for future in concurrent.futures.as_completed(futures):
|
||||
if future.cancelled(): # cancelled at the ceiling: never run, nothing to record
|
||||
continue
|
||||
result = future.result()
|
||||
if any(code in str(result.get("unverified", "")) for code in ACCOUNT_ERRORS):
|
||||
# The account, not the candidate: nothing after this can succeed. Stop, record nothing.
|
||||
account_error = str(result["unverified"])[:200]
|
||||
stopped = True
|
||||
for other in futures:
|
||||
other.cancel()
|
||||
continue
|
||||
spent += float(result.get("usd", 0.0))
|
||||
sink.write(json.dumps(result) + "\n")
|
||||
sink.flush()
|
||||
done.append(result)
|
||||
if spent >= args.budget_usd and not stopped:
|
||||
stopped = True
|
||||
for other in futures:
|
||||
other.cancel()
|
||||
finally:
|
||||
shutil.rmtree(checkout, ignore_errors=True)
|
||||
verified = {tuple(d["candidate"][:2]) for d in done if "findings" in d}
|
||||
findings = [f for d in done for f in d.get("findings", [])]
|
||||
unverified = [d["unverified"] for d in done if "unverified" in d]
|
||||
complete = len(verified) >= len(todo) and not account_error
|
||||
result = {
|
||||
"root": "/case", "families": [case["family"]], "questions": len(todo), "completed": len(verified),
|
||||
"completed_regions": sorted(f"{p}:{fn}" for p, fn in verified), "unverified": unverified[:20],
|
||||
"usd": round(sum(float(d.get("usd", 0.0)) for d in done), 4), "seconds": round(time.monotonic() - started, 1),
|
||||
"findings": sorted(findings, key=lambda f: f["site"]),
|
||||
} # fmt: skip
|
||||
if complete:
|
||||
out.write_text(json.dumps(result, indent=1) + "\n")
|
||||
tokens = {k: sum(int(d.get("usage", {}).get(k, 0)) for d in done) for k in ("prompt_tokens", "prompt_cache_hit_tokens", "completion_tokens")}
|
||||
result["usage"] = tokens | {"escalated": sum(1 for d in done if d.get("escalated"))}
|
||||
print(json.dumps({"case": case["id"], "pin": label, "candidates": len(todo), "verified": len(verified), "findings": len(findings), "unverified": len(unverified), "escalated": result["usage"]["escalated"], "cache_share": round(tokens["prompt_cache_hit_tokens"] / max(tokens["prompt_tokens"], 1), 2), "usd": result["usd"], "total_usd": round(spent, 4), "complete": complete}), flush=True)
|
||||
if account_error:
|
||||
print(json.dumps({"stopped": "account", "error": account_error, "spent_usd": round(spent, 4)}), flush=True)
|
||||
return 2
|
||||
if stopped or spent >= args.budget_usd:
|
||||
print(json.dumps({"stopped": "budget", "spent_usd": round(spent, 4)}), flush=True)
|
||||
break
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Bulk reader: answer a question about one or more files with the cheap chat model, citing line numbers.
|
||||
|
||||
The Portal pattern (Spotify, 2026-09): reading five files to answer one question is I/O, not reasoning, so a
|
||||
cheap model reads and the expensive one gets the answer. Files are sent in numbered chunks; the answer must cite
|
||||
`path:line`, which is what a summary otherwise lacks. Reasoning stays with the caller: this reports what the
|
||||
files say, it does not decide anything.
|
||||
|
||||
Usage: .venv/bin/python benchmarks/dev/bulk_read.py "<question>" FILE [FILE ...] [--max-usd 0.05]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
CHUNK_LINES = 400
|
||||
PROMPT = (
|
||||
"Answer the question using only the files below (numbered lines). Be specific and brief; cite every claim as "
|
||||
"path:line. If the files do not answer it, say so.\n\nQuestion: {question}\n\n{files}"
|
||||
)
|
||||
|
||||
|
||||
def numbered(path: Path) -> list[str]:
|
||||
lines = path.read_text(errors="ignore").splitlines()
|
||||
chunks = []
|
||||
for start in range(0, max(len(lines), 1), CHUNK_LINES):
|
||||
body = "\n".join(f"{n + 1:5d}| {line}" for n, line in enumerate(lines[start : start + CHUNK_LINES], start=start))
|
||||
chunks.append(f"--- {path} (lines {start + 1}-{min(start + CHUNK_LINES, len(lines))})\n{body}")
|
||||
return chunks
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("question")
|
||||
parser.add_argument("files", nargs="+", type=Path)
|
||||
parser.add_argument("--max-usd", type=float, default=0.05)
|
||||
args = parser.parse_args()
|
||||
from openultrasast.config import load_config
|
||||
from openultrasast.model.endpoint import DEFAULT_DETECTOR_MODEL
|
||||
from openultrasast import tool_hunter
|
||||
|
||||
load_config()
|
||||
client = tool_hunter.resolve_hunter_client()
|
||||
if client is None:
|
||||
print("no chat client: set DEEPSEEK_API_KEY", file=sys.stderr)
|
||||
return 2
|
||||
chunks = [chunk for path in args.files for chunk in numbered(path)]
|
||||
answers = []
|
||||
for chunk in chunks:
|
||||
if float(client.cost_usd()) >= args.max_usd:
|
||||
answers.append(f"[stopped at the ${args.max_usd} ceiling before: {chunk.splitlines()[0]}]")
|
||||
break
|
||||
response = client.complete(model=DEFAULT_DETECTOR_MODEL, messages=[{"role": "user", "content": PROMPT.format(question=args.question, files=chunk)}], tools=[], timeout_seconds=90)
|
||||
answers.append((response.content or "").strip())
|
||||
print("\n\n".join(a for a in answers if a))
|
||||
print(f"\n[bulk_read: {len(chunks)} chunk(s), ${float(client.cost_usd()):.4f}]", file=sys.stderr)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env python3
|
||||
"""PreToolUse hook: a whole-file Read of more than LIMIT lines is denied and redirected.
|
||||
|
||||
Measured 2026-09-29: 90% of a development session's tokens were tool I/O -- inline scripts and what they
|
||||
printed -- not reasoning. Whole-file reads are the largest single item that a summary could replace. A Read with
|
||||
`offset`/`limit` is a deliberate range and passes; a file at or under LIMIT lines passes.
|
||||
|
||||
Wired in .claude/settings.local.json (PreToolUse, matcher Read). Stdin: the hook JSON; stdout: a permission
|
||||
decision only when denying. Never fails the tool: any error here exits 0 silently.
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
|
||||
LIMIT = 350
|
||||
|
||||
|
||||
def main() -> int:
|
||||
try:
|
||||
event = json.load(sys.stdin)
|
||||
except (json.JSONDecodeError, OSError):
|
||||
return 0
|
||||
if event.get("tool_name") != "Read":
|
||||
return 0
|
||||
tool_input = event.get("tool_input") or {}
|
||||
path = tool_input.get("file_path")
|
||||
if not path or tool_input.get("limit") or tool_input.get("offset"):
|
||||
return 0
|
||||
try:
|
||||
with open(path, "rb") as handle:
|
||||
lines = sum(1 for _ in handle)
|
||||
except OSError:
|
||||
return 0
|
||||
if lines <= LIMIT:
|
||||
return 0
|
||||
reason = (
|
||||
f"{path} has {lines} lines (limit {LIMIT} for a whole-file read). Read a range with offset/limit, or ask "
|
||||
f"the bulk reader for what you need: .venv/bin/python benchmarks/dev/bulk_read.py '<question>' {path}"
|
||||
)
|
||||
print(json.dumps({"hookSpecificOutput": {"hookEventName": "PreToolUse", "permissionDecision": "deny", "permissionDecisionReason": reason}}))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Where a Claude Code session's tokens go: tool input vs tool output vs text, from the transcript.
|
||||
|
||||
The 2026-09-29 measurement of this project's main session: tool_use input 47.5% (inline scripts), tool results
|
||||
42%, assistant text 6%, thinking 2%, user 2%. Re-run after a workflow change to see whether it moved.
|
||||
|
||||
Usage: python benchmarks/dev/token_report.py [TRANSCRIPT.jsonl] (default: the newest under ~/.claude/projects)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import collections
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def newest() -> Path:
|
||||
candidates = sorted(Path.home().glob(".claude/projects/*/*.jsonl"), key=lambda p: p.stat().st_mtime)
|
||||
if not candidates:
|
||||
raise SystemExit("no transcript found")
|
||||
return candidates[-1]
|
||||
|
||||
|
||||
def main() -> int:
|
||||
path = Path(sys.argv[1]) if len(sys.argv) > 1 else newest()
|
||||
chars: collections.Counter[str] = collections.Counter()
|
||||
tools: collections.Counter[str] = collections.Counter()
|
||||
for line in path.open():
|
||||
try:
|
||||
entry = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
content = (entry.get("message") or {}).get("content")
|
||||
kind = entry.get("type", "?")
|
||||
if isinstance(content, str):
|
||||
chars[kind] += len(content)
|
||||
elif isinstance(content, list):
|
||||
for block in content:
|
||||
block_type = block.get("type")
|
||||
if block_type == "text":
|
||||
chars[f"{kind}:text"] += len(block.get("text", ""))
|
||||
elif block_type == "tool_use":
|
||||
tools[str(block.get("name"))] += 1
|
||||
chars["tool_use_input"] += len(json.dumps(block.get("input")))
|
||||
elif block_type == "tool_result":
|
||||
result = block.get("content")
|
||||
chars["tool_result"] += len(result if isinstance(result, str) else json.dumps(result))
|
||||
elif block_type == "thinking":
|
||||
chars["thinking"] += len(block.get("thinking", ""))
|
||||
total = sum(chars.values()) or 1
|
||||
print(f"{path.name}: {total / 1e6:.1f}M chars unique (~{total / 4e6:.2f}M tokens once; re-sent every turn)")
|
||||
for key, value in chars.most_common():
|
||||
print(f" {key:16s} {value / 1e6:6.2f}M {value / total:5.1%}")
|
||||
print("tool calls:", sum(tools.values()), dict(tools.most_common(5)))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,89 @@
|
||||
<?php
|
||||
// Benign twin of benchmarks/fixtures/php-vulnerable/app.php: the same handlers, each one repaired the way its
|
||||
// upstream fix did it. Every quick-mode finding here is a false alert (benchmarks/manifests/php-benign.toml).
|
||||
|
||||
$conn = mysqli_connect('localhost', 'app', 'app', 'app');
|
||||
|
||||
function order_by_code($code)
|
||||
{
|
||||
global $wpdb;
|
||||
return $wpdb->get_var($wpdb->prepare("SELECT id FROM {$wpdb->prefix}orders WHERE code = %s LIMIT 1", $code));
|
||||
}
|
||||
|
||||
function find_user($conn)
|
||||
{
|
||||
$stmt = mysqli_prepare($conn, 'SELECT * FROM users WHERE name = ?');
|
||||
mysqli_stmt_bind_param($stmt, 's', $_GET['name']);
|
||||
mysqli_stmt_execute($stmt);
|
||||
return mysqli_stmt_get_result($stmt);
|
||||
}
|
||||
|
||||
function members_sorted($order)
|
||||
{
|
||||
$order = in_array(strtoupper($order), array('ASC', 'DESC'), true) ? strtoupper($order) : 'ASC';
|
||||
return ' ORDER BY u.user_login ' . ('DESC' === $order ? 'DESC' : 'ASC') . ' ';
|
||||
}
|
||||
|
||||
function create_form($data)
|
||||
{
|
||||
return 'INSERT INTO forms (id, label) VALUES (' . intval($data['id']) . ', "x")';
|
||||
}
|
||||
|
||||
function captcha_cleanup($db, $userAgent)
|
||||
{
|
||||
$delete = sprintf(
|
||||
"DELETE FROM captcha WHERE useragent = '%s'",
|
||||
$db->escape($userAgent)
|
||||
);
|
||||
$db->query($delete);
|
||||
}
|
||||
|
||||
function ping()
|
||||
{
|
||||
system('ping -c 1 ' . escapeshellarg($_GET['host']));
|
||||
}
|
||||
|
||||
function calculate($formula)
|
||||
{
|
||||
return is_numeric($formula) ? (float) $formula : 0.0;
|
||||
}
|
||||
|
||||
function restore_session()
|
||||
{
|
||||
return json_decode($_COOKIE['session'], true);
|
||||
}
|
||||
|
||||
function greet()
|
||||
{
|
||||
echo 'Hello ' . htmlspecialchars($_GET['name'], ENT_QUOTES);
|
||||
}
|
||||
|
||||
function page()
|
||||
{
|
||||
$pages = array('home', 'about');
|
||||
include in_array($_GET['page'], $pages, true) ? $_GET['page'] . '.php' : 'home.php';
|
||||
}
|
||||
|
||||
function download()
|
||||
{
|
||||
readfile('/var/www/files/' . basename($_GET['file']));
|
||||
}
|
||||
|
||||
function fetch_preview()
|
||||
{
|
||||
$url = filter_var($_GET['url'], FILTER_VALIDATE_URL);
|
||||
$ch = curl_init(wp_http_validate_url($url) ? $url : 'about:blank');
|
||||
return curl_exec($ch);
|
||||
}
|
||||
|
||||
function go_back()
|
||||
{
|
||||
wp_safe_redirect(wp_validate_redirect(wp_unslash($_GET['next']), home_url()));
|
||||
}
|
||||
|
||||
function find_order($conn)
|
||||
{
|
||||
$id = intval($_POST['id']);
|
||||
$sql = 'SELECT * FROM orders WHERE id = ' . $id;
|
||||
return mysqli_query($conn, $sql);
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
<?php
|
||||
// Intentionally vulnerable PHP fixture for the quick-mode PHP rules. Do not deploy.
|
||||
// Each handler has one labeled weakness (benchmarks/manifests/php-vulnerable.toml); the fixed twin is
|
||||
// benchmarks/fixtures/php-benign/app.php.
|
||||
|
||||
$conn = mysqli_connect('localhost', 'app', 'app', 'app');
|
||||
|
||||
function order_by_code($code)
|
||||
{
|
||||
global $wpdb;
|
||||
return $wpdb->get_var("SELECT id FROM {$wpdb->prefix}orders WHERE code = '" . $code . "' LIMIT 1");
|
||||
}
|
||||
|
||||
function find_user($conn)
|
||||
{
|
||||
return mysqli_query($conn, "SELECT * FROM users WHERE name = '" . $_GET['name'] . "'");
|
||||
}
|
||||
|
||||
function members_sorted($order)
|
||||
{
|
||||
$order = esc_sql($order); // escaping does not protect an unquoted ORDER BY
|
||||
return " ORDER BY u.user_login {$order} ";
|
||||
}
|
||||
|
||||
function create_form($data)
|
||||
{
|
||||
return 'INSERT INTO forms (id, label) VALUES (' . $data['id'] . ', "x")';
|
||||
}
|
||||
|
||||
function captcha_cleanup($db, $userAgent)
|
||||
{
|
||||
$delete = sprintf(
|
||||
"DELETE FROM captcha WHERE useragent = '%s'",
|
||||
$userAgent
|
||||
);
|
||||
$db->query($delete);
|
||||
}
|
||||
|
||||
function ping()
|
||||
{
|
||||
system('ping -c 1 ' . $_GET['host']);
|
||||
}
|
||||
|
||||
function calculate($formula)
|
||||
{
|
||||
eval('$value = ' . $formula . ';');
|
||||
return $value;
|
||||
}
|
||||
|
||||
function restore_session()
|
||||
{
|
||||
return unserialize($_COOKIE['session']);
|
||||
}
|
||||
|
||||
function greet()
|
||||
{
|
||||
echo 'Hello ' . $_GET['name'];
|
||||
}
|
||||
|
||||
function page()
|
||||
{
|
||||
include $_GET['page'] . '.php';
|
||||
}
|
||||
|
||||
function download()
|
||||
{
|
||||
readfile('/var/www/files/' . $_GET['file']);
|
||||
}
|
||||
|
||||
function fetch_preview()
|
||||
{
|
||||
$ch = curl_init($_GET['url']);
|
||||
return curl_exec($ch);
|
||||
}
|
||||
|
||||
function go_back()
|
||||
{
|
||||
header('Location: ' . $_GET['next']);
|
||||
}
|
||||
|
||||
// Not visible to a line rule: the value reaches the sink through a variable on another line.
|
||||
function find_order($conn)
|
||||
{
|
||||
$id = $_POST['id'];
|
||||
$sql = "SELECT * FROM orders WHERE id = '" . $id . "'";
|
||||
return mysqli_query($conn, $sql);
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
<?php
|
||||
// Safe equivalents. None of these lines should produce a finding.
|
||||
|
||||
function order_by_code_safe($code)
|
||||
{
|
||||
global $wpdb;
|
||||
return $wpdb->get_var($wpdb->prepare("SELECT id FROM {$wpdb->prefix}orders WHERE code = %s LIMIT 1", $code));
|
||||
}
|
||||
|
||||
function orders_in_safe($ids)
|
||||
{
|
||||
global $wpdb;
|
||||
$ids_format = implode(', ', array_fill(0, count($ids), '%d'));
|
||||
return $wpdb->prepare('SELECT * FROM orders WHERE id IN (' . $ids_format . ')', $ids);
|
||||
}
|
||||
|
||||
function page_limit_safe($page)
|
||||
{
|
||||
return ' LIMIT ' . intval($page) . ', 20';
|
||||
}
|
||||
|
||||
function quoted_escaped_safe($name)
|
||||
{
|
||||
return "SELECT * FROM users WHERE name = '" . esc_sql($name) . "'";
|
||||
}
|
||||
|
||||
function ping_safe()
|
||||
{
|
||||
system('ping -c 1 ' . escapeshellarg($_GET['host']));
|
||||
}
|
||||
|
||||
function version_safe()
|
||||
{
|
||||
exec('git describe --tags', $output);
|
||||
return $output;
|
||||
}
|
||||
|
||||
function greet_safe()
|
||||
{
|
||||
echo 'Hello ' . htmlspecialchars($_GET['name'], ENT_QUOTES);
|
||||
}
|
||||
|
||||
function download_safe()
|
||||
{
|
||||
readfile('/var/www/files/' . basename($_GET['file']));
|
||||
}
|
||||
|
||||
function restore_safe($raw)
|
||||
{
|
||||
return unserialize($raw, ['allowed_classes' => false]);
|
||||
}
|
||||
|
||||
function query_args_safe()
|
||||
{
|
||||
$args = [];
|
||||
$args['include'] = [(int) $_REQUEST['p']];
|
||||
return $args;
|
||||
}
|
||||
|
||||
function comment_only()
|
||||
{
|
||||
// system('rm -rf ' . $_GET['dir']) would be dangerous but this is a comment
|
||||
# eval($_POST['code']) is a comment too
|
||||
return 'ok';
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,346 @@
|
||||
"""Run and score the pre-registered evaluation of the independent population (protocol-v1.md).
|
||||
|
||||
`run` exports each case's four pins (vulnerable, fixed, benign base, benign tip) with `git archive` and scans each
|
||||
for the case's family through the production scan (`benchmarks/push/finding_dump.py`) inside the shipped image,
|
||||
one container at a time. It is resumable: a scan whose result exists is not repeated. A scan that asked no
|
||||
question is an instrument failure and is retried once.
|
||||
|
||||
`score` applies protocol-v1.md exactly -- matching, detection, fixed-side and benign false alerts, unanswered --
|
||||
and draws the precision sample (every third pooled vulnerable-pin finding, or all of them if 15 or fewer).
|
||||
|
||||
Protocol v2 (protocol-v2.md, population-v2.toml) scans the vulnerable and fixed pins twice each and credits a
|
||||
finding at a declared site as well; `--population population-v2.toml` selects it for both commands.
|
||||
|
||||
Usage:
|
||||
python benchmarks/independent/evaluate.py run --source <frozen analyzer tree> [--only ID,...] [--population F]
|
||||
python benchmarks/independent/evaluate.py score [--population F]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
ROOT = HERE.parents[1]
|
||||
POPULATION = HERE / "population-v1.toml"
|
||||
CACHE = Path.home() / ".cache" / "openultrasast" / "independent"
|
||||
RESULTS = Path.home() / "ousast-results" / "independent-v1"
|
||||
DEADLINE = 2400
|
||||
LINE_WINDOW = 10
|
||||
|
||||
|
||||
def cases() -> list[dict]:
|
||||
return tomllib.loads(POPULATION.read_text())["case"]
|
||||
|
||||
|
||||
def version() -> str:
|
||||
return POPULATION.stem.removeprefix("population-")
|
||||
|
||||
|
||||
def use(population: Path) -> None:
|
||||
"""Point the module at a population file and its own results directory."""
|
||||
global POPULATION, RESULTS
|
||||
POPULATION = population if population.is_absolute() else HERE / population.name
|
||||
RESULTS = Path.home() / "ousast-results" / f"independent-{version()}"
|
||||
|
||||
|
||||
def pins(case: dict) -> dict[str, str]:
|
||||
if version() != "v1":
|
||||
# Protocol v2: the vulnerable and fixed pins twice each, the benign pair once.
|
||||
return {
|
||||
"vulnerable_a": case["vulnerable"],
|
||||
"vulnerable_b": case["vulnerable"],
|
||||
"fixed_a": case["fixed"],
|
||||
"fixed_b": case["fixed"],
|
||||
"benign_base": case["benign"]["base"],
|
||||
"benign_tip": case["benign"]["tip"],
|
||||
}
|
||||
return {
|
||||
"vulnerable": case["vulnerable"],
|
||||
"fixed": case["fixed"],
|
||||
"benign_base": case["benign"]["base"],
|
||||
"benign_tip": case["benign"]["tip"],
|
||||
}
|
||||
|
||||
|
||||
def export(case: dict, label: str, commit: str) -> Path:
|
||||
target = RESULTS / "checkouts" / case["id"] / label
|
||||
if target.exists():
|
||||
shutil.rmtree(target)
|
||||
target.mkdir(parents=True)
|
||||
archive = target.parent / f"{label}.tar"
|
||||
with archive.open("wb") as stream:
|
||||
subprocess.run(["git", "-C", str(CACHE / case["id"]), "archive", commit], stdout=stream, check=True)
|
||||
subprocess.run(["tar", "-xf", str(archive), "-C", str(target)], check=True)
|
||||
archive.unlink()
|
||||
if not any(target.rglob("*")):
|
||||
raise RuntimeError(f"{case['id']} {label}: the export is empty")
|
||||
return target
|
||||
|
||||
|
||||
def scan(case: dict, label: str, checkout: Path, source: Path, out: Path) -> dict:
|
||||
command = [
|
||||
"docker", "run", "--rm", "--init", "--network", "none", "--memory", "3g", "--entrypoint", "python",
|
||||
"-e", "OUSAST_LOG_LEVEL=WARNING", "-e", "PYTHONPATH=/new/src",
|
||||
"-v", f"{source / 'src'}:/new/src:ro",
|
||||
"-v", f"{source / 'benchmarks' / 'push' / 'finding_dump.py'}:/dump.py:ro",
|
||||
"-v", f"{checkout}:/case:ro", "-v", f"{out.parent}:/out",
|
||||
"openultrasast:dev", "/dump.py", "--root", "/case", "--families", case["family"],
|
||||
"--out", f"/out/{out.name}", "--deadline", str(DEADLINE), "--max-regions", "5000",
|
||||
] # fmt: skip
|
||||
completed = subprocess.run(command, capture_output=True, text=True, timeout=DEADLINE + 900, check=False)
|
||||
(out.parent / f"{out.stem}.log").write_text((completed.stdout or "") + "\n" + (completed.stderr or "")[-20000:])
|
||||
if not out.is_file():
|
||||
return {"instrument_error": f"no result (exit {completed.returncode})"}
|
||||
result = json.loads(out.read_text())
|
||||
if not result.get("questions"):
|
||||
return {"instrument_error": "no question was asked"}
|
||||
return result
|
||||
|
||||
|
||||
def run(args: argparse.Namespace) -> int:
|
||||
only = {c.strip() for c in args.only.split(",") if c.strip()}
|
||||
for case in cases():
|
||||
if only and case["id"] not in only:
|
||||
continue
|
||||
for label, commit in pins(case).items():
|
||||
out = RESULTS / "scans" / f"{case['id']}--{label}.json"
|
||||
if out.is_file():
|
||||
continue
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
for attempt in (1, 2):
|
||||
checkout = export(case, label, commit)
|
||||
result = scan(case, label, checkout, args.source, out)
|
||||
shutil.rmtree(checkout, ignore_errors=True)
|
||||
if "instrument_error" not in result:
|
||||
break
|
||||
print(json.dumps({"case": case["id"], "pin": label, "attempt": attempt, **result}), flush=True)
|
||||
out.unlink(missing_ok=True)
|
||||
else:
|
||||
out.write_text(json.dumps({"instrument_error": result["instrument_error"], "questions": 0, "findings": []}) + "\n")
|
||||
summary = json.loads(out.read_text())
|
||||
keys = ("questions", "completed", "seconds")
|
||||
progress = {"case": case["id"], "pin": label, "findings": len(summary.get("findings", [])), **{k: summary.get(k) for k in keys}}
|
||||
print(json.dumps(progress), flush=True)
|
||||
return 0
|
||||
|
||||
|
||||
def hunks(case: dict, side: str) -> dict[str, list[tuple[int, int]]]:
|
||||
"""Changed line ranges per file on one side of the fix (`old` = vulnerable, `new` = fixed)."""
|
||||
diff = subprocess.run(
|
||||
["git", "-C", str(CACHE / case["id"]), "diff", "-U0", case["vulnerable"], case["fixed"]], capture_output=True, text=True, check=True
|
||||
).stdout
|
||||
ranges: dict[str, list[tuple[int, int]]] = {}
|
||||
current = ""
|
||||
for line in diff.splitlines():
|
||||
if (line.startswith("--- a/") and side == "old") or (line.startswith("+++ b/") and side == "new"):
|
||||
current = line[6:]
|
||||
elif line.startswith("@@") and current:
|
||||
match = re.search(r"-(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))?", line)
|
||||
if match:
|
||||
start, count = (match.group(1), match.group(2)) if side == "old" else (match.group(3), match.group(4))
|
||||
first, length = int(start), int(count) if count is not None else 1
|
||||
ranges.setdefault(current, []).append((first, first + max(length, 1) - 1))
|
||||
return ranges
|
||||
|
||||
|
||||
def named_functions(case: dict) -> set[str]:
|
||||
names: set[str] = set()
|
||||
for field in ("sink", "fix_site"):
|
||||
_, _, rest = str(case.get(field, "")).partition("::")
|
||||
names.update(n.split("::")[-1] for n in re.findall(r"[A-Za-z_][A-Za-z0-9_:]*(?=\s*(?:\(|,|;|$))", rest.split("(")[0] + ","))
|
||||
return {n for n in names if n not in {"used", "by", "source", "callers", "and", "from", "the"}}
|
||||
|
||||
|
||||
def matches(finding: dict, case: dict, ranges: dict[str, list[tuple[int, int]]], functions: set[str]) -> bool:
|
||||
path, _, rest = finding["site"].partition(":")
|
||||
line_text, _, function = rest.partition(":")
|
||||
if finding.get("family") != case["family"] or path not in ranges:
|
||||
return False
|
||||
if function in functions:
|
||||
return True
|
||||
line = int(line_text) if line_text.isdigit() else -1
|
||||
return any(first - LINE_WINDOW <= line <= last + LINE_WINDOW for first, last in ranges[path])
|
||||
|
||||
|
||||
def wilson(successes: int, total: int) -> list[float] | None:
|
||||
if not total:
|
||||
return None
|
||||
z, p = 1.96, successes / total
|
||||
centre, spread = (
|
||||
(p + z * z / (2 * total)) / (1 + z * z / total),
|
||||
z * math.sqrt(p * (1 - p) / total + z * z / (4 * total * total)) / (1 + z * z / total),
|
||||
)
|
||||
return [round(max(0.0, centre - spread), 3), round(min(1.0, centre + spread), 3)]
|
||||
|
||||
|
||||
def score(args: argparse.Namespace) -> int:
|
||||
per_case, pool = [], []
|
||||
for case in cases():
|
||||
scans = {
|
||||
label: json.loads((RESULTS / "scans" / f"{case['id']}--{label}.json").read_text())
|
||||
for label in pins(case)
|
||||
if (RESULTS / "scans" / f"{case['id']}--{label}.json").is_file()
|
||||
}
|
||||
if len(scans) < 4:
|
||||
per_case.append({"id": case["id"], "status": "incomplete", "scans": sorted(scans)})
|
||||
continue
|
||||
old, new, functions = hunks(case, "old"), hunks(case, "new"), named_functions(case)
|
||||
vulnerable, fixed = scans["vulnerable"].get("findings", []), scans["fixed"].get("findings", [])
|
||||
detected = [f["site"] for f in vulnerable if matches(f, case, old, functions)]
|
||||
fixed_alerts = [f["site"] for f in fixed if matches(f, case, new, functions)]
|
||||
base_regions = {tuple(f["site"].split(":")[0::2]) for f in scans["benign_base"].get("findings", [])}
|
||||
benign_alerts = [
|
||||
f["site"] for f in scans["benign_tip"].get("findings", []) if tuple(f["site"].split(":")[0::2]) not in base_regions
|
||||
]
|
||||
answered = any(r.split(":")[0] in old for r in scans["vulnerable"].get("completed_regions", []))
|
||||
errors = {label: s["instrument_error"] for label, s in scans.items() if "instrument_error" in s}
|
||||
per_case.append(
|
||||
{
|
||||
"id": case["id"], "family": case["family"], "language": case["language"],
|
||||
"detected": bool(detected), "detected_sites": detected,
|
||||
"unanswered": not detected and not answered, "instrument_errors": errors,
|
||||
"fixed_side_alerts": fixed_alerts, "benign_alerts": benign_alerts,
|
||||
"completion": {label: [s.get("completed"), s.get("questions")] for label, s in scans.items()},
|
||||
"seconds": {label: s.get("seconds") for label, s in scans.items()},
|
||||
}
|
||||
) # fmt: skip
|
||||
pool += [{"case": case["id"], **f} for f in vulnerable]
|
||||
pool.sort(key=lambda f: (f["case"], f["site"]))
|
||||
sample = pool if len(pool) <= 15 else pool[0::3]
|
||||
scored = [c for c in per_case if c.get("status") != "incomplete"]
|
||||
detected = sum(c["detected"] for c in scored)
|
||||
result = {
|
||||
"protocol": "protocol-v1.md",
|
||||
"cases_scored": len(scored), "cases_incomplete": [c["id"] for c in per_case if c.get("status") == "incomplete"],
|
||||
"recall": [detected, len(scored)], "recall_wilson95": wilson(detected, len(scored)),
|
||||
"fixed_side_alerts": sum(len(c["fixed_side_alerts"]) for c in scored),
|
||||
"benign_alerts": sum(len(c["benign_alerts"]) for c in scored),
|
||||
"pooled_vulnerable_findings": len(pool), "precision_sample_size": len(sample),
|
||||
"cases": per_case, "precision_sample": sample,
|
||||
} # fmt: skip
|
||||
(RESULTS / "score.json").write_text(json.dumps(result, indent=2) + "\n")
|
||||
print(json.dumps({k: v for k, v in result.items() if k not in ("cases", "precision_sample")}, indent=1))
|
||||
return 0
|
||||
|
||||
|
||||
def matches_v2(finding: dict, case: dict, ranges: dict[str, list[tuple[int, int]]]) -> bool:
|
||||
"""protocol-v2.md: v1's rule with the record's declared site functions, or a declared `path::function`."""
|
||||
path, _, rest = finding["site"].partition(":")
|
||||
line_text, _, function = rest.partition(":")
|
||||
if finding.get("family") != case["family"]:
|
||||
return False
|
||||
sites = set(case.get("sites", []))
|
||||
if f"{path}::{function}" in sites:
|
||||
return True
|
||||
if path not in ranges:
|
||||
return False
|
||||
if function in {site.partition("::")[2] for site in sites}:
|
||||
return True
|
||||
line = int(line_text) if line_text.isdigit() else -1
|
||||
return any(first - LINE_WINDOW <= line <= last + LINE_WINDOW for first, last in ranges[path])
|
||||
|
||||
|
||||
def stability(one: list[dict], two: list[dict]) -> tuple[int, int]:
|
||||
"""(findings in exactly one run, findings in either)."""
|
||||
a, b = {f["site"] for f in one}, {f["site"] for f in two}
|
||||
return len(a ^ b), len(a | b)
|
||||
|
||||
|
||||
def score_v2() -> int:
|
||||
per_case, pool, unstable_pool = [], [], 0
|
||||
changing, total_questions, total_completed = [0, 0], 0, 0
|
||||
for case in cases():
|
||||
path = lambda label: RESULTS / "scans" / f"{case['id']}--{label}.json" # noqa: E731
|
||||
scans = {label: json.loads(path(label).read_text()) for label in pins(case) if path(label).is_file()}
|
||||
if len(scans) < 6:
|
||||
per_case.append({"id": case["id"], "status": "incomplete", "scans": sorted(scans)})
|
||||
continue
|
||||
old, new = hunks(case, "old"), hunks(case, "new")
|
||||
va, vb = scans["vulnerable_a"].get("findings", []), scans["vulnerable_b"].get("findings", [])
|
||||
fa, fb = scans["fixed_a"].get("findings", []), scans["fixed_b"].get("findings", [])
|
||||
hit_a = {f["site"] for f in va if matches_v2(f, case, old)}
|
||||
hit_b = {f["site"] for f in vb if matches_v2(f, case, old)}
|
||||
fixed_alerts = sorted({f["site"] for f in fa + fb if matches_v2(f, case, new)})
|
||||
base_regions = {tuple(f["site"].split(":")[0::2]) for f in scans["benign_base"].get("findings", [])}
|
||||
# protocol-v2.md: benign alerts and the precision pool are findings OF THE CASE FAMILY. The engine's dump
|
||||
# emits only that family, so this was invisible until the hunter, whose findings span every family.
|
||||
benign_alerts = [
|
||||
f["site"]
|
||||
for f in scans["benign_tip"].get("findings", [])
|
||||
if f.get("family") == case["family"] and tuple(f["site"].split(":")[0::2]) not in base_regions
|
||||
]
|
||||
site_files = {site.partition("::")[0] for site in case.get("sites", [])}
|
||||
covered = set(old) | site_files
|
||||
answered = any(r.split(":")[0] in covered for label in ("vulnerable_a", "vulnerable_b") for r in scans[label].get("completed_regions", []))
|
||||
diff_v, union_v = stability(va, vb)
|
||||
diff_f, union_f = stability(fa, fb)
|
||||
changing[0] += diff_v + diff_f
|
||||
changing[1] += union_v + union_f
|
||||
for scan in scans.values():
|
||||
total_questions += int(scan.get("questions") or 0)
|
||||
total_completed += int(scan.get("completed") or 0)
|
||||
detected = sorted(hit_a & hit_b)
|
||||
per_case.append(
|
||||
{
|
||||
"id": case["id"], "family": case["family"], "language": case["language"],
|
||||
"detected": bool(detected), "detected_sites": detected,
|
||||
"unstable": bool(hit_a ^ hit_b) and not detected, "unstable_sites": sorted(hit_a ^ hit_b),
|
||||
"unanswered": not (hit_a | hit_b) and not answered,
|
||||
"instrument_errors": {label: s["instrument_error"] for label, s in scans.items() if "instrument_error" in s},
|
||||
"fixed_side_alerts": fixed_alerts, "benign_alerts": benign_alerts,
|
||||
"stability": {"vulnerable": [diff_v, union_v], "fixed": [diff_f, union_f]},
|
||||
"completion": {label: [s.get("completed"), s.get("questions")] for label, s in scans.items()},
|
||||
"seconds": {label: s.get("seconds") for label, s in scans.items()},
|
||||
}
|
||||
) # fmt: skip
|
||||
stable = {f["site"] for f in vb if f.get("family") == case["family"]}
|
||||
pool += [{"case": case["id"], **f} for f in va if f.get("family") == case["family"] and f["site"] in stable]
|
||||
unstable_pool += len({f["site"] for f in va if f.get("family") == case["family"]} ^ stable)
|
||||
pool.sort(key=lambda f: (f["case"], f["site"]))
|
||||
sample = pool if len(pool) <= 15 else pool[0::3]
|
||||
scored = [c for c in per_case if c.get("status") != "incomplete"]
|
||||
detected = sum(c["detected"] for c in scored)
|
||||
result = {
|
||||
"protocol": "protocol-v2.md",
|
||||
"cases_scored": len(scored), "cases_incomplete": [c["id"] for c in per_case if c.get("status") == "incomplete"],
|
||||
"recall": [detected, len(scored)], "recall_wilson95": wilson(detected, len(scored)),
|
||||
"unstable_detections": [c["id"] for c in scored if c["unstable"]],
|
||||
"fixed_side_alerts": sum(len(c["fixed_side_alerts"]) for c in scored),
|
||||
"benign_alerts": sum(len(c["benign_alerts"]) for c in scored),
|
||||
"completion": [total_completed, total_questions],
|
||||
"stability": changing,
|
||||
"pooled_stable_vulnerable_findings": len(pool), "unstable_vulnerable_findings": unstable_pool,
|
||||
"precision_sample_size": len(sample),
|
||||
"cases": per_case, "precision_sample": sample,
|
||||
} # fmt: skip
|
||||
(RESULTS / "score.json").write_text(json.dumps(result, indent=2) + "\n")
|
||||
print(json.dumps({k: v for k, v in result.items() if k not in ("cases", "precision_sample")}, indent=1))
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
runner = sub.add_parser("run")
|
||||
runner.add_argument("--source", type=Path, required=True, help="a frozen export of the analyzer (freeze_source.sh)")
|
||||
runner.add_argument("--only", default="")
|
||||
runner.add_argument("--population", type=Path, default=POPULATION)
|
||||
scorer = sub.add_parser("score")
|
||||
scorer.add_argument("--population", type=Path, default=POPULATION)
|
||||
args = parser.parse_args()
|
||||
use(args.population)
|
||||
if args.command == "run":
|
||||
return run(args)
|
||||
return score(args) if version() == "v1" else score_v2()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,334 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"population": "population-v1.toml",
|
||||
"cases": [
|
||||
{
|
||||
"id": "wp-ultimate-member-sqli",
|
||||
"repo": "https://github.com/ultimatemember/ultimatemember",
|
||||
"vulnerable": "a49b3061ded1630e4f6860b4ebece5ccf0a493d2",
|
||||
"fixed": "476de92f4c9aaa56c3e2f5527b242ce0476f4655",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "includes/core/class-member-directory-meta.php",
|
||||
"sink_functions": [
|
||||
"ajax_get_members"
|
||||
],
|
||||
"sink_sha256_vulnerable": "fc6f64e3570f69637a05d747c24bc6fd5e5b350e0a9b1639ef1114d76177c719",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "2f7a6a913a1ac0c0bed728c8b53d4a32435bafe4f50756f7d8fe30effc789fd7",
|
||||
"fix_changed_files": 1,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "13151ef558eb6998720cb1ee5f78b5f17535c16c",
|
||||
"tip": "9130a3226ffec45569a2872ffac338200bd1bfcb",
|
||||
"subject": "- wpcs;",
|
||||
"manual": true
|
||||
},
|
||||
"benign_review": "replaced: the proposal ea316d04 changed account-status/deactivation logic (authorization-adjacent); 9130a322 is a WPCS reformat of class-dependencies.php"
|
||||
},
|
||||
{
|
||||
"id": "wp-learnpress-sqli",
|
||||
"repo": "https://github.com/LearnPress/learnpress",
|
||||
"vulnerable": "73b3cbceef8669056e9da5e2f2c78944eb139af5",
|
||||
"fixed": "aaf42fe0c6bf05dcf2ff125750b7bd836e76fa3c",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "inc/Databases/class-lp-db.php",
|
||||
"sink_functions": [
|
||||
"execute"
|
||||
],
|
||||
"sink_sha256_vulnerable": "5a1e22fa525b56771676e6dd7562c796146e52b96e4e0c11471bf2f0d1d329ce",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "5a1e22fa525b56771676e6dd7562c796146e52b96e4e0c11471bf2f0d1d329ce",
|
||||
"fix_changed_files": 4,
|
||||
"fix_touches_sink_file": false,
|
||||
"fix_touches_fix_site": true,
|
||||
"benign_proposal": {
|
||||
"base": "aaf42fe0c6bf05dcf2ff125750b7bd836e76fa3c",
|
||||
"tip": "179ed7760e6ae5fa46fbe9c5f1324278a42ece04",
|
||||
"subject": "= 4.2.7.1 = ~ Added: lpOnElementReady function js",
|
||||
"files": [
|
||||
"assets/src/js/loadAJAX.js",
|
||||
"assets/src/js/utils.js",
|
||||
"inc/class-lp-assets.php"
|
||||
],
|
||||
"lines": 50
|
||||
},
|
||||
"benign_review": "accepted: JS utility and asset registration"
|
||||
},
|
||||
{
|
||||
"id": "phpmyfaq-captcha-ua-sqli",
|
||||
"repo": "https://github.com/thorsten/phpMyFAQ",
|
||||
"vulnerable": "11583abcf4787543d00c25d6c7926af6f509faf0",
|
||||
"fixed": "545bdffb11244a4741cd29ac909a849c9a6a2e53",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "phpmyfaq/src/phpMyFAQ/Captcha/BuiltinCaptcha.php",
|
||||
"sink_functions": [
|
||||
"garbageCollector",
|
||||
"saveCaptcha"
|
||||
],
|
||||
"sink_sha256_vulnerable": "87e468ef69ea87d20d2bafe4ec3a2b5a796fa96faaaac114334e98e72f85a5fb",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "37c38d1fc7a8ee0dc3a79b69f3fdfb2c9e0f3b90eef078c027e8884b2975dfbc",
|
||||
"fix_changed_files": 2,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "bf781788ab6b5c2967a09d66216d7580778a2b5a",
|
||||
"tip": "a4601ae5f01761dc0bb91b8c0c3901be91987ba3",
|
||||
"subject": "build: updated version strings to 4.2",
|
||||
"files": [
|
||||
"CHANGELOG.md",
|
||||
"docker-compose.yml",
|
||||
"docs/update.md",
|
||||
"mkdocs.yml",
|
||||
"package.json",
|
||||
"phpmyfaq/src/phpMyFAQ/Setup/AbstractSetup.php",
|
||||
"phpmyfaq/src/phpMyFAQ/Setup/Installer.php",
|
||||
"phpmyfaq/src/phpMyFAQ/Setup/Update.php",
|
||||
"phpmyfaq/src/phpMyFAQ/Setup/UpdateRunner.php",
|
||||
"phpmyfaq/src/phpMyFAQ/System.php",
|
||||
"scripts/version.sh",
|
||||
"tests/phpMyFAQ/SystemTest.php"
|
||||
],
|
||||
"lines": 157
|
||||
},
|
||||
"benign_review": "accepted: version strings"
|
||||
},
|
||||
{
|
||||
"id": "yeswiki-bazar-form-sqli",
|
||||
"repo": "https://github.com/YesWiki/yeswiki",
|
||||
"vulnerable": "6f222915df36fca628fc83325cb0feed8dc2aca0",
|
||||
"fixed": "fe7244bffcbeffd42777496ebc2a6f6118cc2388",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "tools/bazar/services/FormManager.php",
|
||||
"sink_functions": [
|
||||
"create"
|
||||
],
|
||||
"sink_sha256_vulnerable": "2051be959575f429ad51367ea6a2e592bc3eb0e86756e93cca3715a942084d3c",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "f71d844466f0466d8b3e5963670c29b6730f5e675de985b466137a03ad3e1ef9",
|
||||
"fix_changed_files": 2,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "03f27fdde5f89f90de76cd3f2160cff82e7bffac",
|
||||
"tip": "70d240c0b43f7e7b0bac55fc0431fb4c2d88b49c",
|
||||
"subject": "fix(contact): send shared page by mail",
|
||||
"files": [
|
||||
"tools/contact/libs/contact.functions.php"
|
||||
],
|
||||
"lines": 7
|
||||
},
|
||||
"benign_review": "accepted: mail receiver string-to-array normalisation and comments"
|
||||
},
|
||||
{
|
||||
"id": "signalk-remote-conn-ssrf",
|
||||
"repo": "https://github.com/SignalK/signalk-server",
|
||||
"vulnerable": "1c479cc0d2d26d14ed74416cff36fa7336c7bf9e",
|
||||
"fixed": "2816b0dda1bdde893d85c969a9c5950e7825eee7",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "src/serverroutes.ts",
|
||||
"sink_functions": [
|
||||
"makeRemoteRequest"
|
||||
],
|
||||
"sink_sha256_vulnerable": "10b2224bfdc0308bf1b7f92ee2f37aaabcfdc6578cda59d245e3e97ddea8185f",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "9aeb864d90df914181683ada2bd49a8e4a86cda4a1a2caa160025225001801bd",
|
||||
"fix_changed_files": 5,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "8af993d45c898edfe28f820f1726b6fc33b27cad",
|
||||
"tip": "07416b79601c5d901f671f70a94bd57bed90bad8",
|
||||
"subject": "fix: allow multiple deltaHandler next calls (#2818)",
|
||||
"files": [
|
||||
"src/deltachain.ts",
|
||||
"test/deltachain.ts"
|
||||
],
|
||||
"lines": 61
|
||||
},
|
||||
"benign_review": "accepted: delta handler chaining fix"
|
||||
},
|
||||
{
|
||||
"id": "saltcorn-sync-sqli",
|
||||
"repo": "https://github.com/saltcorn/saltcorn",
|
||||
"vulnerable": "e7e09254a3c69dc6582eda9e5b5769935db108af",
|
||||
"fixed": "2b6c8a2eab88955ea6b12b1b4e5c2e613faefef5",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "packages/server/routes/sync.js",
|
||||
"sink_functions": [
|
||||
"getSyncRows",
|
||||
"getDelRows"
|
||||
],
|
||||
"sink_sha256_vulnerable": "b69b79171574ba7d8ca4b02d5c0ffcadeb0f885faab016d8066ebf753d839edf",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "5b5d34758caca6601e327953191ca30475193e389bea542549a91f82af2bf8dc",
|
||||
"fix_changed_files": 5,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "5c619403c40447bf56ce662570d3a3ffbf781019",
|
||||
"tip": "df114765fd36d280395b2129f1703fa98c712d06",
|
||||
"subject": "updated at in trigger.findAllWithTableName",
|
||||
"files": [
|
||||
"packages/saltcorn-data/models/trigger.ts"
|
||||
],
|
||||
"lines": 2
|
||||
},
|
||||
"benign_review": "accepted: adds a constant column to a SQL select -- a query edited without any input, a useful negative control"
|
||||
},
|
||||
{
|
||||
"id": "mongo-express-bson-rce",
|
||||
"repo": "https://github.com/mongo-express/mongo-express",
|
||||
"vulnerable": "ea02b364d43f179f191fc91fb9962efdb0843a8d",
|
||||
"fixed": "d8c9bda46a204ecba1d35558452685cd0674e6f2",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "lib/bson.js",
|
||||
"sink_functions": [
|
||||
"toBSON"
|
||||
],
|
||||
"sink_sha256_vulnerable": "09032fcec4e4b0782f94bb69bd946afeee9794991daa4b6a9d3c7aa0b1b5531c",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "a9fa97927fbbb8460902813beb8b99e7d23cda6be7f3761420259aaaf9268640",
|
||||
"fix_changed_files": 6,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "9550ae2529258b47a2b637d8e64e69a0d4d38ad8",
|
||||
"tip": "5b9b34f8553d004ceb7b666bbf566b406a2a3153",
|
||||
"subject": "Merge pull request #538 from damien-malescot/master",
|
||||
"files": [
|
||||
"lib/routes/document.js",
|
||||
"lib/views/document.html"
|
||||
],
|
||||
"lines": 7
|
||||
},
|
||||
"benign_review": "accepted: UUID document-delete fix"
|
||||
},
|
||||
{
|
||||
"id": "mockoon-admin-api-cors",
|
||||
"repo": "https://github.com/mockoon/mockoon",
|
||||
"vulnerable": "b42bdfb7f82e83f0e81bea8e6fe41adf5ec82585",
|
||||
"fixed": "c420b5a56918475b8663977b51e5f986e45b3299",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "packages/commons-server/src/libs/server/admin-api.ts",
|
||||
"sink_functions": [
|
||||
"middleware"
|
||||
],
|
||||
"sink_sha256_vulnerable": "1beb42bf35daa8c7291a1f75976a7012cdd29411d909218a27544ec06eae92f9",
|
||||
"sink_evidence_found": true,
|
||||
"sink_sha256_fixed": "81866aec8e3a67f3aef78c40980193dab841934102fce53ca4f53c73c2ed6eab",
|
||||
"fix_changed_files": 34,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "de69f5e30f89e17918855d147f4e8c055d5b2ba9",
|
||||
"tip": "895961070b55a14d290ad50232ab9c6d800c3cad",
|
||||
"subject": "Fix serverless lib returning a 404 when no body is present (#2274)",
|
||||
"files": [
|
||||
"packages/commons-server/src/libs/server/server.ts",
|
||||
"packages/commons-server/test/specs/server/server-body-parsing.test.ts"
|
||||
],
|
||||
"lines": 40
|
||||
},
|
||||
"benign_review": "accepted: serverless 404 on empty body"
|
||||
},
|
||||
{
|
||||
"id": "alerta-query-sqli",
|
||||
"repo": "https://github.com/alerta/alerta",
|
||||
"vulnerable": "71c098d8bb8091e76edaefeef4662eb77c753ce7",
|
||||
"fixed": "fdd52cd1abad8d02d1dfb8ecdcdbb43b6af3b883",
|
||||
"problems": [
|
||||
"no benign change found automatically; choose one by hand"
|
||||
],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "alerta/database/backends/postgres/queryparser.py",
|
||||
"sink_functions": [
|
||||
"__repr__"
|
||||
],
|
||||
"sink_sha256_vulnerable": "40884d7e83cc38db30a03b11f9686c8cb4b629c2de74191175516cb4c4f1ad3e",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "d9c13317bb36208bbe83bd305bed1199a9e7a586659ca8256479f6158b9d0ce6",
|
||||
"fix_changed_files": 3,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "b7111d23ae9aefb5064ef107976f62e3836217b6",
|
||||
"tip": "71c098d8bb8091e76edaefeef4662eb77c753ce7",
|
||||
"subject": "fix(deps): replace pkg_resources with importlib.metadata (#2041)",
|
||||
"manual": true
|
||||
},
|
||||
"benign_review": "replaced: no neutral commit after the fix (proposals were login, TLS-verification and filter changes); 71c098d8 replaces pkg_resources with importlib.metadata, both sides still vulnerable"
|
||||
},
|
||||
{
|
||||
"id": "wger-next-open-redirect",
|
||||
"repo": "https://github.com/wger-project/wger",
|
||||
"vulnerable": "d30e85b23fb1307ac6579e9e3ae393c50470cea7",
|
||||
"fixed": "0ba371bbb50cd5668820f240995405e1b2483384",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "wger/core/views/user.py",
|
||||
"sink_functions": [
|
||||
"trainer_login",
|
||||
"use_browser_language"
|
||||
],
|
||||
"sink_sha256_vulnerable": "185582190d001b1670738dde2264dac5e6fef2253cc8595dc862c74a270e906f",
|
||||
"sink_functions_missing": [
|
||||
"use_browser_language"
|
||||
],
|
||||
"sink_sha256_fixed": "c0952e4b1990081906cb90a597a83cabafd1a3f8e2d124c4c9b1e2ab80756534",
|
||||
"fix_changed_files": 4,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "47d646c24178783eb6787ecf26e79dc8f4a394c6",
|
||||
"tip": "c1144533bf9bca487ff905d925b567151d89c1aa",
|
||||
"subject": "Formatting",
|
||||
"files": [
|
||||
"wger/core/api/serializers.py"
|
||||
],
|
||||
"lines": 5
|
||||
},
|
||||
"benign_review": "accepted: formatting"
|
||||
},
|
||||
{
|
||||
"id": "opencve-notification-ssrf",
|
||||
"repo": "https://github.com/opencve/opencve",
|
||||
"vulnerable": "928bbadbb1672051bcfc2a6289a9e6a8f029c106",
|
||||
"fixed": "0af7f894debc95c67fa53e7ba5a6cdecb10c34f0",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "web/projects/notifications.py",
|
||||
"sink_functions": [
|
||||
"_http_post_json"
|
||||
],
|
||||
"sink_sha256_vulnerable": "392b560fdbfe29f0d56661a5a1e74afcca426ce6658eb96d1c0ff12c8cacb1ad",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "e8cc8b410e1e43bbf5356066a800c61a32f7d13f03c449512c761dd67553e933",
|
||||
"fix_changed_files": 8,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "0c8101082ba20628b28fcec9c4611745d32202c2",
|
||||
"tip": "965dc348e225308edabdfef3ecb76f7d73a0c162",
|
||||
"subject": "fix: unlink automation executions before cleaning expired reports",
|
||||
"files": [
|
||||
"scheduler/dags/includes/constants.py",
|
||||
"scheduler/tests/tasks/test_reports.py"
|
||||
],
|
||||
"lines": 118
|
||||
},
|
||||
"benign_review": "accepted: scheduler cleanup of automation executions"
|
||||
}
|
||||
],
|
||||
"reviewed_by": "assistant review of each benign diff; selection of cases by the maintainer",
|
||||
"frozen_on": "2026-09-24"
|
||||
}
|
||||
@@ -0,0 +1,475 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"population": "population-v2.toml",
|
||||
"cases": [
|
||||
{
|
||||
"id": "pgadmin-maintenance-sqli",
|
||||
"repo": "https://github.com/pgadmin-org/pgadmin4",
|
||||
"vulnerable": "d112dc3b962218efc3613fa727d54f0fc5b1d527",
|
||||
"fixed": "cf53953d9ad66c182a08e662c1ba29c5831b486b",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "web/pgadmin/tools/maintenance/__init__.py",
|
||||
"sink_functions": [
|
||||
"create_maintenance_job"
|
||||
],
|
||||
"sink_sha256_vulnerable": "9fa2828e8ff9d3af71b9194a8903eee6301e61f8af33c942f1a447ebbf829ec9",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "e841eedc2e859240c87899b5de1e5a077e36b6eaa4251c82435a425812bf6be6",
|
||||
"fix_changed_files": 4,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "e7eaef0dc002c254e4dd15705df800c448e8eb4c",
|
||||
"tip": "a50a553b0a7e529e5d87f52bdc6d3a5ee3e8b9e8",
|
||||
"subject": "chore: feature tests use sys.executable; sync yarn.lock to package.json",
|
||||
"files": [
|
||||
"web/regression/feature_utils/app_starter.py",
|
||||
"web/yarn.lock"
|
||||
],
|
||||
"lines": 48
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "qwed-sympy-eval",
|
||||
"repo": "https://github.com/QWED-AI/qwed-verification",
|
||||
"vulnerable": "06801f68cbf9c9b5e54cc17f2a61cb2792da7a26",
|
||||
"fixed": "6066b68c0c4f4cc2c3771824822aaa864d082ef8",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "src/qwed_new/api/main.py",
|
||||
"sink_functions": [
|
||||
"verify_math"
|
||||
],
|
||||
"sink_sha256_vulnerable": "bff26eca2c749dbd6bbc20cfd912ef077bd2ab3c62d87b9caea5f6f989a463df",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "7632a218b3ce20eb50ebf76e3f9d0110a9152f9a38bad9f47b752294fe7a505d",
|
||||
"fix_changed_files": 8,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "62e74ad0a1760abbb9986292fef3e2e6125b1187",
|
||||
"tip": "89498ca5eb621a66316b2cf2de1e1e8fdaf6d79d",
|
||||
"subject": "Merge pull request #211 from QWED-AI/fix/codeql-redos-data-uri",
|
||||
"files": [
|
||||
".coverage",
|
||||
"src/qwed_new/core/output_sanitizer.py",
|
||||
"tests/test_output_sanitizer.py"
|
||||
],
|
||||
"lines": 70
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "contextforge-prompt-ssti",
|
||||
"repo": "https://github.com/IBM/mcp-context-forge",
|
||||
"vulnerable": "0f8e0ad679f1fff685f486e17f642ac863c8ca9e",
|
||||
"fixed": "4d31004661858a2e99055b3c0c9f14218b8f7120",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "mcpgateway/services/prompt_service.py",
|
||||
"sink_functions": [
|
||||
"_render_template",
|
||||
"_compile_jinja_template"
|
||||
],
|
||||
"sink_sha256_vulnerable": "0a9752890c5324a7ff01bd15890b376b35226cc81d8853525e41f8b31fb66cb6",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "dd91186b8ee3eed6373761554ad5873e3436b7213305d1d1104f209c6be213af",
|
||||
"fix_changed_files": 27,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "1f8fd58980f4565d4788ffc3587d6e3bd16b3342",
|
||||
"tip": "9938c58f7819c526932c4ae4ad646a9c028cb19e",
|
||||
"subject": "fix: add rust coverage reports (#4458)",
|
||||
"files": [
|
||||
"Makefile",
|
||||
"crates/mcp_runtime/DEVELOPING.md",
|
||||
"tests/unit/test_makefile_rust_targets.py"
|
||||
],
|
||||
"lines": 81
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "lmdeploy-media-url-ssrf",
|
||||
"repo": "https://github.com/InternLM/lmdeploy",
|
||||
"vulnerable": "71648d239c978452720a6d6692fe914efc8a7b94",
|
||||
"fixed": "03c313006d17cc3feae86b633c44206a997c44db",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "lmdeploy/vl/media/connection.py",
|
||||
"sink_functions": [
|
||||
"_load_http_url"
|
||||
],
|
||||
"sink_sha256_vulnerable": "697f169c03cf175c338fee2dddaa81a7649913ed2908e1602bd845806fd19646",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "ec8efc50b1749326454731e395d92ab5430e6dac3cf1f334c067ab25a2eb566b",
|
||||
"fix_changed_files": 12,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "b6a29db4d2d218838048975977f125dc9b1ac75d",
|
||||
"tip": "3d9d23087b59fa44afe83deb03ee7cce04df3b4d",
|
||||
"subject": "fix(benchmark): seed NumPy RNG so --seed controls random-mode lengths (#4785)",
|
||||
"files": [
|
||||
"benchmark/benchmark_guided.py",
|
||||
"benchmark/profile_pipeline_api.py",
|
||||
"benchmark/profile_throughput.py"
|
||||
],
|
||||
"lines": 3
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "winml-cli-serve-cors",
|
||||
"repo": "https://github.com/microsoft/winml-cli",
|
||||
"vulnerable": "02179d185f0e3ac23518236b5ca1f61fd37ef680",
|
||||
"fixed": "f4073e0ef4700a25b623487e7e45c421ca0b9993",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "src/winml/modelkit/serve/cli_api.py",
|
||||
"sink_functions": [],
|
||||
"sink_sha256_vulnerable": "efab45cd65ef1cd0a64070fe104dfdcfc2f298c98def37973c0f22cd0140949f",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "c616799bdaace01883b9e392a3c118eb54295f827eae7f8d7cafcfd40674c012",
|
||||
"fix_changed_files": 12,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "8d631f6f1e5db26a04e8c13045280408c6f984dc",
|
||||
"tip": "be3e59dd412d4918c5a852aa4d8f0207c34aaf6f",
|
||||
"subject": "Remove preload ORT DLL (undo PR 689) (#1380)",
|
||||
"files": [
|
||||
"src/winml/modelkit/__init__.py",
|
||||
"tests/unit/test_modelkit_init.py"
|
||||
],
|
||||
"lines": 158
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "reactpress-findall-column-sqli",
|
||||
"repo": "https://github.com/fecommunity/reactpress",
|
||||
"vulnerable": "c3dd10681b3d2b5e224f7ac647b3769557603975",
|
||||
"fixed": "78ecb70af1c021455c05fdcbe137212c70e310d6",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "server/src/modules/article/article.service.ts",
|
||||
"sink_functions": [
|
||||
"findAll"
|
||||
],
|
||||
"sink_sha256_vulnerable": "33fb3c644ab4ee3ea469cb190a58eacdc2f34f8f2379c04437f3df4d7ede7901",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "8fe3e41ec60351a004455c49d5110fb969acdf2a883b9f22505e66d56c84a3b5",
|
||||
"fix_changed_files": 9,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "78ecb70af1c021455c05fdcbe137212c70e310d6",
|
||||
"tip": "70c7dc39e1a885a1ac7d051c57a409432905fe6c",
|
||||
"subject": "fix(server): correct column whitelist to match entity definitions",
|
||||
"files": [
|
||||
"server/src/utils/query-whitelist.util.ts"
|
||||
],
|
||||
"lines": 23
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "fuxa-tdengine-escape-sqli",
|
||||
"repo": "https://github.com/frangoteam/FUXA",
|
||||
"vulnerable": "193a888f8e3f805325072829b1644a2c768c948b",
|
||||
"fixed": "b78f6159d02a56ce5ff48207feb936afb3534fc8",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "server/runtime/storage/tdengine/index.js",
|
||||
"sink_functions": [
|
||||
"getDaqValue"
|
||||
],
|
||||
"sink_sha256_vulnerable": "e129a1520f736ef53de1c813ca4295b2b9de09f8041477a4e96bcf242b375d0e",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "7f89162c781081034a2b6e525c0a2e29f976ee984651b2345b2b5fefdd9ac456",
|
||||
"fix_changed_files": 2,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "e787e4140b5e71c5b02e23c295fa08b8df0edc53",
|
||||
"tip": "bf998eeee3158e338724c71979b7cdafcfa74b32",
|
||||
"subject": "Button control with configurable icon (#2349)",
|
||||
"files": [
|
||||
"client/src/app/_models/hmi.ts",
|
||||
"client/src/app/framework/icon-selector/icon-selector.component.scss",
|
||||
"client/src/app/gauges/controls/html-button/html-button.component.ts",
|
||||
"client/src/app/gauges/gauge-property/gauge-property.component.html",
|
||||
"client/src/app/gauges/gauge-property/gauge-property.component.scss",
|
||||
"client/src/app/gauges/gauge-property/gauge-property.component.ts",
|
||||
"client/src/assets/i18n/en.json"
|
||||
],
|
||||
"lines": 92
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "budibase-mongo-template-nosqli",
|
||||
"repo": "https://github.com/Budibase/budibase",
|
||||
"vulnerable": "b7e63df737eede85d951aa9665b0c0778ece7ccb",
|
||||
"fixed": "dd8c0654b35cd89ce3645f2355f4bf2ff9a5dd80",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "packages/server/src/sdk/workspace/queries/queries.ts",
|
||||
"sink_functions": [
|
||||
"enrichContext"
|
||||
],
|
||||
"sink_sha256_vulnerable": "27233a212e8227bb0dc5f0045aa64b8afa0ced563b174ddd17f8ff5013a516cb",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "e127d41a1e33b89173b69e802395d753f3cfc0cda06bffa3ae6952e910102ce8",
|
||||
"fix_changed_files": 4,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "a4423fc5bf237e6101c2e8348ef5372fff6484cd",
|
||||
"tip": "272c6a61a2a386688792f9e26c2665f2f431a5c2",
|
||||
"subject": "Merge pull request #18865 from Budibase/feature/18784-ai-credit",
|
||||
"files": [
|
||||
"packages/builder/src/components/usage/Usage.svelte",
|
||||
"packages/builder/src/settings/pages/usage.svelte",
|
||||
"packages/shared-core/src/helpers/tests/url.spec.ts",
|
||||
"packages/shared-core/src/helpers/url.ts"
|
||||
],
|
||||
"lines": 92
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "unleash-addon-webhook-ssrf",
|
||||
"repo": "https://github.com/Unleash/unleash",
|
||||
"vulnerable": "b9f3217d327f24094b4352130baa30387f85cffe",
|
||||
"fixed": "2100db76af3473f13e6fb40096cf17a9c2b741a1",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "src/lib/addons/addon.ts",
|
||||
"sink_functions": [
|
||||
"fetchRetry"
|
||||
],
|
||||
"sink_sha256_vulnerable": "a52627ff0742bd8c296a9f63753174b334f74aba50e013cc8bc18d60e6929aa1",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "7b405bfdf4a981787e70c21f4559e8df94d55f6a56cfc0ab130c7ff8a18549de",
|
||||
"fix_changed_files": 21,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "d4efb3a31616483c438606edbecce13b868fc587",
|
||||
"tip": "cb4e6250743fac364bae50b985b2b8348def7a2b",
|
||||
"subject": "Merge commit '232b5d575c7d455ce7ac767183e61fd389dd8aac' into automation/sync-oss-upstream",
|
||||
"files": [
|
||||
"frontend/src/component/providers/UnleashFlagProvider/UnleashFlagProvider.tsx"
|
||||
],
|
||||
"lines": 13
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "medplum-external-auth-open-redirect",
|
||||
"repo": "https://github.com/medplum/medplum",
|
||||
"vulnerable": "5d4a8654b02aa67c444f96ae1eb5663c09854006",
|
||||
"fixed": "7ae10035ddadde4dba7b18d3156553940465b3a1",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "packages/server/src/auth/external.ts",
|
||||
"sink_functions": [
|
||||
"externalCallbackHandler"
|
||||
],
|
||||
"sink_sha256_vulnerable": "3ab597c5f268210f56232b07094d80771dce0bc3b78996e9246bb31a0930bd53",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "74cc58f9a399550945d7a2266c8bd4dac48a852378e4eea56fb5f2a1deb72084",
|
||||
"fix_changed_files": 7,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "0dd335e8520b859563626d9bf50d602a5f3ad79a",
|
||||
"tip": "20790b17d76f335e60dd703f65d4eb13bbb4754d",
|
||||
"subject": "tweak(cdk): make `minHealthyPercent` 100 (#8836)",
|
||||
"files": [
|
||||
"packages/cdk/src/backend.ts"
|
||||
],
|
||||
"lines": 4
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "cct-studio-server-cmdi",
|
||||
"repo": "https://github.com/davila7/claude-code-templates",
|
||||
"vulnerable": "acdb5893412a81eaec9862ee24a4613bf296ab45",
|
||||
"fixed": "bc4618b07232633c1c0aac12a43e436268d31783",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "cli-tool/src/sandbox-server.js",
|
||||
"sink_functions": [
|
||||
"executeLocalTask",
|
||||
"checkAndInstallAgent"
|
||||
],
|
||||
"sink_sha256_vulnerable": "e2c24dcf02b90e5056c2b7065ce46027ef02139991117ee70f589a3a971e974d",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "2414d562c0714689978808349499af8cadf7cf39a28a534194f2ff40954aa25e",
|
||||
"fix_changed_files": 4,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": null
|
||||
},
|
||||
{
|
||||
"id": "wp-gopay-log-filter-sqli",
|
||||
"repo": "https://github.com/gopaycommunity/gopay-woocommerce-integration",
|
||||
"vulnerable": "626bc8b5f8d33de6eefd921e3a05ed52ac320bcd",
|
||||
"fixed": "7190c985370f9758f226c2f2addd79532a2f93d5",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "admin/views/log.php",
|
||||
"sink_functions": [],
|
||||
"sink_sha256_vulnerable": "541b921b7810a7b563a4ffefd2ac296e5b37f81d970c5fd9dd787f107c0593b6",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "bdc6178b33d87bcc149019555239668bd71a9bd0f9c30c3ec396ee32974810f4",
|
||||
"fix_changed_files": 1,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "2b5fe58d6dadbfecb01ed34611aed8502d11e612",
|
||||
"tip": "88b10bda704a37e88121e22dc22f4e5019097272",
|
||||
"subject": "GPY025-136: fix i18n and remove translation calls on non-literal text",
|
||||
"files": [
|
||||
"includes/class-gopay-gateway-options.php",
|
||||
"includes/class-gopay-wc-blocks-support.php"
|
||||
],
|
||||
"lines": 11
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wp-user-registration-login-redirect",
|
||||
"repo": "https://github.com/themegrill/user-registration",
|
||||
"vulnerable": "e52cf151cb229852e6209fb3a627aa51485139c1",
|
||||
"fixed": "75c940dab15fd54d5a3fbb9f2213767b89467b08",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "includes/functions-ur-core.php",
|
||||
"sink_functions": [
|
||||
"ur_process_login"
|
||||
],
|
||||
"sink_sha256_vulnerable": "7ec30d26e49ed039448d4e2616000987db57d2067533e0b2015fe9621e38e57f",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "867fccbaa0df3344523aff6516102cdb22e1e7cd8b2eb7b482cdf9e9e3184b06",
|
||||
"fix_changed_files": 8,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "064dd7ab1585082b9b5cf078f2d22c0234f25e48",
|
||||
"tip": "3b6acc4695e93d9d48969ed894c21419bd09b33a",
|
||||
"subject": "Fix - Deactivate Licence button did nothing when the pro-activation transient was set (#1418)",
|
||||
"files": [
|
||||
"includes/class-ur-plugin-updater.php"
|
||||
],
|
||||
"lines": 4
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wp-directorist-avatar-ssrf",
|
||||
"repo": "https://github.com/sovware/directorist",
|
||||
"vulnerable": "77cfc9bc8b453f97ab7e720330706fa4ef056867",
|
||||
"fixed": "1a67825d5cf6e7c768c3a692da1d18bae732bea9",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "includes/rest-api/functions.php",
|
||||
"sink_functions": [
|
||||
"directorist_rest_upload_image_from_url"
|
||||
],
|
||||
"sink_sha256_vulnerable": "fb9e13afbf7f1b856d59a75acca4fc8ebbadb06908af5e409d2b2e7eb470e4b4",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "c451a4853a2ec8a3038741af558f4024cf34edc919a07dd3b16c8f665f834d0f",
|
||||
"fix_changed_files": 7,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": null
|
||||
},
|
||||
{
|
||||
"id": "wp-groundhogg-confirm-redirect",
|
||||
"repo": "https://github.com/groundhoggwp/groundhogg",
|
||||
"vulnerable": "9472a1dd79d4556c40eece501a61930793f170d3",
|
||||
"fixed": "028247d4b8ec9ba051d3eb48f1466a759bc16bfd",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "templates/preferences.php",
|
||||
"sink_functions": [],
|
||||
"sink_sha256_vulnerable": "7887dbbac602984f3c8f754c526dc0656e547c2e66f00e0eb417d56c1af85554",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "081f9b70f08905b7c2c909306747aaacb4a885d7563f455b54638a3a70c611d8",
|
||||
"fix_changed_files": 1,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "61c8a03f11b6030c630be6ae0b0c2ad641eb733a",
|
||||
"tip": "e3a69f1c2814c6bebed940505894008748566b93",
|
||||
"subject": "output a square image regardless of image dimensions",
|
||||
"files": [
|
||||
"includes/utils/html.php"
|
||||
],
|
||||
"lines": 13
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "froxlor-admin-ip-second-order-sqli",
|
||||
"repo": "https://github.com/froxlor/froxlor",
|
||||
"vulnerable": "9bf03de643a2edee4642bfc6c94db1f05253da02",
|
||||
"fixed": "a1eaca5a1601c8a30e00814a4fc73ad0c185f89e",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "lib/Froxlor/Api/Commands/IpsAndPorts.php",
|
||||
"sink_functions": [
|
||||
"listing",
|
||||
"listingCount"
|
||||
],
|
||||
"sink_sha256_vulnerable": "d1494998117a922d6370c9acbc68c9f547207dfae435c9e23a1d4af1786b0854",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "c4db47c2b4b7d62f6494989c30bd682bd1a39136b6b545d79dad8d33fbff6863",
|
||||
"fix_changed_files": 3,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "b595f061fbb68874423809fec9f2a42bf6b34175",
|
||||
"tip": "9d274828bdc4eefec0058a5a3a25bf4eb69d0f70",
|
||||
"subject": "avoid warnings regarding php8.5 in config/database.php; adjust phpunit for froxlor/framework testing",
|
||||
"files": [
|
||||
"composer.json",
|
||||
"config/database.php",
|
||||
"phpunit.xml",
|
||||
"tests/Feature/ExampleTest.php",
|
||||
"tests/Unit/ExampleTest.php"
|
||||
],
|
||||
"lines": 62
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sulu-smart-content-sql2-injection",
|
||||
"repo": "https://github.com/sulu/sulu",
|
||||
"vulnerable": "d061094f5b7bb1d5e974544fce30bede9c7adf8e",
|
||||
"fixed": "d19c01487af8c3de2fb3aa145856707a6367392c",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "src/Sulu/Component/Content/SmartContent/QueryBuilder.php",
|
||||
"sink_functions": [
|
||||
"buildCategoriesWhere",
|
||||
"buildWhere"
|
||||
],
|
||||
"sink_sha256_vulnerable": "421e860f75a4953bef494c24b64e951dc0e3748c7d7f592d73ee419af46c73cc",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "764798a416b14f85c24c9ff7f6342e78fde837dbf1d74207c4284bb162318b99",
|
||||
"fix_changed_files": 2,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "2c4ee09545cf1c9a400006fcaca3b8499cc9a161",
|
||||
"tip": "87d30d8f061102ce114556d72d628224e64529e5",
|
||||
"subject": "Merge branch '2.6' of https://github.com/sulu/sulu into 3.0",
|
||||
"files": [
|
||||
"config/packages/security.yaml",
|
||||
"src/Sulu/Bundle/SecurityBundle/Security/AuthenticationHandler.php",
|
||||
"src/Sulu/Bundle/SecurityBundle/Tests/Application/config/config.yml",
|
||||
"src/Sulu/Bundle/SecurityBundle/Tests/Functional/Controller/LoginControllerTest.php",
|
||||
"src/Sulu/Bundle/SecurityBundle/Tests/Functional/Security/AuthenticationHandlerTest.php",
|
||||
"src/Sulu/Component/Content/Tests/Unit/Types/LinkTest.php",
|
||||
"src/Sulu/Component/Content/Types/Link.php"
|
||||
],
|
||||
"lines": 183
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,561 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"population": "population-v3-php.toml",
|
||||
"cases": [
|
||||
{
|
||||
"id": "gitlist-commandline-cmdi",
|
||||
"repo": "https://github.com/klaussilveira/gitlist",
|
||||
"vulnerable": "f67609d52c1812fa8a7ed80eae5e795cfd72115f",
|
||||
"fixed": "88cf2866083d5f7c20d9d565c45f828a7ad1516b",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "src/SCM/System/Git/CommandLine.php",
|
||||
"sink_functions": [
|
||||
"getDefaultBranch"
|
||||
],
|
||||
"sink_sha256_vulnerable": "a0aa9125dabc9748274b3762b9bd1ad117400b1b8d446593afdc3e6f38951e26",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "3f1c45d9f0ee68c517bf89fa3fe65c0ead84351ed5d9329ac5d6a4b53d575344",
|
||||
"fix_changed_files": 2,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "ce5806c849f453d7200c1f488ee6aba19eef9a4c",
|
||||
"tip": "ecc8e18e03d9b547e82b7c7c148cb00ba01c0cc9",
|
||||
"subject": "Merge pull request #902 from zoredache/sort_repositories",
|
||||
"files": [
|
||||
"src/App/Controller/Repository.php"
|
||||
],
|
||||
"lines": 2,
|
||||
"side": "before the vulnerable pin"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "qloapps-customermessage-bo-query-sqli",
|
||||
"repo": "https://github.com/Qloapps/QloApps",
|
||||
"vulnerable": "5b4d65cbb8570a4ba46364ee0f2ded79d294ab84",
|
||||
"fixed": "123c97c110b7053ea3297d8e58fe95b3c3536560",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sites": [
|
||||
"classes/CustomerMessage.php::searchByName"
|
||||
],
|
||||
"site_basis": {
|
||||
"classes/CustomerMessage.php::searchByName": "resolved-from-fix-diff"
|
||||
},
|
||||
"site_files_unresolved": {},
|
||||
"sink_file": "classes/CustomerMessage.php",
|
||||
"sink_functions": [
|
||||
"searchByName"
|
||||
],
|
||||
"sink_sha256_vulnerable": "f4037f17c83d248639dbb6cce36f35e3b7eaae557dd631bc9268a97415c46534",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "0464ba7047c393a2c873f301717258397bfc50c2b4f423e98f33d0c954678d41",
|
||||
"fix_changed_files": 5,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "884563ccff83fe42b4689ba32e605276278c3989",
|
||||
"tip": "e43b45009508eb1465709d4af0f549faacd1d817",
|
||||
"subject": "Merge pull request #1735 from amitsharma322/gli-3007",
|
||||
"files": [
|
||||
"classes/PaymentModule.php",
|
||||
"controllers/admin/AdminOrdersController.php"
|
||||
],
|
||||
"lines": 23,
|
||||
"side": "after the fix"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "roundcube-markasjunk-cmd-learn-rce",
|
||||
"repo": "https://github.com/roundcube/roundcubemail",
|
||||
"vulnerable": "a1afb8fd1f00ed4cb9376c072bb5ca5ded64495e",
|
||||
"fixed": "b8f90e28a46d42e79a69568cba897f8f4223d9cd",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sites": [
|
||||
"plugins/markasjunk/drivers/cmd_learn.php::_do_salearn"
|
||||
],
|
||||
"site_basis": {
|
||||
"plugins/markasjunk/drivers/cmd_learn.php::_do_salearn": "resolved-from-fix-diff"
|
||||
},
|
||||
"site_files_unresolved": {},
|
||||
"sink_file": "plugins/markasjunk/drivers/cmd_learn.php",
|
||||
"sink_functions": [
|
||||
"_do_salearn"
|
||||
],
|
||||
"sink_sha256_vulnerable": "f4f46e54fa8a19d20ab4000e0f363afd544c6f9f73b806c7b8176fd8348dea83",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "3082db4ed8a79b472ebc2b7d42e34db054737297a81cf595b652a6ba4e34f8eb",
|
||||
"fix_changed_files": 2,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "1cddafded973c4bf6faa173accaf8f52ca1ac0d0",
|
||||
"tip": "747f544708ed1d698b677d333346164a9451aeda",
|
||||
"subject": "Simplify",
|
||||
"files": [
|
||||
"tests/Framework/TnefDecoderTest.php"
|
||||
],
|
||||
"lines": 6,
|
||||
"side": "after the fix"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "maccms10-template-blacklist-rce",
|
||||
"repo": "https://github.com/magicblack/maccms10",
|
||||
"vulnerable": "2cabecda29947e8f9ea7cd3aa47ab90ed817605b",
|
||||
"fixed": "71ad3bb29570e110d8e973acff68040a3050ddf0",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sites": [
|
||||
"application/admin/controller/Template.php::info"
|
||||
],
|
||||
"site_basis": {
|
||||
"application/admin/controller/Template.php::info": "resolved-from-fix-diff"
|
||||
},
|
||||
"site_files_unresolved": {},
|
||||
"sink_file": "application/admin/controller/Template.php",
|
||||
"sink_functions": [
|
||||
"info"
|
||||
],
|
||||
"sink_sha256_vulnerable": "02e0ff0d172a25fc35b5b386c4280cf65bf3980a612cd833020e43f2dddb5d3c",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "310d667aa2a99f554f49649b0ff1549b0c7f35bf1a0c167a86569c49167f5cc9",
|
||||
"fix_changed_files": 1,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "d9f09011a1523d0bfaed5dea9b2139461f2b0ea8",
|
||||
"tip": "668ba418e50e96ab32ac612f3b6528beb172b3b7",
|
||||
"subject": "fix #1356 \u4fee\u6b63\u6ce8\u518cAPI\u6821\u9a8c\u5668\u89e3\u6790\u9519\u8bef\u53ca\u7f3a\u53c2\u62a5\u9519",
|
||||
"files": [
|
||||
"application/api/controller/User.php",
|
||||
"application/common/model/User.php",
|
||||
"tests/regression/user_register_validate.php",
|
||||
"\"\\350\\257\\264\\346\\230\\216\\346\\226\\207\\346\\241\\243/API\\346\\216\\245\\345\\217\\243\\350\\257\\264\\346\\230\\216V2.txt\""
|
||||
],
|
||||
"lines": 166,
|
||||
"side": "after the fix"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "mantisbt-tokenizer-eval-rce",
|
||||
"repo": "https://github.com/mantisbt/mantisbt",
|
||||
"vulnerable": "e7041ecc80c8d35f76a7ea1027a8a231244f7dc5",
|
||||
"fixed": "78c0af63d1fe0118004744cab21ca3bf2cea0f5c",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sites": [
|
||||
"core/classes/Tokenizer.class.php::__construct"
|
||||
],
|
||||
"site_basis": {
|
||||
"core/classes/Tokenizer.class.php::__construct": "resolved-from-fix-diff"
|
||||
},
|
||||
"site_files_unresolved": {
|
||||
"adm_config_set.php": "the fix does not change this file"
|
||||
},
|
||||
"sink_file": "core/classes/Tokenizer.class.php",
|
||||
"sink_functions": [
|
||||
"__construct"
|
||||
],
|
||||
"sink_sha256_vulnerable": "cda7f81f7fcb4675c7bf5fc3f8dfd41ccb48ad891e95a85cf5edae90f0cbb1df",
|
||||
"sink_evidence_found": true,
|
||||
"sink_sha256_fixed": "f555c63e2dc15dbef7ff2c4db8d7c47848c8bc7abe981364a11f65c5e44b8fb0",
|
||||
"fix_changed_files": 1,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "088fa767bce246b6d2f083f32d937f22c8de4767",
|
||||
"tip": "d6ca719b591f70acfa711fc41705388557b6ac04",
|
||||
"subject": "Improve Workflow Graph display.",
|
||||
"files": [
|
||||
"workflow_graph_img.php"
|
||||
],
|
||||
"lines": 14,
|
||||
"side": "after the fix"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sharp-editor-iframe-srcdoc-xss",
|
||||
"repo": "https://github.com/code16/sharp",
|
||||
"vulnerable": "67743149309054493ec8f5505173fc57aaff760f",
|
||||
"fixed": "ec509a22c808a5bd9dfad6a0a85c92ce6f411e21",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sites": [
|
||||
"src/Utils/Sanitization/FormatsSanitizedValue.php::sanitizer",
|
||||
"src/Utils/Sanitization/FormatsSanitizedValue.php::isEncodingNeeded"
|
||||
],
|
||||
"site_basis": {
|
||||
"src/Utils/Sanitization/FormatsSanitizedValue.php::sanitizer": "resolved-from-fix-diff",
|
||||
"src/Utils/Sanitization/FormatsSanitizedValue.php::isEncodingNeeded": "resolved-from-fix-diff"
|
||||
},
|
||||
"site_files_unresolved": {},
|
||||
"sink_file": "src/Utils/Sanitization/FormatsSanitizedValue.php",
|
||||
"sink_functions": [
|
||||
"sanitizer",
|
||||
"isEncodingNeeded"
|
||||
],
|
||||
"sink_sha256_vulnerable": "7e65b0fbcd481c6322f83ae46523b069c4a62383cbf088e7be60be6ace4bcc9a",
|
||||
"sink_evidence_found": true,
|
||||
"sink_sha256_fixed": "59a2845f4a8b419f8504b2cb539087eb3dc3fb8bb0ec7c950330ba951c6c75ad",
|
||||
"fix_changed_files": 2,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "ec509a22c808a5bd9dfad6a0a85c92ce6f411e21",
|
||||
"tip": "7c601fffe930fc321a5d8bcb25cb072d50a6fc08",
|
||||
"subject": "Update version to v9.22.5",
|
||||
"files": [
|
||||
"src/SharpInternalServiceProvider.php"
|
||||
],
|
||||
"lines": 2,
|
||||
"side": "after the fix"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "privatebin-jsonld-request-uri-injection",
|
||||
"repo": "https://github.com/PrivateBin/PrivateBin",
|
||||
"vulnerable": "e0dd4c025c19a182b6a4c6fb77a8bf81ceff6899",
|
||||
"fixed": "75f056dcda955d94c17ec5a4f8c54a9b7bfcee07",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "lib/Request.php",
|
||||
"sink_functions": [
|
||||
"getRequestUri"
|
||||
],
|
||||
"sink_sha256_vulnerable": "bd35439be97ac659d94b1d0cde13f70644f99468bfc16ddc2fb214a845bdb7b4",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "ea17a346372b8259bad1ac7edb1268378168d0de0628cd53b948f52038fd9c1e",
|
||||
"fix_changed_files": 4,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "75f056dcda955d94c17ec5a4f8c54a9b7bfcee07",
|
||||
"tip": "3035d6e5d3cceb7f06f7b7bb80968727879c52b0",
|
||||
"subject": "incrementing version",
|
||||
"files": [
|
||||
"CHANGELOG.md",
|
||||
"Makefile",
|
||||
"README.md",
|
||||
"SECURITY.md",
|
||||
"doc/Installation.md",
|
||||
"js/package.json",
|
||||
"lib/Controller.php"
|
||||
],
|
||||
"lines": 18,
|
||||
"side": "after the fix"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "openstamanager-group-name-html-injection",
|
||||
"repo": "https://github.com/devcode-it/openstamanager",
|
||||
"vulnerable": "57a2734e2121aa8fcd456eff7c59f03e29e69176",
|
||||
"fixed": "0b88ad1963871abfa5e134004ca0c74c496d8d2f",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "modules/utenti/edit.php",
|
||||
"sink_functions": [],
|
||||
"sink_sha256_vulnerable": "8524dd092b2fdbf68339b80e374c4108fd6925a146223b325370f4ee8414a016",
|
||||
"sink_evidence_found": true,
|
||||
"sink_sha256_fixed": "290d5fde2df7713eb592361f3b0abfd3524a3b83bf99ef0fafdf80f0ee29612b",
|
||||
"fix_changed_files": 4,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "0b88ad1963871abfa5e134004ca0c74c496d8d2f",
|
||||
"tip": "1832b85d0e570e0b873c7c23dcfcb6b5d1365a28",
|
||||
"subject": "fix: ricerca riferimenti in importazione fattura per riga con articolo non esistente",
|
||||
"files": [
|
||||
"plugins/importFE/actions.php"
|
||||
],
|
||||
"lines": 2,
|
||||
"side": "after the fix"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wp-geo-my-wp-info-window-lfi",
|
||||
"repo": "https://github.com/Fitoussi/geo-my-wp",
|
||||
"vulnerable": "1dbb7b1672a45a8f88fdcd379ceb07b6cf2beceb",
|
||||
"fixed": "5a768bf1c6e44ded83a65be8789f3587515665ac",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sites": [
|
||||
"plugins/posts-locator/includes/gmw-posts-locator-functions.php::gmw_posts_locator_ajax_info_window_loader",
|
||||
"includes/gmw-functions.php::gmw_ajax_info_window_init"
|
||||
],
|
||||
"site_basis": {
|
||||
"plugins/posts-locator/includes/gmw-posts-locator-functions.php::gmw_posts_locator_ajax_info_window_loader": "advisory-named",
|
||||
"includes/gmw-functions.php::gmw_ajax_info_window_init": "resolved-from-fix-diff"
|
||||
},
|
||||
"site_files_unresolved": {},
|
||||
"sink_file": "plugins/posts-locator/includes/gmw-posts-locator-functions.php",
|
||||
"sink_functions": [
|
||||
"gmw_posts_locator_ajax_info_window_loader"
|
||||
],
|
||||
"sink_sha256_vulnerable": "605e777dab7653792d5ae2f2ef6c776f76d1fb1a271d0446c65f6ebf1dfb8461",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "32166bf3941e0a5048b58c49884d513a1be9b11722245f3b7d4d072846400ec2",
|
||||
"fix_changed_files": 5,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "9c394482d1509ca9222365a91117a10b19619be8",
|
||||
"tip": "85bcd85541c777b6fd8580cb1915504a2a2b84cf",
|
||||
"subject": "Release 4.5.5.2",
|
||||
"files": [
|
||||
"geo-my-wp.php",
|
||||
"readme.txt"
|
||||
],
|
||||
"lines": 17,
|
||||
"side": "before the vulnerable pin"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "kirby-realpath-sibling-traversal",
|
||||
"repo": "https://github.com/getkirby/kirby",
|
||||
"vulnerable": "e90257fd7504053e13e93a1e9a5a9a5137aed966",
|
||||
"fixed": "2b6fab950bc89d505ea89576bf71c8de614cc3b2",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "src/Filesystem/Dir.php",
|
||||
"sink_functions": [
|
||||
"realpath"
|
||||
],
|
||||
"sink_sha256_vulnerable": "d17a09853356ed1d2e08421e3130a4fd0e68dc6c40f3734c796d425ea135f8d3",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "e0ec9c78eff595b6076ca396646a097a7f37a4f49212c2bc7ef4aa0d68448652",
|
||||
"fix_changed_files": 4,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "bbc7c251efcbb01a72f5786af15964e596dc6c4d",
|
||||
"tip": "65dfca053d78bac574c277404adbc8175b2844d8",
|
||||
"subject": "fix: Improve content salt message",
|
||||
"files": [
|
||||
"i18n/translations/en.json",
|
||||
"tests/Panel/Areas/SystemTest.php"
|
||||
],
|
||||
"lines": 4,
|
||||
"side": "before the vulnerable pin"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "mineadmin-appstore-identifier-traversal",
|
||||
"repo": "https://github.com/mineadmin/MineAdmin",
|
||||
"vulnerable": "ac0dcad616c5d304c249d1db4a6675f65cf736dc",
|
||||
"fixed": "ca41902a2a5422676227e5088f4cc1dec06044f1",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "plugin/mine-admin/app-store/src/Service/Service.php",
|
||||
"sink_functions": [
|
||||
"download",
|
||||
"install",
|
||||
"unInstall"
|
||||
],
|
||||
"sink_sha256_vulnerable": "d06a053c26f7918a179ead518f70d79d60c2211bcbe751185b95cead8a8af22e",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "b4f2cd97704d2eb76eae07287fd2251d5e072de1a5cba333857a7cc1fd23747c",
|
||||
"fix_changed_files": 2,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "1f99cfd595f9621f4be572d970821f3509acfe33",
|
||||
"tip": "15deedde8c106842d9c3584d8680e2f65adaaac1",
|
||||
"subject": "Upgrade the MineAdmin to 3.2-alpha.1, and update Hyperf to the 3.2 release.",
|
||||
"files": [
|
||||
"composer.json",
|
||||
"config/autoload/cache.php"
|
||||
],
|
||||
"lines": 105,
|
||||
"side": "after the fix"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "pagible-admin-proxy-dns-rebinding-ssrf",
|
||||
"repo": "https://github.com/aimeos/pagible",
|
||||
"vulnerable": "03f03b2f81b20b1c1625ff227235c43791ddbb33",
|
||||
"fixed": "09a8205d513ec89ed22cdd7bdae0f4c181cee082",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"inferred_fix_changes_sites": {
|
||||
"admin/src/Controllers/AdminController.php::proxy": true,
|
||||
"core/src/Utils.php::isValidUrl": true
|
||||
},
|
||||
"sink_file": "admin/src/Controllers/AdminController.php",
|
||||
"sink_functions": [
|
||||
"proxy"
|
||||
],
|
||||
"sink_sha256_vulnerable": "cedb2973084f6ba825df0f4cf0d9edc1270c25e236155da0a1f7dddb8fa21340",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "eb6a56affa39cb0aa88e894227c2dc871f6a7dc508a8121d205cb8bb3c2105a9",
|
||||
"fix_changed_files": 3,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "981b8d63e52e7ba8875a9a52c1865eb3bc662154",
|
||||
"tip": "d7577efd68e9c5e36436c8572bfed0df2be2c634",
|
||||
"subject": "Publish related items (#210)",
|
||||
"files": [
|
||||
"core/src/Commands/Publish.php",
|
||||
"core/src/Models/Element.php",
|
||||
"core/src/Models/Page.php",
|
||||
"graphql/src/GraphQL/Mutations/PubElement.php",
|
||||
"graphql/src/GraphQL/Mutations/PubPage.php",
|
||||
"graphql/tests/GraphqlPageTest.php",
|
||||
"mcp/src/Tools/PublishElement.php",
|
||||
"mcp/src/Tools/PublishPage.php"
|
||||
],
|
||||
"lines": 44,
|
||||
"side": "after the fix"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wp-graphql-updatepost-publish-bypass",
|
||||
"repo": "https://github.com/wp-graphql/wp-graphql",
|
||||
"vulnerable": "d53b0e566f20a8df0664559cb6432d52a7cd285c",
|
||||
"fixed": "55441663eaa33c3f2e05de038c8286c845916461",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sites": [
|
||||
"plugins/wp-graphql/src/Mutation/PostObjectUpdate.php::mutate_and_get_payload"
|
||||
],
|
||||
"site_basis": {
|
||||
"plugins/wp-graphql/src/Mutation/PostObjectUpdate.php::mutate_and_get_payload": "resolved-from-fix-diff"
|
||||
},
|
||||
"site_files_unresolved": {},
|
||||
"sink_file": "plugins/wp-graphql/src/Mutation/PostObjectUpdate.php",
|
||||
"sink_functions": [
|
||||
"mutate_and_get_payload"
|
||||
],
|
||||
"sink_sha256_vulnerable": "e39fe898c11102ea902c0db1a97cfc641c676d88d54bbb12d50c67ac9f9e4447",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "744928c8eeefdd40b1c7f48ce6ae7fb71f07ff26a95b26321725eaf4bffebdde",
|
||||
"fix_changed_files": 4,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "b583bb6ec87016965085afc7ea586f114c711c68",
|
||||
"tip": "cc12aed380414545a005048bfcebe4ba50e9c8f9",
|
||||
"subject": "fix: point acf.wpgraphql.com links at the wpgraphql.com docs portal (#4251)",
|
||||
"files": [
|
||||
"plugins/wp-graphql-acf/README.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/advanced-link.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/button-group.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/checkbox.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/code-editor.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/color-picker.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/countries.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/currencies.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/date-picker.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/date-range-picker.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/datetimepicker.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/email.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/file.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/gallery.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/google-map.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/group.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/image-selector.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/image-sizes.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/image.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/languages.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/link.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/radio-button.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/range.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/select.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/taxonomies.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/taxonomy-terms.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/taxonomy.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/text-area.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/text.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/time-picker.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/true-false.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/url.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/user-roles.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/user.md",
|
||||
"plugins/wp-graphql-acf/docs/field-types/wysiwyg.md",
|
||||
"plugins/wp-graphql-acf/readme.txt",
|
||||
"plugins/wp-graphql/docs/compatibility-guide.md",
|
||||
"plugins/wp-graphql/src/Admin/AdminNotices.php",
|
||||
"plugins/wp-graphql/src/Admin/Extensions/Registry.php"
|
||||
],
|
||||
"lines": 90,
|
||||
"side": "before the vulnerable pin"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sylius-mollie-thankyou-order-idor",
|
||||
"repo": "https://github.com/Sylius/MolliePlugin",
|
||||
"vulnerable": "7400f1489a4b699df97ced9499acde4faaa476d5",
|
||||
"fixed": "01316b3ad3cf82e3c5ad160115d0a2cf89174e49",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sink_file": "src/Controller/Shop/PageRedirectController.php",
|
||||
"sink_functions": [
|
||||
"thankYouAction"
|
||||
],
|
||||
"sink_sha256_vulnerable": "60578839af598e79b4f3792ad31e8a2c80a360c53fd75609bc25acfb68c4fcd9",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "950d09980903953d802456cafed8a3efb4f9f0e953cdf49482e9ab4a9ac68de6",
|
||||
"fix_changed_files": 7,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "7cc02612b4c6b4403a2b0a1bcef1eb7c5382f505",
|
||||
"tip": "45b01569853d16d04ffd50df281e2fd607650864",
|
||||
"subject": "[UPMERGE] 3.2 -> 3.3 (#333)",
|
||||
"files": [
|
||||
"src/Client/MollieApiClient.php",
|
||||
"src/SyliusMolliePlugin.php",
|
||||
"src/Twig/Extension/MolliePluginLatestVersion.php"
|
||||
],
|
||||
"lines": 21,
|
||||
"side": "before the vulnerable pin"
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wp-podlove-generation-guard-poi",
|
||||
"repo": "https://github.com/podlove/podlove-publisher",
|
||||
"vulnerable": "4eedbd66796942ab052de2dbd6311a24cc75c4a7",
|
||||
"fixed": "ea878175f2612d54888d9e2aa49ae4db93b724fd",
|
||||
"problems": [],
|
||||
"vulnerable_is_ancestor_of_fixed": true,
|
||||
"sites": [
|
||||
"lib/modules/shownotes/rest_api.php::create_link_item",
|
||||
"lib/image_cache/generation_guard.php::__destruct",
|
||||
"lib/model/base.php::to_array"
|
||||
],
|
||||
"site_basis": {
|
||||
"lib/modules/shownotes/rest_api.php::create_link_item": "advisory-named",
|
||||
"lib/image_cache/generation_guard.php::__destruct": "advisory-named",
|
||||
"lib/model/base.php::to_array": "resolved-from-fix-diff"
|
||||
},
|
||||
"site_files_unresolved": {},
|
||||
"inferred_fix_changes_sites": {
|
||||
"lib/modules/shownotes/rest_api.php::create_link_item": true,
|
||||
"lib/image_cache/generation_guard.php::__destruct": false
|
||||
},
|
||||
"sink_file": "lib/modules/shownotes/rest_api.php",
|
||||
"sink_functions": [
|
||||
"create_link_item"
|
||||
],
|
||||
"sink_sha256_vulnerable": "5023f7950ad450c6f0e9ca9bec474f161a509b0b3ddbf8f9f495593738dd8a05",
|
||||
"sink_functions_missing": [],
|
||||
"sink_sha256_fixed": "316f85861d95e63a9a7cc1f089e2b6c3f1ac61aa92092e8b279534b226fe042f",
|
||||
"fix_changed_files": 9,
|
||||
"fix_touches_sink_file": true,
|
||||
"fix_touches_fix_site": false,
|
||||
"benign_proposal": {
|
||||
"base": "af187ad4bba2739839a0aa1e6cf2cabd6f9b99f5",
|
||||
"tip": "50da8db0f5a85afaf27cd6199940be0403b368ec",
|
||||
"subject": "fix PHP warning",
|
||||
"files": [
|
||||
"lib/modules/contributors/contributor_list_table.php"
|
||||
],
|
||||
"lines": 2,
|
||||
"side": "after the fix"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,366 @@
|
||||
"""Verify each reserved case by READING its source, and propose a benign change -- without scanning anything.
|
||||
|
||||
Freezing the independent population (pre-push-safety-net 15.1) needs, per case: both pins present and in the
|
||||
right order, the vulnerable sink confirmed where the advisory says it is, the fix touching that file, and a
|
||||
benign change from the same history. This does exactly that with git and file reads. It never imports or runs
|
||||
the analyzer, its queries, or any project tooling: a case this script touched is still unscanned.
|
||||
|
||||
The benign change is a PROPOSAL for review, chosen mechanically: the first commit after the fix, on the first
|
||||
-parent line, that changes source files of the case's language, does not touch the sink file, does not talk
|
||||
about security in its message, and stays small. A reviewer confirms or replaces it before the status becomes
|
||||
`frozen`.
|
||||
|
||||
Usage: python benchmarks/independent/freeze.py [--population population-vN.toml] [--cache DIR] [--only ID,...]
|
||||
Writes: benchmarks/independent/freeze-vN.json (v1 by default)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
POPULATION = HERE / "population-v1.toml"
|
||||
|
||||
|
||||
def freeze_record(population: Path) -> Path:
|
||||
"""`population-v2.toml` -> `freeze-v2.json`, next to it."""
|
||||
return population.with_name("freeze-" + population.stem.removeprefix("population-") + ".json")
|
||||
|
||||
|
||||
EXTENSIONS = {
|
||||
"php": (".php",),
|
||||
"javascript": (".js", ".mjs", ".cjs", ".jsx"),
|
||||
"typescript": (".ts", ".tsx"),
|
||||
"python": (".py",),
|
||||
}
|
||||
# A benign change must not be a security change in disguise. The first pass let through "Restrict access to
|
||||
# routine logs to owners only", "querybuilder missing the valid_filter_params check" and "add access mode and
|
||||
# scopes to API tokens" -- access control and input validation, which is exactly what a benign control must not be.
|
||||
SECURITY_WORDS = re.compile(
|
||||
r"secur|cve|ghsa|vuln|inject|xss|csrf|ssrf|sanitiz|escap|exploit|attack|auth|permission|cors|access|restrict|harden"
|
||||
r"|token|scope|valid|filter|privileg|role|owner|password|secret|redirect|sql|query|login|log in|ldap|session|ssl|tls|verif|cert|crypt|hash|cookie",
|
||||
re.I,
|
||||
)
|
||||
MAX_BENIGN_LINES = 200
|
||||
|
||||
|
||||
def git(repo: Path, *args: str, check: bool = True) -> str:
|
||||
done = subprocess.run(["git", "-C", str(repo), *args], capture_output=True, text=True, check=False)
|
||||
if check and done.returncode != 0:
|
||||
raise RuntimeError(f"git {' '.join(args)}: {done.stderr.strip()[:300]}")
|
||||
return done.stdout
|
||||
|
||||
|
||||
def clone(url: str, target: Path) -> None:
|
||||
if (target / ".git").is_dir():
|
||||
git(target, "fetch", "--quiet", "origin")
|
||||
return
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
subprocess.run(["git", "clone", "--quiet", "--filter=blob:none", "--no-checkout", url, str(target)], check=True)
|
||||
|
||||
|
||||
def blob(repo: Path, commit: str, path: str) -> bytes | None:
|
||||
done = subprocess.run(["git", "-C", str(repo), "show", f"{commit}:{path}"], capture_output=True, check=False)
|
||||
return done.stdout if done.returncode == 0 else None
|
||||
|
||||
|
||||
def sink_parts(sink: str) -> tuple[str, list[str]]:
|
||||
"""The sink file and the function names the record claims, from `path :: A::b (...), c`."""
|
||||
path, _, rest = sink.partition("::")
|
||||
names = re.findall(r"[A-Za-z_][A-Za-z0-9_]*(?=\s*(?:\(|,|$|;))", rest.split("(")[0] + ",")
|
||||
functions = [n for n in names if n not in {"used", "by", "source", "callers", "and", "from", "the"}]
|
||||
return path.strip(), functions
|
||||
|
||||
|
||||
PHP_FUNCTION = re.compile(r"\bfunction\s*&?\s*([A-Za-z_][A-Za-z0-9_]*)\s*\(")
|
||||
|
||||
|
||||
def php_code(text: str) -> str:
|
||||
"""`text` with comments, string contents and inline HTML blanked to spaces, newlines kept (line numbers hold).
|
||||
|
||||
Enough to find function bodies by brace matching; heredoc/nowdoc bodies are blanked too. Not a parser: a file
|
||||
that defeats it yields no function, and the site falls back to `<global>`, which the freeze record shows."""
|
||||
out, i, n, php = list(text), 0, len(text), False
|
||||
|
||||
def blank(start: int, end: int) -> None:
|
||||
for k in range(start, min(end, n)):
|
||||
if out[k] != "\n":
|
||||
out[k] = " "
|
||||
|
||||
while i < n:
|
||||
if not php:
|
||||
j = text.find("<?", i)
|
||||
blank(i, n if j < 0 else j)
|
||||
if j < 0:
|
||||
break
|
||||
php, i = True, j + (5 if text.startswith("<?php", j) else 2)
|
||||
continue
|
||||
if text.startswith("?>", i):
|
||||
php, i = False, i + 2
|
||||
elif text.startswith("//", i) or text[i] == "#" and not text.startswith("#[", i):
|
||||
j = min(k for k in (text.find("\n", i), text.find("?>", i), n) if k >= 0)
|
||||
blank(i, j)
|
||||
i = j
|
||||
elif text.startswith("/*", i):
|
||||
j = text.find("*/", i + 2)
|
||||
j = n if j < 0 else j + 2
|
||||
blank(i, j)
|
||||
i = j
|
||||
elif text.startswith("<<<", i):
|
||||
label = re.match(r"<<<\s*['\"]?([A-Za-z_][A-Za-z0-9_]*)", text[i:])
|
||||
end = re.compile(r"^\s*" + re.escape(label.group(1)) + r"\b", re.M) if label else None
|
||||
found = end.search(text, text.find("\n", i) + 1) if end else None
|
||||
j = found.end() if found else i + 3
|
||||
blank(i, j)
|
||||
i = j
|
||||
elif text[i] in "'\"":
|
||||
quote, j = text[i], i + 1
|
||||
while j < n and text[j] != quote:
|
||||
j += 2 if text[j] == "\\" else 1
|
||||
blank(i + 1, j)
|
||||
i = j + 1
|
||||
else:
|
||||
i += 1
|
||||
return "".join(out)
|
||||
|
||||
|
||||
def php_functions(text: str) -> list[tuple[str, int, int]]:
|
||||
"""(name, first line, last line) of every named function or method body in a PHP file, 1-based."""
|
||||
code, spans = php_code(text), []
|
||||
for match in PHP_FUNCTION.finditer(code):
|
||||
brace, semi = code.find("{", match.end()), code.find(";", match.end())
|
||||
if brace < 0 or 0 <= semi < brace: # abstract or interface method: no body
|
||||
continue
|
||||
depth, k = 0, brace
|
||||
while k < len(code):
|
||||
depth += {"{": 1, "}": -1}.get(code[k], 0)
|
||||
if depth == 0:
|
||||
break
|
||||
k += 1
|
||||
spans.append((match.group(1), code.count("\n", 0, match.start()) + 1, code.count("\n", 0, k) + 1))
|
||||
return spans
|
||||
|
||||
|
||||
def enclosing(spans: list[tuple[str, int, int]], line: int, inserted: bool = False) -> str:
|
||||
"""The innermost named function containing `line`, or `<global>`. For a pure insertion, `line` is the line the
|
||||
new code follows, so a function whose closing brace is on that line does not contain it."""
|
||||
inside = [(last - first, name) for name, first, last in spans if first <= line <= last and not (inserted and line == last)]
|
||||
return min(inside)[1] if inside else "<global>"
|
||||
|
||||
|
||||
def old_hunks(repo: Path, vulnerable: str, fixed: str, path: str) -> list[tuple[int, int, bool]]:
|
||||
"""Vulnerable-side line ranges the fix changes in `path`, and whether each is a pure insertion (then the range
|
||||
is the line it follows, at least 1)."""
|
||||
diff = git(repo, "diff", "-U0", vulnerable, fixed, "--", path)
|
||||
ranges = []
|
||||
for start, count in re.findall(r"^@@ -(\d+)(?:,(\d+))? \+", diff, re.M):
|
||||
first, size = int(start), 1 if count == "" else int(count)
|
||||
ranges.append((max(first, 1), max(first, 1), True) if size == 0 else (first, first + size - 1, False))
|
||||
return ranges
|
||||
|
||||
|
||||
def changed_functions(repo: Path, vulnerable: str, fixed: str, path: str) -> list[str]:
|
||||
"""The functions (or `<global>`) of `path` at the vulnerable pin that the fix's hunks touch, in hunk order."""
|
||||
text = (blob(repo, vulnerable, path) or b"").decode(errors="replace")
|
||||
spans, names = php_functions(text), []
|
||||
for first, last, inserted in old_hunks(repo, vulnerable, fixed, path):
|
||||
for line in range(first, last + 1):
|
||||
name = enclosing(spans, line, inserted)
|
||||
# New code added between functions (a new helper) changes no vulnerable-side code: no site.
|
||||
if name not in names and not (inserted and name == "<global>"):
|
||||
names.append(name)
|
||||
return names
|
||||
|
||||
|
||||
def resolve_sites(repo: Path, case: dict[str, object], changed: list[str]) -> dict[str, object]:
|
||||
"""Decision 2 of population v3: a file the advisory names without a function gets, as its declared site, the
|
||||
function(s) the fix's diff of that file changes. Advisory-named sites are kept and marked as such."""
|
||||
declared = [str(s) for s in case.get("sites", [])] # type: ignore[union-attr]
|
||||
recorded = case.get("site_basis") or {} # a frozen population keeps each site's basis
|
||||
basis = {site: recorded.get(site, "advisory-named") for site in declared} # type: ignore[union-attr]
|
||||
unresolved = {}
|
||||
named_files = {site.partition("::")[0] for site in declared}
|
||||
for path in [str(f) for f in case.get("site_files", [])]: # type: ignore[union-attr]
|
||||
if path in named_files:
|
||||
continue
|
||||
if path not in changed:
|
||||
unresolved[path] = "the fix does not change this file"
|
||||
continue
|
||||
for name in changed_functions(repo, str(case["vulnerable"]), str(case["fixed"]), path):
|
||||
basis.setdefault(f"{path}::{name}", "resolved-from-fix-diff")
|
||||
return {"sites": list(basis), "site_basis": basis, "site_files_unresolved": unresolved}
|
||||
|
||||
|
||||
def fix_changes_sites(repo: Path, case: dict[str, object], sites: list[str]) -> dict[str, bool]:
|
||||
"""Per declared site: does the fix's diff change a line inside that function (or the file, for `<global>`)?"""
|
||||
result = {}
|
||||
for site in sites:
|
||||
path, _, name = site.partition("::")
|
||||
hunks = old_hunks(repo, str(case["vulnerable"]), str(case["fixed"]), path)
|
||||
if name == "<global>" or not hunks:
|
||||
result[site] = bool(hunks)
|
||||
continue
|
||||
spans = php_functions((blob(repo, str(case["vulnerable"]), path) or b"").decode(errors="replace"))
|
||||
result[site] = any(enclosing(spans, line, inserted) == name for first, last, inserted in hunks for line in range(first, last + 1))
|
||||
return result
|
||||
|
||||
|
||||
def benign(
|
||||
repo: Path, fixed: str, sink_file: str, extensions: tuple[str, ...], strict: bool = False, vulnerable: str = ""
|
||||
) -> dict[str, object] | None:
|
||||
"""The first acceptable change after the fix (see the module docstring).
|
||||
|
||||
`strict` (a single-language population, v3) tightens the proposal: the change is measured against its first
|
||||
parent, so a merge counts with everything it brings in, and every message it brings in is checked for security
|
||||
words (a branch-sync merge can carry the fix itself); it must change a file of the population's language; and
|
||||
when no commit after the fix qualifies, the history before the vulnerable pin is searched, newest first."""
|
||||
head = git(repo, "rev-parse", "origin/HEAD", check=False).strip() or git(repo, "rev-parse", "HEAD").strip()
|
||||
span = [f"{fixed}..{head}"]
|
||||
if subprocess.run(["git", "-C", str(repo), "merge-base", "--is-ancestor", fixed, head], check=False).returncode:
|
||||
# A fix on a maintenance branch (population v3: kirby 4.x, roundcube 1.6): the default branch's commits
|
||||
# after the fix date, not its whole history outside the branch.
|
||||
span = [f"--since={git(repo, 'log', '-1', '--format=%cI', fixed).strip()}", head]
|
||||
commits = git(repo, "rev-list", "--first-parent", "--reverse", *span, check=False).split()[:400]
|
||||
after = set(commits)
|
||||
if strict and vulnerable:
|
||||
commits += git(repo, "rev-list", "--first-parent", "--max-count=400", f"{vulnerable}^", check=False).split()
|
||||
for commit in commits:
|
||||
parent = git(repo, "rev-parse", f"{commit}^", check=False).strip()
|
||||
if not parent:
|
||||
continue
|
||||
if strict:
|
||||
message = git(repo, "log", "--format=%s%n%b", f"{parent}..{commit}")
|
||||
stat = git(repo, "diff", "--numstat", parent, commit).splitlines()
|
||||
else:
|
||||
message = git(repo, "log", "-1", "--format=%s%n%b", commit)
|
||||
stat = git(repo, "show", "--numstat", "--format=", commit).splitlines()
|
||||
if SECURITY_WORDS.search(message):
|
||||
continue
|
||||
files = [line.split("\t") for line in stat if line.count("\t") == 2]
|
||||
if not files or any(f[2] == sink_file for f in files):
|
||||
continue
|
||||
source = [f for f in files if f[2].endswith(extensions)]
|
||||
lines = sum(int(a) + int(b) for a, b, _ in files if a.isdigit() and b.isdigit())
|
||||
if source and 0 < lines <= MAX_BENIGN_LINES:
|
||||
subject = git(repo, "log", "-1", "--format=%s", commit).strip()
|
||||
side = "after the fix" if commit in after else "before the vulnerable pin"
|
||||
return {"base": parent, "tip": commit, "subject": subject[:120], "files": [f[2] for f in files], "lines": lines, "side": side}
|
||||
return None
|
||||
|
||||
|
||||
def verify(case: dict[str, object], cache: Path) -> dict[str, object]:
|
||||
url, vulnerable, fixed = str(case["repo"]), str(case["vulnerable"]), str(case["fixed"])
|
||||
repo = cache / str(case["id"])
|
||||
clone(url, repo)
|
||||
record: dict[str, object] = {"id": case["id"], "repo": url, "vulnerable": vulnerable, "fixed": fixed, "problems": []}
|
||||
problems: list[str] = record["problems"] # type: ignore[assignment]
|
||||
for pin in (vulnerable, fixed):
|
||||
if git(repo, "cat-file", "-t", pin, check=False).strip() != "commit":
|
||||
problems.append(f"pin {pin[:12]} is not a commit in the repository")
|
||||
if problems:
|
||||
return record
|
||||
ordered = subprocess.run(["git", "-C", str(repo), "merge-base", "--is-ancestor", vulnerable, fixed], check=False).returncode == 0
|
||||
record["vulnerable_is_ancestor_of_fixed"] = ordered
|
||||
if not ordered:
|
||||
problems.append("the vulnerable pin is not an ancestor of the fixed pin")
|
||||
changed = git(repo, "diff", "--name-only", vulnerable, fixed).splitlines()
|
||||
sites = [str(site) for site in case.get("sites", [])] # type: ignore[union-attr]
|
||||
if case.get("site_files"):
|
||||
# Population v3: files the advisory names without a function are resolved from the fix's diff.
|
||||
record.update(resolve_sites(repo, case, changed))
|
||||
sites = record["sites"] # type: ignore[assignment]
|
||||
if not sites:
|
||||
problems.append("no declared site: the advisory names no function and the fix changes no named file")
|
||||
if case.get("fix_named_by_advisory") is False:
|
||||
# An inferred fix must change the declared sink file and at least one declared site function.
|
||||
basis = record.get("site_basis") or {}
|
||||
named = [s for s in sites if basis.get(s, "advisory-named") == "advisory-named"] # type: ignore[union-attr]
|
||||
touched = fix_changes_sites(repo, case, named)
|
||||
record["inferred_fix_changes_sites"] = touched
|
||||
if not any(touched.values()):
|
||||
problems.append("the inferred fix changes none of the declared site functions: drop the case")
|
||||
if sites:
|
||||
# Declared `path::function` sites (population v2) are what the verification reads; the first names the
|
||||
# sink file, and every function declared in that file must appear in it.
|
||||
sink_file = sites[0].partition("::")[0]
|
||||
functions = [site.partition("::")[2] for site in sites if site.partition("::")[0] == sink_file and not site.endswith("::<global>")]
|
||||
else:
|
||||
sink_file, functions = sink_parts(str(case["sink"]))
|
||||
record["sink_file"], record["sink_functions"] = sink_file, functions
|
||||
before, after = blob(repo, vulnerable, sink_file), blob(repo, fixed, sink_file)
|
||||
if before is None:
|
||||
problems.append(f"sink file {sink_file} is absent at the vulnerable pin")
|
||||
else:
|
||||
record["sink_sha256_vulnerable"] = hashlib.sha256(before).hexdigest()
|
||||
text = before.decode(errors="replace")
|
||||
evidence = str(case.get("sink_evidence", ""))
|
||||
if evidence:
|
||||
record["sink_evidence_found"] = evidence in text
|
||||
if evidence not in text:
|
||||
problems.append(f"sink evidence {evidence!r} is absent from {sink_file} at the vulnerable pin")
|
||||
else:
|
||||
missing = [f for f in functions if f.split("::")[-1] not in text]
|
||||
record["sink_functions_missing"] = missing
|
||||
if functions and len(missing) == len(functions):
|
||||
problems.append(f"none of {functions} appears in {sink_file} at the vulnerable pin")
|
||||
if after is not None:
|
||||
record["sink_sha256_fixed"] = hashlib.sha256(after).hexdigest()
|
||||
record["fix_changed_files"] = len(changed)
|
||||
record["fix_touches_sink_file"] = sink_file in changed
|
||||
fix_site = str(case.get("fix_site", "")).partition("::")[0].strip()
|
||||
record["fix_touches_fix_site"] = bool(fix_site) and fix_site in changed
|
||||
if sink_file not in changed and not record["fix_touches_fix_site"]:
|
||||
problems.append(f"the fix changes neither {sink_file} nor a declared fix_site")
|
||||
languages = [str(case["language"]), *[str(x) for x in case.get("also", [])]] # type: ignore[union-attr]
|
||||
extensions = tuple(e for lang in languages for e in EXTENSIONS.get(lang, ()))
|
||||
strict = bool(case.get("_population_language"))
|
||||
if strict:
|
||||
extensions = EXTENSIONS.get(str(case["_population_language"]), extensions)
|
||||
record["benign_proposal"] = benign(repo, fixed, sink_file, extensions, strict, vulnerable)
|
||||
if record["benign_proposal"] is None and not case.get("benign"):
|
||||
problems.append("no benign change found automatically; choose one by hand")
|
||||
return record
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--cache", type=Path, default=Path.home() / ".cache" / "openultrasast" / "independent")
|
||||
parser.add_argument("--only", default="")
|
||||
parser.add_argument("--population", type=Path, default=POPULATION, help="a population-vN.toml in this directory")
|
||||
args = parser.parse_args()
|
||||
population = args.population if args.population.is_absolute() else HERE / args.population.name
|
||||
data = tomllib.loads(population.read_text())
|
||||
cases = data["case"]
|
||||
for case in cases:
|
||||
# A single-language population (v3) gets the strict benign proposal (see `benign`).
|
||||
case["_population_language"] = data.get("language", "")
|
||||
only = {c.strip() for c in args.only.split(",") if c.strip()}
|
||||
records = []
|
||||
for case in cases:
|
||||
if only and case["id"] not in only:
|
||||
continue
|
||||
try:
|
||||
record = verify(case, args.cache)
|
||||
except (RuntimeError, subprocess.CalledProcessError) as error:
|
||||
record = {"id": case["id"], "problems": [f"could not verify: {error}"]}
|
||||
records.append(record)
|
||||
print(
|
||||
json.dumps(
|
||||
{"id": record["id"], "problems": record["problems"], "benign": (record.get("benign_proposal") or {}).get("subject")}
|
||||
),
|
||||
flush=True,
|
||||
)
|
||||
freeze_record(population).write_text(
|
||||
json.dumps({"schema_version": 1, "population": population.name, "cases": records}, indent=2) + "\n"
|
||||
)
|
||||
return 0 if all(not r["problems"] for r in records) else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,55 @@
|
||||
# Hunter experiment on population v2 (pre-registered)
|
||||
|
||||
Written and committed on 2026-09-28, BEFORE the tool hunter is run on any v2 pin. It is the deciding experiment
|
||||
of the strategy review (`.kiro/specs/pre-push-safety-net/strategy-review-2026-09-28.md`): does a model with
|
||||
repository tools find what the static engine did not (v2: recall 0/17, precision 0/17)?
|
||||
|
||||
## What is under test
|
||||
|
||||
- `tool_hunter.run_tool_hunter` as it stands: default system prompt, default tools (`read_file`, `grep_repo`,
|
||||
`find_refs`), `DEFAULT_MAX_STEPS` (4), the final-answer turn. Its prompt, tools and parsing are unchanged since
|
||||
the 2026-09-06 measurement; the only changes today are transport (cf57995 and this commit: the client reaches
|
||||
DeepSeek, and the endpoint is priced so the budget can see spending).
|
||||
- Model: `deepseek-flash` (DeepSeek, the project's chat provider), through `resolve_hunter_client()`.
|
||||
- Runner: `benchmarks/independent/hunt.py` at the commit that introduces this file.
|
||||
|
||||
## Scope handed to the model, per pin
|
||||
|
||||
The regions the static scan's discovery and ranker produce for the case's family (`regions_for` over
|
||||
`analyze_entry_points`, the same as `finding_dump.py`), sorted by rank, top 20, as hunter hotspots in four groups
|
||||
of five: one bounded hunt per group. Nothing else: not the fix diff, not the advisory, not the case's declared
|
||||
sites, not the case id.
|
||||
|
||||
## Pins
|
||||
|
||||
Protocol v2's six per case: vulnerable a/b, fixed a/b, benign base and tip. Each is a `git archive` of the pin.
|
||||
|
||||
## Turning a hunter answer into a scoreable finding
|
||||
|
||||
- `site` = `path:line:function`, with the function the model reported, or empty.
|
||||
- `family` = the model's own family and title mapped by the fixed keyword table in `hunt.py` (`FAMILY_WORDS`,
|
||||
checked in that order: untrusted_destination, config_secrets, injection; anything else is `other`).
|
||||
|
||||
## Scoring
|
||||
|
||||
Exactly protocol v2 (`protocol-v2.md`), via `evaluate.score_v2` over the hunter's results: the same matching
|
||||
(v1's changed-line rule or a declared `path::function` site), detection in BOTH vulnerable runs, a fixed-side
|
||||
alert in EITHER fixed run, benign alerts, stability, and the precision sample (every third stable
|
||||
vulnerable-pin finding) adjudicated to v1's standard. A finding with no function can match only by the
|
||||
changed-line rule. A pin whose four hunts all fail is an instrument failure, reported as such.
|
||||
|
||||
"Questions" for the hunter are its hunts (4 per pin), so completion is hunts completed / hunts run; it is not
|
||||
comparable to the engine's question completion and is reported only as an instrument check.
|
||||
|
||||
## Cost
|
||||
|
||||
Hard ceiling $10 of recorded model spend, enforced by the runner; a pilot on a v1 development case
|
||||
(mongo-express, 4 hunts) cost $0.031 and 57 s. If the ceiling stops the run, the unscanned pins are reported
|
||||
as not run.
|
||||
|
||||
## What may not happen
|
||||
|
||||
- No change to the hunter's prompt, tools, step budget, parsing, scope rule or family table before this result
|
||||
is committed.
|
||||
- Whatever is learned here is a finding about the hunter as it is; tuning it on v2 afterwards is a retune and
|
||||
needs a new population to qualify.
|
||||
@@ -0,0 +1,92 @@
|
||||
{
|
||||
"measurement": "ai-service-plane first increment (Requirement 6): repo-facts + verify (2 passes) + agree for the 46-candidate validation set, executed by the ax-backed reconciler on google/ax + Agent Substrate (kind, gVisor) on the 7 GB host",
|
||||
"date": "2026-09-29",
|
||||
"run": {
|
||||
"name": "validation-46",
|
||||
"manifest": "plane/runs/validation-46.yaml",
|
||||
"tasks": 60,
|
||||
"workers": 2,
|
||||
"status": "done",
|
||||
"results": "~/ousast-results/plane/validation-46/",
|
||||
"model": "deepseek-flash",
|
||||
"runner_image": "localhost:5001/ousast-runner@sha256:200e335877b52d68ea0bc53b4c9ba49ef2ee19e8a5ec2c874148b969123ef7ab"
|
||||
},
|
||||
"reference": "benchmarks/independent/verifier-batched-check-2026-09-29.json (script pipeline, same cases, candidates, pins and recorded triage)",
|
||||
"gate": {
|
||||
"declared_sites_agreed": {
|
||||
"plane": 14,
|
||||
"reference": 16,
|
||||
"of": 20,
|
||||
"required": ">= 16",
|
||||
"met": false
|
||||
},
|
||||
"cost_per_candidate_with_recorded_triage_usd": {
|
||||
"plane": 0.0194,
|
||||
"plane_over_46": 0.0181,
|
||||
"reference": 0.022,
|
||||
"required": "< 0.022",
|
||||
"met": true
|
||||
},
|
||||
"result": "NOT MET (recall criterion)"
|
||||
},
|
||||
"detail": {
|
||||
"agreed_candidates": 23,
|
||||
"disputed_candidates": 10,
|
||||
"candidates_after_recorded_triage": 43,
|
||||
"plane_usd": 0.7471,
|
||||
"recorded_triage_usd": 0.0872,
|
||||
"usd_with_triage": 0.8343,
|
||||
"reference_usd": 1.0123,
|
||||
"calls": 357,
|
||||
"prompt_tokens": 3820395,
|
||||
"cache_hit_tokens": 2385152,
|
||||
"output_tokens": 62257,
|
||||
"turns_per_hunt": 5.37,
|
||||
"hunts": 54,
|
||||
"per_case_sites_agreed_vs_reference": {
|
||||
"budibase-mongo-template-nosqli": "1/2 vs 2/2 (read: flagged by neither plane pass; the reference agreed on it)",
|
||||
"froxlor-admin-ip-second-order-sqli": "0/3 vs 2/3 (all three flagged by pass b only)",
|
||||
"unleash-addon-webhook-ssrf": "1/1 vs 0/1 (reference found it in one pass only)",
|
||||
"fuxa-tdengine-escape-sqli": "0/1 vs 0/1 (getDaqValue: neither plane pass; the reference found it in one pass)",
|
||||
"winml-cli-serve-cors": "0/1 vs 0/1 (triage drops the only candidate)",
|
||||
"all_other_cases": "equal"
|
||||
},
|
||||
"declared_sites_flagged_in_either_pass": {
|
||||
"plane": 17,
|
||||
"reference": 19,
|
||||
"of": 20,
|
||||
"definition": "declared sites whose candidate pass a or pass b flagged; reference = 16 agreed + fuxa, froxlor validateIpAddresses, unleash found in one pass"
|
||||
},
|
||||
"declared_sites_covered_by_candidates": {
|
||||
"plane": 19,
|
||||
"of": 20,
|
||||
"definition": "agree's metric declared_sites_matched: candidates whose site falls on a declared site, flagged or not (winml has no candidate)"
|
||||
}
|
||||
},
|
||||
"reading": "Detection is lower, not only agreement: plane passes flagged 17 of 20 declared sites in at least one pass against the reference's 19 (budibase read and fuxa getDaqValue were flagged by neither plane pass), and two-pass agreement moved the three froxlor sites (pass b only) into 'disputed'; unleash moved the other way. Cost fell 17% (62% of prompt tokens were cache hits). By the pre-registered rule the increment is NOT MET and the design is revisited before task 9.",
|
||||
"dev_token_report": {
|
||||
"transcript": "the whole development session (c09e30b2), not the increment alone",
|
||||
"unique_chars": "7.3M (~1.81M tokens)",
|
||||
"tool_use_input_pct": 45.4,
|
||||
"tool_result_pct": 40.1,
|
||||
"tool_io_pct": 85.5,
|
||||
"previous_tool_io_pct": 90,
|
||||
"tool_calls": 2701
|
||||
},
|
||||
"live_defects_found_by_running_on_ax": [
|
||||
"no Substrate WorkerPool",
|
||||
"tag-pinned images rejected",
|
||||
"ax CLI/server skew",
|
||||
"unknown-field rejection (atespace, commit)",
|
||||
"server-set fields in AX_TASK_YAML",
|
||||
"tasks created Suspended; no Completed phase",
|
||||
"runner exited after the command",
|
||||
"golden-snapshot boot ran the command",
|
||||
"egress gateway not deployed; deny-by-default per-actor policy",
|
||||
"gateway dials the address the actor dialled",
|
||||
"workspace clone before the actor was running",
|
||||
"32 KB env limit on inlined inputs",
|
||||
"63-byte template name limit"
|
||||
],
|
||||
"correction": "2026-09-30: the first version of this record read agree's declared_sites_matched (coverage by candidates, 19) as 'found in either pass' and concluded detection was unchanged; recounted from agreed.json rows it is 17 vs the reference's 19."
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
{
|
||||
"measurement": "ai-service-plane increment re-measured after the design revisit (Requirement 6.4): passes a and b as recorded in plane-increment-1.json, plus a tie-break pass c on the 10 candidates a and b disputed; agreement 2-of-3. Executed on google/ax by rerunning the same Run (the 60 done tasks were reused, 30 new tasks ran).",
|
||||
"date": "2026-09-30",
|
||||
"run": {
|
||||
"name": "validation-46",
|
||||
"tasks": 90,
|
||||
"status": "done",
|
||||
"runner_image": "localhost:5001/ousast-runner@sha256:fb9a743027518976659fd11505e58cc3fc84fb42e07627196e13b76ac01d0617"
|
||||
},
|
||||
"previous": "benchmarks/independent/plane-increment-1.json",
|
||||
"gate": {
|
||||
"declared_sites_agreed": {
|
||||
"plane": 16,
|
||||
"reference": 16,
|
||||
"of": 20,
|
||||
"required": ">= 16",
|
||||
"met": true
|
||||
},
|
||||
"cost_per_candidate_with_recorded_triage_usd": {
|
||||
"plane_over_46": 0.0205,
|
||||
"plane_over_43_after_triage": 0.02195,
|
||||
"reference_over_46": 0.022,
|
||||
"required": "< 0.022",
|
||||
"met": true,
|
||||
"note": "the reference divides by the 46 candidates before triage; on that basis the plane is 7% cheaper, on the 43 after triage it is 0.2% under"
|
||||
},
|
||||
"result": "MET"
|
||||
},
|
||||
"detail": {
|
||||
"tiebreak": {
|
||||
"asked": 10,
|
||||
"flipped_to_agreed": 5,
|
||||
"declared_sites_recovered": [
|
||||
"froxlor listing",
|
||||
"froxlor listingCount"
|
||||
],
|
||||
"declared_site_still_disputed": "froxlor validateIpAddresses (b only)",
|
||||
"non_declared_flipped_to_agreed": [
|
||||
"pgadmin enable_disable",
|
||||
"pgadmin get_job_args_value",
|
||||
"pgadmin run_job"
|
||||
]
|
||||
},
|
||||
"agreed_candidates": {
|
||||
"two_pass": 23,
|
||||
"two_of_three": 28
|
||||
},
|
||||
"declared_sites_flagged_in_any_pass": {
|
||||
"plane": 17,
|
||||
"reference": 19,
|
||||
"of": 20,
|
||||
"missed_by_every_plane_pass": [
|
||||
"budibase read",
|
||||
"fuxa getDaqValue"
|
||||
]
|
||||
},
|
||||
"usd": {
|
||||
"plane": 0.8567,
|
||||
"recorded_triage": 0.0872,
|
||||
"with_triage": 0.9439,
|
||||
"reference": 1.0123
|
||||
}
|
||||
},
|
||||
"reading": "The plane on ax matches the script pipeline's pre-registered recall (16/20 agreed) at 7% lower cost, so Requirement 6.2 is met after the 6.4 revisit. Two caveats travel with the result: (1) the tie-break is a design change chosen after seeing the first measurement on the same validation set, so this compares plane with script like for like and qualifies nothing on an independent population; (2) the plane's passes flag fewer declared sites than the reference's (17 vs 19 in any pass), and 2-of-3 also promoted three non-declared pgadmin candidates, whose truth is unassessed (an earlier pgadmin spot check found 3 of 5 findings truly privileged). Precision is not part of this gate and remains the open problem."
|
||||
}
|
||||
@@ -0,0 +1,189 @@
|
||||
# Independent evaluation population v1 (pre-push-safety-net task 15.1, milestone M4).
|
||||
#
|
||||
# RESERVED, NOT SCANNED. Every case below was chosen from public advisories and fix diffs by web research alone:
|
||||
# no repository was cloned, no scanner or project tooling was run on any of them, and none appears among the
|
||||
# 262 repositories referenced anywhere in this project's benchmarks. They exist to answer the question the
|
||||
# development subjects can no longer answer honestly -- PMPro, WP Statistics, MW WP Form, NodeGoat and VAmPI all
|
||||
# shaped the rules they would be measured by.
|
||||
#
|
||||
# Rules for this file:
|
||||
# * Do not scan, trace, probe or census any of these repositories until the population is FROZEN (status
|
||||
# below) and the freeze commit is recorded. A scan before that makes the case a development subject.
|
||||
# * Freezing requires, per case: the vulnerable and fixed pins verified in the checkout, the sink confirmed at
|
||||
# the vulnerable pin by reading source, and a BENIGN change chosen from the same history without scanning.
|
||||
# * No rule, fact or query change may cite anything learned from these cases before their evaluation is
|
||||
# recorded; after it, a retune needs a new untouched population (release-milestones M4).
|
||||
#
|
||||
# Selected by the maintainer on 2026-09-24 from 15 researched candidates (menu selection); rejected alternates
|
||||
# and the research notes are in the research log referenced below.
|
||||
|
||||
schema_version = 1
|
||||
status = "frozen"
|
||||
frozen_on = "2026-09-24"
|
||||
freeze_record = "freeze-v1.json"
|
||||
reserved_on = "2026-09-24"
|
||||
selected_by = "maintainer"
|
||||
research = "web research of GitHub advisories and fix diffs via the GitHub API; no clone, no scan"
|
||||
multi_language_members = ["wp-ultimate-member-sqli", "wp-learnpress-sqli", "phpmyfaq-captcha-ua-sqli", "yeswiki-bazar-form-sqli"]
|
||||
also_reserved = ["../push/ghost-reserved.toml"]
|
||||
|
||||
[[case]]
|
||||
id = "wp-ultimate-member-sqli"
|
||||
repo = "https://github.com/ultimatemember/ultimatemember"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2024-1071", "GHSA-mh7h-gpmx-fggj"]
|
||||
vulnerable = "a49b3061ded1630e4f6860b4ebece5ccf0a493d2"
|
||||
fixed = "476de92f4c9aaa56c3e2f5527b242ce0476f4655"
|
||||
sink = "includes/core/class-member-directory-meta.php :: Member_Directory_Meta::ajax_get_members (ORDER BY built from esc_sql'd sort values)"
|
||||
privilege = "unauthenticated (only with the custom usermeta table option on)"
|
||||
license = "GPL"
|
||||
notes = "Release pair 2.8.2 (e1885a73) -> 2.8.3 (a6a7d13a) spans 94 commits; the commit pair is the clean one. c61008bd is a larger follow-up refactor of the same file."
|
||||
|
||||
benign = { base = "13151ef558eb6998720cb1ee5f78b5f17535c16c", tip = "9130a3226ffec45569a2872ffac338200bd1bfcb", subject = "- wpcs;" }
|
||||
[[case]]
|
||||
id = "wp-learnpress-sqli"
|
||||
repo = "https://github.com/LearnPress/learnpress"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2024-8522", "GHSA-3w3r-r6g6-w8x5", "CVE-2024-8529", "GHSA-98wp-w76v-f75p"]
|
||||
vulnerable = "73b3cbceef8669056e9da5e2f2c78944eb139af5"
|
||||
fixed = "aaf42fe0c6bf05dcf2ff125750b7bd836e76fa3c"
|
||||
sink = "inc/Databases/class-lp-db.php :: LP_Database::execute (implode of only_fields into SELECT); source REST get_courses c_only_fields/c_fields"
|
||||
privilege = "unauthenticated (REST /wp-json/learnpress/v1/courses)"
|
||||
license = "GPL"
|
||||
fix_site = "inc/Models/Courses.php :: handle_params_for_query_courses (fields quoted as identifiers); the REST controller stops assigning c_only_fields directly"
|
||||
notes = "The sink is where the SQL runs (class-lp-db.php:607 implode of only_fields); the fix is on the source side and does not change that file. It also touches assets/src/js/loadAJAX.js: the change itself is multi-language."
|
||||
|
||||
benign = { base = "aaf42fe0c6bf05dcf2ff125750b7bd836e76fa3c", tip = "179ed7760e6ae5fa46fbe9c5f1324278a42ece04", subject = "= 4.2.7.1 = ~ Added: lpOnElementReady function js" }
|
||||
[[case]]
|
||||
id = "phpmyfaq-captcha-ua-sqli"
|
||||
repo = "https://github.com/thorsten/phpMyFAQ"
|
||||
language = "php"
|
||||
also = ["typescript", "javascript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-46364", "GHSA-289f-fq7w-6q2w"]
|
||||
vulnerable = "11583abcf4787543d00c25d6c7926af6f509faf0"
|
||||
fixed = "545bdffb11244a4741cd29ac909a849c9a6a2e53"
|
||||
sink = "phpmyfaq/src/phpMyFAQ/Captcha/BuiltinCaptcha.php :: garbageCollector, saveCaptcha (User-Agent and client IP in sprintf SQL)"
|
||||
privilege = "unauthenticated (GET /api/captcha)"
|
||||
license = "MPL-2.0"
|
||||
notes = "The advisory's commit b9f25109 is the vulnerable master commit the reporter tested, not the fix. Release pair 4.1.1 (f791858b) -> 4.1.2 (66792f25)."
|
||||
|
||||
benign = { base = "bf781788ab6b5c2967a09d66216d7580778a2b5a", tip = "a4601ae5f01761dc0bb91b8c0c3901be91987ba3", subject = "build: updated version strings to 4.2" }
|
||||
[[case]]
|
||||
id = "yeswiki-bazar-form-sqli"
|
||||
repo = "https://github.com/YesWiki/yeswiki"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-46670", "GHSA-jwvv-qr7q-cv8j"]
|
||||
vulnerable = "6f222915df36fca628fc83325cb0feed8dc2aca0"
|
||||
fixed = "fe7244bffcbeffd42777496ebc2a6f6118cc2388"
|
||||
sink = "tools/bazar/services/FormManager.php :: FormManager::create (bn_id_nature unquoted in INSERT); source FormController::displayAll imported-form JSON"
|
||||
privilege = "unauthenticated"
|
||||
license = "AGPL-3.0"
|
||||
notes = "Four-line fix: intval plus a Guard::isAllowed check. Default branch doryphore-dev."
|
||||
|
||||
benign = { base = "03f27fdde5f89f90de76cd3f2160cff82e7bffac", tip = "70d240c0b43f7e7b0bac55fc0431fb4c2d88b49c", subject = "fix(contact): send shared page by mail" }
|
||||
[[case]]
|
||||
id = "signalk-remote-conn-ssrf"
|
||||
repo = "https://github.com/SignalK/signalk-server"
|
||||
language = "typescript"
|
||||
family = "untrusted_destination"
|
||||
advisory = ["CVE-2026-55591", "GHSA-q59x-jc9f-gfqf"]
|
||||
vulnerable = "1c479cc0d2d26d14ed74416cff36fa7336c7bf9e"
|
||||
fixed = "2816b0dda1bdde893d85c969a9c5950e7825eee7"
|
||||
sink = "src/serverroutes.ts :: makeRemoteRequest (req.body.host/port to http(s).request)"
|
||||
privilege = "unauthenticated by default (security not configured); admin otherwise"
|
||||
license = "Apache-2.0"
|
||||
notes = "Fix adds a schema, an ssrfGuard module and requestId validation (~450 added lines, mostly guard code and tests)."
|
||||
|
||||
benign = { base = "8af993d45c898edfe28f820f1726b6fc33b27cad", tip = "07416b79601c5d901f671f70a94bd57bed90bad8", subject = "fix: allow multiple deltaHandler next calls (#2818)" }
|
||||
[[case]]
|
||||
id = "saltcorn-sync-sqli"
|
||||
repo = "https://github.com/saltcorn/saltcorn"
|
||||
language = "javascript"
|
||||
also = ["typescript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-41478", "GHSA-jp74-mfrx-3qvh"]
|
||||
vulnerable = "e7e09254a3c69dc6582eda9e5b5769935db108af"
|
||||
fixed = "2b6c8a2eab88955ea6b12b1b4e5c2e613faefef5"
|
||||
sink = "packages/server/routes/sync.js :: getSyncRows, getDelRows (maxLoadedId and timestamps in SQL template literals)"
|
||||
privilege = "authenticated low-privilege user"
|
||||
license = "MIT"
|
||||
notes = "Large multi-package repository; the fix is on the main line (v1.4.6 is a separate backport), so use the commit pair."
|
||||
|
||||
benign = { base = "5c619403c40447bf56ce662570d3a3ffbf781019", tip = "df114765fd36d280395b2129f1703fa98c712d06", subject = "updated at in trigger.findAllWithTableName" }
|
||||
[[case]]
|
||||
id = "mongo-express-bson-rce"
|
||||
repo = "https://github.com/mongo-express/mongo-express"
|
||||
language = "javascript"
|
||||
family = "injection"
|
||||
advisory = ["CVE-2019-10758", "GHSA-h47j-hc6x-h3qq"]
|
||||
vulnerable = "ea02b364d43f179f191fc91fb9962efdb0843a8d"
|
||||
fixed = "d8c9bda46a204ecba1d35558452685cd0674e6f2"
|
||||
sink = "lib/bson.js :: toBSON (vm.runInNewContext over request text); callers lib/routes/collection.js, lib/routes/document.js"
|
||||
privilege = "authenticated (HTTP basic auth, default admin:pass)"
|
||||
license = "MIT"
|
||||
notes = "Code injection, not SQL. The advisory's first reference 7d365141 is only the v0.54.0 release commit."
|
||||
|
||||
benign = { base = "9550ae2529258b47a2b637d8e64e69a0d4d38ad8", tip = "5b9b34f8553d004ceb7b666bbf566b406a2a3153", subject = "Merge pull request #538 from damien-malescot/master" }
|
||||
[[case]]
|
||||
id = "mockoon-admin-api-cors"
|
||||
repo = "https://github.com/mockoon/mockoon"
|
||||
language = "typescript"
|
||||
family = "config_secrets"
|
||||
advisory = ["CVE-2026-59148", "GHSA-rqx4-3f6q-3x2v"]
|
||||
vulnerable = "b42bdfb7f82e83f0e81bea8e6fe41adf5ec82585"
|
||||
fixed = "c420b5a56918475b8663977b51e5f986e45b3299"
|
||||
sink = "packages/commons-server/src/libs/server/admin-api.ts :: admin-API middleware (Access-Control-Allow-Origin: *)"
|
||||
sink_evidence = "'Access-Control-Allow-Origin': '*'"
|
||||
privilege = "unauthenticated"
|
||||
license = "MIT"
|
||||
notes = "Weaker fit: the fix commit touches 34 files; only one hunk in admin-api.ts is the CORS fix."
|
||||
|
||||
benign = { base = "de69f5e30f89e17918855d147f4e8c055d5b2ba9", tip = "895961070b55a14d290ad50232ab9c6d800c3cad", subject = "Fix serverless lib returning a 404 when no body is present (#2274)" }
|
||||
[[case]]
|
||||
id = "alerta-query-sqli"
|
||||
repo = "https://github.com/alerta/alerta"
|
||||
language = "python"
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-34400", "GHSA-8prr-286p-4w7j"]
|
||||
vulnerable = "71c098d8bb8091e76edaefeef4662eb77c753ce7"
|
||||
fixed = "fdd52cd1abad8d02d1dfb8ecdcdbb43b6af3b883"
|
||||
sink = "alerta/database/backends/postgres/queryparser.py :: SearchTerm.__repr__ (f-string WHERE) used by utils.py QueryBuilder.from_params"
|
||||
privilege = "API user; possibly unauthenticated if AUTH_REQUIRED defaults to False (unverified)"
|
||||
license = "Apache-2.0"
|
||||
notes = "The advisory's aeba85a3 introduced the parser; the fix is fdd52cd1."
|
||||
|
||||
benign = { base = "b7111d23ae9aefb5064ef107976f62e3836217b6", tip = "71c098d8bb8091e76edaefeef4662eb77c753ce7", subject = "fix(deps): replace pkg_resources with importlib.metadata (#2041)" }
|
||||
[[case]]
|
||||
id = "wger-next-open-redirect"
|
||||
repo = "https://github.com/wger-project/wger"
|
||||
language = "python"
|
||||
family = "untrusted_destination"
|
||||
advisory = ["GHSA-vqv8-j3mj-wjxj"]
|
||||
vulnerable = "d30e85b23fb1307ac6579e9e3ae393c50470cea7"
|
||||
fixed = "0ba371bbb50cd5668820f240995405e1b2483384"
|
||||
sink = "wger/core/views/user.py :: trainer_login; wger/core/views/languages.py :: use_browser_language (request.GET next redirected unchecked)"
|
||||
privilege = "use_browser_language: unauthenticated; trainer_login: authenticated trainer"
|
||||
license = "AGPL-3.0"
|
||||
notes = "No CVE. A later commit cfa27a7f fixes the same pattern in core/middleware.py; not part of this fix."
|
||||
|
||||
benign = { base = "47d646c24178783eb6787ecf26e79dc8f4a394c6", tip = "c1144533bf9bca487ff905d925b567151d89c1aa", subject = "Formatting" }
|
||||
[[case]]
|
||||
id = "opencve-notification-ssrf"
|
||||
repo = "https://github.com/opencve/opencve"
|
||||
language = "python"
|
||||
family = "untrusted_destination"
|
||||
advisory = ["CVE-2026-62282", "GHSA-ch3g-4xvr-674q"]
|
||||
vulnerable = "928bbadbb1672051bcfc2a6289a9e6a8f029c106"
|
||||
fixed = "0af7f894debc95c67fa53e7ba5a6cdecb10c34f0"
|
||||
sink = "web/projects/notifications.py :: _http_post_json (webhook and Slack test POST to a user-supplied URL)"
|
||||
privilege = "authenticated (can configure project notifications)"
|
||||
license = "BUSL-1.1 (verified 2026-10-02: github.com/opencve/opencve LICENSE -- Business Source License 1.1, not OSI; non-production use permitted by the Additional Use Grant; change date 2030-08-14 to Apache-2.0). Published by maintainer decision 2026-10-02: the record holds metadata only (repository, pins, detection), no OpenCVE code."
|
||||
notes = "Selected by the maintainer despite the licence question; results must not be redistributed until it is checked."
|
||||
benign = { base = "0c8101082ba20628b28fcec9c4611745d32202c2", tip = "965dc348e225308edabdfef3ecb76f7d73a0c162", subject = "fix: unlink automation executions before cleaning expired reports" }
|
||||
@@ -0,0 +1,338 @@
|
||||
# Independent evaluation population v2 (milestone M4). Reserved 2026-09-26.
|
||||
#
|
||||
# RESERVED, NOT SCANNED. Chosen by web research alone (research-2026-09-26.md): advisories taken newest-first per
|
||||
# language under fixed criteria, never by how easy a case looks for this analyzer; 261 advisories examined, 224
|
||||
# rejected with reasons. No repository was cloned, no scanner or project tooling run on any of them, and none is
|
||||
# referenced anywhere in this project's development tree (checked against every github.com URL under benchmarks/,
|
||||
# src/, tests/ and .kiro/, which includes the v1 population).
|
||||
#
|
||||
# Why a v2: v1 (2026-09-24) was evaluated and then used to retune the analyzer (file-owned gaps, Node vm sinks,
|
||||
# Django sources, quoted-only SQL escapes, guards, .gitignore semantics). A retune cannot be qualified on the
|
||||
# population that informed it.
|
||||
#
|
||||
# Rules for this file are v1's: no scan, trace, probe or census of these repositories before the population is
|
||||
# FROZEN and protocol-v2.md is committed; no rule, fact or query change may cite these cases before their
|
||||
# evaluation is recorded.
|
||||
#
|
||||
# `sites` are the sink or execution locations the ADVISORY names, declared here before any scan. Protocol v2
|
||||
# credits a finding at one of them as well as one matching the fix's changed lines.
|
||||
#
|
||||
# Selected by the maintainer on 2026-09-26 ("balanced" rule-based selection, 17 cases -- the menu called it 16; the
|
||||
# list it showed held 17). Licences marked "verify" must be checked before any result is published.
|
||||
|
||||
schema_version = 1
|
||||
status = "frozen"
|
||||
frozen_on = "2026-09-26"
|
||||
freeze_record = "freeze-v2.json"
|
||||
reserved_on = "2026-09-26"
|
||||
selected_by = "maintainer"
|
||||
research = "research-2026-09-26.md"
|
||||
protocol = "protocol-v2.md"
|
||||
multi_language_members = ["wp-gopay-log-filter-sqli", "wp-user-registration-login-redirect", "wp-directorist-avatar-ssrf", "wp-groundhogg-confirm-redirect"]
|
||||
also_reserved = ["population-v1.toml", "../push/ghost-reserved.toml"]
|
||||
|
||||
[[case]]
|
||||
id = "pgadmin-maintenance-sqli"
|
||||
repo = "https://github.com/pgadmin-org/pgadmin4"
|
||||
language = "python"
|
||||
also = ["javascript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-7815", "GHSA-hp84-p2gq-6fvr"]
|
||||
vulnerable = "d112dc3b962218efc3613fa727d54f0fc5b1d527"
|
||||
fixed = "cf53953d9ad66c182a08e662c1ba29c5831b486b"
|
||||
ancestry = "parent"
|
||||
sink = "web/pgadmin/tools/maintenance/templates/maintenance/sql/command.sql (string concatenation of vacuum_index_cleanup, vacuum_parallel, buffer_usage_limit, reindex_tablespace) rendered in web/pgadmin/tools/maintenance/__init__.py :: create_maintenance_job() → render_template(...) → BatchProcess(cmd=psql, args=[..., '--command', query])"
|
||||
source = "JSON body of POST /maintenance/job/<sid>/<did>"
|
||||
sites = ["web/pgadmin/tools/maintenance/__init__.py::create_maintenance_job"]
|
||||
privilege = "authenticated (tools_maintenance permission)"
|
||||
license = "PostgreSQL (verified 2026-10-02: LICENSE, d112dc3b962218efc3613fa727d54f0fc5b1d527)"
|
||||
confidence = "high"
|
||||
benign = { base = "e7eaef0dc002c254e4dd15705df800c448e8eb4c", tip = "a50a553b0a7e529e5d87f52bdc6d3a5ee3e8b9e8", subject = "chore: feature tests use sys.executable; sync yarn.lock to package.json", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "qwed-sympy-eval"
|
||||
repo = "https://github.com/QWED-AI/qwed-verification"
|
||||
language = "python"
|
||||
also = []
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-55585", "GHSA-q27q-98j4-9pfv"]
|
||||
vulnerable = "06801f68cbf9c9b5e54cc17f2a61cb2792da7a26"
|
||||
fixed = "6066b68c0c4f4cc2c3771824822aaa864d082ef8"
|
||||
ancestry = "first parent (fixed is the merge of PR #200; the PR commit `dc9d4db72ca4b4ae3f96d0e6a0c27a9e38a06f61` is a direct child of the vulnerable SHA)"
|
||||
sink = "src/qwed_new/api/main.py :: verify_math() → sympy.parsing.sympy_parser.parse_expr(expression_normalized) (also left_str/right_str); second path src/qwed_new/core/batch.py :: _verify_item() → parse_expr(item.query)"
|
||||
source = "POST /verify/math JSON expression; POST /verify/batch item query"
|
||||
sites = ["src/qwed_new/api/main.py::verify_math", "src/qwed_new/core/batch.py::_verify_item"]
|
||||
privilege = "authenticated tenant (API key; self-signup /auth/signup is on by default)"
|
||||
license = "Apache-2.0"
|
||||
confidence = "high"
|
||||
benign = { base = "5214f2c558641f4d02d3d75abf039f164e393131", tip = "637a880a8333875543090894fc13bf092783d90e", subject = "feat: emit VC v1.0 outputs from Docker action entrypoint (#313)", how = "chosen by hand after review" }
|
||||
|
||||
[[case]]
|
||||
id = "contextforge-prompt-ssti"
|
||||
repo = "https://github.com/IBM/mcp-context-forge"
|
||||
language = "python"
|
||||
also = ["javascript"]
|
||||
family = "injection"
|
||||
advisory = ["GHSA-vwf3-4xxj-qg6h"]
|
||||
vulnerable = "0f8e0ad679f1fff685f486e17f642ac863c8ca9e"
|
||||
fixed = "4d31004661858a2e99055b3c0c9f14218b8f7120"
|
||||
ancestry = "parent"
|
||||
sink = "mcpgateway/services/prompt_service.py :: PromptService._render_template() → _compile_jinja_template() → _get_jinja_env().from_string(template).render() (plain jinja2.Environment)"
|
||||
source = "prompt template stored through register_prompt (prompts create/update API), rendered on prompt get"
|
||||
sites = ["mcpgateway/services/prompt_service.py::_render_template", "mcpgateway/services/prompt_service.py::_compile_jinja_template"]
|
||||
privilege = "authenticated (permission to create or update prompts)"
|
||||
license = "Apache-2.0"
|
||||
confidence = "high"
|
||||
benign = { base = "1f8fd58980f4565d4788ffc3587d6e3bd16b3342", tip = "9938c58f7819c526932c4ae4ad646a9c028cb19e", subject = "fix: add rust coverage reports (#4458)", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "lmdeploy-media-url-ssrf"
|
||||
repo = "https://github.com/InternLM/lmdeploy"
|
||||
language = "python"
|
||||
also = []
|
||||
family = "untrusted_destination"
|
||||
advisory = ["GHSA-39wr-7q6h-cf68"]
|
||||
vulnerable = "71648d239c978452720a6d6692fe914efc8a7b94"
|
||||
fixed = "03c313006d17cc3feae86b633c44206a997c44db"
|
||||
ancestry = "parent"
|
||||
sink = "lmdeploy/vl/media/connection.py :: _load_http_url() → requests.Session().get(url_spec.geturl(), allow_redirects=True), gated by _is_safe_url() (the urlparse host differs from the requests host)"
|
||||
source = "image_url/video/audio URLs in POST /v1/chat/completions → serve/processors/multimodal.py :: MultimodalProcessor._parse_multimodal_item() → load_from_url()"
|
||||
sites = ["lmdeploy/vl/media/connection.py::_load_http_url"]
|
||||
privilege = "unauthenticated by default (api_keys is optional, default None)"
|
||||
license = "Apache-2.0"
|
||||
confidence = "medium"
|
||||
benign = { base = "b6a29db4d2d218838048975977f125dc9b1ac75d", tip = "3d9d23087b59fa44afe83deb03ee7cce04df3b4d", subject = "fix(benchmark): seed NumPy RNG so --seed controls random-mode lengths (#4785)", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "winml-cli-serve-cors"
|
||||
repo = "https://github.com/microsoft/winml-cli"
|
||||
language = "python"
|
||||
also = []
|
||||
family = "config_secrets"
|
||||
advisory = ["CVE-2026-84452", "GHSA-96p9-rh4f-92cf"]
|
||||
vulnerable = "02179d185f0e3ac23518236b5ca1f61fd37ef680"
|
||||
fixed = "f4073e0ef4700a25b623487e7e45c421ca0b9993"
|
||||
ancestry = "parent"
|
||||
sink = "src/winml/modelkit/serve/cli_api.py :: module level app.add_middleware(CORSMiddleware, allow_origins=[\"*\"], ...) (same in serve/app.py :: create_app())"
|
||||
source = "any web page in the victim's browser calling POST /v1/cli/{command} on 127.0.0.1"
|
||||
sites = ["src/winml/modelkit/serve/cli_api.py::<global>", "src/winml/modelkit/serve/app.py::create_app"]
|
||||
privilege = "unauthenticated (cross-origin from any website)"
|
||||
license = "MIT"
|
||||
confidence = "high"
|
||||
benign = { base = "8d631f6f1e5db26a04e8c13045280408c6f984dc", tip = "be3e59dd412d4918c5a852aa4d8f0207c34aaf6f", subject = "Remove preload ORT DLL (undo PR 689) (#1380)", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "reactpress-findall-column-sqli"
|
||||
repo = "https://github.com/fecommunity/reactpress"
|
||||
language = "typescript"
|
||||
also = ["javascript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-61685", "GHSA-wmw4-mw6x-6vfm"]
|
||||
vulnerable = "c3dd10681b3d2b5e224f7ac647b3769557603975"
|
||||
fixed = "78ecb70af1c021455c05fdcbe137212c70e310d6"
|
||||
ancestry = "parent"
|
||||
sink = "server/src/modules/article/article.service.ts :: ArticleService.findAll() (query-string key interpolated as article.${key} into andWhere); same pattern in comment/file/page/knowledge services"
|
||||
source = "@Query() queryParams in ArticleController.findAll() (GET /api/article)"
|
||||
sites = ["server/src/modules/article/article.service.ts::findAll"]
|
||||
privilege = "unauthenticated"
|
||||
license = "MIT"
|
||||
confidence = "high"
|
||||
benign = { base = "295f5de807ed1ebec7b6b155cce16b24a743f247", tip = "903d4b2e2de1c259843f3a8c67701fa7d50d2205", subject = "docs: update documentation links to new site URL", how = "chosen by hand after review" }
|
||||
|
||||
[[case]]
|
||||
id = "fuxa-tdengine-escape-sqli"
|
||||
repo = "https://github.com/frangoteam/FUXA"
|
||||
language = "javascript"
|
||||
also = ["typescript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-47720", "GHSA-h9fj-c2qr-76g2"]
|
||||
vulnerable = "193a888f8e3f805325072829b1644a2c768c948b"
|
||||
fixed = "b78f6159d02a56ce5ff48207feb936afb3534fc8"
|
||||
ancestry = "parent"
|
||||
sink = "server/runtime/storage/tdengine/index.js :: TDengine.getDaqValue() (escapeTdString(tagid) doubles quotes but not backslashes, then SQL string for cursor.query)"
|
||||
source = "sids of GET /api/daq (server/api/daq/index.js) or Socket.IO DAQ_QUERY"
|
||||
sites = ["server/runtime/storage/tdengine/index.js::getDaqValue"]
|
||||
privilege = "unauthenticated (guest level; socket handler has no auth check)"
|
||||
license = "MIT"
|
||||
confidence = "medium"
|
||||
benign = { base = "e787e4140b5e71c5b02e23c295fa08b8df0edc53", tip = "bf998eeee3158e338724c71979b7cdafcfa74b32", subject = "Button control with configurable icon (#2349)", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "budibase-mongo-template-nosqli"
|
||||
repo = "https://github.com/Budibase/budibase"
|
||||
language = "typescript"
|
||||
also = ["javascript"]
|
||||
family = "injection"
|
||||
advisory = ["GHSA-pmpg-2mxq-6xwr", "GHSA-qw6m-8fw2-2v64"]
|
||||
vulnerable = "b7e63df737eede85d951aa9665b0c0778ece7ccb"
|
||||
fixed = "dd8c0654b35cd89ce3645f2355f4bf2ff9a5dd80"
|
||||
ancestry = "parent"
|
||||
sink = "packages/server/src/sdk/workspace/queries/queries.ts :: enrichContext() (handlebars processStringSync(..., {noEscaping: true}) into the json template, then JSON.parse) → packages/server/src/integrations/mongodb.ts :: MongoIntegration.read() (collection.find(json))"
|
||||
source = "query parameters / bindings (e.g. currentUser.email, request parameters) of POST /api/v2/queries/:queryId (packages/server/src/api/controllers/query/index.ts :: executeV2)"
|
||||
sites = ["packages/server/src/sdk/workspace/queries/queries.ts::enrichContext", "packages/server/src/integrations/mongodb.ts::read"]
|
||||
privilege = "authenticated app user (BASIC role)"
|
||||
license = "GPL-3.0 (root LICENSE says \"GPLv3 overall\"; server package.json GPL-3.0; GitHub NOASSERTION)"
|
||||
confidence = "medium"
|
||||
benign = { base = "a4423fc5bf237e6101c2e8348ef5372fff6484cd", tip = "272c6a61a2a386688792f9e26c2665f2f431a5c2", subject = "Merge pull request #18865 from Budibase/feature/18784-ai-credit", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "unleash-addon-webhook-ssrf"
|
||||
repo = "https://github.com/Unleash/unleash"
|
||||
language = "typescript"
|
||||
also = ["javascript"]
|
||||
family = "untrusted_destination"
|
||||
advisory = ["CVE-2026-63004", "GHSA-5vf6-jrqr-78fj"]
|
||||
vulnerable = "b9f3217d327f24094b4352130baa30387f85cffe"
|
||||
fixed = "2100db76af3473f13e6fb40096cf17a9c2b741a1"
|
||||
ancestry = "parent"
|
||||
sink = "src/lib/addons/addon.ts :: Addon.fetchRetry() (ky(url, ...)), called from webhook.ts / slack / teams / datadog / new-relic addons"
|
||||
source = "addon parameters.url saved through the admin addons API (CREATE_ADDON/UPDATE_ADDON), fired on feature events"
|
||||
sites = ["src/lib/addons/addon.ts::fetchRetry"]
|
||||
privilege = "authenticated (CREATE_ADDON/UPDATE_ADDON root permission)"
|
||||
license = "AGPL-3.0"
|
||||
confidence = "high"
|
||||
benign = { base = "d4efb3a31616483c438606edbecce13b868fc587", tip = "cb4e6250743fac364bae50b985b2b8348def7a2b", subject = "Merge commit '232b5d575c7d455ce7ac767183e61fd389dd8aac' into automation/sync-oss-upstream", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "medplum-external-auth-open-redirect"
|
||||
repo = "https://github.com/medplum/medplum"
|
||||
language = "typescript"
|
||||
also = ["javascript"]
|
||||
family = "untrusted_destination"
|
||||
advisory = ["CVE-2026-53728", "GHSA-m44r-7c5h-m6mj"]
|
||||
vulnerable = "5d4a8654b02aa67c444f96ae1eb5663c09854006"
|
||||
fixed = "7ae10035ddadde4dba7b18d3156553940465b3a1"
|
||||
ancestry = "parent"
|
||||
sink = "packages/server/src/auth/external.ts :: externalCallbackHandler() (res.redirect(redirectUrl) with URI accepted by getClientRedirectUri(client, body.redirectUri, true) prefix match in packages/server/src/oauth/clients.ts)"
|
||||
source = "redirectUri inside JSON state query parameter of GET /auth/external"
|
||||
sites = ["packages/server/src/auth/external.ts::externalCallbackHandler"]
|
||||
privilege = "unauthenticated (victim completes external IdP login)"
|
||||
license = "Apache-2.0"
|
||||
confidence = "high"
|
||||
benign = { base = "0dd335e8520b859563626d9bf50d602a5f3ad79a", tip = "20790b17d76f335e60dd703f65d4eb13bbb4754d", subject = "tweak(cdk): make `minHealthyPercent` 100 (#8836)", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "cct-studio-server-cmdi"
|
||||
repo = "https://github.com/davila7/claude-code-templates"
|
||||
language = "javascript"
|
||||
also = ["typescript", "python"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-73222", "GHSA-79wm-x847-7cvg"]
|
||||
vulnerable = "acdb5893412a81eaec9862ee24a4613bf296ab45"
|
||||
fixed = "bc4618b07232633c1c0aac12a43e436268d31783"
|
||||
ancestry = "parent"
|
||||
sink = "cli-tool/src/sandbox-server.js :: executeLocalTask() (spawn('claude', [finalPrompt], {shell: true})), route handler POST /api/install-agent and checkAndInstallAgent() (spawn('npx', [..., agentName, ...], {shell: true}))"
|
||||
source = "req.body.prompt, req.body.agentName, req.body.agent"
|
||||
sites = ["cli-tool/src/sandbox-server.js::executeLocalTask", "cli-tool/src/sandbox-server.js::checkAndInstallAgent"]
|
||||
privilege = "unauthenticated (server on 0.0.0.0, wildcard CORS)"
|
||||
license = "MIT"
|
||||
confidence = "high"
|
||||
benign = { base = "c2ed4066bc7ea66806e90d6c0b97988495ff44b2", tip = "025520a8bfc8ed05e9a197bf53fa2f871f666dea", subject = "feat(dashboard): hide Loops from navigation menu (#738)", how = "chosen by hand after review" }
|
||||
|
||||
[[case]]
|
||||
id = "wp-gopay-log-filter-sqli"
|
||||
repo = "https://github.com/gopaycommunity/gopay-woocommerce-integration"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-75959", "GHSA-88c4-2f34-57h5"]
|
||||
vulnerable = "626bc8b5f8d33de6eefd921e3a05ed52ac320bcd"
|
||||
fixed = "7190c985370f9758f226c2f2addd79532a2f93d5"
|
||||
ancestry = "parent"
|
||||
sink = "admin/views/log.php :: file scope (included by admin/class-gopay-gateway-admin.php), two $wpdb->get_results(sprintf(\"... REGEXP '[\\w\\W]*%s[\\w\\W]*' ...\", ..., strtoupper($log_table_filter)))"
|
||||
source = "filter_input(INPUT_GET, 'log_table_filter')"
|
||||
sites = ["admin/views/log.php::<global>"]
|
||||
privilege = "admin (shop manager, manage_woocommerce)"
|
||||
license = "GPL-2.0-or-later (verified 2026-10-02: gopay-gateway.php header, 626bc8b5f8d33de6eefd921e3a05ed52ac320bcd)"
|
||||
confidence = "high"
|
||||
benign = { base = "2b5fe58d6dadbfecb01ed34611aed8502d11e612", tip = "88b10bda704a37e88121e22dc22f4e5019097272", subject = "GPY025-136: fix i18n and remove translation calls on non-literal text", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "wp-user-registration-login-redirect"
|
||||
repo = "https://github.com/themegrill/user-registration"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
family = "untrusted_destination"
|
||||
advisory = ["CVE-2026-80072", "GHSA-w78c-fx7q-9x8m"]
|
||||
vulnerable = "e52cf151cb229852e6209fb3a627aa51485139c1"
|
||||
fixed = "75c940dab15fd54d5a3fbb9f2213767b89467b08"
|
||||
ancestry = "parent"
|
||||
sink = "includes/functions-ur-core.php :: ur_process_login() (wp_redirect(wp_validate_redirect($redirect, $redirect)), and wp_send_json_success(['message'=>$redirect]) on the AJAX path)"
|
||||
source = "$post['redirect'] (login form POST) or wp_get_raw_referer()"
|
||||
sites = ["includes/functions-ur-core.php::ur_process_login"]
|
||||
privilege = "unauthenticated attacker, victim logs in"
|
||||
license = "GPL-3.0-or-later (verified 2026-10-02: license.txt, e52cf151cb229852e6209fb3a627aa51485139c1)"
|
||||
confidence = "high"
|
||||
benign = { base = "064dd7ab1585082b9b5cf078f2d22c0234f25e48", tip = "3b6acc4695e93d9d48969ed894c21419bd09b33a", subject = "Fix - Deactivate Licence button did nothing when the pro-activation transient was set (#1418)", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "wp-directorist-avatar-ssrf"
|
||||
repo = "https://github.com/sovware/directorist"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
family = "untrusted_destination"
|
||||
advisory = ["CVE-2026-84046", "GHSA-6q63-3w56-m722"]
|
||||
vulnerable = "77cfc9bc8b453f97ab7e720330706fa4ef056867"
|
||||
fixed = "1a67825d5cf6e7c768c3a692da1d18bae732bea9"
|
||||
ancestry = "parent"
|
||||
sink = "includes/rest-api/functions.php :: directorist_rest_upload_image_from_url() (download_url($image_url))"
|
||||
source = "$request['avater']['src'] in includes/rest-api/Version1/class-users-controller.php :: update_user_meta_fields() (REST user update)"
|
||||
sites = ["includes/rest-api/functions.php::directorist_rest_upload_image_from_url"]
|
||||
privilege = "authenticated (subscriber)"
|
||||
license = "GPL-3.0-or-later (verified 2026-10-02: license.txt + directorist-base.php header, 77cfc9bc8b453f97ab7e720330706fa4ef056867)"
|
||||
confidence = "medium"
|
||||
benign = { base = "61ddcdf84f18cbe7749036a912d2931cef8cbd5a", tip = "986becd484f8f2d9b60604737d4fb4bf08fa6d1b", subject = "Replaced Location with Address on the All Listings map view (#2280)", how = "chosen by hand after review" }
|
||||
|
||||
[[case]]
|
||||
id = "wp-groundhogg-confirm-redirect"
|
||||
repo = "https://github.com/groundhoggwp/groundhogg"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
family = "untrusted_destination"
|
||||
advisory = ["CVE-2026-81741", "GHSA-m96f-c5vh-h7cm"]
|
||||
vulnerable = "9472a1dd79d4556c40eece501a61930793f170d3"
|
||||
fixed = "028247d4b8ec9ba051d3eb48f1466a759bc16bfd"
|
||||
ancestry = "parent (follow-up is the direct child of the fix)"
|
||||
sink = "templates/preferences.php :: file scope, case 'confirm': (wp_redirect($redirect_to))"
|
||||
source = "get_url_var('redirect_to') (query string, only through esc_url_raw(sanitize_text_field(...)))"
|
||||
sites = ["templates/preferences.php::<global>"]
|
||||
privilege = "unauthenticated (crafted preferences link)"
|
||||
license = "GPL-3.0-or-later (verified 2026-10-02: groundhogg.php header, 9472a1dd79d4556c40eece501a61930793f170d3)"
|
||||
confidence = "high"
|
||||
benign = { base = "61c8a03f11b6030c630be6ae0b0c2ad641eb733a", tip = "e3a69f1c2814c6bebed940505894008748566b93", subject = "output a square image regardless of image dimensions", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "froxlor-admin-ip-second-order-sqli"
|
||||
repo = "https://github.com/froxlor/froxlor"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-54348", "GHSA-w27m-rmmf-g5w4"]
|
||||
vulnerable = "9bf03de643a2edee4642bfc6c94db1f05253da02"
|
||||
fixed = "a1eaca5a1601c8a30e00814a4fc73ad0c185f89e"
|
||||
ancestry = "parent (ancestor of tag 2.3.8)"
|
||||
sink = "lib/Froxlor/Api/Commands/IpsAndPorts.php :: IpsAndPorts::listing() / listingCount() (implode(\", \", json_decode($this->getUserDetail('ip'), true)) into WHERE id IN (...)); also lib/Froxlor/Api/Commands/Domains.php :: validateIpAddresses()"
|
||||
source = "ipaddress API parameter of Admins.add / Admins.update (lib/Froxlor/Api/Commands/Admins.php), stored as JSON in panel_admins.ip, read back via getUserDetail('ip')"
|
||||
sites = ["lib/Froxlor/Api/Commands/IpsAndPorts.php::listing", "lib/Froxlor/Api/Commands/IpsAndPorts.php::listingCount", "lib/Froxlor/Api/Commands/Domains.php::validateIpAddresses"]
|
||||
privilege = "admin (API key with change_serversettings)"
|
||||
license = "LGPL-2.1 (per GitHub)"
|
||||
confidence = "high"
|
||||
benign = { base = "b595f061fbb68874423809fec9f2a42bf6b34175", tip = "9d274828bdc4eefec0058a5a3a25bf4eb69d0f70", subject = "avoid warnings regarding php8.5 in config/database.php; adjust phpunit for froxlor/framework testing", how = "automatic proposal, reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "sulu-smart-content-sql2-injection"
|
||||
repo = "https://github.com/sulu/sulu"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
family = "injection"
|
||||
advisory = ["CVE-2026-92692", "GHSA-jg26-q8hg-3pq4"]
|
||||
vulnerable = "d061094f5b7bb1d5e974544fce30bede9c7adf8e"
|
||||
fixed = "d19c01487af8c3de2fb3aa145856707a6367392c"
|
||||
ancestry = "parent (2.6 branch; ancestor of tag 2.6.25)"
|
||||
sink = "src/Sulu/Component/Content/SmartContent/QueryBuilder.php :: QueryBuilder::buildCategoriesWhere() (also buildTagsWhere(), buildAudienceTargeting()), string-concatenated into the SQL2 WHERE clause built by buildWhere()"
|
||||
source = "?categories= query parameter, read in src/Sulu/Component/Category/Request/CategoryRequestHandler.php :: getCategories() (explode + trim only), passed by src/Sulu/Component/SmartContent/ContentType.php as websiteCategories"
|
||||
sites = ["src/Sulu/Component/Content/SmartContent/QueryBuilder.php::buildCategoriesWhere", "src/Sulu/Component/Content/SmartContent/QueryBuilder.php::buildWhere"]
|
||||
privilege = "unauthenticated (any public page with a category-filtered Smart Content block)"
|
||||
license = "MIT"
|
||||
confidence = "high"
|
||||
benign = { base = "777f4eb47d45dab420c82b43ae6f513638294de0", tip = "6621a429c5284bd05ee1b60f333aff83297fecde", subject = "Removing unused service argument (#7169)", how = "chosen by hand after review" }
|
||||
@@ -0,0 +1,457 @@
|
||||
# Independent evaluation population v3 (PHP). Reserved 2026-09-30.
|
||||
#
|
||||
# RESERVED, NOT SCANNED. Chosen by web research alone (research-2026-09-30-php.md, candidates-v3-php.json):
|
||||
# advisories published 2026-06-01 or later, taken newest first per family under criteria fixed before any
|
||||
# advisory was read, never by how easy a case looks for this analyzer; 470 advisories examined, 26 candidates.
|
||||
# No repository was cloned, no scanner or project tooling run on any of them, and none is referenced anywhere in
|
||||
# this project's development tree (checked against every github.com URL under benchmarks/, src/, tests/ and
|
||||
# .kiro/, which includes populations v1 and v2).
|
||||
#
|
||||
# Why a v3: the PHP quick-mode rules were written on 2026-09-30 while looking at PMPro, population-v1's PHP cases,
|
||||
# Ultimate Member, MediaWiki and WP Statistics, so no PHP number measured so far qualifies them, and population v2
|
||||
# is reserved for other qualification. v3 is PHP cases nobody in this project has looked at.
|
||||
#
|
||||
# HOLD-OUT RULES (v1's and v2's): no scanner, quick rule, engine, plane task or model is run on any of these
|
||||
# repositories, and no trace, probe or census of them is made, before the population is FROZEN and protocol-v3.md
|
||||
# is committed. Freezing reads git history, file lists and the fix commit's diff only. No rule, fact, query,
|
||||
# prompt or scoring change may cite these cases before their evaluation is recorded; any analyzer change made
|
||||
# after a v3 result is a retune that v3 cannot qualify. tests/test_independent_population.py fails the moment one
|
||||
# of these repositories is named anywhere under benchmarks/, src/ or tests/ outside this directory.
|
||||
#
|
||||
# SELECTION RULE (research-2026-09-30-php.md, "Recommended balanced selection"), applied mechanically: family
|
||||
# targets injection 5, output_encoding 3, path 3, untrusted_destination 2, access_control 2, deserialization 1;
|
||||
# within a family, candidates whose fix commit the advisory names come first, then newest first by publication;
|
||||
# take the first n.
|
||||
#
|
||||
# Maintainer's decision 2026-09-30, "Recommended 16, minus v2 mentions": the advisories that
|
||||
# research-2026-09-26.md (population v2's research) names in its passed-over note are excluded, and each slot
|
||||
# they held goes to the next candidate in its family by the same rule. Excluded as v2 mentions: Winter, elFinder,
|
||||
# Easy!Appointments (all three in the recommended 16), and Grav, Backpack CRUD and Koel (alternates).
|
||||
# - injection: winter-filter-numberrange-sqli -> mantisbt-tokenizer-eval-rce (next named fix, 2026-07-15).
|
||||
# - untrusted_destination: elfinder-fsock-dns-rebinding-ssrf -> pagible-admin-proxy-dns-rebinding-ssrf (koel,
|
||||
# next in line, is a v2 mention; pagible's fix is inferred). easyappointments-caldav-ssrf -> EMPTY: the family
|
||||
# has no remaining candidate among the 26 (the research's further alternates were never verified and hold no
|
||||
# record), so the slot is left empty rather than bending the rule. The population holds 15 cases.
|
||||
#
|
||||
# `sites` are the sink or execution locations the ADVISORY names (`path::function`, `<global>` for file scope),
|
||||
# declared before any scan. Where an advisory names only a file (`site_files`), the function is resolved at
|
||||
# freeze from the fix commit's diff of that file (maintainer's decision 2, "Resolve the function at freeze"), and
|
||||
# the freeze record gives each site's basis. Protocol v3 credits a finding at a declared site as well as one
|
||||
# matching the fix's changed lines.
|
||||
#
|
||||
# Fixes the advisory does not name (`fix_named_by_advisory = false`) are confirmed at freeze: the diff must change
|
||||
# the declared sink file and function, or the case is dropped and the next candidate taken by the rule.
|
||||
#
|
||||
# Licences marked "verify" must be checked before any result is published. Kirby is under a proprietary licence:
|
||||
# it stays in the population, but its results are unpublishable without the licensor's permission.
|
||||
#
|
||||
# FROZEN 2026-09-30 (freeze.py, freeze-v3-php.json; git history, file lists and fix diffs only). No case dropped:
|
||||
# both inferred fixes change a declared site function (pagible: proxy and isValidUrl; podlove: create_link_item,
|
||||
# while GenerationGuard::__destruct is not in the diff). Sites resolved from the fix diff are marked in
|
||||
# `site_basis`; a file the fix does not change stays unresolved (`site_files_unresolved`). Benign controls come
|
||||
# from freeze.py's strict proposal (single-language population: PHP change, merges judged with everything they
|
||||
# bring in, history before the vulnerable pin when nothing after the fix qualifies), reviewed on commit messages
|
||||
# and file names; five proposals were replaced by hand, with the reason in `how`.
|
||||
|
||||
schema_version = 1
|
||||
status = "frozen"
|
||||
frozen_on = "2026-09-30"
|
||||
freeze_record = "freeze-v3-php.json"
|
||||
language = "php"
|
||||
reserved_on = "2026-09-30"
|
||||
selected_by = "maintainer"
|
||||
selection = "Recommended 16, minus v2 mentions (15 cases; one untrusted_destination slot empty)"
|
||||
research = "research-2026-09-30-php.md"
|
||||
candidates = "candidates-v3-php.json"
|
||||
protocol = "protocol-v3.md"
|
||||
multi_language_members = ["wp-geo-my-wp-info-window-lfi", "wp-graphql-updatepost-publish-bypass", "wp-podlove-generation-guard-poi"]
|
||||
also_reserved = ["population-v1.toml", "population-v2.toml", "../push/ghost-reserved.toml"]
|
||||
|
||||
[[case]]
|
||||
id = "gitlist-commandline-cmdi"
|
||||
repo = "https://github.com/klaussilveira/gitlist"
|
||||
language = "php"
|
||||
also = []
|
||||
wordpress = false
|
||||
family = "injection"
|
||||
subtype = "OS command / argument injection"
|
||||
advisory = ["CVE-2026-82668", "GHSA-v64j-pgc3-qgc9"]
|
||||
published = "2026-08-31"
|
||||
vulnerable = "f67609d52c1812fa8a7ed80eae5e795cfd72115f"
|
||||
fixed = "88cf2866083d5f7c20d9d565c45f828a7ad1516b"
|
||||
ancestry = "parent; ancestor of tag 3.0.0-beta"
|
||||
fix_named_by_advisory = true
|
||||
sink = "`getDefaultBranch` of src/SCM/System/Git/CommandLine.php (git command line)"
|
||||
source = "repository content/arguments reaching the git command line (not stated precisely)"
|
||||
sites = ["src/SCM/System/Git/CommandLine.php::getDefaultBranch"]
|
||||
site_files = []
|
||||
site_derivation = "file and function quoted by the advisory (VulDB-generated text)"
|
||||
privilege = "not stated ('remotely')"
|
||||
license = "BSD-3-Clause"
|
||||
confidence = "medium"
|
||||
caveats = ["VulDB auto-generated description; the fix message ('Prevent textconv') suggests git config/attribute abuse rather than shell metacharacters, and the patched lines may not be in getDefaultBranch -- verify at freeze"]
|
||||
benign = { base = "ce5806c849f453d7200c1f488ee6aba19eef9a4c", tip = "ecc8e18e03d9b547e82b7c7c148cb00ba01c0cc9", subject = "Merge pull request #902 from zoredache/sort_repositories", how = "automatic proposal (before the vulnerable pin), reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "qloapps-customermessage-bo-query-sqli"
|
||||
repo = "https://github.com/Qloapps/QloApps"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
wordpress = false
|
||||
family = "injection"
|
||||
subtype = "SQL injection"
|
||||
advisory = ["CVE-2026-75497", "GHSA-5w6r-2mxc-fq3v"]
|
||||
published = "2026-08-25"
|
||||
vulnerable = "5b4d65cbb8570a4ba46364ee0f2ded79d294ab84"
|
||||
fixed = "123c97c110b7053ea3297d8e58fe95b3c3536560"
|
||||
ancestry = "parent"
|
||||
fix_named_by_advisory = true
|
||||
sink = "database query in CustomerMessage.php built from the `bo_query` parameter"
|
||||
source = "`bo_query` request parameter"
|
||||
sites = ["classes/CustomerMessage.php::searchByName"]
|
||||
site_files = ["classes/CustomerMessage.php"]
|
||||
site_basis = { "classes/CustomerMessage.php::searchByName" = "resolved-from-fix-diff" }
|
||||
site_derivation = "the advisory names 'CustomerMessage.php' (basename) and the parameter; the classes/ path is from the fix commit; no function named"
|
||||
privilege = "administrator"
|
||||
license = "OSL-3.0"
|
||||
confidence = "medium"
|
||||
caveats = ["one fix covers two advisories (Address.php is GHSA-mj48); repo 129 MB, near the size cap", "function-level site to be resolved at freeze"]
|
||||
benign = { base = "5c52b864878848a625a5d9cfbb6e3925ab853ad1", tip = "a8f6071627bf97a77d715c861c1157fded5e9e60", subject = "Merge pull request #1813 from Azhar-Webkul/gli-3095", how = "chosen by hand after review: proposal e43b4500 changes payment-status logic; next passing change taken" }
|
||||
|
||||
[[case]]
|
||||
id = "roundcube-markasjunk-cmd-learn-rce"
|
||||
repo = "https://github.com/roundcube/roundcubemail"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
wordpress = false
|
||||
family = "injection"
|
||||
subtype = "OS command injection"
|
||||
advisory = ["CVE-2026-74997", "GHSA-q5rh-mmp4-ppgq"]
|
||||
published = "2026-08-17"
|
||||
vulnerable = "a1afb8fd1f00ed4cb9376c072bb5ca5ded64495e"
|
||||
fixed = "b8f90e28a46d42e79a69568cba897f8f4223d9cd"
|
||||
ancestry = "parent; ancestor of tag 1.6.18"
|
||||
fix_named_by_advisory = true
|
||||
sink = "the markasjunk plugin's cmd_learn driver runs a shell command with placeholder replacement values"
|
||||
source = "crafted placeholder replacement values (message/user data) when a message is marked as junk"
|
||||
sites = ["plugins/markasjunk/drivers/cmd_learn.php::_do_salearn"]
|
||||
site_files = ["plugins/markasjunk/drivers/cmd_learn.php"]
|
||||
site_basis = { "plugins/markasjunk/drivers/cmd_learn.php::_do_salearn" = "resolved-from-fix-diff" }
|
||||
site_derivation = "the advisory names 'the cmd_learn driver of the markasjunk plugin'; the file path follows from that name and is confirmed by the fix commit; no function named"
|
||||
privilege = "authenticated webmail user"
|
||||
license = "GPL-3.0"
|
||||
confidence = "medium"
|
||||
caveats = ["only instances that enable the markasjunk plugin with the cmd_learn driver"]
|
||||
benign = { base = "1cddafded973c4bf6faa173accaf8f52ca1ac0d0", tip = "747f544708ed1d698b677d333346164a9451aeda", subject = "Simplify", how = "automatic proposal (after the fix), reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "maccms10-template-blacklist-rce"
|
||||
repo = "https://github.com/magicblack/maccms10"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
wordpress = false
|
||||
family = "injection"
|
||||
subtype = "code injection (template → PHP, blacklist bypass)"
|
||||
advisory = ["CVE-2026-71232", "GHSA-cv48-22qx-mpmx"]
|
||||
published = "2026-08-05"
|
||||
vulnerable = "2cabecda29947e8f9ea7cd3aa47ab90ed817605b"
|
||||
fixed = "71ad3bb29570e110d8e973acff68040a3050ddf0"
|
||||
ancestry = "parent"
|
||||
fix_named_by_advisory = true
|
||||
sink = "ThinkPHP `{if condition=...}` compiled into raw PHP; the admin template editor only blocks a function blacklist"
|
||||
source = "template content saved through the admin template editor"
|
||||
sites = ["application/admin/controller/Template.php::info"]
|
||||
site_files = ["application/admin/controller/Template.php"]
|
||||
site_basis = { "application/admin/controller/Template.php::info" = "resolved-from-fix-diff" }
|
||||
site_derivation = "file path quoted by the advisory; no function named"
|
||||
privilege = "administrator"
|
||||
license = "GitHub: NOASSERTION; no licence declared by the project [verify before publishing]"
|
||||
confidence = "medium"
|
||||
caveats = ["guard bypass (incomplete blacklist); the fix is still a blacklist", "licence not declared on GitHub"]
|
||||
benign = { base = "6ace264920bc50a07944c163b17a8fd8a3f7687f", tip = "c9f8d0e848489a2dc73daf1106748d2cd94d8138", subject = "修复 meilisearch 排序规则语法兼容问题", how = "chosen by hand after review: proposal 668ba418 fixes a registration-API validator (input validation); rejected" }
|
||||
|
||||
[[case]]
|
||||
id = "mantisbt-tokenizer-eval-rce"
|
||||
repo = "https://github.com/mantisbt/mantisbt"
|
||||
language = "php"
|
||||
also = []
|
||||
wordpress = false
|
||||
family = "injection"
|
||||
subtype = "code injection (eval)"
|
||||
advisory = ["CVE-2026-49273", "GHSA-v84x-qvhg-f36r"]
|
||||
published = "2026-07-15"
|
||||
vulnerable = "e7041ecc80c8d35f76a7ea1027a8a231244f7dc5"
|
||||
fixed = "78c0af63d1fe0118004744cab21ca3bf2cea0f5c"
|
||||
ancestry = "parent; ancestor of tag release-2.28.4"
|
||||
fix_named_by_advisory = true
|
||||
sink = "`Tokenizer` (reached through `ConfigParser`) calls `eval()` with a `return;` prefix; PHP hoists class declarations past the return"
|
||||
source = "non-string configuration value submitted to adm_config_set.php (Manage Configuration)"
|
||||
sites = ["core/classes/Tokenizer.class.php::__construct"]
|
||||
site_files = ["core/classes/Tokenizer.class.php", "adm_config_set.php"]
|
||||
site_basis = { "core/classes/Tokenizer.class.php::__construct" = "resolved-from-fix-diff" }
|
||||
site_files_unresolved = { "adm_config_set.php" = "the fix does not change this file" }
|
||||
sink_evidence = "eval("
|
||||
site_derivation = "the advisory names adm_config_set.php, ConfigParser, Tokenizer and eval(); the Tokenizer file path and the enclosing function are not named (path from the fix commit)"
|
||||
privilege = "administrator"
|
||||
license = "GitHub: none; GPL per project [verify before publishing]"
|
||||
confidence = "high"
|
||||
caveats = ["function-level site must be resolved at freeze by reading the file (the advisory names only the class)"]
|
||||
benign = { base = "088fa767bce246b6d2f083f32d937f22c8de4767", tip = "d6ca719b591f70acfa711fc41705388557b6ac04", subject = "Improve Workflow Graph display.", how = "automatic proposal (after the fix), reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "sharp-editor-iframe-srcdoc-xss"
|
||||
repo = "https://github.com/code16/sharp"
|
||||
language = "php"
|
||||
also = ["typescript"]
|
||||
wordpress = false
|
||||
family = "output_encoding"
|
||||
subtype = "stored XSS (sanitizer allow-list)"
|
||||
advisory = ["CVE-2026-61823", "GHSA-qxg3-46rw-79j8"]
|
||||
published = "2026-09-25"
|
||||
vulnerable = "67743149309054493ec8f5505173fc57aaff760f"
|
||||
fixed = "ec509a22c808a5bd9dfad6a0a85c92ce6f411e21"
|
||||
ancestry = "parent; ancestor of tag v9.22.5"
|
||||
fix_named_by_advisory = true
|
||||
sink = "Symfony HtmlSanitizer config allows the iframe `srcdoc` attribute; the value is rendered as live HTML"
|
||||
source = "Editor field content saved by a user with edit permission"
|
||||
sites = ["src/Utils/Sanitization/FormatsSanitizedValue.php::sanitizer", "src/Utils/Sanitization/FormatsSanitizedValue.php::isEncodingNeeded"]
|
||||
site_files = ["src/Utils/Sanitization/FormatsSanitizedValue.php"]
|
||||
site_basis = { "src/Utils/Sanitization/FormatsSanitizedValue.php::sanitizer" = "resolved-from-fix-diff", "src/Utils/Sanitization/FormatsSanitizedValue.php::isEncodingNeeded" = "resolved-from-fix-diff" }
|
||||
sink_evidence = "srcdoc"
|
||||
site_derivation = "file path quoted by the advisory; no function named"
|
||||
privilege = "authenticated editor"
|
||||
license = "MIT"
|
||||
confidence = "medium"
|
||||
caveats = ["the flaw is a sanitizer allow-list entry, not a missing escape call"]
|
||||
benign = { base = "ec509a22c808a5bd9dfad6a0a85c92ce6f411e21", tip = "7c601fffe930fc321a5d8bcb25cb072d50a6fc08", subject = "Update version to v9.22.5", how = "automatic proposal (after the fix), reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "privatebin-jsonld-request-uri-injection"
|
||||
repo = "https://github.com/PrivateBin/PrivateBin"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
wordpress = false
|
||||
family = "output_encoding"
|
||||
subtype = "reflected JSON injection (CWE-116)"
|
||||
advisory = ["CVE-2026-55891", "GHSA-xrjc-c68j-hp7w"]
|
||||
published = "2026-08-28"
|
||||
vulnerable = "e0dd4c025c19a182b6a4c6fb77a8bf81ceff6899"
|
||||
fixed = "75f056dcda955d94c17ec5a4f8c54a9b7bfcee07"
|
||||
ancestry = "first parent of a merge; ancestor of tag 2.0.5"
|
||||
fix_named_by_advisory = true
|
||||
sink = "`Controller::_jsonld()` splices `$_urlBase` into a .jsonld template with `str_replace()`"
|
||||
source = "`$_SERVER['REQUEST_URI']` via `Request::getRequestUri()` (FILTER_SANITIZE_URL keeps quotes and angle brackets)"
|
||||
sites = ["lib/Request.php::getRequestUri", "lib/Controller.php::_jsonld"]
|
||||
site_files = []
|
||||
site_derivation = "Class::method names quoted by the advisory; paths lib/*.php from the fix commit"
|
||||
privilege = "unauthenticated"
|
||||
license = "GitHub: NOASSERTION; project files state a zlib-style licence [verify before publishing]"
|
||||
confidence = "high"
|
||||
caveats = ["the fix changes Request.php only; Controller::_jsonld is a declared site outside the diff", "response type is JSON-LD, not HTML", "licence NOASSERTION on GitHub (zlib-style per project, verify)"]
|
||||
benign = { base = "75f056dcda955d94c17ec5a4f8c54a9b7bfcee07", tip = "3035d6e5d3cceb7f06f7b7bb80968727879c52b0", subject = "incrementing version", how = "automatic proposal (after the fix), reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "openstamanager-group-name-html-injection"
|
||||
repo = "https://github.com/devcode-it/openstamanager"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
wordpress = false
|
||||
family = "output_encoding"
|
||||
subtype = "stored HTML injection"
|
||||
advisory = ["CVE-2026-44701", "GHSA-crx4-7mmq-j74j"]
|
||||
published = "2026-08-26"
|
||||
vulnerable = "57a2734e2121aa8fcd456eff7c59f03e29e69176"
|
||||
fixed = "0b88ad1963871abfa5e134004ca0c74c496d8d2f"
|
||||
ancestry = "parent; ancestor of tag v2.11-beta"
|
||||
fix_named_by_advisory = true
|
||||
sink = "`'<span class=\"text-primary\">'.$group->getTranslation('title').'</span>'` in modules/utenti/edit.php"
|
||||
source = "group name saved when creating a user group"
|
||||
sites = ["modules/utenti/edit.php::<global>"]
|
||||
site_files = []
|
||||
sink_evidence = "$group->getTranslation('title')"
|
||||
site_derivation = "file and code quoted by the advisory"
|
||||
privilege = "authenticated user able to create groups"
|
||||
license = "GPL-3.0"
|
||||
confidence = "high"
|
||||
caveats = ["second advisory commit (7c9eafe9, global GET/POST filtering) is not in the pinned pair"]
|
||||
benign = { base = "0b88ad1963871abfa5e134004ca0c74c496d8d2f", tip = "1832b85d0e570e0b873c7c23dcfcb6b5d1365a28", subject = "fix: ricerca riferimenti in importazione fattura per riga con articolo non esistente", how = "automatic proposal (after the fix), reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "wp-geo-my-wp-info-window-lfi"
|
||||
repo = "https://github.com/Fitoussi/geo-my-wp"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
wordpress = true
|
||||
family = "path"
|
||||
subtype = "local file inclusion"
|
||||
advisory = ["CVE-2026-85200", "GHSA-6hw9-7gx5-mw88"]
|
||||
published = "2026-09-12"
|
||||
vulnerable = "1dbb7b1672a45a8f88fdcd379ceb07b6cf2beceb"
|
||||
fixed = "5a768bf1c6e44ded83a65be8789f3587515665ac"
|
||||
ancestry = "parent"
|
||||
fix_named_by_advisory = true
|
||||
sink = "include of a file chosen through `gmw_posts_locator_ajax_info_window_loader`"
|
||||
source = "AJAX request parameters (unauthenticated)"
|
||||
sites = ["plugins/posts-locator/includes/gmw-posts-locator-functions.php::gmw_posts_locator_ajax_info_window_loader", "includes/gmw-functions.php::gmw_ajax_info_window_init"]
|
||||
site_files = ["includes/gmw-functions.php"]
|
||||
site_basis = { "plugins/posts-locator/includes/gmw-posts-locator-functions.php::gmw_posts_locator_ajax_info_window_loader" = "advisory-named", "includes/gmw-functions.php::gmw_ajax_info_window_init" = "resolved-from-fix-diff" }
|
||||
site_derivation = "function named by the advisory text; files from the advisory's own trac reference links (gmw-functions.php#L1945/L1987, gmw-posts-locator-functions.php#L964/L975); the function-to-file pairing is an assumption to confirm at freeze"
|
||||
privilege = "unauthenticated"
|
||||
license = "GitHub: none; GPL per project [verify before publishing]"
|
||||
confidence = "medium"
|
||||
caveats = ["WordPress plugin; GitHub shows no licence (plugin header GPL presumed, verify)", "release commit (small)"]
|
||||
benign = { base = "9c394482d1509ca9222365a91117a10b19619be8", tip = "85bcd85541c777b6fd8580cb1915504a2a2b84cf", subject = "Release 4.5.5.2", how = "automatic proposal (before the vulnerable pin), reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "kirby-realpath-sibling-traversal"
|
||||
repo = "https://github.com/getkirby/kirby"
|
||||
language = "php"
|
||||
also = []
|
||||
wordpress = false
|
||||
family = "path"
|
||||
subtype = "path traversal (prefix containment check)"
|
||||
advisory = ["CVE-2026-75592", "GHSA-6j4c-mgqr-qv76"]
|
||||
published = "2026-09-02"
|
||||
vulnerable = "e90257fd7504053e13e93a1e9a5a9a5137aed966"
|
||||
fixed = "2b6fab950bc89d505ea89576bf71c8de614cc3b2"
|
||||
ancestry = "parent; ancestor of tag 4.9.5 (not of 5.5.2)"
|
||||
fix_named_by_advisory = true
|
||||
sink = "`Dir::realpath()` / `F::realpath()` containment check accepts a sibling directory sharing the prefix (site vs site2)"
|
||||
source = "media handler request path for not-yet-generated thumbnails"
|
||||
sites = ["src/Filesystem/Dir.php::realpath", "src/Filesystem/F.php::realpath"]
|
||||
site_files = []
|
||||
site_derivation = "`Dir::realpath()` and `F::realpath()` named by the advisory; paths confirmed by the fix commit"
|
||||
privilege = "unauthenticated"
|
||||
license = "Kirby licence (proprietary, GitHub NOASSERTION) -- not OSI; results unpublishable without permission"
|
||||
confidence = "medium"
|
||||
caveats = ["only deployments whose index root has a sibling directory with the same name prefix", "Kirby is under a proprietary licence (GitHub NOASSERTION); must be cleared before publishing"]
|
||||
benign = { base = "7ee5b79dd8eea6aa3d5a7eb0c5411a6631af88f1", tip = "d333470b9d9df7a48ee8d0bc2249db36aa9dd68c", subject = "style: Fix coding style", how = "chosen by hand after review: proposals 65dfca05/0901d0cb concern the content salt, 7bbefc1e updates vendored dependencies; rejected" }
|
||||
|
||||
[[case]]
|
||||
id = "mineadmin-appstore-identifier-traversal"
|
||||
repo = "https://github.com/mineadmin/MineAdmin"
|
||||
language = "php"
|
||||
also = []
|
||||
wordpress = false
|
||||
family = "path"
|
||||
subtype = "path traversal"
|
||||
advisory = ["CVE-2026-55224", "GHSA-59xm-4m8c-g3xj"]
|
||||
published = "2026-08-18"
|
||||
vulnerable = "ac0dcad616c5d304c249d1db4a6675f65cf736dc"
|
||||
fixed = "ca41902a2a5422676227e5088f4cc1dec06044f1"
|
||||
ancestry = "parent; the fixed commit is tag v3.2.0-alpha.2"
|
||||
fix_named_by_advisory = true
|
||||
sink = "`BASE_PATH . '/plugin/' . $params['identifier']` in download()/install()/unInstall(), then `Plugin::install($params['identifier'])`"
|
||||
source = "`identifier` request parameter of the app-store plugin endpoints"
|
||||
sites = ["plugin/mine-admin/app-store/src/Service/Service.php::download", "plugin/mine-admin/app-store/src/Service/Service.php::install", "plugin/mine-admin/app-store/src/Service/Service.php::unInstall"]
|
||||
site_files = []
|
||||
site_derivation = "file path and methods quoted by the advisory"
|
||||
privilege = "authenticated admin with app-store access (verify)"
|
||||
license = "Apache-2.0"
|
||||
confidence = "high"
|
||||
caveats = ["Hyperf framework (not Laravel/Symfony)"]
|
||||
benign = { base = "c57753e3f547683dc3e14836c3563a51ba4edaee", tip = "ff33969ef5ee378654dacab89e6f4b1928f4edd1", subject = "Fix/service logstash namespace (#663)", how = "chosen by hand after review: proposal 15deedde is a framework/dependency upgrade; rejected" }
|
||||
|
||||
[[case]]
|
||||
id = "pagible-admin-proxy-dns-rebinding-ssrf"
|
||||
repo = "https://github.com/aimeos/pagible"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
wordpress = false
|
||||
family = "untrusted_destination"
|
||||
subtype = "SSRF (DNS rebinding guard bypass)"
|
||||
advisory = ["CVE-2026-49262", "GHSA-mmj8-wcvw-6789"]
|
||||
published = "2026-06-26"
|
||||
vulnerable = "03f03b2f81b20b1c1625ff227235c43791ddbb33"
|
||||
fixed = "09a8205d513ec89ed22cdd7bdae0f4c181cee082"
|
||||
ancestry = "parent; ancestor of tag 0.10.4"
|
||||
fix_named_by_advisory = false
|
||||
inferred_fix_confirmed = ["admin/src/Controllers/AdminController.php::proxy", "core/src/Utils.php::isValidUrl"]
|
||||
sink = "`AdminController::proxy` → Guzzle request after `\\Aimeos\\Cms\\Utils::isValidUrl($url)`"
|
||||
source = "`url` query parameter of `/cmsproxy`"
|
||||
sites = ["admin/src/Controllers/AdminController.php::proxy", "core/src/Utils.php::isValidUrl"]
|
||||
site_files = []
|
||||
site_derivation = "class::method names quoted by the advisory; paths from the fix commit"
|
||||
privilege = "authenticated (proxy token)"
|
||||
license = "MIT"
|
||||
confidence = "medium"
|
||||
caveats = ["guard bypass (DNS rebinding)"]
|
||||
benign = { base = "981b8d63e52e7ba8875a9a52c1865eb3bc662154", tip = "d7577efd68e9c5e36436c8572bfed0df2be2c634", subject = "Publish related items (#210)", how = "automatic proposal (after the fix), reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "wp-graphql-updatepost-publish-bypass"
|
||||
repo = "https://github.com/wp-graphql/wp-graphql"
|
||||
language = "php"
|
||||
also = ["javascript"]
|
||||
wordpress = true
|
||||
family = "access_control"
|
||||
subtype = "missing object-level capability check"
|
||||
advisory = ["CVE-2026-88974", "GHSA-5mmc-8pc9-wggg"]
|
||||
published = "2026-09-23"
|
||||
vulnerable = "d53b0e566f20a8df0664559cb6432d52a7cd285c"
|
||||
fixed = "55441663eaa33c3f2e05de038c8286c845916461"
|
||||
ancestry = "parent; ancestor of tag wp-graphql/v2.22.2"
|
||||
fix_named_by_advisory = true
|
||||
sink = "`updatePost` mutation checks only `$post_type_object->cap->edit_posts`"
|
||||
source = "GraphQL `updatePost` mutation input (status PUBLISH, published post id)"
|
||||
sites = ["plugins/wp-graphql/src/Mutation/PostObjectUpdate.php::mutate_and_get_payload"]
|
||||
site_files = ["plugins/wp-graphql/src/Mutation/PostObjectUpdate.php"]
|
||||
site_basis = { "plugins/wp-graphql/src/Mutation/PostObjectUpdate.php::mutate_and_get_payload" = "resolved-from-fix-diff" }
|
||||
site_derivation = "the advisory names src/Mutation/PostObjectUpdate.php; the repository is a monorepo (plugins/wp-graphql/ prefix from the fix commit); no function named"
|
||||
privilege = "Contributor"
|
||||
license = "GPL-3.0"
|
||||
confidence = "medium"
|
||||
caveats = ["fix commit also carries an unrelated Model.php change", "function-level site to resolve at freeze"]
|
||||
benign = { base = "b583bb6ec87016965085afc7ea586f114c711c68", tip = "cc12aed380414545a005048bfcebe4ba50e9c8f9", subject = "fix: point acf.wpgraphql.com links at the wpgraphql.com docs portal (#4251)", how = "automatic proposal (before the vulnerable pin), reviewed" }
|
||||
|
||||
[[case]]
|
||||
id = "sylius-mollie-thankyou-order-idor"
|
||||
repo = "https://github.com/Sylius/MolliePlugin"
|
||||
language = "php"
|
||||
also = []
|
||||
wordpress = false
|
||||
family = "access_control"
|
||||
subtype = "IDOR (missing ownership check)"
|
||||
advisory = ["CVE-2026-68501", "GHSA-x83g-979r-f5fh"]
|
||||
published = "2026-07-31"
|
||||
vulnerable = "7400f1489a4b699df97ced9499acde4faaa476d5"
|
||||
fixed = "01316b3ad3cf82e3c5ad160115d0a2cf89174e49"
|
||||
ancestry = "first parent of a merge; ancestor of tag v3.3.1"
|
||||
fix_named_by_advisory = true
|
||||
sink = "`findOneBy(['id' => $orderId])` without ownership/session check, returning the order token (302 Location) and QR code"
|
||||
source = "`orderId` of GET /{_locale}/thank-you and /{_locale}/get-code"
|
||||
sites = ["src/Controller/Shop/PageRedirectController.php::thankYouAction", "src/Controller/Shop/QrCodeAction.php::fetchQrCodeFromOrder"]
|
||||
site_files = []
|
||||
site_derivation = "Class::method names quoted by the advisory; paths from the fix commit"
|
||||
privilege = "unauthenticated"
|
||||
license = "MIT"
|
||||
confidence = "high"
|
||||
caveats = ["merge commit; one fix covers two advisories (webhook forgery GHSA-rc52 is the other)"]
|
||||
benign = { base = "19f22cd673fde2b6ac750326fecbfe5fc4c06b9e", tip = "91254cc9c9821acbb3c785918759feedd827bdb9", subject = "Add support for Swish gateway (#324)", how = "chosen by hand after review: proposal 45b01569 is a branch upmerge; rejected" }
|
||||
|
||||
[[case]]
|
||||
id = "wp-podlove-generation-guard-poi"
|
||||
repo = "https://github.com/podlove/podlove-publisher"
|
||||
language = "php"
|
||||
also = ["javascript", "typescript"]
|
||||
wordpress = true
|
||||
family = "deserialization"
|
||||
subtype = "PHP object injection → arbitrary file deletion (in-plugin POP chain)"
|
||||
advisory = ["CVE-2026-16099", "GHSA-q83h-6954-x259"]
|
||||
published = "2026-08-16"
|
||||
vulnerable = "4eedbd66796942ab052de2dbd6311a24cc75c4a7"
|
||||
fixed = "ea878175f2612d54888d9e2aa49ae4db93b724fd"
|
||||
ancestry = "linear chain of 3 commits (4eedbd66 → d49c4d7f → b59d7962 → ea878175); ancestor of tag 4.5.4"
|
||||
fix_named_by_advisory = false
|
||||
inferred_fix_confirmed = ["lib/modules/shownotes/rest_api.php::create_link_item"]
|
||||
sink = "unserialize() of model data (lib/model/base.php#L552) instantiating `Podlove\\ImageCache\\GenerationGuard`, whose `__destruct()` calls `wp_delete_file()`"
|
||||
source = "shownotes REST input handled by `create_link_item` (contributor+)"
|
||||
sites = ["lib/modules/shownotes/rest_api.php::create_link_item", "lib/image_cache/generation_guard.php::__destruct", "lib/model/base.php::to_array"]
|
||||
site_files = ["lib/model/base.php"]
|
||||
site_basis = { "lib/modules/shownotes/rest_api.php::create_link_item" = "advisory-named", "lib/image_cache/generation_guard.php::__destruct" = "advisory-named", "lib/model/base.php::to_array" = "resolved-from-fix-diff" }
|
||||
site_derivation = "create_link_item and GenerationGuard::__destruct named by the advisory; files from the advisory's own trac reference links"
|
||||
privilege = "Contributor"
|
||||
license = "MIT"
|
||||
confidence = "medium"
|
||||
caveats = ["fix inferred from file history (three commits)", "the advisory text describes the impact (file deletion) under CWE-502"]
|
||||
benign = { base = "af187ad4bba2739839a0aa1e6cf2cabd6f9b99f5", tip = "50da8db0f5a85afaf27cd6199940be0403b368ec", subject = "fix PHP warning", how = "automatic proposal (after the fix), reviewed" }
|
||||
@@ -0,0 +1,63 @@
|
||||
# Evaluation protocol v1 for the independent population (pre-registered)
|
||||
|
||||
Written and committed on 2026-09-24, BEFORE the first scan of any case in `population-v1.toml`. It fixes what is
|
||||
measured, how a result is scored and what is reported, so that no rule of the scoring can be chosen after seeing
|
||||
an outcome. Changing it after the first scan makes the run exploratory, and it must then be reported as such.
|
||||
|
||||
## What is under test
|
||||
|
||||
- The analyzer at the commit that introduces the evaluation runner, exported with
|
||||
`benchmarks/push/freeze_source.sh`; its hash is recorded in the result.
|
||||
- Per case, only the case's own family (`injection`, `untrusted_destination` or `config_secrets`), through the
|
||||
production discovery and scan (`benchmarks/push/finding_dump.py`), unbounded region count (5,000), a 2,400 s
|
||||
deadline per scan, one engine container at a time.
|
||||
- Four scans per case: the vulnerable pin, the fixed pin, the benign base and the benign tip. Each checkout is a
|
||||
`git archive` of the pin, with no project tooling run on it.
|
||||
|
||||
## Scoring, per case
|
||||
|
||||
A finding MATCHES the case when its file is one the fix changes (`git diff --name-only vulnerable fixed`), its
|
||||
family is the case family, and either its function is one the record names for the sink or fix site, or its line
|
||||
lies within 10 lines of a hunk the fix changes on that side.
|
||||
|
||||
- **Detected** (recall): at least one matching finding at the vulnerable pin.
|
||||
- **Fixed-side false alert**: a matching finding at the fixed pin.
|
||||
- **Benign false alert**: any finding of the case family at the benign tip whose (file, function) has no finding
|
||||
at the benign base.
|
||||
- **Unanswered**: the scan completed no question covering the sink file's function. It counts as NOT detected,
|
||||
and the reason is reported. A scan with no questions at all, or unreadable input, is an instrument failure: it
|
||||
is rerun once, and if it fails again it is reported as a failure, never as a clean result.
|
||||
|
||||
## Precision
|
||||
|
||||
All findings of the case family at the VULNERABLE pin, across the population, are pooled and sorted by
|
||||
(case, site). Every third one, starting from the first, is adjudicated against the source, as are all of them if
|
||||
the pool holds 15 or fewer. The same standard as the PMPro adjudications applies:
|
||||
|
||||
- **True**: attacker-controlled data reaches the sink without a sufficient guard.
|
||||
- **True, privileged**: the same, reachable only with a privileged role.
|
||||
- **False, guarded**: a guard makes the flow safe.
|
||||
- **False, not attacker**: the value does not come from an attacker.
|
||||
|
||||
The matched case findings are part of the pool and are adjudicated like any other.
|
||||
|
||||
## What is reported
|
||||
|
||||
Per case: detected / unanswered, fixed-side and benign false alerts, completion (questions completed / total)
|
||||
for each of the four scans, and seconds. Overall:
|
||||
|
||||
- recall with a Wilson 95% interval;
|
||||
- sampled precision with a Wilson 95% interval;
|
||||
- total fixed-side and benign false alerts.
|
||||
|
||||
These are reported against the M4 profile: at least 95% precision, at least 90% recall, at least 95% completion,
|
||||
and zero fixed or benign false alerts. A gate that is not met is reported as not met.
|
||||
|
||||
## What may not happen
|
||||
|
||||
- No rule, fact, query or scoring change informed by these results before the result is committed.
|
||||
- Any change made after it is a retune. A retune can only be qualified on a NEW untouched population, as
|
||||
release-milestones M4 requires.
|
||||
- The OpenCVE case is not published until its licence is checked. Checked 2026-10-02: BUSL-1.1 (not OSI; non-production
|
||||
use permitted; converts to Apache-2.0 on 2030-08-14). The maintainer decided to publish the case record, which holds
|
||||
metadata only (repository, pins, detection) and no OpenCVE code.
|
||||
@@ -0,0 +1,90 @@
|
||||
# Evaluation protocol v2 for the independent population v2 (pre-registered)
|
||||
|
||||
Written and committed on 2026-09-26, BEFORE the first scan of any case in `population-v2.toml`. It fixes what is
|
||||
measured, how a result is scored and what is reported, so that no rule of the scoring can be chosen after seeing
|
||||
an outcome. Changing it after the first scan makes the run exploratory, and it must then be reported as such.
|
||||
|
||||
Differences from protocol v1, each decided by the maintainer before any v2 case was scanned, and each a lesson
|
||||
of the v1 run rather than of any v2 case:
|
||||
|
||||
- **Declared sites.** v1 credited a finding only in a file the fix changed. alerta and YesWiki (v1) execute their
|
||||
SQL in a helper the fix does not touch, so a correct report where the query runs could never count. Each v2
|
||||
case therefore declares, from its advisory and before any scan, the `sites` (`path::function`) where the
|
||||
vulnerable operation happens; a finding there is credited too.
|
||||
- **Two runs.** The engine's path selection is not deterministic: on v1 retunes one verdict flipped in 2 of 4
|
||||
identical runs, and a PMPro finding came and went between runs. The vulnerable and the fixed pin are scanned
|
||||
twice each, and the difference is reported.
|
||||
|
||||
## What is under test
|
||||
|
||||
- The analyzer at the commit that introduces the v2 evaluation runner, exported with
|
||||
`benchmarks/push/freeze_source.sh`; its hash is recorded in the result.
|
||||
- Per case, only the case's own family (`injection`, `untrusted_destination` or `config_secrets`), through the
|
||||
production discovery and scan (`benchmarks/push/finding_dump.py`), unbounded region count (5,000), a 2,400 s
|
||||
deadline per scan, one engine container at a time.
|
||||
- Six scans per case: the vulnerable pin twice (runs a, b), the fixed pin twice (a, b), the benign base and the
|
||||
benign tip once each. Each checkout is a `git archive` of the pin, with no project tooling run on it.
|
||||
|
||||
## Scoring, per case
|
||||
|
||||
A finding MATCHES the case when its family is the case family and either
|
||||
1. its file is one the fix changes (`git diff --name-only vulnerable fixed`) and either its function is one the
|
||||
record names in `sites`, or its line lies within 10 lines of a hunk the fix changes on that side (v1's rule);
|
||||
or
|
||||
2. its `path::function` equals one of the record's declared `sites` (`<global>` for file scope).
|
||||
|
||||
- **Detected** (recall): a matching finding in BOTH vulnerable runs.
|
||||
- **Unstable detection**: a matching finding in exactly one vulnerable run. Reported; counts as NOT detected.
|
||||
- **Fixed-side false alert**: a matching finding in EITHER fixed run.
|
||||
- **Benign false alert**: any finding of the case family at the benign tip whose (file, function) has no finding
|
||||
at the benign base.
|
||||
- **Unanswered**: neither vulnerable run completed a question covering a file the fix changes or a declared site
|
||||
file. It counts as NOT detected, and the reason is reported. A scan with no questions at all, or unreadable
|
||||
input, is an instrument failure: it is rerun once, and if it fails again it is reported as a failure, never as
|
||||
a clean result.
|
||||
|
||||
## Stability
|
||||
|
||||
Per pin with two runs, the findings present in exactly one run divided by the findings present in either is
|
||||
reported, per case and pooled. It is not a gate; it says how far a single run can be trusted.
|
||||
|
||||
## Precision
|
||||
|
||||
All findings of the case family present in BOTH vulnerable runs, across the population, are pooled and sorted by
|
||||
(case, site). Every third one, starting from the first, is adjudicated against the source, as are all of them if
|
||||
the pool holds 15 or fewer. Findings present in only one run are counted and reported, not adjudicated. The
|
||||
standard is v1's:
|
||||
|
||||
- **True**: attacker-controlled data reaches the sink without a sufficient guard.
|
||||
- **True, privileged**: the same, reachable only with a privileged role.
|
||||
- **False, guarded**: a guard makes the flow safe.
|
||||
- **False, not attacker**: the value does not come from an attacker.
|
||||
|
||||
The matched case findings are part of the pool and are adjudicated like any other.
|
||||
|
||||
## What is reported
|
||||
|
||||
Per case: detected / unstable / unanswered, fixed-side and benign false alerts, completion (questions completed /
|
||||
total) for each of the six scans, stability, and seconds. Overall:
|
||||
|
||||
- recall with a Wilson 95% interval;
|
||||
- sampled precision with a Wilson 95% interval;
|
||||
- total fixed-side and benign false alerts;
|
||||
- completion over all scans;
|
||||
- pooled stability.
|
||||
|
||||
These are reported against the M4 profile: at least 95% precision, at least 90% recall, at least 95% completion,
|
||||
and zero fixed or benign false alerts. A gate that is not met is reported as not met.
|
||||
|
||||
## What may not happen
|
||||
|
||||
- No rule, fact, query or scoring change informed by these results before the result is committed.
|
||||
- Any change made after it is a retune. A retune can only be qualified on a NEW untouched population.
|
||||
- No result of a case whose licence is marked "verify" is published until the licence is checked.
|
||||
- 2026-10-02: the five v2 cases that carried the "verify" marker were checked read-only at their pinned
|
||||
vulnerable commits (LICENSE / license.txt / plugin header `License:` line via the GitHub API) and the
|
||||
`license` field now records the SPDX id with the evidence: pgadmin-maintenance-sqli is PostgreSQL;
|
||||
wp-gopay-log-filter-sqli is GPL-2.0-or-later; wp-user-registration-login-redirect, wp-directorist-avatar-ssrf
|
||||
and wp-groundhogg-confirm-redirect are GPL-3.0-or-later. All five are OSI-approved. The OpenCVE case in v1
|
||||
is BSL-1.1, which is not OSI-approved; non-production use is permitted by its Additional Use Grant, and its
|
||||
change date is 2030-08-14 (to Apache-2.0).
|
||||
@@ -0,0 +1,173 @@
|
||||
# Evaluation protocol v3 for the independent PHP population v3 (pre-registered)
|
||||
|
||||
Written and committed on 2026-09-30, AFTER `population-v3-php.toml` was frozen and BEFORE any scanner, quick rule,
|
||||
engine, plane task or model has been run on any v3 repository. It fixes what is measured, how a result is scored
|
||||
and what is reported, so that no rule of the scoring can be chosen after seeing an outcome. Changing it after the
|
||||
first scan makes the run exploratory, and it must then be reported as such.
|
||||
|
||||
Differences from protocol v2, each decided by the maintainer on 2026-09-30 before any v3 case was scanned:
|
||||
|
||||
- **Six families.** v2 scored `injection`, `untrusted_destination` and `config_secrets`. v3 pre-registers matching
|
||||
for `injection`, `output_encoding`, `path`, `untrusted_destination`, `access_control` and `deserialization`
|
||||
("Extend protocol v3 to all six"), with one rule for all of them (below).
|
||||
- **Resolved sites.** Where an advisory names only a file, freeze read that file's diff at the fix commit and
|
||||
declared the changed function(s) as the site ("Resolve the function at freeze"). `site_basis` in the population
|
||||
says which sites the advisory named and which were resolved from the fix diff; both are declared sites and are
|
||||
credited alike. A file the fix does not change was not resolved (`site_files_unresolved`) and is not a site.
|
||||
- **Three instruments**, each scored on its own (below).
|
||||
|
||||
## The population
|
||||
|
||||
`population-v3-php.toml`, status `frozen`, record `freeze-v3-php.json`: 15 cases (injection 5, output_encoding 3,
|
||||
path 3, untrusted_destination 1, access_control 2, deserialization 1). The second untrusted_destination slot is
|
||||
empty by the selection rule and stays empty; recall is over the 15 cases.
|
||||
|
||||
## What is under test
|
||||
|
||||
All three instruments at ONE analyzer commit: the commit that introduces the v3 evaluation runner, exported with
|
||||
`benchmarks/push/freeze_source.sh`; its hash, the runner image digest and the model ids are recorded in the result.
|
||||
No instrument's configuration may be changed between instruments or between cases.
|
||||
|
||||
1. **Quick PHP rules** -- `src/openultrasast/ruleset/php/rules.toml` through the production quick scan
|
||||
(`ousast scan <checkout> --mode quick`, default configuration: `min_emit_priority` and `min_emit_precision`
|
||||
0.0). Scored: the ENABLED rules. The shadow rules (`shadow_findings.json`) are scored the same way and reported
|
||||
separately as exploratory; they do not enter any gate. A quick finding's family is fixed by this table, and a
|
||||
rule not in it has no family and never matches:
|
||||
|
||||
| family | rules |
|
||||
|---|---|
|
||||
| injection | `wpdb-sql-composition`, `sql-unquoted-concat`, `sql-unquoted-interpolation`, `sql-sprintf-unescaped`, `command-injection`; shadow `sql-call-composition`, `code-eval` |
|
||||
| path | `file-inclusion-input`, `path-input` |
|
||||
| untrusted_destination | `ssrf-input`, `open-redirect` |
|
||||
| deserialization | shadow `unserialize` |
|
||||
| output_encoding | shadow `xss-echo-input` |
|
||||
| access_control | none |
|
||||
|
||||
A finding's site is `path:line:function` from the finding's `path`, `line` and `function_name`.
|
||||
2. **The Joern engine** -- the production discovery and scan (`benchmarks/push/finding_dump.py`) inside the
|
||||
shipped image, per case only the case's own family (`--families <family>`), 5,000 regions, a 2,400 s deadline
|
||||
per scan, one engine container at a time, `--network none --memory 3g`, as `evaluate.py run` does.
|
||||
3. **The plane pipeline** -- the ai-service plane as implemented at that commit: candidates -> triage -> verify
|
||||
pass a and pass b -> tie-break pass c on the candidates a and b dispute -> agreement. A candidate is a finding
|
||||
when a and b both flag it, or, for a disputed candidate only, when at least 2 of the 3 passes flag it; pass c
|
||||
alone never decides. Model: `deepseek-flash` (DeepSeek) for triage and all three verify passes, as in
|
||||
`plane/models/deepseek-flash.yaml`. Site: the candidate's anchor `path:line:function` (line 0 when the model
|
||||
names another file); family: the case's family.
|
||||
|
||||
Per case and instrument, six pins, protocol v2's: the vulnerable pin twice (runs a, b), the fixed pin twice (a,
|
||||
b), the benign base and the benign tip once each. Each checkout is a `git archive` of the pin. For the plane,
|
||||
"run a" and "run b" are two complete, independent executions of the whole pipeline (each with its own a/b/c
|
||||
passes). The quick rules are deterministic: their two runs must be identical, and a difference is an instrument
|
||||
failure.
|
||||
|
||||
## Instrument checks (before any number is believed)
|
||||
|
||||
- Every scan records the files and bytes it read. A scan that read no file of the checkout, asked no question
|
||||
(engine), or completed no task (plane) is an instrument failure: it is rerun once, and if it fails again it is
|
||||
reported as a failure, never as a clean result.
|
||||
- A scan finishing implausibly fast for its instrument (an engine scan under the JVM start time) is treated as a
|
||||
failed launch and checked before it is scored.
|
||||
- The plane is smoke-tested once, before the run, on a development case (never a v3 case), to show that model
|
||||
calls succeed and are priced; the smoke test's cost counts against the budget.
|
||||
|
||||
## Scoring, per case (all six families, all three instruments)
|
||||
|
||||
A finding MATCHES the case when its family is the case family (exact string) and either
|
||||
1. its file is one the fix changes (`git diff --name-only vulnerable fixed`) and either its function is one the
|
||||
record names in `sites`, or its line lies within 10 lines of a hunk the fix changes on that side; or
|
||||
2. its `path::function` equals one of the record's declared `sites` (`<global>` for file scope), whatever the
|
||||
site's basis.
|
||||
|
||||
This is `evaluate.matches_v2` unchanged; `evaluate.py --population population-v3-php.toml` selects the six-pin
|
||||
protocol and `score_v2`. There is no cross-family credit: a finding of another family never matches, even at a
|
||||
declared site.
|
||||
|
||||
What a match means, per family (for the adjudicator and the report; the matching rule above is the same):
|
||||
|
||||
- **injection** -- the interpreter call (SQL, OS command, code evaluation, template compilation) or the function
|
||||
that builds its argument.
|
||||
- **output_encoding** -- the rendering or sanitizing point (an echo/print of the value, a response body, a
|
||||
sanitizer configuration) at a declared site or in the fix's range. families.toml's limits hold: a clean result
|
||||
says nothing about template-mediated escaping.
|
||||
- **path** -- the filesystem operation, the inclusion, or the containment check the fix changes.
|
||||
- **untrusted_destination** -- the outbound request or redirect, or the destination guard the fix changes.
|
||||
- **access_control** -- an absence bug: the finding names the protected operation that lacks its guard, at the
|
||||
declared function where the guard belongs or in the fix's changed lines, where the fix adds it.
|
||||
- **deserialization** -- the deserializing call, the entry that feeds it, or the gadget method the advisory
|
||||
names.
|
||||
|
||||
Outcomes, as protocol v2:
|
||||
|
||||
- **Detected** (recall): a matching finding in BOTH vulnerable runs.
|
||||
- **Unstable detection**: a matching finding in exactly one vulnerable run. Reported; counts as NOT detected.
|
||||
- **Fixed-side false alert**: a matching finding in EITHER fixed run.
|
||||
- **Benign false alert**: any finding of the case family at the benign tip whose (file, function) has no finding
|
||||
at the benign base. Some benign pairs precede the vulnerable pin (`how` in the population says so); they are
|
||||
scored the same way.
|
||||
- **Unanswered**: neither vulnerable run completed a question, rule pass or task covering a file the fix changes
|
||||
or a declared site file. Counts as NOT detected, with the reason.
|
||||
- **Not covered**: the instrument has no rule or query for the case family (quick rules: access_control; any
|
||||
family the engine or plane cannot emit). Counts as NOT detected, reported separately from misses.
|
||||
- **Not run**: a pin the budget ceiling stopped. Counts as NOT detected and as not completed.
|
||||
|
||||
## Stability and precision
|
||||
|
||||
Stability: per pin with two runs, findings in exactly one run over findings in either, per case and pooled. Not a
|
||||
gate.
|
||||
|
||||
Precision, per instrument: all findings of the case family present in BOTH vulnerable runs, across the
|
||||
population, pooled and sorted by (case, site); every third one, starting from the first, is adjudicated against
|
||||
the source, or all of them if the pool holds 15 or fewer. Findings in only one run are counted, not adjudicated.
|
||||
The standard is v1's: True; True, privileged; False, guarded; False, not attacker. Matched case findings are part
|
||||
of the pool.
|
||||
|
||||
## Gates
|
||||
|
||||
These are reported against the M4 profile: at least 95% precision, at least 90% recall, at least 95% completion,
|
||||
and zero fixed or benign false alerts. A gate that is not met is reported as not met.
|
||||
|
||||
- The gates apply to EACH instrument separately, on its own findings: the quick rules (enabled rules only), the
|
||||
Joern engine, and the plane pipeline. The plane pipeline is the product's decision path; its result is the
|
||||
M4 answer for PHP. The quick rules and the engine are reported against the same gates as components.
|
||||
- Completion is, per instrument: quick rules -- files scanned over PHP files in the checkout; engine -- questions
|
||||
completed over questions asked (v2); plane -- tasks completed over tasks run, with budget-stopped pins counted
|
||||
as not completed.
|
||||
- Recall is detected cases over 15, with a Wilson 95% interval; precision is the sampled fraction True (either
|
||||
kind), with a Wilson 95% interval. With 15 cases, 90% recall needs 14 detections.
|
||||
|
||||
## Budget
|
||||
|
||||
Model spend (the plane only; the quick rules and the engine call no model) is metered by the plane's
|
||||
`MeteredClient` (`OUSAST_BUDGET_USD`, `OUSAST_BUDGET_CALLS`) and recorded per task.
|
||||
|
||||
- Hard ceiling for the whole v3 evaluation: **$60** of recorded spend, smoke test and triage included.
|
||||
- Per case: **$6** across its six pins; per pass, the plane's own per-pass ceiling at the runner commit.
|
||||
- Reference: $0.02 per candidate for two verify passes with triage (plane-increment-2.json); the largest v2 case
|
||||
cost $8.91 in the earlier script pipeline.
|
||||
- When a ceiling stops a case, its remaining pins are "not run"; the run continues with the next case until the
|
||||
total ceiling. The ceilings are not raised after the first v3 task has started.
|
||||
|
||||
## What is reported
|
||||
|
||||
Per case and instrument: detected / unstable / unanswered / not covered / not run, fixed-side and benign false
|
||||
alerts, completion for each of the six scans, stability, seconds and (plane) dollars. Overall, per instrument:
|
||||
recall and precision with Wilson intervals, total fixed-side and benign false alerts, completion, pooled
|
||||
stability, spend. Shadow quick rules are reported separately as exploratory.
|
||||
|
||||
## What may not happen
|
||||
|
||||
- No rule, fact, query, prompt, model, budget or scoring change informed by these results before the result is
|
||||
committed.
|
||||
- Any analyzer change made after a v3 result -- to rules, queries, facts, prompts, models, the plane's tasks or
|
||||
the pipeline's agreement rule -- is a retune. v3 cannot qualify it; a retune can only be qualified on a NEW
|
||||
untouched population.
|
||||
- No v3 repository may be named by a test, fixture, rule or measurement outside `benchmarks/independent/`
|
||||
(`tests/test_independent_population.py`).
|
||||
- No result of a case whose licence is marked "verify" is published until the licence is checked. Kirby is not
|
||||
OSI-licensed: its results are unpublishable without the licensor's permission.
|
||||
|
||||
## Maintainer confirmation (2026-09-30, before any scan)
|
||||
|
||||
The maintainer confirmed the two choices made while drafting: the model-spend ceiling of $60 in total and $6 per
|
||||
case (a case stopped by its cap counts as not detected), and shadow quick rules reported as exploratory only,
|
||||
outside every gate. No v3 repository had been scanned when this was confirmed.
|
||||
@@ -0,0 +1,151 @@
|
||||
# Independent evaluation population: candidate real-world cases
|
||||
|
||||
Research date: 2026-09-24. Selected by the maintainer: 11 of these 15 (see population-v1.toml); not selected: The Events Calendar, Tutor LMS, pyLoad, unfurl. Method: web research only. I read GitHub advisories (GHSA/CVE), commit pages, compare views and diffs through the GitHub REST API (`gh api`) and the web. I did not clone any repository or run any scanner or project tooling.
|
||||
|
||||
Exclusion check: I compared every repo URL below against `used-repos.txt` (case-insensitive exact match on `https://github.com/<owner>/<repo>`). None of them is on the list. None is NodeGoat, VAmPI, PMPro, WP Statistics, MW WP Form, libpng, a deliberately vulnerable training app or a benchmark suite. I did not use Ghost.
|
||||
|
||||
"Diff seen" means I read the patch hunks of the named fix commit through the GitHub API. "Vuln" is the parent of the fix commit unless a tag is given. Tags come with the commit SHA they resolve to.
|
||||
|
||||
## Summary table
|
||||
|
||||
| # | id | repo | lang | family | advisory | vulnerable (commit / tag) | fixed (commit / tag) | sink file :: function | privilege | diff seen |
|
||||
|---|----|------|------|--------|----------|---------------------------|----------------------|-----------------------|-----------|-----------|
|
||||
| 1 | wp-ultimate-member-sqli | https://github.com/ultimatemember/ultimatemember | PHP+JS (WP plugin) | SQLi | CVE-2024-1071 / GHSA-mh7h-gpmx-fggj | tag 2.8.2 = `e1885a73a8ad542cfd2ed60aef38a6bce4ab8553`; commit-level parent `a49b3061ded1630e4f6860b4ebece5ccf0a493d2` | `476de92f4c9aaa56c3e2f5527b242ce0476f4655` (+ `c61008bdbe1ce8de4597b41c7aad8dfda88f527d`); tag 2.8.3 = `a6a7d13a433b24e468ee10e778e3da5668b6ef59` | includes/core/class-member-directory-meta.php :: `Member_Directory_Meta::ajax_get_members()` (`$this->sql_order` ORDER BY) | unauthenticated | yes |
|
||||
| 2 | wp-learnpress-sqli | https://github.com/LearnPress/learnpress | PHP+JS (WP plugin) | SQLi | CVE-2024-8522 / GHSA-3w3r-r6g6-w8x5 (+ CVE-2024-8529 / GHSA-98wp-w76v-f75p) | `73b3cbceef8669056e9da5e2f2c78944eb139af5` (4.2.7) | `aaf42fe0c6bf05dcf2ff125750b7bd836e76fa3c` (4.2.7.1) | inc/Databases/class-lp-db.php :: `LP_Database::execute()` (`implode(',', $filter->only_fields)` into SELECT); source inc/jwt/rest-api/version1/class-lp-rest-courses-v1-controller.php :: `get_courses()` | unauthenticated (REST `/wp-json/learnpress/v1/courses`) | yes |
|
||||
| 3 | wp-events-calendar-sqli | https://github.com/the-events-calendar/the-events-calendar | PHP+JS (WP plugin) | SQLi | CVE-2024-8275 / GHSA-v6x4-m48j-jm68 | tag 6.6.4 = `4e4baa3fee7ef49abdd4cb277d963d555ffab4c7` | tag 6.6.4.1 = `419965a87d99195ad3fca22738e19dd287051f1c` (fix hunk in `e128ec5e2a25fe4fdc4542a9023918bf915b4084`) | src/Events/Custom_Tables/V1/WP_Query/Custom_Tables_Query.php :: `redirect_posts_orderby()` / `parse_orderby()` | unauthenticated per Wordfence (reached through `tribe_has_next_event` `order`) | yes |
|
||||
| 4 | wp-tutor-lms-coupon-sqli | https://github.com/themeum/tutor | PHP+JS/TS (WP plugin) | SQLi | CVE-2025-13673 / GHSA-f7q9-gg7h-5rff | tag v3.9.6 = `71187a525dbdbdb045137af702307f05dda6a7a6` (= parent of fix) | `e4a25f69f0fb63a46d57eff6ba5b58378a4c088d` (3.9.7, no GitHub tag) | helpers/QueryHelper.php :: `prepare_where_clause()` / `make_clause()`; source models/CouponModel.php :: `get_coupon_details_for_checkout()`, `get_coupon_by_code()` | unauthenticated (checkout `coupon_code`) | yes (mapping partly inferred) |
|
||||
| 5 | phpmyfaq-captcha-ua-sqli | https://github.com/thorsten/phpMyFAQ | PHP+TS/JS | SQLi | CVE-2026-46364 / GHSA-289f-fq7w-6q2w | `11583abcf4787543d00c25d6c7926af6f509faf0`; tag 4.1.1 = `f791858b2aa4b811f5b19d10c20ba5c4a30786de` | `545bdffb11244a4741cd29ac909a849c9a6a2e53`; tag 4.1.2 = `66792f2575b07d65896b504d47deb5f4f903f9ff` | phpmyfaq/src/phpMyFAQ/Captcha/BuiltinCaptcha.php :: `garbageCollector()`, `saveCaptcha()` | unauthenticated (`GET /api/captcha`, User-Agent header) | yes |
|
||||
| 6 | yeswiki-bazar-form-sqli | https://github.com/YesWiki/yeswiki | PHP+JS | SQLi | CVE-2026-46670 / GHSA-jwvv-qr7q-cv8j | `6f222915df36fca628fc83325cb0feed8dc2aca0`; tag v4.6.3 = `5e8fc114d01d558dc4799b95219549ccf02ebd11` | `fe7244bffcbeffd42777496ebc2a6f6118cc2388`; tag v4.6.4 = `c1bae9fc260eb3e4346f3b0bc65ab609d1cf783a` | tools/bazar/services/FormManager.php :: `FormManager::create()`; source tools/bazar/controllers/FormController.php :: `displayAll()` | unauthenticated | yes |
|
||||
| 7 | signalk-remote-conn-ssrf | https://github.com/SignalK/signalk-server | TypeScript (Node/Express) | SSRF | CVE-2026-55591 / GHSA-q59x-jc9f-gfqf | `1c479cc0d2d26d14ed74416cff36fa7336c7bf9e`; tag v2.27.0 = `4592992aaf7c44548842be9390b651cfb816eaf4` | `2816b0dda1bdde893d85c969a9c5950e7825eee7`; tag v2.28.0 = `3645160b235364e1fd3afe1f2f6b2270b8194b7d` | src/serverroutes.ts :: `makeRemoteRequest()` (from `testSignalKConnection`, `requestAccess`, `checkAccessRequest` routes) | unauthenticated by default (security not configured); admin otherwise | yes |
|
||||
| 8 | saltcorn-sync-sqli | https://github.com/saltcorn/saltcorn | JavaScript/TS (Node/Express) | SQLi | CVE-2026-41478 / GHSA-jp74-mfrx-3qvh | `e7e09254a3c69dc6582eda9e5b5769935db108af`; tag v1.4.5 = `92d24ca5ce8480d3b0cf1067e33113f2a790030c` | `2b6c8a2eab88955ea6b12b1b4e5c2e613faefef5`; tag v1.4.6 = `d49e6a947de777053a203a27ab4d9e1a1754ccd0` (backport) | packages/server/routes/sync.js :: `getSyncRows()`, `getDelRows()` (+ packages/saltcorn-data/models/table.ts sync_info queries) | authenticated low-privilege user | yes |
|
||||
| 9 | mongo-express-bson-rce | https://github.com/mongo-express/mongo-express | JavaScript (Node/Express) | code injection (vm eval) | CVE-2019-10758 / GHSA-h47j-hc6x-h3qq | `ea02b364d43f179f191fc91fb9962efdb0843a8d`; tag v0.53.0 = `9eb49994f3842260f7c6db75592ea6ba9376fad6` | `d8c9bda46a204ecba1d35558452685cd0674e6f2`; tag v0.54.0 = `7d365141deadbd38fa961cd835ce68eab5731494` | lib/bson.js :: `toBSON()` (`vm.runInNewContext` over request text); callers lib/routes/collection.js, lib/routes/document.js | authenticated (HTTP basic auth, default admin:pass) | yes |
|
||||
| 10 | mockoon-admin-api-cors | https://github.com/mockoon/mockoon | TypeScript (Node/Express) | permissive CORS | CVE-2026-59148 / GHSA-rqx4-3f6q-3x2v | `b42bdfb7f82e83f0e81bea8e6fe41adf5ec82585`; tag v9.6.1 = `1c874584c058b891f6affbb44d5a5223f3f8f812` | `c420b5a56918475b8663977b51e5f986e45b3299`; tag v9.7.0 = `df02ba3b0d9e8b88651230f3b38d8e6fa3679b9b` | packages/commons-server/src/libs/server/admin-api.ts :: admin-API middleware (`app.use(`${adminApiPrefix}*`)`, `Access-Control-Allow-Origin: *`) | unauthenticated | yes (large diff, one CORS hunk) |
|
||||
| 11 | pyload-add-package-sqli | https://github.com/pyload/pyload | Python (Flask) | SQLi | CVE-2025-55156 / GHSA-pwh4-6r3m-j2rf | `a123accec0618798db77f9f5bc66838b741d9f58` | `134edcdf6e2a10c393743c254da3d9d90b74258f` (0.5.0b3.dev91) | src/pyload/core/database/file_database.py :: `FileDatabaseMethods.update_link_info()`; source src/pyload/webui/app/blueprints/json_blueprint.py :: `add_package` | authenticated (ADD permission) | yes |
|
||||
| 12 | alerta-query-sqli | https://github.com/alerta/alerta | Python (Flask) | SQLi | CVE-2026-34400 / GHSA-8prr-286p-4w7j | `71c098d8bb8091e76edaefeef4662eb77c753ce7`; tag v9.0.4 = `b7111d23ae9aefb5064ef107976f62e3836217b6` | `fdd52cd1abad8d02d1dfb8ecdcdbb43b6af3b883`; tag v9.1.0 = `f4312727783dad64198b62ed778bbca98a5144a3` | alerta/database/backends/postgres/queryparser.py :: `SearchTerm.__repr__` (f-string WHERE); used by alerta/database/backends/postgres/utils.py :: `QueryBuilder.from_params()` | API user (API key when `AUTH_REQUIRED`; see notes) | yes |
|
||||
| 13 | wger-next-open-redirect | https://github.com/wger-project/wger | Python (Django) | open redirect | GHSA-vqv8-j3mj-wjxj (no CVE) | `d30e85b23fb1307ac6579e9e3ae393c50470cea7`; tag 2.5 = `c1fca6bdf77d8ed80048d1b73a8da80cf36ee751` | `0ba371bbb50cd5668820f240995405e1b2483384`; tag 2.6 = `e1d70bcc38cd56ae4a254dca1713c404d069f319` | wger/core/views/user.py :: `trainer_login()`; wger/core/views/languages.py :: `use_browser_language()` | `use_browser_language`: unauthenticated; `trainer_login`: authenticated trainer | yes |
|
||||
| 14 | unfurl-flask-debug | https://github.com/obsidianforensics/unfurl | Python (Flask) | permissive config (debug on) | GHSA-vg9h-jx4v-cwx2 (no CVE) | `6b8e97853090c909936a18d41228168b7a35fe94` | `4c0a07ab1e9af3a1ddf0e7f47153ec9ba77946dd` | unfurl/app.py :: `web_app()` → `UnfurlApp.__init__` `app.run(debug=...)` | unauthenticated (Werkzeug debugger) | yes |
|
||||
| 15 | opencve-notification-ssrf | https://github.com/opencve/opencve | Python (Django) | SSRF | CVE-2026-62282 / GHSA-ch3g-4xvr-674q | `928bbadbb1672051bcfc2a6289a9e6a8f029c106` | `0af7f894debc95c67fa53e7ba5a6cdecb10c34f0`; tag v3.0.0 = `928943aa87ceac789061114159d74c2067cc6ae6` | web/projects/notifications.py :: `_http_post_json()` (webhook and Slack test); forms in web/projects/forms.py | authenticated (can configure project notifications) | yes |
|
||||
|
||||
Mix: 6 PHP (all six repos ship JS, four are WordPress plugins), 4 JS/TS, 5 Python. By family: 10 injection (9 SQLi: #1–6, #8, #11, #12; 1 code injection: #9), 3 untrusted destination (SSRF #7 and #15, open redirect #13), 2 permissive configuration (CORS #10, debug #14). By privilege: 10 unauthenticated or unauthenticated by default (#1–7, #10, #14, plus #13's `use_browser_language` sink) and 5 authenticated (#8, #9, #11, #15, and #12 depending on `AUTH_REQUIRED`).
|
||||
|
||||
## Per-candidate notes
|
||||
|
||||
### 1. wp-ultimate-member-sqli (PHP+JS, WordPress)
|
||||
- Repo: https://github.com/ultimatemember/ultimatemember (GPL, PHP 2.7 MB + JS 0.6 MB).
|
||||
- Advisory: CVE-2024-1071, GHSA-mh7h-gpmx-fggj (unreviewed GHSA mirror of the Wordfence record). Affects 2.1.3–2.8.2, CVSS 9.8.
|
||||
- Flaw: `ajax_get_members()` builds `$this->sql_order = " ORDER BY CAST(umm_sort.um_value AS {$custom_sort_type}) {$custom_sort_order} "` and similar ORDER BY strings. Values from the member-directory `sorting` request pass only through `esc_sql()`. That does not protect a context without quotes. Only exploitable when the "custom usermeta table" option is on.
|
||||
- Fix: `476de92f` checks `$order` / `$custom_sort_order` against `ASC`/`DESC` and `$custom_sort_type` against `$this->sort_data_types`. `c61008bd` ("fixed security issue with mysql query escape") is a larger WPCS refactor of the same file. Use `476de92f` as the minimal fix and the 2.8.2 → 2.8.3 tags for a release-level pair.
|
||||
- Confidence: high. I read both diffs. The 2.8.2...2.8.3 compare has 94 commits, so a tag pair gives a noisy diff. The commit pair is cleaner.
|
||||
|
||||
### 2. wp-learnpress-sqli (PHP+JS, WordPress)
|
||||
- Repo: https://github.com/LearnPress/learnpress (PHP 5.9 MB, JS 1.4 MB).
|
||||
- Advisories: CVE-2024-8522 (`c_only_fields`, CVSS 10.0) and CVE-2024-8529 (`c_fields`). Both are fixed by the same commit.
|
||||
- Flaw: the REST controller `get_courses()` put `explode(',', $params['c_only_fields'])` straight into `$filter->only_fields`. `LP_Database::execute()` then runs `implode(',', array_unique($filter->only_fields))` into the SELECT column list (class-lp-db.php around line 607).
|
||||
- Fix `aaf42fe0`: `Courses::handle_params_for_query_courses()` now runs every field through `$wpdb->prepare('%i', $field)` (identifier quoting), and the controller's direct assignment is removed. It also touches `assets/src/js/loadAJAX.js`, so the commit is itself multi-language.
|
||||
- Confidence: high. I read the diff and checked the sink line at the vulnerable commit.
|
||||
|
||||
### 3. wp-events-calendar-sqli (PHP+JS, WordPress)
|
||||
- Repo: https://github.com/the-events-calendar/the-events-calendar (large plugin: PHP 10 MB, JS 0.7 MB; license field NOASSERTION, the plugin is GPL). **It has a git submodule `common` (tribe-common), so a checkout needs `--recursive`.**
|
||||
- Advisory: CVE-2024-8275, GHSA-v6x4-m48j-jm68. The `order` parameter of `tribe_has_next_event`, ≤ 6.6.4.
|
||||
- Flaw: `redirect_posts_orderby()` split ORDER BY fragments with a regex and kept any trailing `order` token as is. `parse_orderby()` inserted `$order` into `"ID $order, ..."`.
|
||||
- Fix (6.6.4.1, commit `e128ec5e`): adds `sanitize_order()` (only ASC/DESC) and applies it at both sites. It also bumps the submodule.
|
||||
- Privilege: Wordfence rates it unauthenticated. Reachability in practice depends on how the template tag's arguments reach WP_Query. The ground-truth sink is still unambiguous.
|
||||
- Confidence: high for the diff. Medium for how exploitable it is from a clean install.
|
||||
|
||||
### 4. wp-tutor-lms-coupon-sqli (PHP+JS/TS, WordPress)
|
||||
- Repo: https://github.com/themeum/tutor (GPL-3.0; PHP 3.8 MB, TS 3.3 MB, JS 0.4 MB).
|
||||
- Advisory: CVE-2025-13673, GHSA-f7q9-gg7h-5rff. ≤ 3.9.6, unauthenticated `coupon_code`. Partial fixes shipped in 3.9.4 and 3.9.6.
|
||||
- Flaw: `QueryHelper::prepare_where_clause()` built values as `"'" . $val . "'"`. The only escaping was an `esc_sql()` in `get_coupon_details_for_checkout()`. Other coupon paths (`get_coupon_by_code()`, `get_coupon_usage_count()`, `get_coupon_applications()`) pass `$coupon_code` through without escaping.
|
||||
- Fix `e4a25f69` (parent = tag v3.9.6): `make_clause()` now binds every value through `$wpdb->prepare('%s'|'%d', ...)`, and the manual quoting and `esc_sql` are removed.
|
||||
- Confidence: medium. I read the diff and it is the only SQL change right after v3.9.6. The trac changeset cited by the CVE (3469242) returned 403, so the exact exploited call chain from the checkout request to QueryHelper is my inference. There is no v3.9.7 tag on GitHub. Use the commit pair.
|
||||
|
||||
### 5. phpmyfaq-captcha-ua-sqli (PHP+TS/JS)
|
||||
- Repo: https://github.com/thorsten/phpMyFAQ (MPL-2.0; PHP 14.7 MB, TS 1.7 MB).
|
||||
- Advisory: CVE-2026-46364, GHSA-289f-fq7w-6q2w. ≤ 4.1.1. The advisory's commit reference `b9f25109` is the **vulnerable** master commit the reporter tested, not the fix.
|
||||
- Flaw: the constructor stores `$request->headers->get('user-agent')` and `getClientIp()`. `garbageCollector()` (DELETE) and `saveCaptcha()` (SELECT/INSERT) put them into `sprintf` SQL without escaping. Both run on every `GET /api/captcha`.
|
||||
- Fix `545bdffb` ("fix: added missing escaping"): adds `escapeQueryValue()` for userAgent, ip, language and code.
|
||||
- Confidence: high. I read the diff in the 4.1.1...4.1.2 compare and the commit.
|
||||
|
||||
### 6. yeswiki-bazar-form-sqli (PHP+JS)
|
||||
- Repo: https://github.com/YesWiki/yeswiki (AGPL-3.0; PHP 4 MB, JS 0.8 MB). Default branch `doryphore-dev`.
|
||||
- Advisory: CVE-2026-46670, GHSA-jwvv-qr7q-cv8j. < 4.6.4.
|
||||
- Flaw: `FormManager::create()` concatenates `$data['bn_id_nature']` unquoted into `INSERT ... VALUES (`. The data comes from `$_POST['imported-form']` JSON in `FormController::displayAll()`, and that path had no permission check.
|
||||
- Fix `fe7244bf`: `intval($data['bn_id_nature'])` plus a `Guard::isAllowed('saisie_formulaire')` check. The diff is 4 lines.
|
||||
- Confidence: high. I read the diff. Related YesWiki SQLi advisories (GHSA-qg78-vmvc-fhjw, GHSA-f58v-p6j9-24c2, GHSA-4pf7-cc4r-g63h) could add more cases from the same repo.
|
||||
|
||||
### 7. signalk-remote-conn-ssrf (TypeScript, Node/Express)
|
||||
- Repo: https://github.com/SignalK/signalk-server (Apache-2.0, about 24 MB).
|
||||
- Advisory: CVE-2026-55591, GHSA-q59x-jc9f-gfqf. ≤ 2.27.0.
|
||||
- Flaw: `makeRemoteRequest(host, port, useTLS, selfsignedcert, path)` passes `req.body.host`/`port` to `http(s).request` without validation. It is reached from three admin routes.
|
||||
- Fix `2816b0dd`: TypeBox schema validation (new src/remoteConnectionSchemas.ts), a new src/ssrfGuard.ts (BlockList for loopback, link-local and metadata, plus a DNS lookup guard) wired into `makeRemoteRequest`, and UUID validation of `requestId`.
|
||||
- Confidence: high. I read the diff. It is medium-sized, about 450 added lines, mostly new guard code and tests.
|
||||
|
||||
### 8. saltcorn-sync-sqli (JavaScript/TS, Node/Express)
|
||||
- Repo: https://github.com/saltcorn/saltcorn (MIT). It is a monorepo of packages and on the large side (about 340 MB of git data), but the relevant code is in `packages/server` and `packages/saltcorn-data`.
|
||||
- Advisory: CVE-2026-41478, GHSA-jp74-mfrx-3qvh. < 1.4.6 / < 1.5.6 / < 1.6.0-beta.5.
|
||||
- Flaw: `getSyncRows()` interpolates `${syncInfo.maxLoadedId}` (from `req.body.syncInfos[table].maxLoadedId`) and `${user.id}` into SQL template literals. `getDelRows()` does the same with timestamps.
|
||||
- Fix `2b6c8a2e`: integer and date validation plus `$1`-style parameterised queries in sync.js and table.ts.
|
||||
- Confidence: high. I read the diff. The fix commit is on the main line, and v1.4.6 is a separate backport. For a clean pair, use the commit and its parent `e7e09254`.
|
||||
|
||||
### 9. mongo-express-bson-rce (JavaScript, Node/Express)
|
||||
- Repo: https://github.com/mongo-express/mongo-express (MIT, about 16 MB).
|
||||
- Advisory: CVE-2019-10758, GHSA-h47j-hc6x-h3qq (CWE-94/78). < 0.54.0.
|
||||
- Flaw: `lib/bson.js` `toBSON()` evaluates user-supplied document/query text with `vm.runInNewContext` in a BSON sandbox. The sandbox can be escaped, which gives RCE through the `/checkValid` and document/collection routes.
|
||||
- Fix `d8c9bda4`: replaces the vm-based parser with `mongodb-query-parser`. It removes 101 lines from bson.js, deletes lib/json.js and changes collection.js.
|
||||
- Confidence: high. I read the diff. It is an older case (2019), useful as a known-answer eval sink. Note that the advisory's first reference, `7d365141`, is only the version-bump or release commit (= tag v0.54.0).
|
||||
|
||||
### 10. mockoon-admin-api-cors (TypeScript, Node/Express)
|
||||
- Repo: https://github.com/mockoon/mockoon (MIT; monorepo of app, cli, commons-server and serverless; about 42 MB).
|
||||
- Advisory: CVE-2026-59148, GHSA-rqx4-3f6q-3x2v (CWE-942/306/352/732). < 9.7.0.
|
||||
- Flaw: the admin API middleware sets `Access-Control-Allow-Origin: *` with all methods on every `/mockoon-admin/*` endpoint. The admin API is on by default and has no authentication.
|
||||
- Fix `c420b5a5`: adds a configurable `adminApiCorsOrigins` allowlist (no header by default, 403 on preflight), optional admin API auth, and an env-var prefix check.
|
||||
- Confidence: high that the CORS hunk is the fix. The commit is **large (34 files)**, but the permissive-CORS sink sits in one clearly identifiable hunk of admin-api.ts. It is a weaker fit for "small diff" than the others.
|
||||
|
||||
### 11. pyload-add-package-sqli (Python, Flask)
|
||||
- Repo: https://github.com/pyload/pyload (AGPL-3.0, GitHub reports NOASSERTION; about 50 MB).
|
||||
- Advisory: CVE-2025-55156, GHSA-pwh4-6r3m-j2rf. < 0.5.0b3.dev91.
|
||||
- Flaw: `update_link_info()` builds `"','".join(x[3] for x in data)` into `f"SELECT id FROM links WHERE url IN ('{statuses}')"`. The URLs come from `add_links` on `/json/add_package`. Chain: json_blueprint `add_package` → `Api.add_package` → `FileManager.add_links` → `InfoThread` → `update_file_info` → `update_link_info`.
|
||||
- Fix `134edcdf`: `?` placeholders. The diff is 5 lines. The source-to-sink path crosses a thread, which makes it a good cross-thread taint test.
|
||||
- Confidence: high. I read the diff. The same repo also has CVE-2023-0297 (GHSA-pf38-5p22-x6h6, unauthenticated js2py code injection, fix `7d73ba7919e594d783b3411d7ddb87885aea782d`, parent `6aa71cd99a6f510de7137d17ec99f930916704c0`). I kept it as an alternate only, because that fix just calls `js2py.disable_pyimport()`. The `eval_js` sink stays in the fixed tree, so a sink-based analyzer would still correctly flag the fixed version.
|
||||
|
||||
### 12. alerta-query-sqli (Python, Flask)
|
||||
- Repo: https://github.com/alerta/alerta (Apache-2.0, about 11 MB).
|
||||
- Advisory: CVE-2026-34400, GHSA-8prr-286p-4w7j. < 9.1.0. The advisory's `aeba85a3` reference (2018) is the commit that **introduced** the parser. The fix is `fdd52cd1`.
|
||||
- Flaw: the Postgres query-string parser (`?q=`) turned parsed search terms into SQL with f-strings in `SearchTerm.__repr__`. `QueryBuilder.from_params()` then used that as a WHERE clause with an empty `qvars`.
|
||||
- Fix: new `to_sql(params)` methods produce `%(name)s` placeholders, and `from_params()` passes the parsed vars.
|
||||
- Privilege: alerta requires an API key or login when `AUTH_REQUIRED=True`. I believe the default is `False`, which would make this unauthenticated on a default install, but I did not check settings.py. Record it as "API user, possibly unauthenticated".
|
||||
- Confidence: high for the diff. The fix is about 100 lines in one file plus tests.
|
||||
|
||||
### 13. wger-next-open-redirect (Python, Django)
|
||||
- Repo: https://github.com/wger-project/wger (AGPL-3.0, about 91 MB with a JS frontend).
|
||||
- Advisory: GHSA-vqv8-j3mj-wjxj (no CVE assigned). ≤ 2.5, fixed in 2.6.
|
||||
- Flaw: `trainer_login()` returns `HttpResponseRedirect(request.GET['next'])`. `use_browser_language()` redirects to `remove_language_code(request.GET.get('next','/'))`. Neither checks host or scheme.
|
||||
- Fix `0ba371bb`: `url_has_allowed_host_and_scheme()` at both sites. The diff is 20 lines of code plus tests.
|
||||
- Two sinks, two privilege levels: `use_browser_language` is unauthenticated, `trainer_login` needs a trainer. A later follow-up, `cfa27a7fc78c682ad4244020505dce4825fe1e26`, fixes the same pattern in `wger/core/middleware.py` (AuthProxyMiddleware). I found no separate advisory for it. It is optional as a third sink, but it is not in the 2.6 fix.
|
||||
- Confidence: high. I read the diff.
|
||||
|
||||
### 14. unfurl-flask-debug (Python, Flask)
|
||||
- Repo: https://github.com/obsidianforensics/unfurl (Apache-2.0, about 24 MB).
|
||||
- Advisory: GHSA-vg9h-jx4v-cwx2 (critical, CWE-489, no CVE). The GHSA lists "patched: none" as a package version, but it references the fix commit.
|
||||
- Flaw: `web_app()` reads `debug` with `config[...].get('debug')`, which returns a string. Any non-empty string, including `"False"`, is truthy, so `app.run(debug=...)` always enables the Werkzeug debugger.
|
||||
- Fix `4c0a07ab`: `getboolean('debug')` and `debug = false` in unfurl.ini. The diff is 2 lines.
|
||||
- Caveat: the function default `debug='True'` is unchanged in the fixed version, so a run without an `[UNFURL_APP]` section still starts in debug mode. The fixed revision may therefore still, correctly, be flagged for debug-on by a strict analyzer.
|
||||
- Confidence: high. I read the diff.
|
||||
|
||||
### 15. opencve-notification-ssrf (Python, Django)
|
||||
- Repo: https://github.com/opencve/opencve (GitHub license: NOASSERTION. I believe OpenCVE v2+ is under a non-OSI license (BSL), but I did not verify this. Check before use).
|
||||
- Advisory: CVE-2026-62282, GHSA-ch3g-4xvr-674q. The advisory's version ranges are confusing ("≤ 1.5.0, patched 3.0.0"). Use the commit pair.
|
||||
- Flaw: webhook and Slack notification "test" requests POST to a user-supplied URL (`_http_post_json` with urllib) and do not restrict the destination.
|
||||
- Fix `0af7f894`: new web/opencve/utils/ssrf.py (`validate_public_http_url`, `safe_request` with DNS pinning), `clean_url`/`clean_webhook_url` validators in forms.py, and notifications.py switched to `safe_request`.
|
||||
- Confidence: high for the diff. The license is the concern. It is the weakest Python pick; drop it if an OSS license is mandatory.
|
||||
|
||||
## Considered and rejected (for traceability)
|
||||
- vantage6 CVE-2024-23823 (permissive CORS, fix `70bb4e1d889230a841eb364d6c03accd7dd01a41`): the fix only adds a `cors_allowed_origins` option whose default is still `"*"`, so the fixed version is still permissive by default.
|
||||
- FlaskBB CVE-2026-46556 (SSRF): the advisory has no patched version. The only related commit (`e87e585f`) just sets `allow_redirects=False`.
|
||||
- SillyTavern CVE-2026-46372 (SSRF): the fix is a global request filter that is off by default.
|
||||
- NodeBB CVE-2025-50979 (SQLi): no patched version or fix commit is listed, and I could not confirm the claim.
|
||||
- Etherpad CVE-2026-55087 (open redirect): the referenced commits introduced the code. The fix is inside a large "assorted hardening" PR (#7784).
|
||||
- EverShop CVE-2023-46943 (hardcoded HMAC secret): the fix is a large dev→main merge commit.
|
||||
- claude-code-router CVE-2025-57755 (CORS): no tags, and I found no identifiable fix commit.
|
||||
- calibre-web CVE-2022-0339 (SSRF): the fix commit is mixed with unrelated Kobo-sync changes. calibre-web SQLi CVE-2022-30765 (GHSA-8ppf-x4gr-2x7g) could be a Python alternate, but I did not check its diff.
|
||||
- Tutor LMS PRs #3011/#3014/#3018 (SQLi hardening, Sept 2026): no CVE linked yet.
|
||||
@@ -0,0 +1,658 @@
|
||||
# Independent evaluation population v2: candidate real-world cases
|
||||
|
||||
Research date: 2026-09-26. Method: web research only. I read GitHub advisories (GHSA/CVE), commit pages, compare views and diffs through the GitHub REST API (`gh api`, read-only). I did not clone any candidate repository or run any scanner or project tooling on one. The work ran in four parallel lanes (Python, JS/TS, PHP apps, WordPress plugins) under one shared rule sheet, reproduced below. Every candidate was then re-verified from the merged list.
|
||||
|
||||
## Selection method (anti-bias)
|
||||
- **Enumeration.** Reviewed GitHub advisories per ecosystem (pip, npm, composer): `gh api "/advisories?ecosystem=<eco>&type=reviewed&cwes=<C>&sort=published&direction=desc"` for CWE 89, 78, 77, 94, 95, 1336, 943, 90, 918, 601, 489, 215, 942, 295, 798, 321, 259, 1188, 16, 346, 1004. The results were merged, deduplicated, limited to publications on or after 2023-01-01 and sorted newest first.
|
||||
- WordPress plugins: the newest 300 *unreviewed* advisories per CWE (89, 78, 94, 918, 601, 798, 942), kept only when the description mentions a WordPress plugin, then worked newest first. Each plugin's slug was searched on GitHub for a real development repository; SVN mirrors do not count.
|
||||
- The CWE filter is the only prefilter. It removes out-of-family advisories before anyone sees them, so those are not counted.
|
||||
- **Order.** Every lane took each advisory that met the criteria, strictly newest first, until its quota was filled:
|
||||
- Python 7, JS/TS 7, PHP 4, WordPress 5 (rows marked `quota`).
|
||||
- After the quota, each lane continued newest first for `untrusted_destination` and `config_secrets` only, until it had one extra of each or had examined 40 more advisories (rows marked `extra`).
|
||||
- The quota phase came out heavy on SSRF and redirects (about half the reviewed advisories of 2026 are SSRF in AI and agent servers). So the Python, JS and PHP lanes ran a third pass, `continuation`: newest first, injection only (Python) or CWE-89/943 only (JS, PHP), until 2 more were taken. Python then ran a CWE-89 pass until 1 SQLi was taken.
|
||||
- At no point was a case chosen or skipped for how easy it would be to detect.
|
||||
- **Shared rules:**
|
||||
- In scope means a web application or service. For Python that is Flask, Django or FastAPI (Starlette counts). For JS/TS it is a Node server (Express, Koa, NestJS, Next API routes, plain http). For PHP it is any web application or plugin. Libraries, SDKs, CLIs, MCP stdio tool servers and other frameworks (Zope/Plone, aiohttp, WsgiDAV) were rejected.
|
||||
- Guard-bypass and incomplete-fix advisories are allowed, with the caveat "vulnerable pin already has a guard".
|
||||
- A fix spread over up to 3 commits is allowed. A fix inside a large unrelated squash or release commit is rejected.
|
||||
- One case per repository: the newest qualifying advisory wins.
|
||||
- **Exclusion check.** I compared every candidate repo against `used-repos-v2.txt` (case-insensitive, `github.com/owner/repo`), which covers the v1 population and all development subjects. There were no hits. No candidate is a training app, CTF, benchmark suite or toy fork.
|
||||
- **Verification (merged list, re-run independently of the lanes).** For all 37 pairs:
|
||||
- `gh api repos/O/R/commits/<SHA> --jq .sha` returned both SHAs.
|
||||
- For 36 pairs, the vulnerable SHA is listed in the fixed commit's `parents`. For qwed, the fixed SHA is a merge and the vulnerable SHA is its first parent.
|
||||
- For LightRAG, `compare` reports the fixed SHA 3 commits ahead of the vulnerable SHA and 0 behind, so the vulnerable SHA is an ancestor.
|
||||
|
||||
"Diff seen" means the patch hunks of the named fix commit were read through the API, and the sink was read at the vulnerable pin (`contents/<path>?ref=<vuln>`).
|
||||
|
||||
## Summary table
|
||||
|
||||
| # | id | phase | repo | lang (+also) | family | advisory | vulnerable SHA | fixed SHA | ancestry | sink file :: function | source | privilege | license | diff seen | confidence |
|
||||
|---|----|-------|------|--------------|--------|----------|----------------|-----------|----------|-----------------------|--------|-----------|---------|-----------|------------|
|
||||
| 1 | lightrag-md-image-ssrf | quota | https://github.com/HKUDS/LightRAG | Python/FastAPI (+TS web UI) | untrusted_destination (SSRF, guard bypass) | CVE-2026-85740 / GHSA-vv3m-f8x4-7377 | `480f4a01741ec1d246842af68e09555a51ebf78f` | `c598545aa0084a427bcb19a84ade5e0ec31fa673` | ancestor (3 commits: `a2586217` → `9207e7fd` → `c598545a`, linear) | lightrag/parser/markdown/parser.py :: `_MarkdownImageResolver._download()` → `_build_guarded_opener().open(req)`, gated by the module function `_validated_addresses()` (image `src` URL from an uploaded markdown/textpack document) | document upload through the FastAPI API (image URLs inside the uploaded markdown) | authenticated (API key through `combined_auth`) | MIT | yes | medium |
|
||||
| 2 | lmdeploy-media-url-ssrf | quota | https://github.com/InternLM/lmdeploy | Python/FastAPI (+C++/CUDA) | untrusted_destination (SSRF, guard bypass) | GHSA-39wr-7q6h-cf68 (no CVE) | `71648d239c978452720a6d6692fe914efc8a7b94` | `03c313006d17cc3feae86b633c44206a997c44db` | parent | lmdeploy/vl/media/connection.py :: `_load_http_url()` → `requests.Session().get(url_spec.geturl(), allow_redirects=True)`, gated by `_is_safe_url()` (the `urlparse` host differs from the `requests` host) | `image_url`/video/audio URLs in `POST /v1/chat/completions` → serve/processors/multimodal.py :: `MultimodalProcessor._parse_multimodal_item()` → `load_from_url()` | unauthenticated by default (`api_keys` is optional, default None) | Apache-2.0 | yes | medium |
|
||||
| 3 | litellm-user-config-ssrf | quota | https://github.com/BerriAI/litellm | Python/FastAPI (+TS admin UI) | untrusted_destination (SSRF) | CVE-2026-59823 / GHSA-hx8v-g79f-8w5f | `93faf321df159b1ef65a98cc29a70958a96d184a` | `47214be317f45a2d2ac08258362e0481411eb427` | parent | litellm/proxy/route_llm_request.py :: `_route_user_config_request()` (`litellm.Router(**user_config)` then the provider call to the attacker's `api_base`); the only guard is litellm/proxy/auth/auth_utils.py :: `is_request_body_safe()` (`banned_params`) | JSON body of `POST /chat/completions` etc., key `user_config` | authenticated (any valid virtual key) | NOASSERTION (MIT core + enterprise dir) | yes | medium |
|
||||
| 4 | open-webui-playwright-ssrf | quota | https://github.com/open-webui/open-webui | Python/FastAPI (+Svelte/TS/JS) | untrusted_destination (SSRF, DNS rebinding) | CVE-2026-87996 / GHSA-4v28-j6q3-5m4r | `73c1f5806aeb6345dad5de8f5aa26d1f3d0bef80` | `27402ff210bfa253445720920dfb86b15a00327b` | parent | backend/open_webui/retrieval/web/utils.py :: `SafePlaywrightURLLoader._intercept_navigation_sync()` / `_intercept_navigation()` (`validate_url(req.url)` then `route.fetch()`, where the browser re-resolves the host) | URL submitted for RAG ingestion or web search | authenticated (any verified user) | Open WebUI License (BSD-3 plus branding clause, not OSI; GitHub: NOASSERTION) | yes | medium |
|
||||
| 5 | winml-cli-serve-cors | quota | https://github.com/microsoft/winml-cli | Python/FastAPI (+HTML) | config_secrets (wildcard CORS on an unauthenticated local API) | CVE-2026-84452 / GHSA-96p9-rh4f-92cf | `02179d185f0e3ac23518236b5ca1f61fd37ef680` | `f4073e0ef4700a25b623487e7e45c421ca0b9993` | parent | src/winml/modelkit/serve/cli_api.py :: module level `app.add_middleware(CORSMiddleware, allow_origins=["*"], ...)` (same in serve/app.py :: `create_app()`) | any web page in the victim's browser calling `POST /v1/cli/{command}` on 127.0.0.1 | unauthenticated (cross-origin from any website) | MIT | yes | high |
|
||||
| 6 | chainlit-mcp-ssrf | quota | https://github.com/Chainlit/chainlit | Python/FastAPI (+TS) | untrusted_destination (SSRF) | CVE-2026-45019 / GHSA-hvfh-5mj3-5f3j | `b8506a47fa59d7eaba40ad6d27496c2605a64cb9` | `0565fd0eccb915fce159929598b053ed79f6e0c9` | parent | backend/chainlit/server.py :: `connect_mcp()` → nested `_mcp_session_runner()` → `sse_client(url=...)` / `streamablehttp_client(url=..., headers=...)` (`payload.url`, `payload.headers`) | `POST /mcp` JSON body (`ConnectMCPRequest`) | unauthenticated (needs non-default `features.mcp.enabled = true`) | Apache-2.0 | yes | high |
|
||||
| 7 | contextforge-prompt-ssti | quota | https://github.com/IBM/mcp-context-forge | Python/FastAPI (+JS, Rust, Go) | injection (SSTI, Jinja2) | GHSA-vwf3-4xxj-qg6h (no CVE) | `0f8e0ad679f1fff685f486e17f642ac863c8ca9e` | `4d31004661858a2e99055b3c0c9f14218b8f7120` | parent | mcpgateway/services/prompt_service.py :: `PromptService._render_template()` → `_compile_jinja_template()` → `_get_jinja_env().from_string(template).render()` (plain `jinja2.Environment`) | prompt template stored through `register_prompt` (prompts create/update API), rendered on prompt get | authenticated (permission to create or update prompts) | Apache-2.0 | yes | high |
|
||||
| 8 | lemur-acme-update-ssrf | extra | https://github.com/Netflix/lemur | Python/Flask (+JS) | untrusted_destination (SSRF, stored) | CVE-2026-71303 / GHSA-v5rc-cpwc-cfpr | `e9ade7dad5c9bc1369a831554ed6df3277c9cbae` | `edca0390f930344d65ff4ca37a669c2320e3dfad` | parent | lemur/plugins/lemur_acme/acme_handlers.py :: `AcmeHandler.setup_acme_client_no_retry()` → `ClientV2.get_directory(directory_url, net)` (`acme_url` from the stored authority options) | `PUT /api/1/authorities/<id>` `options` → lemur/authorities/service.py :: `update()` stores it without validation | authenticated (user with a role on the ACME authority) | Apache-2.0 | yes | high |
|
||||
| 9 | glances-restful-cors-credentials | extra | https://github.com/nicolargo/glances | Python/FastAPI (+Vue/JS) | config_secrets (wildcard CORS with credentials) | CVE-2026-68517 / GHSA-fp27-88fp-2phg | `a14fb302a772bf8341b00c3ab8c2b0e95cffc197` | `890858944ab9d03730ec6b1ba42d4015e6d85db5` | parent | glances/outputs/glances_restful_api.py :: `GlancesRestfulApi.__init__()` (`add_middleware(CORSMiddleware, allow_origins=cors_origins, allow_credentials=cors_credentials)` after a guard that only matches `cors_origins == ["*"]`) | configuration file (`[outputs] cors_origins=*,https://x`, `cors_credentials=true`); exploited by any website through the victim's cached Basic auth | unauthenticated cross-origin reader (config-dependent) | LGPL-3.0 per COPYING (GitHub: NOASSERTION) | yes | medium |
|
||||
| 10 | qwed-sympy-eval | continuation | https://github.com/QWED-AI/qwed-verification | Python/FastAPI (+HTML, small Go/TS/Rust SDKs) | injection (code/eval injection through SymPy `parse_expr`) | CVE-2026-55585 / GHSA-q27q-98j4-9pfv | `06801f68cbf9c9b5e54cc17f2a61cb2792da7a26` | `6066b68c0c4f4cc2c3771824822aaa864d082ef8` | first parent (fixed is the merge of PR #200; the PR commit `dc9d4db72ca4b4ae3f96d0e6a0c27a9e38a06f61` is a direct child of the vulnerable SHA) | src/qwed_new/api/main.py :: `verify_math()` → `sympy.parsing.sympy_parser.parse_expr(expression_normalized)` (also `left_str`/`right_str`); second path src/qwed_new/core/batch.py :: `_verify_item()` → `parse_expr(item.query)` | `POST /verify/math` JSON `expression`; `POST /verify/batch` item `query` | authenticated tenant (API key; self-signup `/auth/signup` is on by default) | Apache-2.0 | yes | high |
|
||||
| 11 | xinference-llama3-tool-eval | continuation | https://github.com/xorbitsai/inference | Python/FastAPI (+TS/JS web UI) | injection (code/eval injection) | CVE-2026-61539 / GHSA-x2rj-828p-hx9m | `4cefcaa1e63d7e29d26bcd4be96cb267da113871` | `1b3d220f342ce68d34cec4586d9409d457dadc42` | parent | xinference/model/llm/tool_parsers/llama3_tool_parser.py :: `Llama3ToolParser.extract_tool_calls()` → `eval(model_output, {}, {})`; second site xinference/model/llm/utils.py :: `ChatModelMixin._eval_llama3_chat_arguments()` → `eval(text, {}, {})` | model output steered by the prompt in `POST /v1/chat/completions` with `tools` (xinference/api/restful_api.py) | unauthenticated by default (auth not enabled in the default deployment, per the advisory) | Apache-2.0 | yes | medium |
|
||||
| 12 | pgadmin-maintenance-sqli | continuation | https://github.com/pgadmin-org/pgadmin4 | Python/Flask (+JS/React) | injection (SQLi) | CVE-2026-7815 / GHSA-hp84-p2gq-6fvr | `d112dc3b962218efc3613fa727d54f0fc5b1d527` | `cf53953d9ad66c182a08e662c1ba29c5831b486b` | parent | web/pgadmin/tools/maintenance/templates/maintenance/sql/command.sql (string concatenation of `vacuum_index_cleanup`, `vacuum_parallel`, `buffer_usage_limit`, `reindex_tablespace`) rendered in web/pgadmin/tools/maintenance/__init__.py :: `create_maintenance_job()` → `render_template(...)` → `BatchProcess(cmd=psql, args=[..., '--command', query])` | JSON body of `POST /maintenance/job/<sid>/<did>` | authenticated (`tools_maintenance` permission) | PostgreSQL License (OSI; GitHub: NOASSERTION) | yes | high |
|
||||
| 13 | reactpress-findall-column-sqli | quota | https://github.com/fecommunity/reactpress | TypeScript, NestJS/TypeORM (+JS, CSS, HTML) | injection (SQLi) | CVE-2026-61685 / GHSA-wmw4-mw6x-6vfm | `c3dd10681b3d2b5e224f7ac647b3769557603975` | `78ecb70af1c021455c05fdcbe137212c70e310d6` | parent | server/src/modules/article/article.service.ts :: `ArticleService.findAll()` (query-string key interpolated as `article.${key}` into `andWhere`); same pattern in comment/file/page/knowledge services | `@Query() queryParams` in `ArticleController.findAll()` (`GET /api/article`) | unauthenticated | MIT | yes | high |
|
||||
| 14 | 9router-kiro-region-ssrf | quota | https://github.com/decolua/9router | JavaScript, Next.js API routes (+CSS, Shell) | untrusted_destination (SSRF) | CVE-2026-56678 / GHSA-6mwv-4mrm-5p3m | `f2a7ae20309b4af55023eb11d1c02f63be1b80d1` | `126aa244c5b51b74ab8c7594e3418fcf4437bf6f` | parent | src/lib/oauth/services/kiro.js :: `KiroService.listAvailableProfiles()` (`fetch(\`https://codewhisperer.${region}.amazonaws.com\`)` with the API key as Bearer), reached via `validateApiKey()` | `region` in JSON body of `POST /api/oauth/kiro/api-key` (src/app/api/oauth/kiro/api-key/route.js) | authenticated (dashboard session) | MIT | yes | high |
|
||||
| 15 | omniroute-acp-agent-rce | quota | https://github.com/diegosouzapw/OmniRoute | TypeScript, Next.js API routes (+JS, Python, Shell) | injection (OS command / code) | CVE-2026-88062 / GHSA-hf57-cqmx-p4gr | `c130f2aa1ccc7aaddd7a7685bd6a0e08136dccf1` | `60829241fd64d0317aa6a0dd8cd7a445a5287fed` | parent | src/lib/acp/registry.ts :: `detectAgent()` → `execFileSync(probe.command, probe.args)` with probe from `resolveVersionProbe(binary, versionCommand, true)` | `binary` / `versionCommand` in body of `POST /api/acp/agents` (src/app/api/acp/agents/route.ts → `setCustomAgents` → `refreshAgentCache`) | unauthenticated when `requireLogin=false` or before a management password is set; management session otherwise | MIT | yes | medium |
|
||||
| 16 | cct-studio-server-cmdi | quota | https://github.com/davila7/claude-code-templates | JavaScript, Express (+Python, TS, HTML, Shell, many template languages) | injection (OS command) | CVE-2026-73222 / GHSA-79wm-x847-7cvg | `acdb5893412a81eaec9862ee24a4613bf296ab45` | `bc4618b07232633c1c0aac12a43e436268d31783` | parent | cli-tool/src/sandbox-server.js :: `executeLocalTask()` (`spawn('claude', [finalPrompt], {shell: true})`), route handler `POST /api/install-agent` and `checkAndInstallAgent()` (`spawn('npx', [..., agentName, ...], {shell: true})`) | `req.body.prompt`, `req.body.agentName`, `req.body.agent` | unauthenticated (server on 0.0.0.0, wildcard CORS) | MIT | yes | high |
|
||||
| 17 | unleash-addon-webhook-ssrf | quota | https://github.com/Unleash/unleash | TypeScript, Express (+JS, PLpgSQL, SCSS) | untrusted_destination (SSRF) | CVE-2026-63004 / GHSA-5vf6-jrqr-78fj | `b9f3217d327f24094b4352130baa30387f85cffe` | `2100db76af3473f13e6fb40096cf17a9c2b741a1` | parent | src/lib/addons/addon.ts :: `Addon.fetchRetry()` (`ky(url, ...)`), called from webhook.ts / slack / teams / datadog / new-relic addons | addon `parameters.url` saved through the admin addons API (`CREATE_ADDON`/`UPDATE_ADDON`), fired on feature events | authenticated (CREATE_ADDON/UPDATE_ADDON root permission) | AGPL-3.0 | yes | high |
|
||||
| 18 | nocobase-backup-restore-cmdi | quota | https://github.com/nocobase/nocobase | TypeScript, Koa (+JS, Shell) | injection (OS command) | CVE-2026-55410 / GHSA-p853-83gj-wjj3 | `a89b1ea0218b0d758543734e86f595f98f654314` | `aecc6758810b46dc13790d35ec4a15eea29357cc` | parent | packages/plugins/@nocobase/plugin-backups/src/server/adapters/database.ts :: `PostgresAdapter.restore()` (`-n ${srcSchema}` in pg_restore command string) → module `run()` → `child_process.exec` | `database.schema` in `_metadata.json` of an uploaded backup archive, restore action with `force` (resourcers/backup-cli.ts → managers/restore.ts) | authenticated, backup-management role | repo: NocoBase License (non-OSI, GitHub NOASSERTION); plugin package.json says Apache-2.0 | yes | medium |
|
||||
| 19 | medplum-external-auth-open-redirect | quota | https://github.com/medplum/medplum | TypeScript, Express (+JS, Python, HCL) | untrusted_destination (open redirect) | CVE-2026-53728 / GHSA-m44r-7c5h-m6mj | `5d4a8654b02aa67c444f96ae1eb5663c09854006` | `7ae10035ddadde4dba7b18d3156553940465b3a1` | parent | packages/server/src/auth/external.ts :: `externalCallbackHandler()` (`res.redirect(redirectUrl)` with URI accepted by `getClientRedirectUri(client, body.redirectUri, true)` prefix match in packages/server/src/oauth/clients.ts) | `redirectUri` inside JSON `state` query parameter of `GET /auth/external` | unauthenticated (victim completes external IdP login) | Apache-2.0 | yes | high |
|
||||
| 20 | flowise-ipv4-mapped-ssrf | extra | https://github.com/FlowiseAI/Flowise | TypeScript, Express (+JS, HTML, SCSS) | untrusted_destination (SSRF, guard bypass) | CVE-2026-69257 / GHSA-c6xh-wv4j-ppv5 | `b7a2005fd7b394ba7874aded744fe3730d59ccb3` | `0fc769208395641c1411ccdb9c81416e54802155` | parent | packages/components/src/httpSecurity.ts :: `isDeniedIP()` (deny-list check used by `checkDenyList()` / `resolveAndValidate()` before `secureFetch()` / `secureAxiosRequest()`) | URLs in chatflow node inputs that go through `secureFetch`/`secureAxiosRequest`; attacker-controlled DNS returning `::ffff:<internal IPv4>` | authenticated (flow builder), possibly prediction API when a flow takes a URL from input | Apache-2.0 outside `packages/server/src/enterprise` (GitHub NOASSERTION; commercial licence for enterprise dir) | yes | medium |
|
||||
| 21 | nx-graph-server-wildcard-cors | extra | https://github.com/nrwl/nx | TypeScript, Node `http` server (+JS, Rust, many) | config_secrets (permissive CORS) | CVE-2026-54753 / GHSA-g2r8-wvmj-jf5w | `6ac9f82b04d02ab057b395b0e2b209fb3caf8545` | `e59122c5569cf201561b9995ce4e34a304f4fb03` | parent | packages/nx/src/command-line/graph/graph.ts :: `startServer()` request handler (`res.setHeader('Access-Control-Allow-Origin', '*')` on every response, including `/help` which runs `execSync(target.metadata.help.command)`) | any cross-origin web page the developer visits while `nx graph` runs | unauthenticated (cross-origin, local dev server) | MIT | yes | high |
|
||||
| 22 | budibase-mongo-template-nosqli | continuation | https://github.com/Budibase/budibase | TypeScript, Koa (+JS, Svelte, Shell) | injection (NoSQL) | GHSA-pmpg-2mxq-6xwr (no CVE); same fix family as GHSA-qw6m-8fw2-2v64 | `b7e63df737eede85d951aa9665b0c0778ece7ccb` | `dd8c0654b35cd89ce3645f2355f4bf2ff9a5dd80` | parent | packages/server/src/sdk/workspace/queries/queries.ts :: `enrichContext()` (handlebars `processStringSync(..., {noEscaping: true})` into the `json` template, then `JSON.parse`) → packages/server/src/integrations/mongodb.ts :: `MongoIntegration.read()` (`collection.find(json)`) | query parameters / bindings (e.g. `currentUser.email`, request parameters) of `POST /api/v2/queries/:queryId` (packages/server/src/api/controllers/query/index.ts :: `executeV2`) | authenticated app user (BASIC role) | GPL-3.0 (root LICENSE says "GPLv3 overall"; server package.json GPL-3.0; GitHub NOASSERTION) | yes | medium |
|
||||
| 23 | fuxa-tdengine-escape-sqli | continuation | https://github.com/frangoteam/FUXA | JavaScript, Express + Socket.IO (+TS, HTML, SCSS) | injection (SQLi, escape bypass) | CVE-2026-47720 / GHSA-h9fj-c2qr-76g2 | `193a888f8e3f805325072829b1644a2c768c948b` | `b78f6159d02a56ce5ff48207feb936afb3534fc8` | parent | server/runtime/storage/tdengine/index.js :: `TDengine.getDaqValue()` (`escapeTdString(tagid)` doubles quotes but not backslashes, then SQL string for `cursor.query`) | `sids` of `GET /api/daq` (server/api/daq/index.js) or Socket.IO `DAQ_QUERY` | unauthenticated (guest level; socket handler has no auth check) | MIT | yes | medium |
|
||||
| 24 | contao-feed-reader-ssrf | quota | https://github.com/contao/contao | PHP (+JS, Twig) | untrusted_destination (SSRF) | CVE-2026-57232 / GHSA-87mg-5grr-rhwh | `394d951e7f6a87074ddc1f75f1c0ad9f94d98d68` | `27f6201809553bee767dcef15535bb8f0f4eac5f` | parent (5.7 branch; fixed commit is an ancestor of tag 5.7.9) | core-bundle/src/Controller/FrontendModule/FeedReaderController.php :: `FeedReaderController::getResponse()` (`$this->feedIo->read($url, ...)` over `$model->rss_feed` URLs; HTTP client wired in core-bundle/config/services.yaml `contao.feed.http_adapter`) | `tl_module.rss_feed` field saved through the back end, rendered by any front-end page that holds the module | authenticated back-end user with module-edit rights | LGPL-3.0 | yes | high |
|
||||
| 25 | formie-form-settings-ssrf | quota | https://github.com/verbb/formie | PHP (+JS, TS) | untrusted_destination (SSRF) | CVE-2026-76086 / GHSA-v3f3-cmj4-cvj9 | `788fc375defb4def857c8737f7c92e487da92ca9` | `6735fe4ae8f6a2a76930716ad7876b236f7c530d` | parent (craft-5 branch; ancestor of tag 3.1.31) | src/controllers/IntegrationsController.php :: `IntegrationsController::actionFormSettings()` (`$integration->setAttributes($settings, false)`), then e.g. src/integrations/emailmarketing/ActiveCampaign.php :: `defineClient()` (`base_uri` from `$this->apiUrl`) → src/base/Integration.php :: `Integration::request()` | POST param `settings[...]` (for example `settings[apiUrl]`) to `formie/integrations/form-settings` | authenticated (any logged-in user, including front-end members) | **Craft License (proprietary, not OSI)**; GitHub says NOASSERTION | yes | medium |
|
||||
| 26 | sulu-smart-content-sql2-injection | quota | https://github.com/sulu/sulu | PHP (+JS) | injection (JCR-SQL2 query injection) | CVE-2026-92692 / GHSA-jg26-q8hg-3pq4 | `d061094f5b7bb1d5e974544fce30bede9c7adf8e` | `d19c01487af8c3de2fb3aa145856707a6367392c` | parent (2.6 branch; ancestor of tag 2.6.25) | src/Sulu/Component/Content/SmartContent/QueryBuilder.php :: `QueryBuilder::buildCategoriesWhere()` (also `buildTagsWhere()`, `buildAudienceTargeting()`), string-concatenated into the SQL2 WHERE clause built by `buildWhere()` | `?categories=` query parameter, read in src/Sulu/Component/Category/Request/CategoryRequestHandler.php :: `getCategories()` (explode + trim only), passed by src/Sulu/Component/SmartContent/ContentType.php as `websiteCategories` | unauthenticated (any public page with a category-filtered Smart Content block) | MIT | yes | high |
|
||||
| 27 | freeform-attribute-twig-ssti | quota | https://github.com/solspace/craft-freeform | PHP (+JS, EJS) | injection (SSTI, sandboxed Twig) | CVE-2026-73858 / GHSA-gxrg-x694-283w | `1b1b16a45c289cec7ac0cc2385e9431867427fe2` | `5f7555320635f1cd3b4c478aa0b58e8c3144313b` | parent (ancestor of tag v5.10.14, not of v5.10.13) | packages/plugin/src/Library/Attributes/Attributes.php :: `Attributes::toHtmlTagArray()` (`$twig->render($value, $properties)` through `Library\Helpers\IsolatedTwig`) | submitted form field values that are rendered back as the field's `value` attribute | unauthenticated (public form submission) | **Craft License (proprietary, not OSI)**; GitHub says NOASSERTION | yes | medium |
|
||||
| 28 | smw-uriresolver-open-redirect | extra | https://github.com/SemanticMediaWiki/SemanticMediaWiki | PHP (+JS) | untrusted_destination (open redirect) | CVE-2026-77609 / GHSA-hw3m-8j5x-94ff | `5821b9b865c58d40e5ecd1aa807b79fa32eb5c2f` | `765a6ac22305e2d5c6fc79415bfee7cbd58514cd` | parent (ancestor of tag 7.2.0, not of 7.1.0) | src/MediaWiki/Specials/SpecialURIResolver.php :: `SpecialURIResolver::execute()` (`$out->redirect($title->getFullURL(), '303')`) | subpage of `Special:URIResolver/<x>` (the `$query` argument), decoded and turned into a Title (interwiki prefixes allowed) | unauthenticated | GPL-2.0-or-later (composer.json; GitHub says NOASSERTION) | yes | high |
|
||||
| 29 | pheditor-default-admin-password | extra | https://github.com/pheditor/pheditor | PHP only (single file) | config_secrets (hardcoded default credential) | CVE-2026-55579 / GHSA-p4h7-p9rj-2pq2 | `e538f05b6faec99e5b23726bc9c17d6b57774297` | `a64eb935ce699e43f1ea60e46d47f33e2f87ffd6` | parent (= tag 2.0.5 → 2.0.6 compare, one commit) | pheditor.php :: top level `define('PASSWORD', '<sha512 of "admin">')` (line 11), checked by the top-level login block (`hash('sha512', $_POST['pheditor_password']) === PASSWORD`) | `pheditor_password` POST field on the login form | unauthenticated (default credential) | MIT | yes | low |
|
||||
| 30 | pimcore-studio-datefilter-sqli | continuation | https://github.com/pimcore/studio-backend-bundle | PHP (+Twig, tiny JS) | injection (SQLi, identifier) | CVE-2026-55208 / GHSA-79cw-hfcc-7mw9 | `ea9d329686f5e5aea2eec378d63ac2deb965bb27` | `f532428cfbf4f5d6e299a13cedd5c29541802552` | parent (ancestor of tags v2025.4.6 and v2026.1.6) | src/Listing/Filter/DateFilter.php :: `DateFilter::applyDateFilter()` (`'`' . $key . '` BETWEEN :minTime AND :maxTime'` into `$listing->addConditionParam()`); same pattern in EqualsFilter::apply(), LikeFilter::apply(), src/Note/Service/FilterService.php :: applyFieldFilters() | `columnFilters[].key` in the JSON body of Studio listing endpoints (e.g. `POST /pimcore-studio/api/website-settings`) | authenticated (any Studio back-end user who can reach a listing endpoint) | **Pimcore Open Core License (POCL), not OSI**; GitHub says NOASSERTION | yes | high |
|
||||
| 31 | froxlor-admin-ip-second-order-sqli | continuation | https://github.com/froxlor/froxlor | PHP (+JS, Twig) | injection (SQLi, second-order) | CVE-2026-54348 / GHSA-w27m-rmmf-g5w4 | `9bf03de643a2edee4642bfc6c94db1f05253da02` | `a1eaca5a1601c8a30e00814a4fc73ad0c185f89e` | parent (ancestor of tag 2.3.8) | lib/Froxlor/Api/Commands/IpsAndPorts.php :: `IpsAndPorts::listing()` / `listingCount()` (`implode(", ", json_decode($this->getUserDetail('ip'), true))` into `WHERE id IN (...)`); also lib/Froxlor/Api/Commands/Domains.php :: `validateIpAddresses()` | `ipaddress` API parameter of `Admins.add` / `Admins.update` (lib/Froxlor/Api/Commands/Admins.php), stored as JSON in `panel_admins.ip`, read back via `getUserDetail('ip')` | admin (API key with `change_serversettings`) | LGPL-2.1 (per GitHub) | yes | high |
|
||||
| 32 | wp-s2member-tracking-eval | quota | https://github.com/wpsharks/s2member | PHP (+JS, CSS) | injection (code injection / eval) | CVE-2026-19804 / GHSA-fgx2-6hhq-hwhw | `4e1fe70b342f911ae5cb5c66652f2a2aef45b08b` | `6c622839aa0efe14031ff7c8fcc1f7d3107563ba` | parent | src/includes/classes/tracking-codes.inc.php :: `c_ws_plugin__s2member_tracking_codes::display_signup_tracking_codes()` → src/includes/classes/utilities.inc.php :: `c_ws_plugin__s2member_utilities::evl()` (`eval('?>'.$code)` over a transient holding the admin template with `%%first_name%%` replaced by the payer's first name) | `$paypal['first_name']` from the PayPal return/IPN data, substituted in src/includes/classes/paypal-return-in-subscr-or-wa-w-level.inc.php (line ~315) and stored with `set_transient()` | unauthenticated (needs the admin template to contain `%%first_name%%` and the proxy key, see notes) | GitHub: none; plugin header "GNU General Public License" (GPL, version not stated) | yes | medium |
|
||||
| 33 | wp-directorist-avatar-ssrf | quota | https://github.com/sovware/directorist | PHP (+JS, Vue, TS, CSS) | untrusted_destination (SSRF) | CVE-2026-84046 / GHSA-6q63-3w56-m722 | `77cfc9bc8b453f97ab7e720330706fa4ef056867` | `1a67825d5cf6e7c768c3a692da1d18bae732bea9` | parent | includes/rest-api/functions.php :: `directorist_rest_upload_image_from_url()` (`download_url($image_url)`) | `$request['avater']['src']` in includes/rest-api/Version1/class-users-controller.php :: `update_user_meta_fields()` (REST user update) | authenticated (subscriber) | GitHub: NOASSERTION; plugin header GPLv3 | yes | medium |
|
||||
| 34 | wp-gopay-log-filter-sqli | quota | https://github.com/gopaycommunity/gopay-woocommerce-integration | PHP (+JS, CSS) | injection (SQLi) | CVE-2026-75959 / GHSA-88c4-2f34-57h5 | `626bc8b5f8d33de6eefd921e3a05ed52ac320bcd` | `7190c985370f9758f226c2f2addd79532a2f93d5` | parent | admin/views/log.php :: file scope (included by admin/class-gopay-gateway-admin.php), two `$wpdb->get_results(sprintf("... REGEXP '[\w\W]*%s[\w\W]*' ...", ..., strtoupper($log_table_filter)))` | `filter_input(INPUT_GET, 'log_table_filter')` | admin (shop manager, `manage_woocommerce`) | GitHub: none; plugin header GPLv2 or later | yes | high |
|
||||
| 35 | wp-user-registration-login-redirect | quota | https://github.com/themegrill/user-registration | PHP (+JS, TS, SCSS) | untrusted_destination (open redirect) | CVE-2026-80072 / GHSA-w78c-fx7q-9x8m | `e52cf151cb229852e6209fb3a627aa51485139c1` | `75c940dab15fd54d5a3fbb9f2213767b89467b08` | parent | includes/functions-ur-core.php :: `ur_process_login()` (`wp_redirect(wp_validate_redirect($redirect, $redirect))`, and `wp_send_json_success(['message'=>$redirect])` on the AJAX path) | `$post['redirect']` (login form POST) or `wp_get_raw_referer()` | unauthenticated attacker, victim logs in | GitHub: NOASSERTION; repo has license.txt, plugin header GPLv3 | yes | high |
|
||||
| 36 | wp-groundhogg-confirm-redirect | quota | https://github.com/groundhoggwp/groundhogg | PHP (+JS, SCSS, HTML) | untrusted_destination (open redirect) | CVE-2026-81741 / GHSA-m96f-c5vh-h7cm | `9472a1dd79d4556c40eece501a61930793f170d3` | `028247d4b8ec9ba051d3eb48f1466a759bc16bfd` (+ comment-only follow-up `2f03c8481a2236b0574f67fa8a7415b77556f730`) | parent (follow-up is the direct child of the fix) | templates/preferences.php :: file scope, `case 'confirm':` (`wp_redirect($redirect_to)`) | `get_url_var('redirect_to')` (query string, only through `esc_url_raw(sanitize_text_field(...))`) | unauthenticated (crafted preferences link) | GitHub: none, no LICENSE file; plugin header GPLv3 | yes | high |
|
||||
| 37 | wp-stripe-payments-thankyou-redirect | extra | https://github.com/Arsenal21/stripe-payments | PHP (+JS, CSS) | untrusted_destination (open redirect) | CVE-2026-81423 / GHSA-rvj6-j43w-hcgw | `71806a51e2c2c2ceb2cdaf00cbbde7a1b9da09f4` | `fb1f647deb6c92a2d2125da7a99fcb2914264f9c` | parent | stripe-payments/includes/class-asp-process-ipn-ng.php :: `ASP_Process_IPN_NG::ipn_completed()` (`wp_redirect($this->asp_redirect_url)`) | POST `asp_thankyou_page_url` (base64-decoded) in `ASP_Process_IPN_NG::process_ipn()`, triggered by POST `asp_process_ipn` on `wp_loaded` | unauthenticated (needs the public payment-form nonce) | GitHub: none; plugin header GPLv2 or later | yes | medium |
|
||||
|
||||
**Mix (37 candidates).**
|
||||
- **By language:** 12 Python, 11 JS/TS, 14 PHP. The 14 PHP split into 8 PHP web apps and 6 WordPress plugins, all of which ship JS.
|
||||
- **By family:**
|
||||
- 16 `injection`: SQLi in #12, #13, #23, #30, #31, #34; JCR-SQL2 query injection #26; NoSQL #22; SSTI #7, #27; code/eval #10, #11, #32; OS command #15, #16, #18.
|
||||
- 17 `untrusted_destination`: SSRF in #1, #2, #3, #4, #6, #8, #14, #17, #20, #24, #25, #33; open redirect in #19, #28, #35, #36, #37.
|
||||
- 4 `config_secrets`: wildcard CORS in #5, #9, #21; hardcoded default password #29.
|
||||
- **By phase:** 23 quota, 7 extra, 7 continuation.
|
||||
- **By privilege:** 19 unauthenticated (some only under a non-default setting, see the notes), 16 authenticated and 2 admin (#31, #34).
|
||||
- **Quota check (quota phase alone):** PHP 9 (5 WordPress), JS 7, Python 7. The quota rows alone give 5 injection, 15 untrusted_destination and 3 config_secrets. Most injection candidates come from the continuation passes, which is why those passes were run.
|
||||
|
||||
## Per-candidate notes
|
||||
|
||||
### Python (Flask / Django / FastAPI)
|
||||
|
||||
Source list: `gh api /advisories?ecosystem=pip&type=reviewed&cwes=<c>&sort=published&direction=desc` for CWEs 89, 78, 77, 94, 95, 1336, 943, 90, 918, 601, 489, 215, 942, 295, 798, 321, 259, 1188, 16, 346, 1004, merged and sorted newest first (list-pip.tsv). I worked from the top (2026-09-24) down. The quota of 7 was filled at GHSA-vwf3-4xxj-qg6h (2026-08-25, row 37). After that I continued newest first for `untrusted_destination` and `config_secrets` only, and stopped once each had one extra: Lemur (SSRF) and Glances (CORS). The last row examined is row 72 of list-pip.tsv (GHSA-fp27-88fp-2phg, 2026-08-17).
|
||||
|
||||
#### lightrag-md-image-ssrf (Python, FastAPI)
|
||||
- Repo: https://github.com/HKUDS/LightRAG (MIT, about 117 MB; Python plus a TypeScript web UI).
|
||||
- Advisory: CVE-2026-85740, GHSA-vv3m-f8x4-7377. ≤ 1.5.4, fixed in 1.5.5.
|
||||
- Flaw: the native markdown parser downloads images referenced by an uploaded document (`NATIVE_MD_IMAGE_DOWNLOAD_ENABLED` defaults to True). `_download()` calls `_host_is_public()`, then opens the URL through a guarded opener. Both rely on `_validated_addresses()`, which accepts an address when `ip.is_global` is true. Python classifies NAT64 (`64:ff9b::/96`, `64:ff9b:1::/48`), IPv4-compatible (`::a.b.c.d`) and 6to4 (`2002::/16`) wrappers around internal IPv4 addresses as global, so those addresses pass.
|
||||
- Fix: three consecutive commits on the same day. `a2586217` adds `_unwrap_embedded_ipv4()` and judges `is_global` on the embedded IPv4. `9207e7fd` corrects the RFC 8215 /48 decoding to deny by default. `c598545a` stops the 6to4 branch from loosening the stdlib verdict. The commits touch only parser.py and its tests.
|
||||
- Caveats: the vulnerable pin already has a guard (this is a guard bypass). The fix spans 3 commits, so use `480f4a01` → `c598545a`. Exploitation needs NAT64/DNS64 routing on the host. The sink is in the same file as the fix.
|
||||
- Confidence: medium. I read all three diffs and the sink at the vulnerable pin. The flaw is a subtle address-classification bug rather than a missing check.
|
||||
|
||||
#### lmdeploy-media-url-ssrf (Python, FastAPI)
|
||||
- Repo: https://github.com/InternLM/lmdeploy (Apache-2.0, about 17 MB; Python plus C++/CUDA kernels). This is an inference engine. The in-scope surface is its FastAPI OpenAI-compatible server (`lmdeploy serve api_server`).
|
||||
- Advisory: GHSA-39wr-7q6h-cf68 (no CVE), ≥ 0.12.3, < 0.15.0. **The advisory names no fix commit.** I found `03c31300` ("fix: restrict remote media domains", #4734) through the history of `lmdeploy/vl/media/connection.py`. It is an ancestor of tag v0.15.0 and the only commit in that range that changes the URL parser.
|
||||
- Flaw: `_is_safe_url()` validates `urlparse(url).hostname`, but `requests` connects to a different host for inputs such as `http://127.0.0.1:6666\@1.1.1.1`. `_load_http_url()` then fetches with `allow_redirects=True`, and redirect targets are not re-validated.
|
||||
- Fix: `_is_safe_url()` uses urllib3 `parse_url` (the parser `requests` uses). `_load_http_url()` follows redirects manually and re-validates each hop. The commit also adds an optional `allowed_media_domains` allowlist, plumbed through api.py, cli/serve.py, pipeline.py, the async engines and api_server.py (12 files).
|
||||
- Privilege: `api_keys` defaults to None in `serve()`, so the server is unauthenticated by default.
|
||||
- Confidence: medium. I read the diff and the sink. The fix commit is my inference because the advisory does not reference it. The vulnerable pin already has a guard (parser differential).
|
||||
|
||||
#### litellm-user-config-ssrf (Python, FastAPI)
|
||||
- Repo: https://github.com/BerriAI/litellm. **It is very large (about 1.7 GB of git data).** GitHub reports the license as NOASSERTION: the core is MIT and the `enterprise/` directory has its own license. Verify before publication.
|
||||
- Advisory: CVE-2026-59823, GHSA-hx8v-g79f-8w5f. ≤ 1.83.8, fixed in 1.83.9.
|
||||
- Flaw: `is_request_body_safe()` blocks `api_base`/`base_url` in the request body but not `user_config`. `route_request()` sends any request containing `user_config` to `_route_user_config_request()`, which builds `litellm.Router(**user_config)` and makes the call. An `api_base` nested in that config therefore redirects the proxy's outbound request, together with the provider credentials in the config.
|
||||
- Fix `47214be3`: adds `"user_config"` to `banned_params`. The same commit makes unrelated hardening changes: a ContextVar for internal calls, stripping `proxy_server_request`/`user_api_key_*` metadata keys, formatting churn, and file_search emulation (6 files).
|
||||
- Caveats: the sink (route_llm_request.py) is outside the changed files. The fix is a one-word denylist change.
|
||||
- Confidence: medium. The diff is clear. The outbound HTTP call itself sits deep in provider code, so I name the Router construction as the sink.
|
||||
|
||||
#### open-webui-playwright-ssrf (Python, FastAPI)
|
||||
- Repo: https://github.com/open-webui/open-webui (about 440 MB; Python backend plus a Svelte/TS frontend). **License: "Open WebUI License", which is BSD-3 plus a branding-preservation clause, not OSI-approved. GitHub reports NOASSERTION.**
|
||||
- Advisory: CVE-2026-87996, GHSA-4v28-j6q3-5m4r. ≥ 0.9.6, < 0.11.1.
|
||||
- Flaw: with `WEB_LOADER_ENGINE=playwright` (not the default), the route interceptor checks `validate_url(req.url)` and then lets the browser perform the fetch through `route.fetch()`. The browser resolves the hostname again, so DNS rebinding reaches internal addresses. Redirect hops are checked the same way.
|
||||
- Fix `27402ff2`: the interceptor now performs each request with the project's SSRF-safe requests/aiohttp sessions, which pin the validated address at connect time, and fulfils the page from that response. The change is 2 files.
|
||||
- Caveats: the vulnerable pin already has a guard (a TOCTOU bypass). The sink is a Playwright browser call, not an HTTP client. The feature is not on by default.
|
||||
- Confidence: medium. I read the diff and the vulnerable interceptor.
|
||||
- Two other Open WebUI advisories in the window (GHSA-34r3-9m95-vq73 and GHSA-5x7x-4c3c-qf5w, both fixed by `e3e4bd87`) were not taken, because the repo already has a candidate.
|
||||
|
||||
#### winml-cli-serve-cors (Python, FastAPI)
|
||||
- Repo: https://github.com/microsoft/winml-cli (MIT, about 26 MB). The package is a CLI whose `serve` subcommand runs a FastAPI server that exposes every CLI command over HTTP on 127.0.0.1.
|
||||
- Advisory: CVE-2026-84452, GHSA-96p9-rh4f-92cf (CWE-306/942). < 0.4.0.
|
||||
- Flaw: cli_api.py (module level) and app.py `create_app()` register `CORSMiddleware(allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])`. The API has no authentication, so any website can drive `POST /v1/cli/{command}`. With `--trust-remote-code` this leads to RCE through `AutoConfig.from_pretrained(..., trust_remote_code=True)`.
|
||||
- Fix `f4073e0e`: replaces CORSMiddleware with a new `SameOriginMiddleware` (serve/_security.py), makes the CLI and model-mutation routes loopback-only, and refuses remote-code execution requested over HTTP. It touches 12 files, all related.
|
||||
- Caveat: `allow_credentials=False` at the vulnerable pin. This is wildcard CORS on an API with no authentication, not wildcard CORS combined with credentials.
|
||||
- Confidence: high. I read the diff and the vulnerable module.
|
||||
|
||||
#### chainlit-mcp-ssrf (Python, FastAPI)
|
||||
- Repo: https://github.com/Chainlit/chainlit (Apache-2.0, about 12 MB; Python plus TypeScript).
|
||||
- Advisory: CVE-2026-45019, GHSA-hvfh-5mj3-5f3j. ≥ 2.4.0rc0, ≤ 2.11.1, fixed in 2.12.0.
|
||||
- Flaw: `connect_mcp()` (`POST /mcp`) builds `SseMcpConnection`/`HttpMcpConnection` from `payload.url` and `payload.headers` without any check, and its background runner opens `sse_client()` / `streamablehttp_client()` to that URL. The attacker also controls the forwarded headers.
|
||||
- Fix `0565fd0e` ("Merge commit from fork"): adds `validate_mcp_url()` against a new `features.mcp.user_servers.allowed_urls` allowlist, header filtering and connect-time destination checks, plus the fix for the sibling stdio command-injection advisory. It touches 24 files, all MCP-related (Python backend, React client and tests), about 4k added lines. That makes it large but coherent.
|
||||
- Caveats: the same commit fixes GHSA-w3fx-mc44-mf6j (command injection through the stdio transport, same endpoint). MCP is disabled by default since 2.7.0.
|
||||
- Confidence: high. I read the server.py hunks and the vulnerable handler.
|
||||
|
||||
#### contextforge-prompt-ssti (Python, FastAPI)
|
||||
- Repo: https://github.com/IBM/mcp-context-forge (Apache-2.0, about 450 MB; Python plus JS, Rust, Go).
|
||||
- Advisory: GHSA-vwf3-4xxj-qg6h (no CVE). < 1.0.0.
|
||||
- Flaw: prompt templates supplied by users are compiled with a plain `jinja2.Environment` (`_get_jinja_env()`) and rendered in `PromptService._render_template()`. That allows `__class__`/`__subclasses__` gadget chains, which give code execution on the gateway. The advisory quotes the 0.9.0 layout (`self._jinja_env` built in `__init__`). At the vulnerable pin the environment comes from the module-level `_get_jinja_env()`, and `__init__` stores that.
|
||||
- Fix `4d310046`: `Environment` becomes `SandboxedEnvironment`, and a regex template validator is added. The hunk is small and clear, but it sits inside a squashed feature PR (#4072, "content security US-3 and US-4", 27 files) that also adds malicious-pattern detection for resources and tools.
|
||||
- Caveats: the fix is mixed with related feature work. The template is stored first and rendered later (second order).
|
||||
- Confidence: high for the sink and the fix hunk.
|
||||
|
||||
#### lemur-acme-update-ssrf (Python, Flask). Extra `untrusted_destination`
|
||||
- Repo: https://github.com/Netflix/lemur (Apache-2.0, about 13 MB). **The repository is archived (read-only)** since after this fix. It is a real Netflix product, not a toy.
|
||||
- Advisory: CVE-2026-71303, GHSA-v5rc-cpwc-cfpr (incomplete fix of GHSA-v2wp-frmc-5q3v).
|
||||
- Flaw: `acme_url` is checked against `ACME_DIRECTORY_HOST_ALLOWLIST` only when an authority is created. `PUT /api/1/authorities/<id>` → `authorities.service.update()` stores new `options` as they are. At the next issuance, `AcmeHandler.setup_acme_client_no_retry()` reads `acme_url` and fetches it with `ClientV2.get_directory(directory_url, net)`.
|
||||
- Fix `edca0390`: `update()` re-validates `acme_url` with the now-public `validate_acme_url()`. The code change is 10 lines.
|
||||
- Caveats: stored (second-order) SSRF. The sink is outside the changed files. The outbound call runs inside the third-party `acme` library, called from project code.
|
||||
- Confidence: high. I read the diff, the handler and the PUT view.
|
||||
- The same repo has GHSA-f3qq-49m6-rw8f and GHSA-xpmj-wjcp-6pww (fixes `ed504a83`, `6dcb19b6`), which I did not take (one case per repo).
|
||||
|
||||
#### glances-restful-cors-credentials (Python, FastAPI). Extra `config_secrets`
|
||||
- Repo: https://github.com/nicolargo/glances (about 60 MB; Python plus a Vue web UI). **COPYING is LGPL-3.0; GitHub reports NOASSERTION.**
|
||||
- Advisory: CVE-2026-68517, GHSA-fp27-88fp-2phg (CWE-942). < 4.5.6.
|
||||
- Flaw: `GlancesRestfulApi.__init__()` disables credentials only when `cors_origins == ["*"]`. Starlette treats any list that contains `"*"` as allow-all and reflects the request Origin with `Access-Control-Allow-Credentials: true`. A configuration such as `cors_origins=*,https://trusted` with `cors_credentials=true` therefore exposes authenticated API data to any website.
|
||||
- Fix `89085894`: a membership test (`"*" in cors_origins`) at two sites. The code change is 4 lines, plus a test config and tests.
|
||||
- Caveats: the vulnerable behaviour depends on a non-default operator configuration. The defaults are `cors_origins=["*"]` with `cors_credentials=False`. The vulnerable pin already has a guard. The same repo has newer command-injection advisories (GHSA-59fj-m2j6-hcxh, GHSA-73wf-9vmv-5pv9, GHSA-qcpp-8x79-hhp3). I passed over them because the injection quota was full when I reached them.
|
||||
- Confidence: medium. The diff is clear, but whether the vulnerability exists depends on configuration.
|
||||
|
||||
#### qwed-sympy-eval (Python, FastAPI). Injection continuation
|
||||
- Repo: https://github.com/QWED-AI/qwed-verification (Apache-2.0, about 30 MB, 59 stars, created 2025-12). It is small but a real product (PyPI `qwed`), not a training app.
|
||||
- Advisory: CVE-2026-55585, GHSA-q27q-98j4-9pfv. < 5.1.2.
|
||||
- Flaw: `verify_math()` reads `request.get("expression")`, applies a cosmetic regex, and calls SymPy `parse_expr()`, which uses `eval` internally, with no restricted `global_dict`. `_verify_item()` in batch.py does the same with `item.query` for MATH items.
|
||||
- Fix: PR #200. The fix commit `dc9d4db7` (child of the vulnerable SHA) and its merge `6066b68c` add src/qwed_new/core/safe_parser.py (`safe_parse_expr`, validation of the input's syntax tree plus restricted namespaces) and switch every `parse_expr` call in main.py, batch.py and verifier.py to it. The merge also changes a binary `.coverage` file.
|
||||
- Caveat: the fixed SHA is a merge commit. Its first parent is the vulnerable SHA. The direct parent/child alternative is `06801f68` → `dc9d4db7`.
|
||||
- Confidence: high. I read the diffs and the vulnerable route.
|
||||
|
||||
#### xinference-llama3-tool-eval (Python, FastAPI). Injection continuation
|
||||
- Repo: https://github.com/xorbitsai/inference (Apache-2.0, about 94 MB; Python plus a TS/JS web UI). This is a model-serving platform with a FastAPI OpenAI-compatible REST API.
|
||||
- Advisory: CVE-2026-61539, GHSA-x2rj-828p-hx9m. ≤ 2.5.0, fixed in 2.7.0.
|
||||
- Flaw: when a chat request carries `tools`, the Llama3 tool-call post-processing calls `eval()` on the raw model output, with empty globals, which is not a sandbox. The attacker controls that output through the prompt.
|
||||
- Fix `1b3d220f`: `json.loads`, with a fall back to `ast.literal_eval`, at both eval sites. It touches 4 files and the change is small.
|
||||
- Caveats: the tainted value reaches `eval` only through the model. There is no direct data-flow edge from the HTTP request to the eval argument, so the source-to-sink link is model-mediated. Exploitation needs a Llama3-family model served with the Transformers backend.
|
||||
- Confidence: medium. The diff and sinks are unambiguous. The medium rating reflects the model-mediated source.
|
||||
|
||||
#### pgadmin-maintenance-sqli (Python, Flask). Injection continuation (SQLi)
|
||||
- Repo: https://github.com/pgadmin-org/pgadmin4 (about 370 MB; Python/Flask plus a JS/React UI). **The LICENSE file says PostgreSQL Licence, which is OSI-approved. GitHub reports NOASSERTION.**
|
||||
- Advisory: CVE-2026-7815, GHSA-hp84-p2gq-6fvr. < 9.15. The advisory references the short SHA `cf53953d9`, which I resolved to the full SHA.
|
||||
- Flaw: the Maintenance tool's Jinja SQL template concatenates `vacuum_index_cleanup`, `vacuum_parallel`, `buffer_usage_limit` (inside double quotes) and `reindex_tablespace` (inside double quotes) into a VACUUM/ANALYZE/REINDEX statement. `create_maintenance_job()` renders it and runs `psql --command <query>` through `BatchProcess`. Injected SQL can reach `COPY ... TO PROGRAM`.
|
||||
- Fix `cf53953d`: `validate_maintenance_data()` allow-lists the four fields before rendering, and the template switches `reindex_tablespace` to `conn|qtIdent`. 4 files, 2 of them tests.
|
||||
- Caveats: the sink is a Jinja SQL template executed by an external `psql` process, not a DB-API call. The same repo has GHSA-j74f-g7vx-fh4x (OS command injection in Import/Export, published at the same time). I passed over it because this phase took SQLi only.
|
||||
- Confidence: high. I read the diff and the vulnerable route.
|
||||
|
||||
### JavaScript / TypeScript (Node servers)
|
||||
|
||||
Source list: `list-npm.tsv` (reviewed npm advisories, CWE-prefiltered at fetch time to 89/78/77/94/95/1336/943/90/918/601/489/215/942/295/798/321/259/1188/16/346/1004, published >= 2023-01-01, sorted by publication date descending). Lines 1-59 (2026-09-24 → 2026-08-17) filled the quota of 7. Lines 60-94 (2026-08-14 → 2026-07-31) were the extra phase (RULES.md): only config_secrets and untrusted_destination considered, until +1 of each was taken.
|
||||
|
||||
"Diff seen" = I read the patch hunks of the named fix commit through `gh api repos/O/R/commits/SHA`. Every SHA below was confirmed with `gh api repos/O/R/commits/SHA --jq .sha`. Every fixed commit was confirmed to be an ancestor of the default branch (`compare/<default>...<sha>` = `behind`) unless stated.
|
||||
|
||||
#### reactpress-findall-column-sqli (TypeScript, NestJS)
|
||||
- Repo: https://github.com/fecommunity/reactpress (MIT, about 50 MB, 757 stars). Server in `server/` (NestJS + TypeORM), Next.js client alongside.
|
||||
- Advisory: CVE-2026-61685, GHSA-wmw4-mw6x-6vfm. ≤ 3.6.0, fixed in 3.7.0.
|
||||
- Flaw: `findAll(queryParams)` destructures `page`, `pageSize`, `status` and loops over every remaining query key: `query.andWhere(\`article.${key} LIKE :${key}\`)`. TypeORM binds the value but the key goes into the SQL text as an identifier. The controller passes `@Query() queryParams` straight in. The same loop exists in comment, file, page and knowledge services.
|
||||
- Fix `78ecb70a`: new `server/src/utils/query-whitelist.util.ts` (`filterByWhitelist(entity, params)`), applied in all five services. The same commit also adds HTML sanitisation after markdown rendering, a `JwtAuthGuard` on one comment route and `starter.ts` changes (9 files total).
|
||||
- Confidence: high. I read the diff and the vulnerable `findAll()` and controller at the parent.
|
||||
- Caveat: five parallel sinks; the fix commit also carries unrelated XSS and auth hardening.
|
||||
|
||||
#### 9router-kiro-region-ssrf (JavaScript, Next.js API routes)
|
||||
- Repo: https://github.com/decolua/9router (MIT, about 16 MB, ~30k stars). Next.js app with API routes.
|
||||
- Advisory: CVE-2026-56678, GHSA-6mwv-4mrm-5p3m. ≤ 0.5.2 per advisory, fixed in 0.5.6.
|
||||
- Flaw: the route reads `{ apiKey, region }` from the JSON body and calls `kiroService.validateApiKey(apiKey, region || "us-east-1")`, which calls `listAvailableProfiles()`: `fetch(\`https://codewhisperer.${region}.amazonaws.com\`, { headers: { Authorization: \`Bearer ${accessToken}\` } })`. A region like `evil.host:8443#` moves the request to another host and forwards the key. The route also reflected the upstream error body.
|
||||
- Fix `126aa244`: `assertValidAwsRegion()` (regex `^[a-z]{2}-[a-z]+-\d{1,2}$`) in `constants/oauth.js`, called before every region-interpolated URL in `kiro.js` and `providers.js`; the route stops reflecting upstream errors. 4 files, ~30 lines.
|
||||
- Confidence: high. I read the diff and the vulnerable `listAvailableProfiles()` / `validateApiKey()`.
|
||||
- Caveat: the same function family builds `https://oidc.${region}.amazonaws.com/...` in four more places, all fixed by the same commit; the advisory's exploited path is the codewhisperer one. The repo has three more SSRF advisories and a hardcoded-JWT-secret advisory (GHSA-jphh-m39h-6gwx, CWE-798) that I did not take (one case per repo).
|
||||
|
||||
#### omniroute-acp-agent-rce (TypeScript, Next.js API routes)
|
||||
- Repo: https://github.com/diegosouzapw/OmniRoute (MIT; **very large: GitHub reports about 1 GB, 75 MB of TypeScript**). Default branch `release/v3.8.51`.
|
||||
- Advisory: CVE-2026-88062, GHSA-hf57-cqmx-p4gr. ≤ 3.8.50; the GHSA lists no patched version but references the fix commit.
|
||||
- Flaw: `POST /api/acp/agents` accepts `binary` and `versionCommand`, stores the custom agent and calls `refreshAgentCache()`. `detectAgent()` runs `execFileSync(probe.command, probe.args)`. `resolveVersionProbe()` only checks that the first token equals the (also attacker-supplied) `binary` and that there are no shell metacharacters, so `binary: "node"`, `versionCommand: "node -e ..."` runs arbitrary code.
|
||||
- Fix `60829241`: in `resolveVersionProbe()`, untrusted probes may only carry a single recognised version flag (`SAFE_VERSION_PROBE_ARG`). The same commit closes three other advisories (db-backups auth path, uppercase path authz bypass, a spawn-capable prefix), 10 files.
|
||||
- Privilege: unauthenticated when `requireLogin=false` or during first-run bootstrap; a management session otherwise.
|
||||
- Confidence: medium. The diff and the vulnerable sink are clear; the vulnerable pin already has a guard (metacharacter filter and binary match), the fix bundles four advisories, and the repo size is a practical concern.
|
||||
|
||||
#### cct-studio-server-cmdi (JavaScript, Express)
|
||||
- Repo: https://github.com/davila7/claude-code-templates (MIT, about 250 MB, mostly template content in many languages). The Express server is `cli-tool/src/sandbox-server.js`, started by `npx claude-code-templates --studio`.
|
||||
- Advisory: CVE-2026-73222, GHSA-79wm-x847-7cvg. ≤ 1.29.2, fixed in 1.29.4.
|
||||
- Flaw: two POST routes pass body fields to `child_process.spawn(cmd, args, { shell: true })`: `prompt` (via `finalPrompt` in `executeLocalTask()`) and `agentName` (`/api/install-agent` and `checkAndInstallAgent()`). With `shell: true` the args are joined and parsed by `sh -c`. The server listens on all interfaces and sends `Access-Control-Allow-Origin: *`.
|
||||
- Fix `bc4618b0` ("Merge commit from fork", single parent): drops `shell: true`, adds a slug regex for `agentName`, restricts CORS to the localhost UI origin with a 403 for other origins, binds to 127.0.0.1. One source file (+57/-17) plus version bumps and CHANGELOG.
|
||||
- Confidence: high. I read the diff; the vulnerable code is visible in the removed lines.
|
||||
- Caveat: this is a developer-local server shipped in a CLI package, not a deployed web app. The same fix also removes a wildcard CORS header, so the case carries a secondary config_secrets aspect.
|
||||
|
||||
#### unleash-addon-webhook-ssrf (TypeScript, Express)
|
||||
- Repo: https://github.com/Unleash/unleash (AGPL-3.0, about 313 MB).
|
||||
- Advisory: CVE-2026-63004, GHSA-5vf6-jrqr-78fj. < 7.5.2, 7.6.0–7.6.4, 8.0.0–8.0.1.
|
||||
- Flaw: every addon (webhook, Slack, Teams, Datadog, New Relic) sends the configured `parameters.url` to `Addon.fetchRetry()`, which calls `ky(url, ...)` with no host checks. The webhook addon also forwards the configured `Authorization` header and custom headers. The request is blind, but success and status land in the integration-event log.
|
||||
- Fix `2100db76` (main, 8.x): new `src/lib/addons/validate-url.ts` (private/loopback/link-local/metadata deny by default, allow-list, http/https only) and `fetchPinned()` with a pinned DNS lookup and `redirect: 'manual'` in `addon.ts`. The advisory also lists two backports, `d45f99df` and `d862562a` (7.x lines), which are not ancestors of main.
|
||||
- Confidence: high. I read the addon.ts diff and the vulnerable `fetchRetry()` / webhook call site at the parent.
|
||||
- Caveat: stored input; the request fires later on a feature event, not in the request that sets the URL.
|
||||
|
||||
#### nocobase-backup-restore-cmdi (TypeScript, Koa)
|
||||
- Repo: https://github.com/nocobase/nocobase (**repo licence: "NocoBase License Agreement", non-OSI, GitHub NOASSERTION; the `plugin-backups` package.json declares Apache-2.0**; about 358 MB).
|
||||
- Advisory: CVE-2026-55410, GHSA-p853-83gj-wjj3. `@nocobase/plugin-backups` < 2.1.19.
|
||||
- Flaw: `PostgresAdapter.restore()` builds `${restoreCmd} -U ... -n ${srcSchema} -d ${database} ...` where `srcSchema` is the `database.schema` value read from `_metadata.json` in the uploaded backup, and runs it through the module-level `run()` → `promisify(exec)`. `#restoreSchema()` also puts `srcSchema` into SQL. Reached when the restore action is called with `force`.
|
||||
- Fix: the advisory references `0e1aba1b7b112ffc841588963f7343c00edd9806` (switch from exec strings to spawn argument arrays). **At that commit's parent the schema value is already validated**: `aecc6758` ("reject unsafe PostgreSQL schema names", 2026-06-12, 11 lines) added `assertPostgresSchemaIdentifier()` for `schema` and `schemaOption` in `restore()`. I use `aecc6758` and its parent as the pair because it is the commit that closes the reported path. `0e1aba1b` (parent `3f4ec12fc60157ea15c41ea3f696f0ddb7366e7c`) is the follow-up hardening.
|
||||
- Confidence: medium. I read both diffs and the vulnerable `restore()` / `run()` at `a89b1ea0`. Medium because the advisory's own fix reference points at the later commit, and because of the licence.
|
||||
- Caveat: sink input comes from a file inside an uploaded archive, not directly from a request field. The same repo has another advisory (GHSA-p849-8hwh-84j9, SQLi, 2026-07-31) that I did not take (one case per repo, and it came up in the extra phase).
|
||||
|
||||
#### medplum-external-auth-open-redirect (TypeScript, Express)
|
||||
- Repo: https://github.com/medplum/medplum (Apache-2.0, **large: about 440 MB**, monorepo; the server is `packages/server`).
|
||||
- Advisory: CVE-2026-53728, GHSA-m44r-7c5h-m6mj. `@medplum/core` ≤ 5.1.5 (the flaw is in `packages/server`), fixed in 5.1.6.
|
||||
- Flaw: `externalCallbackHandler()` parses `req.query.state` as JSON and calls `getClientRedirectUri(client, body.redirectUri, true)`. With `allowPartial` the function returns the requested URI whenever it `startsWith` a registered one (`http://cb.example` accepts `http://cb.example.evil.com/`). The exact-match check afterwards only logs. The handler then appends `login` and `code` and calls `res.redirect()`.
|
||||
- Fix `7ae10035`: exact match first; partial match only when the project sets `allow-dangerous-redirect`, and then only same origin with a path prefix (`isAllowedPartialRedirectUri`). Server change is ~40 lines in two files; the rest of the 7 files are FHIR definitions, docs and tests.
|
||||
- Confidence: high. I read the server diff and the vulnerable handler at the parent.
|
||||
- Caveat: the vulnerable pin already has a (prefix-match) guard.
|
||||
|
||||
#### flowise-ipv4-mapped-ssrf (TypeScript, Express) — extra, untrusted_destination
|
||||
- Repo: https://github.com/FlowiseAI/Flowise (Apache-2.0 for everything outside `packages/server/src/enterprise`, commercial licence inside it; GitHub NOASSERTION; about 83 MB).
|
||||
- Advisory: CVE-2026-69257, GHSA-c6xh-wv4j-ppv5. ≤ 3.1.2, fixed in 3.1.3. Bypass of the earlier fix for CVE-2026-31829.
|
||||
- Flaw: `isDeniedIP()` compares the resolved address to CIDR deny entries only when `parsedIp.kind() === parsedRange.kind()`. An IPv4-mapped IPv6 address (`::ffff:169.254.169.254`) has kind `ipv6`, so every IPv4 deny rule is skipped and `secureFetch()` / `secureAxiosRequest()` go ahead.
|
||||
- Fix `0fc76920`: normalise IPv4-mapped addresses (and deny entries) to IPv4 before matching, strip IPv6 brackets in `checkDenyList()` and `resolveAndValidate()`. One source file (+54/-9) plus a new test file.
|
||||
- Confidence: medium. I read the diff and the vulnerable `isDeniedIP()`. The flaw is in the guard itself; the request call sites are in many node implementations and I did not trace a specific one.
|
||||
- Caveat: vulnerable pin already has a guard; exploitation needs attacker-controlled DNS (AAAA record).
|
||||
|
||||
#### nx-graph-server-wildcard-cors (TypeScript, Node http) — extra, config_secrets
|
||||
- Repo: https://github.com/nrwl/nx (MIT; **very large: about 1 GB**).
|
||||
- Advisory: CVE-2026-54753, GHSA-g2r8-wvmj-jf5w. 17.0.4 – < 22.7.2 and 23.0.0-beta.0 – beta.1.
|
||||
- Flaw: the `nx graph` local server (`http.createServer` in `startServer()`) set `Access-Control-Allow-Origin: *` on every response, so any page could read the project graph and the output of `/help`, which runs a target's configured help command with `execSync`.
|
||||
- Fix: the advisory lists two identical 2-line commits. `e59122c5` is on master (parent `6ac9f82b`); `7620c1ff` (parent `0591aa5f`) is the 22.x backport and is not an ancestor of master. I use the master pair.
|
||||
- Confidence: high for the diff (2 lines removed).
|
||||
- Caveat: developer-local server inside a CLI, not a deployed web app. Huge repo.
|
||||
|
||||
#### budibase-mongo-template-nosqli (TypeScript, Koa) — SQLi/NoSQLi continuation phase
|
||||
- Repo: https://github.com/Budibase/budibase (GPL-3.0 per root LICENSE and `packages/server/package.json`; GitHub NOASSERTION; **very large, about 790 MB**; also contains a `pro` package).
|
||||
- Advisory: GHSA-pmpg-2mxq-6xwr (no CVE, published 2026-07-24). `@budibase/server` ≤ 3.38.1; release 3.39.9 referenced. GHSA-qw6m-8fw2-2v64 describes the same flaw through `POST /api/v2/queries/:queryId` and references the PR #18907 merge commit `2d6c1d17`.
|
||||
- Flaw: `enrichContext()` runs every string field of a saved query through handlebars with `noEscaping: true`, then `JSON.parse`s the `json`/`customData`/`requestBody` result. A binding value containing `"` and `{` breaks out of its JSON string and adds MongoDB operators (e.g. `$ne`), so a BASIC user reads documents the builder's filter was meant to hide. `MongoIntegration.read()` passes the parsed object to `collection.find(json)`. `validateQueryInputs()` in the query controller blocks only `{{ }}` in parameter values.
|
||||
- Fix `dd8c0654` ("Fix JSON template binding handling", on the PR branch, ancestor of master): new `processJsonStringSync()` in packages/string-templates that substitutes values in JSON-aware form, used by `enrichContext()` for the three JSON fields.
|
||||
- Confidence: medium. I read the diff and the vulnerable `enrichContext()` and `read()` at the parent. Medium because master already had an interim escaping helper (`jsonEscapeForContext`) when PR #18907 was merged; the merge `2d6c1d17` replaced it. The pair `b7e63df7` → `dd8c0654` is on the feature branch, where the parent has no escaping at all. Use that pair, not the merge.
|
||||
- Caveat: the sink is a datasource integration; the injected structure goes into a MongoDB query, not SQL.
|
||||
|
||||
#### fuxa-tdengine-escape-sqli (JavaScript, Express/Socket.IO) — SQLi/NoSQLi continuation phase
|
||||
- Repo: https://github.com/frangoteam/FUXA (MIT; SCADA/HMI web server; **large: about 600 MB**, mostly the Angular client and assets).
|
||||
- Advisory: CVE-2026-47720, GHSA-h9fj-c2qr-76g2. `fuxa-server` ≤ 1.1.14-1243 per GHSA (no patched package version); the text says fixed in 1.3.2. **The advisory references no commit.** I found the fix from the path history of the connector: `b78f6159` ("fix: tdengine string escaping (#2343)", 2026-05-03). It is in tag v1.3.2 and not in v1.3.1.
|
||||
- Flaw: `escapeTdString()` only doubles single quotes. TDengine treats `\'` as an escaped quote, so a tag id like `x\' OR 1=1--` closes the string and appends SQL in `getDaqValue()`'s SELECT. The ids come from `query.sids` in `GET /api/daq` (guest-level `secureFnc`) or the Socket.IO `DAQ_QUERY` handler, which has no authorization check.
|
||||
- Fix `b78f6159`: also escape backslashes (one line), plus a unit test.
|
||||
- Confidence: medium. I read the diff and the vulnerable connector and DAQ route at the parent. Medium because I inferred the fix commit from history rather than an advisory reference, and because the path only exists when TDengine is the configured DAQ backend.
|
||||
- Caveat: the vulnerable pin already has a guard (incomplete escaping). An earlier commit `5b59296c` (2026-03-08, "parameterize queries") introduced `escapeTdString`; before it the connector concatenated raw values.
|
||||
|
||||
### PHP web applications (not WordPress)
|
||||
|
||||
Source list: `list-composer.tsv` (GitHub Advisory Database, ecosystem composer, reviewed, CWE-prefiltered to the three families, published >= 2023-01-01, newest first). The quota of 4 was filled at the 5th advisory examined (2026-09-23). After that I continued newest first only for the extra `untrusted_destination` and `config_secrets` slots (RULES.md), which gave one extra of each.
|
||||
|
||||
#### contao-feed-reader-ssrf (PHP, Symfony-based CMS)
|
||||
- Repo: https://github.com/contao/contao (LGPL-3.0; monorepo, about 150 MB, PHP 10 MB, JS 0.2 MB, Twig 0.5 MB).
|
||||
- Advisory: CVE-2026-57232, GHSA-87mg-5grr-rhwh. Affects 5.3.35–5.3.47 and 5.4.0–5.7.8.
|
||||
- Flaw: the Feed Reader front-end module splits the module's `rss_feed` text into URLs and passes each to `$this->feedIo->read($url, new Feed())`. The FeedIo adapter used the plain `@psr18.http_client`, with no scheme check and no private-network block, so loopback, RFC1918 and cloud metadata hosts are fetched.
|
||||
- Fix `27f62018` (5.7 line; the 5.3 backport is `53b939ff2c4718e3a1d7c54ddd8886e9370618e4`, parent `96b216faa19cee9a3a21a103300b7c85d8f759d5`): new `core-bundle/src/HttpClient/NoPrivateNetworkExceptRootPagesHttpClient.php` (Symfony `NoPrivateNetworkHttpClient` variant that allows the site's own root pages), and `services.yaml` wires `contao.feed.http_adapter` and the crawler to it.
|
||||
- Caveat: **the sink file is not in the diff.** The fix is a dependency-injection rewiring in YAML plus a new decorator class. `FeedReaderController.php` is byte-identical before and after. The fixed revision is only distinguishable by the service configuration.
|
||||
- Confidence: high. I read the diff and the sink at the vulnerable pin. The source is stored configuration written by a back-end user, not a request parameter at the sink.
|
||||
|
||||
#### formie-form-settings-ssrf (PHP, Craft CMS plugin)
|
||||
- Repo: https://github.com/verbb/formie (about 196 MB; PHP 4.8 MB, JS 0.6 MB, TS small). **License: Craft License (proprietary, commercial); not OSI.** Must be cleared before publishing any result.
|
||||
- Advisory: CVE-2026-76086, GHSA-v3f3-cmj4-cvj9. Incomplete remediation of GHSA-cvpc-hccg-wmw4 (the `form-settings` action was left out of the 3.1.28 permission gate).
|
||||
- Flaw: `actionFormSettings()` had no permission check and called `$integration->setAttributes($settings, false)` with the request's `settings` array (unsafe mass assignment). An attacker can overwrite `apiUrl` (and similar host properties) of a configured integration. `getFormSettings()` then makes the integration call its API through `defineClient()`/`request()`, sending stored credentials to the attacker-chosen host and returning the response body in JSON.
|
||||
- Fix `6735fe4a` (Craft 5 line; Craft 4 line is `dde7799dfa7e4d0a11e28754ad8544dba62d5def`): `requireCpRequest()`, a per-form integration permission check, and an allowlist filter that drops keys containing `url`, `host`, `key`, `token`, etc.
|
||||
- Caveat: the outbound request is several dynamic-dispatch hops away from the changed file (controller → integration subclass `defineClient()` → base `Integration::request()` → Guzzle). The sink is not in the diff. The case is equally a missing-authorization / mass-assignment bug.
|
||||
- Confidence: medium. Diff read and the concrete ActiveCampaign sink checked at the vulnerable pin. The weakness is the licence and the indirect sink.
|
||||
|
||||
#### sulu-smart-content-sql2-injection (PHP, Symfony CMS)
|
||||
- Repo: https://github.com/sulu/sulu (MIT; about 165 MB, PHP 11 MB, JS 5 MB admin UI). Default branch is 3.x; this fix is on the 2.6 line.
|
||||
- Advisory: CVE-2026-92692, GHSA-jg26-q8hg-3pq4. < 2.6.25 and 3.0.0–3.0.7.
|
||||
- Flaw: `CategoryRequestHandler::getCategories()` reads `?categories=`, explodes on commas and trims. `ContentType` passes the list as `websiteCategories`. `QueryBuilder::buildWhere()` → `buildCategoriesWhere()` appends `'page.[' . $name . '] = ' . $category` to a JCR-SQL2 WHERE clause. The tag filter already validated numbers; the category and target-group paths did not.
|
||||
- Fix `d19c0148`: three `(int)` casts in QueryBuilder.php plus a new unit test. The diff is 3 changed lines of code.
|
||||
- Caveat: the query language is JCR-SQL2 (PHPCR content repository, executed through Jackalope), not SQL sent to the RDBMS. The impact is read/enumeration of content nodes and DoS. Record it as query-language injection, not classic SQLi. 3.0.8 has a separate fix commit (not examined).
|
||||
- Confidence: high. I read the diff, the sink, and the source-to-sink path at the vulnerable pin.
|
||||
|
||||
#### freeform-attribute-twig-ssti (PHP, Craft CMS plugin)
|
||||
- Repo: https://github.com/solspace/craft-freeform (about 100 MB, PHP 6 MB). **License: Craft License (proprietary); not OSI.**
|
||||
- Advisory: CVE-2026-73858, GHSA-gxrg-x694-283w. 5.0.0–5.10.13. "Limited" SSTI: the isolated Twig environment only exposed constants such as `PHP_VERSION` and `CRAFT_BASE_PATH`; RCE was not shown.
|
||||
- Flaw: `Attributes::toHtmlTagArray()` renders every attribute key and value through `IsolatedTwig::render()` with the form properties. When a submitted field value comes back as the `value` attribute, attacker text is evaluated as a Twig template.
|
||||
- Fix `5f755532` (PR #1986): skips Twig rendering for the `value`, `name` and `id` attributes. The diff is 5 lines. The commit's author date is 2025-05-13, but it only lands after v5.10.13 (compare: ahead of v5.10.13, behind v5.10.14).
|
||||
- Caveat: attribute keys and other attribute values are still rendered through Twig in the fixed version. It is a deny-list fix. I inferred the exact path from the submission to the `value` attribute from the advisory and did not trace it in the field classes.
|
||||
- Confidence: medium (licence, limited impact, source path partly inferred).
|
||||
|
||||
#### smw-uriresolver-open-redirect (PHP, MediaWiki extension), extra untrusted_destination slot
|
||||
- Repo: https://github.com/SemanticMediaWiki/SemanticMediaWiki (GPL-2.0-or-later per composer.json, GitHub says NOASSERTION; about 66 MB, JS 0.9 MB).
|
||||
- Advisory: CVE-2026-77609, GHSA-hw3m-8j5x-94ff. ≤ 7.1.0, fixed in 7.2.0. The advisory lists no commit. I found the fix through the history of `SpecialURIResolver.php` and confirmed it is between the 7.1.0 and 7.2.0 tags.
|
||||
- Flaw: `execute($query)` decodes the subpage, builds a Title with `TitleFactory::newFromText()` (interwiki prefixes such as `mw:` resolve off-site) and issues `$out->redirect($title->getFullURL(), '303')` with no host check.
|
||||
- Fix `765a6ac2`: new `isLocalRedirectTarget()` compares the target host with the server host at the sink and rejects userinfo. It shows a bad-title page otherwise.
|
||||
- Caveat: the redirect primitive (`OutputPage::redirect`) and URL building live in MediaWiki core, which is not in this repository. A checkout is an extension, not a runnable app.
|
||||
- Confidence: high. I read the diff and the vulnerable function.
|
||||
|
||||
#### pheditor-default-admin-password (PHP, single-file web file editor), extra config_secrets slot
|
||||
- Repo: https://github.com/pheditor/pheditor (MIT, 182 stars, one 97 KB PHP file, not archived).
|
||||
- Advisory: CVE-2026-55579, GHSA-p4h7-p9rj-2pq2. 2.0.1–2.0.5. The advisory lists no commit. The 2.0.5...2.0.6 compare has exactly one commit, `a64eb935`.
|
||||
- Flaw: `define('PASSWORD', ...)` holds the unsalted SHA-512 of `admin`. The login block accepts it and nothing forces a change, so a default install gives unauthenticated access to the editor, upload and terminal.
|
||||
- Fix `a64eb935`: when `PASSWORD` still equals `hash('sha512','admin')`, login shows a forced "change password" form that rewrites the `define` line in the file. The same commit also tightens the terminal command filter (`|`, backtick, newlines), which is the fix for sibling advisory GHSA-wg4w-wr5q-6vjc (command injection).
|
||||
- Caveat: **the hardcoded default hash is still present, unchanged, on line 11 of the fixed file.** The fix is a behavioural gate, not removal of the constant, so the fixed revision still contains the same literal. The commit also mixes in an unrelated command-injection fix. Ground truth for "fixed = clean" is therefore doubtful for this family.
|
||||
- Confidence: low (see caveat). Kept because it is the first config_secrets advisory newest-first. The next config_secrets candidate in the list, not examined because the slot was filled, is Kimai GHSA-jr9p-4h4j-6c58 (default `APP_SECRET` in the Docker image, CWE-1188).
|
||||
|
||||
#### pimcore-studio-datefilter-sqli (PHP, Symfony bundle, SQLi continuation phase)
|
||||
- Repo: https://github.com/pimcore/studio-backend-bundle (about 10 MB, PHP 7.9 MB). It is the REST backend of Pimcore Studio, a Symfony bundle installed into a Pimcore application, so a checkout is not a runnable app by itself. **License: Pimcore Open Core License (POCL, June 2025), not OSI.**
|
||||
- Advisory: CVE-2026-55208, GHSA-79cw-hfcc-7mw9. < 2025.4.6 and 2026.1.0–2026.1.5. The advisory is filed under pimcore/pimcore, but the code and fix are in studio-backend-bundle.
|
||||
- Flaw: listing endpoints map a `columnFilters` array into `ColumnFilter` objects. `DateFilter::applyDateFilter()` wraps the user-supplied `key` in literal backticks and concatenates it into the condition string. Because the bound parameters are the fixed names `:minTime`/`:maxTime`, a backtick in the key breaks out and allows time-based blind SQLi. EqualsFilter, LikeFilter and the Note FilterService share the pattern (there the key also names the bound parameter, which limits exploitation).
|
||||
- Fix `f532428c` (PR #1883): all four sites use `$this->dbResolver->get()->quoteIdentifier(...)`. About 30 changed code lines plus tests.
|
||||
- Caveat: the SQL is executed in Pimcore core (`AbstractListing::addConditionParam()` → Doctrine DBAL), which is not in this repository. The sink in-repo is the string build handed to the listing object.
|
||||
- Confidence: high. Diff read and vulnerable pin checked.
|
||||
|
||||
#### froxlor-admin-ip-second-order-sqli (PHP, server-management panel, SQLi continuation phase)
|
||||
- Repo: https://github.com/froxlor/froxlor (LGPL-2.1 per GitHub; about 30 MB).
|
||||
- Advisory: CVE-2026-54348, GHSA-w27m-rmmf-g5w4. < 2.3.8.
|
||||
- Flaw: `Admins::add()` / `update()` store any `ipaddress` array as `json_encode($ipaddress)` in `panel_admins.ip` (the write itself is a prepared statement). Later, `IpsAndPorts::listing()` and `listingCount()` run `implode(", ", json_decode($this->getUserDetail('ip'), true))` into `WHERE id IN (...)`, and that string reaches `Database::prepare()`. `Domains::validateIpAddresses()` does the same.
|
||||
- Fix `a1eaca5a` ("verify numeric values for ipaddress-id's"): `array_filter(..., 'is_numeric')` and `array_map('intval', ...)` at the store and at both read sites. 3 files, about 20 lines.
|
||||
- Caveat: second-order. The source (API parameter) and sink (a later API call by the poisoned account) are in different requests, joined through the database. The attacker must already be an admin with an API key.
|
||||
- Confidence: high. Diff read, and the sink checked at the vulnerable pin.
|
||||
|
||||
### WordPress plugins (PHP+JS)
|
||||
|
||||
#### wp-s2member-tracking-eval (PHP+JS, WordPress)
|
||||
- Repo: https://github.com/wpsharks/s2member (PHP 4.0 MB, JS 0.15 MB; about 28 MB of git data). Default branch `dev`. It has a git submodule `phings` (build tooling only).
|
||||
- Advisory: CVE-2026-19804, GHSA-fgx2-6hhq-hwhw (Wordfence, unreviewed GHSA). "Up to and including 260814". Unauthenticated RCE.
|
||||
- Flaw: the PayPal return handler replaces `%%first_name%%` (and last/full name) in the admin's Signup Tracking Codes template with the payer's name. The only escaping is `esc_refs()`, which strips regex back-references, not PHP tags. The result is saved in a transient. `display_signup_tracking_codes()` (on `wp_footer`/`login_footer`) reads the transient and runs it through `c_ws_plugin__s2member_utilities::evl()`, which does `eval('?>'.trim($code))`. The same pattern is in `display_modification_tracking_codes()`, `display_ccap_tracking_codes()` and `display_sp_tracking_codes()` (the last one calls `eval("?>".$code)` directly).
|
||||
- Fix `6c622839` ("Update notification template handling", 9 files, small): removes the `evl()`/`eval` calls in all four display functions and echoes the code through `do_shortcode()` behind a new filter. Related template-handling changes in the admin menu pages and PayPal notify handlers are part of the same commit.
|
||||
- Caveats: the RCE chain also needs the site-global proxy verification key, which the advisory says leaked in PayPal Checkout AJAX responses. That leak was addressed separately (probably `37c29337d9256bf17a2bd115ac1d16bf07923061` "Fix PayPal proxy request handling", which I did not read). The advisory's version bound (≤ 260814) does not match the GitHub history, where the eval removal landed on 2026-09-14 and shipped in v260917. The source-to-sink path crosses a WordPress transient (second-order). Confidence medium because of the version mismatch and the two-part chain. The eval sink and its removal are unambiguous.
|
||||
|
||||
#### wp-directorist-avatar-ssrf (PHP+JS, WordPress)
|
||||
- Repo: https://github.com/sovware/directorist (JS 21 MB of built assets, PHP 4.8 MB). Default branch `trunk`. Tags v8.9.4 = `d0fbb4f8c63582d43c0860e9add24d95b04e4a43`, v8.9.5 = `a3aa54a9a58e3c6d7784302dbe48a16ddcc14712` (101 commits apart, so use the commit pair).
|
||||
- Advisory: CVE-2026-84046, GHSA-6q63-3w56-m722 (WPScan). < 8.9.5, subscriber and above.
|
||||
- Flaw: the REST users controller passes `$request['avater']['src']` to `directorist_rest_upload_image_from_url()`, which calls WordPress `download_url($image_url)` after only `wp_parse_url`/`esc_url_raw`. The same helper is also called from the categories, locations and tools code paths.
|
||||
- Fix `1a67825d` ("readme update & security fix", 7 files): adds `directorist_rest_validate_remote_image_url()` (http/https only, rejects localhost, resolves the host and rejects private or reserved ranges) and forces `reject_unsafe_urls` on the download. The same commit also tightens permission checks in the orders, users and favorites controllers.
|
||||
- Caveat: WordPress core's `download_url()` already uses `wp_safe_remote_get()`, which blocks 127/8, 10/8, 172.16/12 and 192.168/16 but not 169.254/16 (cloud metadata). The practical exposure is therefore narrower than the advisory text suggests. Confidence medium for that reason. The sink and the fix are clear.
|
||||
|
||||
#### wp-gopay-log-filter-sqli (PHP+JS, WordPress/WooCommerce)
|
||||
- Repo: https://github.com/gopaycommunity/gopay-woocommerce-integration (small: PHP 0.14 MB; the plugin ships a committed `vendor/` with the GoPay SDK, but the sink is in the plugin's own code). Plugin slug `gopay-gateway`.
|
||||
- Advisory: CVE-2026-75959, GHSA-88c4-2f34-57h5 (Wordfence). ≤ 1.0.36 per advisory, shop manager and above.
|
||||
- Flaw: admin/views/log.php reads `filter_input(INPUT_GET, 'log_table_filter')` and puts `strtoupper()` of it into `sprintf("... REGEXP '[\w\W]*%s[\w\W]*' ...")` for both the COUNT and the page query, without escaping. The view is included from the admin class's log submenu page (capability `manage_woocommerce`).
|
||||
- Fix `7190c985` ("fix authenticated SQL injection in log admin view (CVE-2026-75959)"): one file. Both queries become `$wpdb->prepare(... LIKE %s ...)` with `$wpdb->esc_like()`.
|
||||
- Caveats: the sink is at file scope in a view template, not inside a function. The advisory says ≤ 1.0.36, but the fix commit precedes the 1.0.35 bump (`329113ac`). The commit message names the CVE, so I trust the commit. Confidence high.
|
||||
|
||||
#### wp-user-registration-login-redirect (PHP+JS/TS, WordPress)
|
||||
- Repo: https://github.com/themegrill/user-registration (PHP 4.9 MB, JS 2.5 MB, TS 0.2 MB; about 75 MB). Tags 5.2.7 = `c532e6aee2efc73ab711bdb6f8b36a06674e5d7d`, 5.2.8 = `28e2e931c3c0b13a8e1b577d56d6224c494fdae0`. That pair is 360 commits and 117 files apart, so use the commit pair.
|
||||
- Advisory: CVE-2026-80072, GHSA-w78c-fx7q-9x8m (WPScan). < 5.2.8, unauthenticated.
|
||||
- Flaw: `ur_process_login()` takes `$redirect = esc_url_raw(wp_unslash($post['redirect']))` (or the raw referer), then calls `wp_redirect(wp_validate_redirect($redirect, $redirect))`. The fallback equals the input, so validation is a no-op. On the AJAX-login path it returns `$redirect` in JSON, and the front-end JS navigates to it. The failed-login branch also redirects to `wp_get_raw_referer()` with `wp_redirect`.
|
||||
- Fix `75c940da` ("#1408 Fix - Prevent membership privilege escalation and open redirect after login", 8 files): `wp_validate_redirect($redirect, get_home_url())` before both branches, `wp_safe_redirect` in both success and failure, and an `allowed_redirect_hosts` filter for the admin-configured external URL. The same commit also fixes a membership privilege escalation in modules/membership. Those hunks are separate. A later commit, `e537e78738aff363b5f250253d2b64bf9dc79c80` "Fix - Open redirect after login via unvalidated redirect parameter (#1417)", has an empty diff against its parent (already applied).
|
||||
- Confidence high: the diff is readable and the no-op validation is visible at the vulnerable pin.
|
||||
|
||||
#### wp-groundhogg-confirm-redirect (PHP+JS, WordPress)
|
||||
- Repo: https://github.com/groundhoggwp/groundhogg (PHP 4.8 MB, JS 2.0 MB). Tags 4.7.1 = `3cc86bfa09e8bb85cbf15f7c8affa727c6b65961`, 4.7.2 = `0f19a08505d0bd0ec27dd4668ac39f00ad3d0d5d` (17 commits).
|
||||
- Advisory: CVE-2026-81741, GHSA-m96f-c5vh-h7cm (WPScan). < 4.7.2, unauthenticated.
|
||||
- Flaw: in the email-preferences template, `case 'confirm':` sets `$redirect_to = esc_url_raw(sanitize_text_field(get_url_var('redirect_to')))`, passes it through a filter and calls `wp_redirect($redirect_to)` (with a phpcs ignore comment for SafeRedirect).
|
||||
- Fix `028247d4` ("use safe redirect"): one line, `wp_redirect` → `wp_safe_redirect`. Its child `2f03c848` only removes the phpcs comment.
|
||||
- Caveats: the sink is at file scope in a template. The preferences page needs a contact context (the tracking/permissions link from an email), which the advisory treats as a crafted link. Confidence high. There is no LICENSE file in the repo, but the plugin header says GPLv3.
|
||||
|
||||
#### wp-stripe-payments-thankyou-redirect (PHP+JS, WordPress), extra untrusted_destination pick
|
||||
- Repo: https://github.com/Arsenal21/stripe-payments (PHP 1.5 MB, JS 0.13 MB). The plugin lives in the `stripe-payments/` subdirectory.
|
||||
- Advisory: CVE-2026-81423, GHSA-rvj6-j43w-hcgw (WPScan). < 2.1.4, unauthenticated.
|
||||
- Flaw: `process_ipn()` (hooked on `wp_loaded` when POST `asp_process_ipn` is set) takes POST `asp_thankyou_page_url`, base64-decodes it into `$this->asp_redirect_url`, and `ipn_completed()` ends with `wp_redirect($this->asp_redirect_url)`.
|
||||
- Fix `fb1f647d` (2 files, 5 lines): `wp_validate_redirect(..., home_url('/'))` plus `wp_safe_redirect`.
|
||||
- Caveats: `process_ipn()` first checks the `asp_payment_form_nonce` (a front-end nonce on any page with a payment form), and `ipn_completed()` runs after payment processing (or on the "already processed" branch). The attacker needs a victim to submit such a request. Confidence medium, because of those preconditions. The sink and the fix are unambiguous. Taken as the "+1 untrusted_destination" extra after the 5-case quota, per RULES.md.
|
||||
|
||||
## Open decisions for the maintainer
|
||||
- **Licences that are not OSI or not verified.** These must be resolved before any result is published:
|
||||
- Not OSI: Open WebUI (#4, Open WebUI License), NocoBase (#18, NocoBase License), Formie and Freeform (#25, #27, proprietary Craft License), Pimcore Studio (#30, Pimcore Open Core License). LiteLLM (#3) and Flowise (#20) have enterprise directories under a commercial licence; the core is MIT or Apache.
|
||||
- GitHub reports NOASSERTION, but the repo's own files state an OSI licence: Glances (#9), pgAdmin (#12), Budibase (#22), Semantic MediaWiki (#28).
|
||||
- All six WordPress plugins (#32–#37) have no GitHub licence metadata and rely on the plugin-header GPL declaration. s2Member's header does not state a GPL version.
|
||||
- **Fix commits the advisory does not name (pair inferred from file history):** LMDeploy (#2), FUXA (#23), Semantic MediaWiki (#28). NocoBase (#18) uses an earlier commit than the one the advisory cites, because at the cited commit's parent the reported path was already closed.
|
||||
- **Fixed SHA is a merge commit:** qwed (#10). The PR commit `dc9d4db72ca4b4ae3f96d0e6a0c27a9e38a06f61` is a plain child of the vulnerable SHA if a non-merge pair is preferred.
|
||||
- **Weak ground truth:**
|
||||
- Pheditor (#29, low): the default hash is still in the fixed file.
|
||||
- xinference (#11): the input reaches `eval` only through model output.
|
||||
- s2Member (#32): the advisory's version bound does not match the history, and exploitation needs a second key leak.
|
||||
- Glances (#9) and Chainlit (#6): vulnerable only with a non-default configuration.
|
||||
- Directorist (#33): WordPress core already blocks most private ranges.
|
||||
- **Sink outside the repository or outside the changed files:**
|
||||
- The sink's final call is outside the repo: Semantic MediaWiki (#28, MediaWiki core), Pimcore Studio (#30, Pimcore core), Lemur (#8, the `acme` library).
|
||||
- The sink is outside the fix's changed files: LiteLLM (#3), Contao (#24).
|
||||
- **Scope edge cases:** winml-cli (#5), cct-studio (#16) and nx (#21) are servers started from a developer CLI, not deployed web apps. Lemur (#8) is archived.
|
||||
- **Guard bypasses (the vulnerable pin already has a check):** #1, #2, #4, #9, #15, #19, #20, #23.
|
||||
- **Very large repositories:** LiteLLM (about 1.7 GB), OmniRoute and nx (about 1 GB each), Budibase (about 790 MB), FUXA (about 600 MB), Open WebUI, ContextForge and medplum (about 440–450 MB).
|
||||
- **Family balance:** in strict newest-first order, recent advisories are dominated by SSRF. A selection of 12–15 with "the rest injection" will need most of the 16 injection rows. Taking fewer untrusted_destination rows is a selection choice; no case was excluded for being hard.
|
||||
|
||||
## Rejection log
|
||||
|
||||
Total: 261 advisories examined (Python 72, JS/TS 112, PHP 20, WordPress 57), 37 taken, 224 rejected. Each lane's log follows, one line per advisory. Advisories that were passed over after a lane's family slot filled are listed separately where the lane recorded them, and are not counted as examined.
|
||||
|
||||
### Python (Flask / Django / FastAPI)
|
||||
|
||||
- GHSA-mr95-65j8-9mxp / CVE-2026-57170 (oscal-compass/compliance-trestle, SSTI): library/CLI, not a web app.
|
||||
- GHSA-rr49-f9g6-c9r5 / CVE-2026-57149 (plone/plone.app.portlets, TALES injection): framework out of scope (Zope/Plone).
|
||||
- GHSA-4ghv-53cq-7wp3 / CVE-2026-91129 (home-assistant/core, SSRF): framework out of scope (aiohttp), and a huge repository.
|
||||
- GHSA-v9m3-wfh8-5646 / CVE-2026-77249 (sooperset/mcp-atlassian, SSRF): fix mixed with unrelated changes. The single squash `b0417334` (#1448, 34 files) fixes about ten advisories across the attachment, transport, SSRF, authorization, filter and OAuth layers. The named sink (jira/users.py `_lookup_user_by_permissions`) is not in the diff. The MCP transport is not Flask/Django/FastAPI.
|
||||
- GHSA-hgcf-4mq8-5266 / CVE-2026-77274 (mcp-atlassian, SSRF): same squash as above.
|
||||
- GHSA-6vmq-24h2-pj7j / CVE-2026-77271 (mcp-atlassian, path traversal to RCE): family out of scope (path traversal), same squash.
|
||||
- GHSA-5wf4-jqxh-8gm3 / CVE-2026-77267 (mcp-atlassian, SSRF): same squash as above.
|
||||
- GHSA-6529-c226-h328 / CVE-2026-77261 (mcp-atlassian, SSRF): same squash as above.
|
||||
- GHSA-49xv-9743-pw8w / CVE-2026-77265 (mcp-atlassian, SSRF): same squash as above.
|
||||
- GHSA-ch3g-4xvr-674q / CVE-2026-62282 (opencve/opencve, SSRF): repo in used list.
|
||||
- GHSA-82r6-8w77-94w6 / CVE-2026-63374 (agronholm/anyio, TLS verification): library, not a web app.
|
||||
- GHSA-rqfv-2mw9-78g2 / CVE-2026-59971 (designcomputer/mysql_mcp_server, CWE-306/346): family out of scope (missing Origin/Host validation, DNS rebinding against an MCP SSE transport, not a permissive-configuration setting). The MCP transport is not Flask/Django/FastAPI.
|
||||
- GHSA-34r3-9m95-vq73 / CVE-2026-87999 (open-webui, SSRF): repo already has a candidate.
|
||||
- GHSA-5x7x-4c3c-qf5w / CVE-2026-88001 (open-webui, SSRF through redirect): repo already has a candidate.
|
||||
- GHSA-jf6q-chmf-3h3v / CVE-2026-55073 (kozea/weasyprint, SSRF): library, not a web app.
|
||||
- GHSA-4hhp-h66f-j5j7 / CVE-2026-73560 (vllm-project/vllm, SSRF): the sink (`mimo_v2_omni.py` `_fetch_image`) is reached through the offline `LLM.generate(multi_modal_data=...)` API in the advisory. The HTTP server fetches media through `MediaConnector`, so this is a library path, not a web app.
|
||||
- GHSA-284h-m62q-gf8w / CVE-2026-78676 (gitpython-developers/gitpython, config injection): library, not a web app.
|
||||
- GHSA-6ww7-3frv-cqxh / CVE-2026-78682 (nltk/nltk, SSRF): library, not a web app.
|
||||
- GHSA-4mvj-m6j5-pmf7 / CVE-2026-71428 (Unstructured-IO/unstructured, SSRF): library, not a web app.
|
||||
- GHSA-jrrm-9hc7-2v3h / CVE-2026-62674 (omnigent-ai/omnigent, CWE-94): family out of scope. This is missing authorization on a shared-agent overwrite. Command execution comes from the intended stdio-MCP feature.
|
||||
- GHSA-756x-9hf6-q4h4 / CVE-2026-62675 (omnigent, CWE-94): family out of scope (uploaded agent bundle runs Python tools by design; authorization issue), and the repo was already examined.
|
||||
- GHSA-p3m8-78j2-g5p3 / CVE-2026-62388 (nltk, CWE-1188): library, not a web app.
|
||||
- GHSA-3gqm-fcw5-w839 / CVE-2026-63311 (nltk, SSRF fail-open): library, not a web app.
|
||||
- GHSA-p6gw-4frg-j7jw / CVE-2026-55509 (mar10/wsgidav, SQLi): framework out of scope (its own WSGI server, not Flask/Django/FastAPI).
|
||||
- GHSA-jw39-3688-r4rx / CVE-2026-54757 (compliance-trestle, SSTI): library/CLI, not a web app.
|
||||
- GHSA-6hx8-3wjj-gr8g / CVE-2026-54770 (Pylons/webob, open redirect): library, not a web app.
|
||||
- GHSA-6wvf-77m9-58rm / CVE-2026-37004 (BerriAI/litellm, SSTI in /prompts/test): repo already has a candidate.
|
||||
- GHSA-jrw6-7x4q-w25j / CVE-2026-54569 (senaite/senaite.core, eval injection): framework out of scope (Plone/Zope).
|
||||
- GHSA-mv8m-v9v6-5f94 / CVE-2026-54548 (siemens/kas, SSH host key checking disabled): library/CLI, not a web app.
|
||||
- GHSA-w3fx-mc44-mf6j / CVE-2026-45018 (Chainlit/chainlit, command injection): repo already has a candidate (published 2 minutes before GHSA-hvfh and fixed by the same commit).
|
||||
- GHSA-8cp3-qxj6-px34 (universal-tool-calling-protocol/python-utcp, SSRF): library, not a web app.
|
||||
- GHSA-ppx3-28rw-8fpf / CVE-2026-12210 (python-utcp, SSRF): library, not a web app.
|
||||
- GHSA-9qhg-99ww-9mqc (python-utcp, SSRF through redirect): library, not a web app.
|
||||
- GHSA-pvph-5j39-v8qc / CVE-2026-55532 (MervinPraison/PraisonAI, CWE-346/352): family out of scope (a prefix-match Origin check leading to CSRF on an MCP HTTP transport, not a permissive CORS setting).
|
||||
- GHSA-hmfx-4v44-9qw9 / CVE-2026-55535 (PraisonAI, SSRF by DNS rebinding in `webhook_url`): the fix is mixed with unrelated changes. The single commit `2f9677ab` ("harden input validation and query parameterization", 16 files) fixes about seven advisories across memory, tools, workflows, the jobs, browser and MCP servers. It only turns `gaierror` into a rejection. `executor.py` still re-resolves at send time, so the fixed tree does not remove the rebinding the advisory describes.
|
||||
- GHSA-rg5q-pp8p-f7jm / CVE-2026-55537 (PraisonAI, SSRF by DNS fail-open): same multi-advisory commit as above.
|
||||
- GHSA-vg6p-v9vm-6fgj / CVE-2026-55524 (PraisonAI `praisonaiagents` web_crawl, SSRF): library agent tool, not a web app; same commit.
|
||||
- GHSA-x44h-65qv-cw74 / CVE-2026-55526 (PraisonAI `praisonaiagents` spider_tools, SSRF): library agent tool, not a web app.
|
||||
- GHSA-wj6g-v78p-6fx3 / CVE-2026-55529 (PraisonAI MCP HTTP stream, origin validation): family out of scope (CSRF/origin).
|
||||
- GHSA-8hjw-25cg-g52h / CVE-2026-55523 (PraisonAI web_crawl, SSRF): library agent tool, not a web app.
|
||||
- GHSA-5r34-2g38-6569 / CVE-2026-55525 (PraisonAI web_crawl, SSRF): library agent tool, not a web app.
|
||||
- GHSA-crp9-r7rq-c8cg (nltk): duplicate advisory.
|
||||
- GHSA-qg9p-xrhj-435m (nltk): duplicate advisory.
|
||||
- GHSA-8vh5-mgjj-w6hg (nltk): duplicate advisory.
|
||||
- GHSA-5p3m-vhh6-9236 (eidetic-labs/stigmem, SSRF): negligible real-world use (4 stars, repo created 2026-05, alpha release 0.9.0a11). I treated it as a toy project.
|
||||
- GHSA-p77j-g7h5-r2vw (geolens-io/geolens, 9 CWEs incl. 918): family out of scope (the advisory is mainly authorization and cache-scope data disclosure).
|
||||
- GHSA-f3qq-49m6-rw8f / CVE-2026-70667 (Netflix/lemur, SSRF): repo already has a candidate.
|
||||
- GHSA-xpmj-wjcp-6pww / CVE-2026-70666 (Netflix/lemur, SSRF): repo already has a candidate.
|
||||
|
||||
Injection-only continuation (examined):
|
||||
- GHSA-mw6r-2hvm-4rp2 / CVE-2026-55546 (QWED-AI/qwed-mcp, SymPy eval): MCP stdio server, not a Flask/Django/FastAPI web app.
|
||||
- GHSA-hxmv-c4g6-5fqc / CVE-2026-55522 (MervinPraison/PraisonAI, tools.py autoload in workflow include): library workflow feature, not a web app. The fix is the same multi-advisory commit `2f9677ab`.
|
||||
- GHSA-xm98-3vcf-fph7 / CVE-2026-53710 (IBM/mcp-context-forge, RestrictedPython sandbox bypass): repo already has a candidate.
|
||||
- GHSA-2cp2-2r3c-7p7r / CVE-2026-68508 (hydra-ecosystem/hydra, hydra.utils.instantiate): library, not a web app.
|
||||
|
||||
SQLi-only continuation (CWE-89, pip, newest first from 2026-08-21):
|
||||
- GHSA-7xw9-549r-8jrc (DIRACGrid/DIRAC, SQLi in PilotManager): framework out of scope (DIRAC's own DISET/Tornado RPC services).
|
||||
- GHSA-m4m7-4cw8-62j6 / CVE-2026-61667 (DIRAC, SQLi + eval in FileCatalog): framework out of scope (same repo, DISET services).
|
||||
- GHSA-6xc5-4r68-67fc / CVE-2026-54760 (langroid/langroid, SQLChatAgent blocklist bypass): library, not a web app.
|
||||
- GHSA-pmch-g965-grmr / CVE-2026-50180 (langroid, SQLChatAgent blocklist): library, not a web app.
|
||||
- GHSA-pqrj-4gwg-h5f7 / CVE-2025-66336 (apache/doris-mcp-server, SQLi): MCP server, not a Flask/Django/FastAPI web app.
|
||||
- GHSA-9pc9-4crj-mhpj (eidetic-labs/stigmem, identifier quoting): negligible real-world use (toy project, see above). The advisory also describes defensive hardening rather than a demonstrated injection.
|
||||
- GHSA-82m5-3pcp-hccq / CVE-2026-10105 (agno-agi/agno, ClickHouse `delete_by_metadata`): library vector-DB backend, not a web app. There is no patched version, and the fix sits in PR #7883 with no merged commit reference.
|
||||
- GHSA-mxfr-6hcw-j9rq / CVE-2026-25879 (langroid, prompt-to-SQL): library, not a web app.
|
||||
|
||||
Passed over in the continuation phase and not examined, because their family quota was already met. Listed for completeness, not counted as examined:
|
||||
- Injection, older than the last row the injection continuation reached (GHSA-x2rj, 2026-08-21): GHSA-533j-2v4q-mw5h (langchain-mongodb), GHSA-w47q-945m-q9pc (document-merge-service), GHSA-9gmc-jqmh-3rvm (copier), GHSA-rghg-q7wp-9767 (MONAI), GHSA-59fj-m2j6-hcxh, GHSA-73wf-9vmv-5pv9 (glances), GHSA-3496-9g83-7v6x (sqlparse).
|
||||
- Non-SQLi injection passed over in the SQLi-only phase: GHSA-j74f-g7vx-fh4x (pgadmin4 OS command injection, same repo as the SQLi taken).
|
||||
- Untrusted destination, passed after Lemur filled the extra slot: GHSA-95px-34x5-p37h (MobSF), GHSA-7gwp-5pfp-969j (mlflow).
|
||||
|
||||
Counts: Examined 72, taken 12 (7 quota, 2 extra-family, 3 injection continuation), rejected 60. The first two phases examined 57 advisories (rows 1–72 of list-pip.tsv down to 2026-08-17, with 15 rows passed over). The injection continuation examined 6 of those 15 passed-over rows, which leaves 9 passed over. The CWE-89 continuation examined 9 more entries from pip-89.json (2026-07-13 down to 2026-05-11) and passed over 1 non-SQLi entry (pgadmin GHSA-j74f). In total, 10 rows were passed over without being examined. Advisories outside the CWE filter never entered the list. Their number is not known, because the list was built from per-CWE queries.
|
||||
|
||||
### JavaScript / TypeScript (Node servers)
|
||||
|
||||
Main phase (lines 1-59 of list-npm.tsv):
|
||||
- GHSA-3cj3-hqcr-g934 / CVE-2026-59723 (cline/cline, CWE-346 cross-origin WebSocket hijacking): VS Code extension, not a Node web app; family out of scope.
|
||||
- GHSA-fm8p-53ww-hf6w / CVE-2026-61742 (bytebase/dbhub, 306/346 DNS rebinding): MCP tool server; missing Host/Origin check is out of the three families.
|
||||
- GHSA-cmhj-wh2f-9cgx / CVE-2026-56676 (decolua/9router, SSRF DNS rebinding): repo already has a candidate.
|
||||
- GHSA-798p-78g2-v556 / CVE-2026-61612 (ondata/ckan-mcp-server, SSRF): MCP tool server (input is LLM tool arguments), not a web app.
|
||||
- GHSA-q69g-4hcv-6jg4 / CVE-2026-71538 (cyclonedx/cyclonedx-node-npm, cmdi): CLI, not a web app.
|
||||
- GHSA-q8hw-4fvp-9rwv / CVE-2026-61793 (nuxt-modules/og-image, SSRF): library (Nuxt module), not an app.
|
||||
- GHSA-2h44-8472-frjj / CVE-2026-61559 (zereight/gitlab-mcp, SSRF): MCP tool server.
|
||||
- GHSA-rqx4-3f6q-3x2v / CVE-2026-59148 (mockoon/mockoon, CORS): repo in used list.
|
||||
- GHSA-65h7-9wrw-629c / CVE-2026-59973 (agentfront/frontmcp, SSRF): library / MCP framework.
|
||||
- GHSA-4x45-gxvp-6283 / CVE-2026-59960 (argos-ci/argos-javascript, cmdi): CI client library/CLI.
|
||||
- GHSA-q5wm-mgqx-fv2f / CVE-2026-86074 (n8n-io/n8n, SSRF): no fix commit referenced; 2.38.1...2.38.2 is a 213-file "Bundle 2.x" commit `8c8b059e` (fix mixed with unrelated changes); licence is the non-OSI Sustainable Use License.
|
||||
- GHSA-6xcw-7xm6-48c6 / CVE-2026-86083 (n8n, code injection): same, fix inside 213-file bundle commit.
|
||||
- GHSA-34ff-336r-5q23 / CVE-2026-86082 (n8n, SSRF): same.
|
||||
- GHSA-hw8v-xxg5-vvvx / CVE-2026-86076 (n8n, code injection): same.
|
||||
- GHSA-f6mr-pjwc-34m4 / CVE-2026-88056 (angular/angular platform-server, SSRF): framework library, not an app.
|
||||
- GHSA-8m3c-c648-2xjj (nodemailer, SSRF/file): library.
|
||||
- GHSA-gx45-xrj5-g6c4 / CVE-2026-75911 (hmbown/codewhale, code injection): TUI/CLI.
|
||||
- GHSA-wrj3-vj8c-784f / CVE-2026-75858 (codewhale): TUI/CLI.
|
||||
- GHSA-6v2g-fpxh-pmmh / CVE-2026-75856 (codewhale, SSRF): TUI/CLI.
|
||||
- GHSA-6hxq-p678-4hr2 (masterkale/simplewebauthn, 295): library; attestation-chain validation is not a permissive-config case.
|
||||
- GHSA-fg9p-mrxr-hvq7, GHSA-88f2-fpv8-89q2, GHSA-w727-8j6c-2rj4, GHSA-2h9g-j24r-h63g, GHSA-8j6p-r8jg-mxqh, GHSA-2w86-xfrc-g85r, GHSA-3575-w9fc-c2j6, GHSA-653q-5476-x79g, GHSA-6437-gxhq-pqv8, GHSA-cxq5-97v7-87j8, GHSA-p4cg-3328-rvfg, GHSA-6mr6-jvcr-2f25 (orval-labs/orval, 12 advisories, code injection / SSRF in generated code): code-generator CLI, not a web app.
|
||||
- GHSA-4q39-2jhr-7qx8 / CVE-2026-65842 (udecode/plate, SSRF): library.
|
||||
- GHSA-cpjf-6666-r8fx / CVE-2026-61704 (op-engineering/link-preview-js, SSRF): library.
|
||||
- GHSA-f65p-4m7j-42xc / CVE-2026-75975 (fastify/fast-uri, SSRF): library.
|
||||
- GHSA-fph4-wmhf-6fwf / CVE-2026-75899 (fastify/fast-uri, SSRF): library.
|
||||
- GHSA-83x6-42hr-jc76 / CVE-2026-73845 (ondata/ckan-mcp-server, SSRF): MCP tool server.
|
||||
- GHSA-vq4v-j7r6-jq4m / CVE-2026-82393 (pnpm/pnpm): CLI; path traversal/file write, family out of scope.
|
||||
- GHSA-qxc2-j82w-r537 / CVE-2026-73231 (faker-js/faker, eval): library.
|
||||
- GHSA-3wgp-x9p5-c7cc / CVE-2026-58191 (appium/appium base-driver, XSS/489): library; the flaw is reflected XSS in test routes, family out of scope.
|
||||
- GHSA-g5xc-5w98-jfvm / CVE-2026-55855 (mariadb-connector-nodejs, SQLi in escaping): database driver library.
|
||||
- GHSA-cqhc-2h57-wpxf / CVE-2026-55215 (mariadb-connector-nodejs, TLS): database driver library.
|
||||
- GHSA-86m2-fcxq-5q7c / CVE-2026-55641 (decolua/9router, Host spoofing / SSRF): repo already has a candidate.
|
||||
- GHSA-66mm-25pp-rfff / CVE-2026-77415 (jsonata-js/jsonata, code execution): library.
|
||||
- GHSA-2943-5xfg-gq5f / CVE-2026-77414 (jsonata): library.
|
||||
- GHSA-8gq3-vp5j-2grp / CVE-2026-77413 (jsonata): library.
|
||||
- GHSA-wppf-h75h-6pm6 / CVE-2026-54689 (ihor-sokoliuk/mcp-searxng, SSRF): MCP tool server.
|
||||
- GHSA-q87f-qc2r-2gw4 / CVE-2026-54688 (mcp-searxng, SSRF): MCP tool server.
|
||||
- GHSA-2xhg-73j7-rrgx / CVE-2026-53957 (contentful/contentful-mcp-server, SSRF): MCP tool server.
|
||||
- GHSA-w6x9-28jw-hq7j / CVE-2026-63643 (MagicMirrorOrg/MagicMirror, SSRF calendar): the referenced fix `58c2a5e6` only enforces the IP whitelist on Socket.IO handshakes; the `fetch(url)` sink is unchanged in the fixed version (fix is access control, not destination validation).
|
||||
- GHSA-998g-7v5w-cr7g / CVE-2026-63642 (MagicMirror, SSRF newsfeed): same fix commit, same reason.
|
||||
- GHSA-8g4w-4ffg-8vgx / CVE-2026-56677 (decolua/9router, SSRF OIDC test): repo already has a candidate.
|
||||
|
||||
Extra phase (lines 60-94; only config_secrets and untrusted_destination sought, +1 each):
|
||||
- GHSA-49mq-fc6q-3h46 / CVE-2026-55157 (ooples/token-optimizer-mcp, cmdi): quota filled, family not sought in extra phase (also an MCP tool server).
|
||||
- GHSA-5fpj-28rv-84r7 / CVE-2026-35219 (Budibase/budibase, SSRF): no fix commit referenced; the 3.41.2...3.41.3 compare (47 commits) does not touch the automation steps (fix not identifiable, possibly in the private pro submodule).
|
||||
- GHSA-fjgc-3mj7-8rg8 / CVE-2026-55087 (ether/etherpad, open redirect): fix inside a large "assorted hardening" PR (#7784), as found in the v1 research.
|
||||
- GHSA-vhh6-v828-x62f / CVE-2026-58230 (@sap/approuter, 601): no public source repository.
|
||||
- GHSA-wcx4-wpfv-mc5c (aws/jsii jsii-diff), GHSA-6x64-9x62-f2gx (mermaid), GHSA-9473-5f9j-94wq and GHSA-279x-mwfv-vcqv (nuxt/nuxt), GHSA-f2r8-jv7c-xqmp and GHSA-4f78-qhmw-8j8m (electron): injection family, not sought in extra phase.
|
||||
- GHSA-v64r-4m7r-3mvq / CVE-2026-70605 (electron/electron, SSRF): framework out of scope (Electron).
|
||||
- GHSA-v3j7-r9gq-3gjw / CVE-2026-70604 (electron, 942 CORS): framework out of scope (Electron).
|
||||
- GHSA-9pf5-hg6p-4pwp / CVE-2026-70599 (electron, 346): framework out of scope.
|
||||
- GHSA-x5mm-wm4g-j5xv, GHSA-gcvv-72q8-9v76, GHSA-g366-23fw-ggp6, GHSA-ch52-px8q-f22j, GHSA-wvp2-4qqp-4h3r (tryghost/ghost, 5x SSRF): repo in used list.
|
||||
- GHSA-5xvg-pmgg-3mxr, GHSA-4j8x-x6v7-w9rq, GHSA-x3hf-7cj6-3r4m (Flowise, code injection, listed before the SSRF advisory on the same day): injection family, not sought in extra phase.
|
||||
- GHSA-x6vm-w76m-8j7g, GHSA-vmv7-4m6c-3cg5, GHSA-3769-jgqc-cxm7, GHSA-wg86-r78f-74mp, GHSA-g32j-mmxr-gfq5 (Flowise, code injection): repo already has a candidate; injection not sought.
|
||||
- GHSA-v8fg-2rw7-q452 / CVE-2026-69240 (sequelize, SQLi Oracle): ORM library; injection not sought.
|
||||
- GHSA-mwp4-54f8-5fhr, GHSA-4xrf-jv44-h6hh, GHSA-22jq-vg5j-6vgg (beaugunderson/ip-address, SSRF classification): library; untrusted extra already filled.
|
||||
- GHSA-34pj-2622-jvxq / CVE-2026-53607 (apostrophecms/apostrophe, SSRF via Host header): untrusted_destination extra already filled by Flowise (newer); not examined further.
|
||||
- GHSA-p849-8hwh-84j9 / CVE-2026-52887 (nocobase/nocobase, SQLi): repo already has a candidate; injection not sought.
|
||||
- GHSA-5846-7qm3-r52j / CVE-2026-54729 (hackingrepo/dssrf-js, SSRF): library; untrusted extra already filled.
|
||||
|
||||
SQLi/NoSQLi continuation phase (CWE-89 / CWE-943 entries of list-npm.tsv after GHSA-p849-8hwh-84j9, newest first; requested by the coordinating session because SQLi was thin):
|
||||
- GHSA-pqh8-p93p-2rx7 (dynatrace-oss/dynatrace-mcp, DQL injection): MCP tool server.
|
||||
- GHSA-6wcc-39rp-hh9p / CVE-2026-54658 (hypequery/hypequery, ClickHouse escaping): library (query builder).
|
||||
- GHSA-pmpg-2mxq-6xwr (Budibase, MongoDB NoSQLi): TAKEN (budibase-mongo-template-nosqli).
|
||||
- GHSA-2xgg-r2wc-c5r2 (Budibase, MySQL DESCRIBE backtick): repo already has a candidate.
|
||||
- GHSA-qw6m-8fw2-2v64 (Budibase, MongoDB NoSQLi via query endpoint): repo already has a candidate (same flaw family as the taken case).
|
||||
- GHSA-q6x4-v3qx-85qw / CVE-2026-73300 (Budibase, MySQL multipleStatements): repo already has a candidate.
|
||||
- GHSA-652q-gvq3-74qv (n8n, Snowflake node SQLi): no fix commit referenced; 2.32.0...2.32.1 has only a "Bundle/2.x" backport commit `f69dfc6d` plus two unrelated commits (fix mixed into bundle); non-OSI licence.
|
||||
- GHSA-jqwr-vx3p-r266 (n8n, PostgresTrigger SQLi): same release bundle as above.
|
||||
- GHSA-hwmj-qg4v-cvg9 / CVE-2026-59257 (n8n, MySQL v1 SQLi): 2.28.0...2.28.1 fix is again inside a "Bundle/2.x" commit `08ffbcc1`.
|
||||
- GHSA-2qp2-6frj-p9pq (n8n MySQL v1): duplicate advisory.
|
||||
- GHSA-5c7w-4wm3-85vw (asymmetric-effort/specifyjs, GraphQL tag): library.
|
||||
- GHSA-8qv3-p479-cj62 / CVE-2026-54350 (Budibase, NoSQL operator injection): repo already has a candidate.
|
||||
- GHSA-9ggv-8w38-r7pm (typeorm/typeorm, orderBy SQLi): ORM library.
|
||||
- GHSA-qqf5-x7mj-v43p (Budibase, connector SQLi): repo already has a candidate.
|
||||
- GHSA-jpq7-226w-6cxx / CVE-2026-54313 (n8n, MongoDB node): n8n release-bundle pattern (not re-checked per release); non-OSI licence.
|
||||
- GHSA-c37g-w77q-m4vp / CVE-2026-54310 (n8n, Postgres v1 node): same.
|
||||
- GHSA-98xf-r82g-9mhx / CVE-2026-48121 (langchain-ai/langgraphjs, MongoDBSaver NoSQLi): library.
|
||||
- GHSA-h9fj-c2qr-76g2 / CVE-2026-47720 (frangoteam/FUXA, TDengine SQLi): TAKEN (fuxa-tdengine-escape-sqli). Quota of 2 reached; stopped.
|
||||
|
||||
Counts: Examined 112 advisories (59 in the main phase, 35 in the extra phase, 18 in the SQLi/NoSQLi continuation phase), taken 11 (7 main + 2 extra + 2 SQLi phase), rejected 101. CWE-prefiltered skips: 0 inside the list (the list was CWE-filtered at fetch time, so reviewed npm advisories with other CWEs never entered it and were not counted).
|
||||
|
||||
Families in this lane: injection 6 (2 SQLi, 1 NoSQLi, 3 OS command/code), untrusted_destination 4 (2 SSRF, 1 SSRF guard bypass, 1 open redirect), config_secrets 1 (wildcard CORS). Privilege: unauthenticated 5 (reactpress, cct, medplum, nx, fuxa) + 1 conditional (omniroute); authenticated 5.
|
||||
|
||||
### PHP web applications (not WordPress)
|
||||
|
||||
- GHSA-rw77-vq4g-x3hp / CVE-2026-56738 (thorsten/phpMyFAQ, SQLi): repo in used list.
|
||||
- GHSA-9ccq-2jfg-qw33 / CVE-2026-72702 (getgrav/grav, CWE-346): extra phase; Referer prefix-match origin check, neither a configuration weakness nor a redirect sink as described; untrusted slot already filled by SMW.
|
||||
- GHSA-68jx-f42c-7599 / CVE-2026-19418 (typo3/typo3, CWE-346/352): extra phase; broken access control, family out of scope.
|
||||
- GHSA-qh7h-6c7g-x8m6 (getgrav/grav, CWE-346): duplicate advisory.
|
||||
- GHSA-4f2f-jr2m-j7p4 (typo3, CWE-346): duplicate advisory.
|
||||
- GHSA-f7vp-7xgx-4w4r / CVE-2026-69245 (guzzle/guzzle, CWE-346): library, not a web app.
|
||||
- GHSA-g446-98w2-8p5w / CVE-2026-59883 (guzzle/guzzle, CWE-346): library, not a web app.
|
||||
|
||||
SQLi-only continuation phase (requested after the cross-lane family balance; CWE-89 entries in list-composer.tsv, newest first, resuming after Sulu on 2026-09-23):
|
||||
- GHSA-vjqc-q4mp-2rvf / CVE-2026-79752 (cakephp/cakephp, SQLi): framework library (`FunctionsBuilder`), not a web app.
|
||||
- GHSA-23rh-xw42-fq82 / CVE-2026-55416 (pimcore/pimcore, SQLi): Custom Reports lets a `reports_config` admin write SQL fragments by design (only a keyword blacklist). No fix commit is linked, and `Sql.php` history shows several successive hardening commits (`3fd77334`, `1999fee2`, `62ad7c5a`). No single identifiable fix.
|
||||
- GHSA-fxf7-vhh8-7vpq / CVE-2026-77635 (cakephp/cakephp, SQLi): framework library, not a web app.
|
||||
- GHSA-9x44-4gxf-8c25 / CVE-2026-55634 (pimcore/pimcore, CWE-89+94): primary sink is PHP code generation (field name injected into a generated class file, RCE). The ALTER TABLE SQLi is a secondary sink of the same fix (`a4f8c3cf`). Rejected in the SQLi-only phase as mixed-family ground truth. It would be a valid `injection` (code) case, licence POCL.
|
||||
- (taken) GHSA-79cw-hfcc-7mw9: pimcore-studio-datefilter-sqli.
|
||||
- (taken) GHSA-w27m-rmmf-g5w4: froxlor-admin-ip-second-order-sqli.
|
||||
|
||||
config_secrets alternate, examined on request:
|
||||
- GHSA-jr9p-4h4j-6c58 / CVE-2026-52824 (kimai/kimai, CWE-1188, default `APP_SECRET`): rejected. The advisory links no commit. The fix is in release commit `31a8f887a5cda517db7b4320a7ad997c87d08601` ("Release 2.58", 85 files, fixes four GHSAs plus unrelated changes; parent `8d245ae223fd8fcbd508362dfe7fd04254a066c4`). That is a fix mixed with unrelated changes. The default secret also lives in `Dockerfile` (`ENV APP_SECRET=...`) and `.docker/entrypoint.sh`, not in PHP code. The fix adds a shell `ensureAppSecret()` function, and the bare-metal `.env.dist` default is, per the advisory, unchanged. pheditor stays the only (low-confidence) config_secrets case from this lane.
|
||||
|
||||
Passed over without examination because the injection quota was filled and only the extra families were being sought (2026-09-17 → 2026-07-16, 39 advisories), including: Grav CWE-94 ×5 (GHSA-r94f-hx44-8jqf, -f8wv-xp27-6gq7, -7pgq-cr25-xvc8, -2c4f-86xc-cr74, -fj2p-qj2f-74v5), CakePHP SQLi ×2 (GHSA-vjqc-q4mp-2rvf, -fxf7-vhh8-7vpq; framework library), Chamilo GHSA-g4c3-4g96-6g4m, Pimcore SQLi ×4, Smarty SSRF GHSA-cq55-c7wv-pxmq, elFinder SSRF GHSA-8x3q-jpjh-qh5c, Snipe-IT open redirect GHSA-wg2f-x2c2-c4rp, SilverStripe ×2, Phalcon, Backpack CRUD, LibreNMS ×3, Froxlor, jmespath.php, Winter, CodeIgniter4, PHP_CodeSniffer, Easy!Appointments SSRF, Poweradmin, WPCS, Pheditor ×2 other, PHPSpreadsheet, Koel SSRF ×6 (partly beyond 07-16). Note for the maintainer: the untrusted_destination ones among them (Smarty, elFinder, Snipe-IT, Easy!Appointments, Poweradmin, Koel) are the next-in-line alternates for that family.
|
||||
|
||||
No WordPress plugin appeared in the composer list up to 2026-07-16 (WPCS GHSA-3pwp-g2mj-5p3v is a coding-standards tool, not a plugin).
|
||||
|
||||
Counts: Examined 20, taken 8 (4 quota + 1 extra untrusted_destination + 1 extra config_secrets + 2 SQLi continuation), rejected 12. SQLi continuation phase: 6 examined, 2 taken. Kimai alternate: 1 examined, rejected. Passed over after the quota was filled (other families): 39. CWE-prefiltered skips: the composer list was prefiltered by CWE at fetch time, so out-of-family advisories never entered it (not counted individually).
|
||||
|
||||
### WordPress plugins (PHP+JS)
|
||||
|
||||
- GHSA-7jgm-vpwg-fj48 / CVE-2026-93899 (Better Messages – Chat Rooms, Group Chat, Private Messages & AI Chat Bots, SQLi, published 2026-09-25): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-g4cp-fj5c-7599 / CVE-2026-12037 (Asset CleanUp: Page Speed Booster, SSRF, published 2026-09-25): only GitHub repo is gabelivan/wp-asset-clean-up, last push 2019; fix only on wp.org SVN
|
||||
- GHSA-6rpr-h6cv-7xm2 / CVE-2026-78397 (Link Library, SSRF, published 2026-09-25): the author repo ylefebvre/link-library stops at 7.9.2 (2026-04-26); the fix is in 7.9.6, which is only on wp.org SVN
|
||||
- GHSA-8q4x-8xp2-6ww3 / CVE-2026-18335 (Kirki – Freeform Page Builder, Website Builder & Customizer, SSRF, published 2026-09-24): themeum/kirki exists but last push 2026-04-10 and has no 6.x code; fix (trac changeset 3636487) only on wp.org SVN
|
||||
- GHSA-4rj9-f57h-53hg / CVE-2026-93368 (Rename wp-login.php to anything you want, SQLi, published 2026-09-23): only unrelated/stale repo (ellatrix/rename-wp-login, 2020); fix only on wp.org SVN
|
||||
- GHSA-8743-f7m5-qxwx / CVE-2026-91024 (Booking Manager, SQLi, published 2026-09-23): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-ppcx-fj6x-46xf / CVE-2026-75799 (YAHMAN Add-ons, code injection, published 2026-09-23): file-type validation (upload), family out of scope; also no GitHub repo
|
||||
- GHSA-9rxg-m349-fgqj / CVE-2022-4997 (jet-form-builder-stripe-gateway, SQLi, published 2026-09-23): CVE-2022 id (disclosed 2022, GHSA published 2026); no GitHub repo
|
||||
- GHSA-gg2m-8p7j-v3mx / CVE-2026-92235 (The WP Ultimate Review, code injection, published 2026-09-22): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-h9fx-ggrg-2982 / CVE-2026-16542 (Import and export users and customers, SSRF, published 2026-09-20): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-h6cq-4ghp-3r32 / CVE-2026-87067 (Forminator Forms, code injection, published 2026-09-20): Forminator: no public source repo; also object-injection style (class restriction), family doubtful
|
||||
- GHSA-fv9m-mw9j-43cg / CVE-2026-9855 (Custom Field Template, SQLi, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-wc9h-566w-4rw2 / CVE-2026-7527 (WP Ghost (Hide My WP Ghost) – Security & Firewall, open redirect, published 2026-09-19): johndarrel/hide-my-wp is a release-dump mirror; the fix (<=7.0.02 -> 7.0.08) sits in one 7.0.08 release commit of 60+ files (fix mixed with unrelated changes)
|
||||
- GHSA-jvc3-5jgr-chw6 / CVE-2026-85658 (Paid Membership Plugin, Ecommerce, User Registration Form, Login Form, User Profile &..., code injection, published 2026-09-19): ProfilePress: only profilepress/profilepress-readmes on GitHub; fix only on wp.org SVN
|
||||
- GHSA-cv43-j772-67wp / CVE-2026-6295 (WP Optimizer, SQLi, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-c79v-6v55-pvvh / CVE-2026-13191 (Create, SQLi, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-mw9j-687w-wh97 / CVE-2026-1641 (Wow Elements Addons for Elementor, SSRF, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-fmwg-rf74-rjqj / CVE-2026-4327 (The Welcomizer, code injection, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-5j33-g9fw-4r4f / CVE-2026-88926 (VikRentItems Flexible Rental Management System, SQLi, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-jhj7-pcmr-9q9p / CVE-2026-13200 (Create, SQLi, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-2vrv-wr6r-v79v / CVE-2026-92229 (The Forminator Forms – Contact Form, Payment Form & Custom Form Builder, code injection, published 2026-09-19): Forminator: no public source repo (wpmudev has only forminator-ui); fix only on wp.org SVN
|
||||
- GHSA-3fg9-62hh-37mh / CVE-2026-92807 (Save as PDF Plugin by PDFCrowd, code injection, published 2026-09-19): pdfcrowd/save-as-pdf-addon is the browser add-on, not the WP plugin; fix only on wp.org SVN
|
||||
- GHSA-x57m-9xmj-gh9p / CVE-2026-89274 (WP Recipe Maker, code injection, published 2026-09-19): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-34mw-p6wx-m38q / CVE-2026-85652 (Photo Gallery by 10Web – Mobile-Friendly Image Gallery, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-7wpp-2mgh-3vfp / CVE-2026-75961 (NEX-Forms – Ultimate Forms Plugin for, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-jvcr-7q46-jqm3 / CVE-2026-85705 (Location Manager, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-35gj-4m99-3jxg / CVE-2026-18442 (WCFM Marketplace – Multivendor Marketplace for WooCommerce, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-q2g2-xwwx-rp3m / CVE-2026-17607 (WP Inventory Manager, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-prxh-6hcq-7gvg / CVE-2026-15275 (WP Multi Store Locator Pro, SQLi, published 2026-09-18): wpexpertsio/WP-Multi-Store-Locator last push 2023-12 (Pro advisory); fix not on GitHub
|
||||
- GHSA-xcfp-7r4f-6367 / CVE-2026-17576 (InfiniteWP Client, SQLi, published 2026-09-18): Revmakx/iwp-client last push 2025-09-23, before the fix; fix only on wp.org SVN
|
||||
- GHSA-7v2c-5xw3-f2hc / CVE-2026-12106 (Auto Upload Images, SSRF, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-93x5-88q4-f8c3 / CVE-2026-87775 (Tz Weekly Radio Schedule, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-hm83-7h6c-m5rj / CVE-2026-87774 (Tz Weekly Radio Schedule, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-c3g6-9rgm-rg4m / CVE-2026-90984 (Generate PDF using Contact Form 7, SSRF, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-4778-xmgc-pc88 / CVE-2026-87770 (Price Drop Alert for Woo Commerce, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-rjgv-8pq9-p3c3 / CVE-2026-87771 (Product Question and Answer, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-52cc-m98v-q64v / CVE-2026-87767 (wp shortcut link and advertisement baner, SQLi, published 2026-09-18): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-7mpj-9p6h-7gr6 / CVE-2026-84738 (AF Companion, code injection, published 2026-09-18): file upload validation, family out of scope; also no GitHub repo
|
||||
- GHSA-vfmj-6hfh-5m4v / CVE-2026-87963 (Yo, SQLi, published 2026-09-17): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-xg7f-297r-82hv / CVE-2026-88795 (wpShopGermany IT-RECHT KANZLEI, code injection, published 2026-09-17): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-xhvp-v8j9-2mrf / CVE-2026-86823 (Newsletter, open redirect, published 2026-09-16): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-49j8-gcx6-2f92 / CVE-2026-76559 (WP Import Export Lite, SSRF, published 2026-09-16): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-g78q-54w3-px42 / CVE-2026-76550 (WP Import Export Lite, code injection, published 2026-09-16): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-v7v7-79q5-j8cf / CVE-2026-76551 (WP Import Export Lite, code injection, published 2026-09-16): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-9c57-9fxg-8x9j / CVE-2026-78159 (The Events Calendar, code injection, published 2026-09-12): repo in used list (the-events-calendar)
|
||||
- GHSA-3q5p-wcf4-wvm5 / CVE-2026-14560 (teddy-bear-customize-addon, code injection, published 2026-09-11): file upload validation, family out of scope; also no GitHub repo
|
||||
- GHSA-p236-jmm4-v336 / CVE-2026-18021 (The Beaver Builder Page Builder – Drag and Drop Website Builder, code injection, published 2026-09-08): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-c2fp-562c-5p2q / CVE-2026-12757 (The Email Subscribers & Newsletters – Email Marketing, Post Notifications & Newsletter Plugin..., code injection, published 2026-09-07): icegram/email-subscribers last push 2019; fix only on wp.org SVN
|
||||
- GHSA-368m-gh85-cq79 / CVE-2026-81342 (MasterStudy LMS, open redirect, published 2026-08-29): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-qc2w-xhvw-6xmh / CVE-2026-16296 (Clearfy Cache, open redirect, published 2026-08-04): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
- GHSA-66xh-mw5m-g493 / CVE-2026-14236 (Contact Form 7, open redirect, published 2026-07-27): no GitHub repo / fix only on wp.org SVN (GitHub search found only SVN mirrors or nothing)
|
||||
|
||||
Counts: - Enumeration: `gh api "/advisories?type=unreviewed&cwes=<C>&per_page=100&page=<1..3>&sort=published&direction=desc"` for C in 89, 78, 94, 918, 601, 798, 942. That gives the 300 newest unreviewed advisories per CWE, covering roughly 2026-09-14 to 2026-09-25 for 89/918, and back to 2026-07-27 for 601. I kept those whose description mentions "WordPress" and "plugin" (57 advisories, 2026-07-27 → 2026-09-25) and worked through them newest first. For each, I took the plugin slug from the trac/wordpress.org references and searched GitHub (`search/repositories` by slug, name and vendor). I rejected repos that are only SVN mirrors (common-repository, WordPressBugBounty, WPPlugins, wp-plugins, WordpressPluginDirectory) or that stop before the fix. No WordPress advisory with CWE 798 or 942 appeared in that window.
|
||||
- Examined 57, taken 6 (5 quota + 1 extra untrusted_destination), rejected 51.
|
||||
- Family outcome: 1 code injection, 1 SQLi, 1 SSRF, 3 open redirect, 0 config_secrets. Most WordPress SQLi advisories in the window (Better Messages, Custom Field Template, WP Optimizer, Mediavine Create, Photo Gallery, NEX-Forms, WCFM, InfiniteWP, etc.) have no public development repository, so SQLi is thin in this lane for that reason only.
|
||||
@@ -0,0 +1,842 @@
|
||||
# Independent evaluation population v3 (PHP): candidate real-world cases
|
||||
|
||||
Research date: 2026-09-30. Method: web research only. I read GitHub advisory records (the global advisory API, reviewed and unreviewed), commit pages, compare views, repository metadata and file histories through `gh api` (read-only) and GitHub search. I did not clone, download or check out any candidate repository, did not run any scanner, rule, engine or project tool on one, and did not read candidate source beyond what an advisory itself quotes. Commit pages were read for their metadata (parents, message, changed-file names and line counts), not their patch text.
|
||||
|
||||
Output: this file and `candidates-v3-php.json` (26 candidates, machine-readable, one record per case in the v2 schema plus research fields). Nothing is reserved yet: the maintainer chooses, then a `population-v3.toml` and a protocol are written and frozen as for v2.
|
||||
|
||||
## Why a v3, and what it must not contain
|
||||
|
||||
PHP quick-mode rules were written on 2026-09-30 while looking at PMPro, population-v1's PHP cases, Ultimate Member, MediaWiki and WP Statistics. None of today's PHP numbers qualifies, and population v2 is reserved for other qualification. v3 must be PHP cases nobody in this project has looked at.
|
||||
|
||||
## Selection criteria (fixed before looking at any advisory)
|
||||
|
||||
From the maintainer's brief, unchanged:
|
||||
- **Language** PHP. WordPress plugins allowed but capped at half the population; non-WordPress PHP applications and frameworks included.
|
||||
- **Recency**: advisory published 2026-06-01 or later, taken newest first; a public fix commit or tag diff in a public GitHub repository; identifiable vulnerable and fixed versions.
|
||||
- **Family spread**: at most about 40% injection; include output_encoding, path, untrusted_destination, deserialization, access_control and OS command injection where advisories exist. Families are the ids of `src/openultrasast/ruleset/families.toml`, and an advisory's family is found from its CWE ids through that file's `cwes` lists (so CWE-434 upload and CWE-74 generic injection map to no family).
|
||||
- **Declarable site**: the advisory names the vulnerable function or file, so a `sites` entry can be declared from its text alone.
|
||||
- **Exclusions**: any repository whose `github.com/owner/repo` appears under `benchmarks/`, `src/`, `tests/` or `.kiro/` (341 repositories, extracted by grep and compared case-insensitively); PMPro, Ultimate Member, LearnPress, phpMyFAQ, YesWiki, MediaWiki (including MediaWiki extensions), WP Statistics, froxlor, GoPay, Directorist and Groundhogg by name; repositories over about 150 MB by the GitHub API `size` (153,600 KB); advisories without a fix diff.
|
||||
|
||||
Operational rules, carried over from the v2 research (`research-2026-09-26.md`) and fixed before the first advisory was read:
|
||||
- **In scope** means a web application, CMS, web framework/bundle or plugin whose vulnerable code is reached from HTTP request handling inside the repository. General-purpose libraries and framework APIs whose attacker input only exists in a calling application, developer tools/CLIs, and client-side JavaScript sinks are rejected (they give the analyzer no in-repo source).
|
||||
- **Fix**: at most 3 commits, or one commit that is plainly about this weakness; a fix buried in a large unrelated squash or release commit is rejected, as is a fix that changes no PHP line (the fixed pin must differ in PHP). A fix commit the advisory does not name may be taken from the release compare or the named file's history, and is then marked `fix_named_by_advisory: false`.
|
||||
- **One case per repository.** Old vulnerabilities republished in 2026 under an older CVE id are rejected (v2 did the same).
|
||||
- **Guard-bypass and incomplete-fix advisories are allowed**, with the caveat recorded.
|
||||
- **At no point was a case chosen or skipped for how easy it would be for this analyzer.** No rule or query of the project was consulted.
|
||||
|
||||
### Order and quotas
|
||||
|
||||
- **Family quotas** (targeting about 27 candidates): injection 8, output_encoding 4, path 4, untrusted_destination 4, access_control 4, deserialization 3.
|
||||
- **Passes by family, rarest first**, so that one-case-per-repository does not hand a rare family's only repository to a common family. Rarity was counted from metadata alone (distinct eligible repositories per family after the automatic exclusions, before any advisory text was read): deserialization 1, injection 9, path 10, untrusted_destination 11, access_control 16, output_encoding 21. Within a pass, advisories are taken newest first (by `published_at`); a repository taken in an earlier pass is skipped.
|
||||
- **Sources (lanes)**, consulted in this order within a pass, and each later lane only for a family still short after the earlier ones:
|
||||
- **Lane A** — reviewed GitHub advisories, ecosystem composer, published >= 2026-06-01 (582 records; 386 in the six families). `gh api --paginate "/advisories?ecosystem=composer&type=reviewed&published=>=2026-06-01&sort=published&direction=desc"`.
|
||||
- **Lane B** — unreviewed advisories for the family CWEs of injection, path, deserialization and untrusted_destination (5,246 records), kept when a reference is a `github.com/<o>/<r>/(commit|pull|compare)/` link and the GitHub API reports the repository's language as PHP (92 records on 56 PHP repositories). Output_encoding and access_control were not fetched, because lane A filled them.
|
||||
- **Lane C** — deserialization only: unreviewed CWE-502 advisories whose text names WordPress (42), each plugin searched on GitHub for a development repository (SVN mirrors do not count, as in v2), plus the non-WordPress CWE-502 records naming a PHP product (9). The Patchstack one-line records ('Unauthenticated PHP Object Injection in X <= v') never name a function and were not examined individually.
|
||||
- Lanes B and C were added after lane A left injection 4, path 3 and deserialization 0 against their quotas; the lane rule was fixed then, before any lane-B/C advisory was read.
|
||||
- **Verification** of every candidate: `gh api repos/O/R/commits/<sha>` for the fixed commit (parents, message, changed files); the vulnerable pin is the fixed commit's (first) parent, except where a short linear chain is noted; `compare/<fixed>...<release tag>` confirms the fixed commit is an ancestor of the first fixed release wherever a tag exists; repository size, licence and default branch from `repos/O/R`.
|
||||
|
||||
## Counts
|
||||
|
||||
- **Examined: 470** advisories (lane A 379, lane B 40, lane C 51). **Taken: 26. Rejected: 444.**
|
||||
- Automatic exclusions by metadata count as examined, as in v2; advisories passed over after their family's quota filled were not examined and are listed separately (lane A 7, lane B 34; lane B's 18 untrusted_destination rows were never needed).
|
||||
|
||||
| Rejection reason | Count |
|
||||
|---|---|
|
||||
| repository over ~150 MB | 161 |
|
||||
| one case per repository | 64 |
|
||||
| repository already referenced in the project tree (used list) | 46 |
|
||||
| advisory names no vulnerable file or function | 46 |
|
||||
| library/framework API: no attacker source inside the repository | 42 |
|
||||
| duplicate advisory | 26 |
|
||||
| no usable fix diff (none, unmerged, reverted, release squash, or non-PHP) | 25 |
|
||||
| no public development repository | 10 |
|
||||
| not a web application (developer tool, CLI, game server) | 9 |
|
||||
| excluded by name | 6 |
|
||||
| old vulnerability republished (CVE-2022..2025) | 4 |
|
||||
| sink is client-side JavaScript | 3 |
|
||||
| other: corrupt advisory record: the description is about draytek vi | 1 |
|
||||
| other: personal/student project (`harsh21patel/inventory-management | 1 |
|
||||
|
||||
## Family mix of the 26 candidates
|
||||
|
||||
| Family | Candidates | Of which WordPress | Subtypes |
|
||||
|---|---|---|---|
|
||||
| injection | 8 | 0 | OS command / argument injection; OS command injection; SQL injection; code injection |
|
||||
| output_encoding | 4 | 0 | XSS; reflected JSON injection; stored HTML injection; stored XSS |
|
||||
| path | 4 | 1 | local file inclusion; path traversal |
|
||||
| untrusted_destination | 4 | 0 | SSRF |
|
||||
| access_control | 4 | 2 | IDOR; IDOR in REST permission callback; missing object-level capability check; unvalidated URL-writable properties override provisioning parameters |
|
||||
| deserialization | 2 | 2 | PHP object injection; PHP object injection → arbitrary file deletion |
|
||||
| **total** | **26** | **5** (19%) | |
|
||||
|
||||
Injection is 8 of 26 (31%): SQL 2 (Winter, QloApps), OS command 3 (Backpack, GitList, Roundcube), code/eval 3 (Grav, MantisBT, MacCMS). Untrusted_destination is all SSRF: no open-redirect advisory reached the top of its pass. WordPress is 5 of 26 (19%), under the cap.
|
||||
|
||||
## Candidates
|
||||
|
||||
Full records (sink, source, sites, site derivation, fix notes, caveats, benign proposal) are in `candidates-v3-php.json`. `R` marks the recommended selection below.
|
||||
|
||||
| # | R | id | repo | family | advisory | published | vulnerable → fixed | fixed release | sites (advisory) | licence | MB | WP |
|
||||
|---|---|----|------|--------|----------|-----------|--------------------|---------------|------------------|---------|----|----|
|
||||
| 1 | | grav-blueprint-bare-function-rce | https://github.com/getgrav/grav | injection | CVE-2026-75827 / GHSA-f8wv-xp27-6gq7 | 2026-09-17 | `24e8717ae317` → `af31066e9b6a` | 2.0.15 | `system/src/Grav/Common/Data/Blueprint.php::isSafeDynamicCall`, `system/src/Grav/Common/Data/Blueprint.php::init` | MIT | 34 | |
|
||||
| 2 | | backpack-stats-host-header-cmdi | https://github.com/Laravel-Backpack/CRUD | injection | CVE-2026-54182 / GHSA-mrc5-3mm3-45c5 | 2026-08-20 | `baae0420b6eb` → `282ba2a0b887` | 7.0.36 | `src/Stats.php::makeCurlRequest` | MIT | 76 | |
|
||||
| 3 | R | winter-filter-numberrange-sqli | https://github.com/wintercms/winter | injection | CVE-2026-32593 / GHSA-m7jc-g4rc-jmvh | 2026-08-12 | `555e67c84b50` → `50713de95adf` | 1.2.13 | `modules/backend/widgets/Filter.php::numbersFromAjax` | MIT | 68 | |
|
||||
| 4 | | mantisbt-tokenizer-eval-rce | https://github.com/mantisbt/mantisbt | injection | CVE-2026-49273 / GHSA-v84x-qvhg-f36r | 2026-07-15 | `e7041ecc80c8` → `78c0af63d1fe` | 2.28.4 | file only: `core/classes/Tokenizer.class.php`, `adm_config_set.php` | GitHub: none [verify] | 89 | |
|
||||
| 5 | R | gitlist-commandline-cmdi | https://github.com/klaussilveira/gitlist | injection | CVE-2026-82668 / GHSA-v64j-pgc3-qgc9 | 2026-08-31 | `f67609d52c18` → `88cf2866083d` | 3.0.0-beta | `src/SCM/System/Git/CommandLine.php::getDefaultBranch` | BSD-3-Clause | 6 | |
|
||||
| 6 | R | qloapps-customermessage-bo-query-sqli | https://github.com/Qloapps/QloApps | injection | CVE-2026-75497 / GHSA-5w6r-2mxc-fq3v | 2026-08-25 | `5b4d65cbb857` → `123c97c110b7` | commit 123c97c | file only: `classes/CustomerMessage.php` | OSL-3.0 | 129 | |
|
||||
| 7 | R | roundcube-markasjunk-cmd-learn-rce | https://github.com/roundcube/roundcubemail | injection | CVE-2026-74997 / GHSA-q5rh-mmp4-ppgq | 2026-08-17 | `a1afb8fd1f00` → `b8f90e28a46d` | 1.6.18 / 1.7.3 | file only: `plugins/markasjunk/drivers/cmd_learn.php` | GPL-3.0 | 67 | |
|
||||
| 8 | R | maccms10-template-blacklist-rce | https://github.com/magicblack/maccms10 | injection | CVE-2026-71232 / GHSA-cv48-22qx-mpmx | 2026-08-05 | `2cabecda2994` → `71ad3bb29570` | commit 71ad3bb | file only: `application/admin/controller/Template.php` | GitHub: NOASSERTION [verify] | 29 | |
|
||||
| 9 | R | kirby-realpath-sibling-traversal | https://github.com/getkirby/kirby | path | CVE-2026-75592 / GHSA-6j4c-mgqr-qv76 | 2026-09-02 | `e90257fd7504` → `2b6fab950bc8` | 4.9.5 / 5.5.2 | `src/Filesystem/Dir.php::realpath`, `src/Filesystem/F.php::realpath` | GitHub: NOASSERTION [verify] | 66 | |
|
||||
| 10 | R | mineadmin-appstore-identifier-traversal | https://github.com/mineadmin/MineAdmin | path | CVE-2026-55224 / GHSA-59xm-4m8c-g3xj | 2026-08-18 | `ac0dcad616c5` → `ca41902a2a54` | 3.2.0-alpha.2 | `plugin/mine-admin/app-store/src/Service/Service.php::download`, `plugin/mine-admin/app-store/src/Service/Service.php::install`, `plugin/mine-admin/app-store/src/Service/Service.php::unInstall` | Apache-2.0 | 53 | |
|
||||
| 11 | | facturascripts-uploadedfile-move-traversal | https://github.com/NeoRazorX/facturascripts | path | GHSA-hgjx-r89m-m7v4 | 2026-07-14 | `6baefcdc00c2` → `d3770be64a3e` | not stated | `Core/UploadedFile.php::move` | LGPL-3.0 | 60 | |
|
||||
| 12 | R | wp-geo-my-wp-info-window-lfi | https://github.com/Fitoussi/geo-my-wp | path | CVE-2026-85200 / GHSA-6hw9-7gx5-mw88 | 2026-09-12 | `1dbb7b1672a4` → `5a768bf1c6e4` | 4.5.5.4 | `plugins/posts-locator/includes/gmw-posts-locator-functions.php::gmw_posts_locator_ajax_info_window_loader` | GitHub: none [verify] | 9 | yes |
|
||||
| 13 | R | elfinder-fsock-dns-rebinding-ssrf | https://github.com/Studio-42/elFinder | untrusted_destination | CVE-2026-81889 / GHSA-8x3q-jpjh-qh5c | 2026-08-31 | `dd73e702820c` → `6d997386cd0f` | 2.1.70 | `php/elFinder.class.php::fsock_get_contents` | GitHub: NOASSERTION [verify] | 43 | |
|
||||
| 14 | R | easyappointments-caldav-ssrf | https://github.com/alextselegidis/easyappointments | untrusted_destination | CVE-2026-52840 / GHSA-pm5p-7w5h-jm5q | 2026-07-29 | `725eafa64730` → `4abb10545d83` | 1.6.0 | `application/controllers/Caldav.php::connect_to_server`, `application/libraries/Caldav_sync.php::test_connection` | GPL-3.0 | 49 | |
|
||||
| 15 | | koel-podcast-enclosure-ipv6-ssrf | https://github.com/koel/koel | untrusted_destination | CVE-2026-54494 / GHSA-rjg7-r26h-cfp2 | 2026-07-15 | `8ee1de0f5589` → `5f6ce2cefd08` | 9.7.1 | `app/Helpers/Network.php::isPublicHost`, `app/Values/Podcast/EpisodePlayable.php::createForEpisode` | MIT | 29 | |
|
||||
| 16 | | pagible-admin-proxy-dns-rebinding-ssrf | https://github.com/aimeos/pagible | untrusted_destination | CVE-2026-49262 / GHSA-mmj8-wcvw-6789 | 2026-06-26 | `03f03b2f81b2` → `09a8205d513e` | 0.10.4 | `admin/src/Controllers/AdminController.php::proxy`, `core/src/Utils.php::isValidUrl` | MIT | 73 | |
|
||||
| 17 | R | wp-graphql-updatepost-publish-bypass | https://github.com/wp-graphql/wp-graphql | access_control | CVE-2026-88974 / GHSA-5mmc-8pc9-wggg | 2026-09-23 | `d53b0e566f20` → `55441663eaa3` | 2.22.2 | file only: `plugins/wp-graphql/src/Mutation/PostObjectUpdate.php` | GPL-3.0 | 120 | yes |
|
||||
| 18 | R | sylius-mollie-thankyou-order-idor | https://github.com/Sylius/MolliePlugin | access_control | CVE-2026-68501 / GHSA-x83g-979r-f5fh | 2026-07-31 | `7400f1489a4b` → `01316b3ad3cf` | 2.2.8 / 3.2.4 / 3.3.1 | `src/Controller/Shop/PageRedirectController.php::thankYouAction`, `src/Controller/Shop/QrCodeAction.php::fetchQrCodeFromOrder` | MIT | 9 | |
|
||||
| 19 | | paymenter-checkout-config-override | https://github.com/Paymenter/Paymenter | access_control | CVE-2026-47198 / GHSA-5q4q-834j-g8g4 | 2026-06-30 | `9d305ea444f7` → `75719ae46b40` | 1.5.1 | file only: `app/Livewire/Products/Checkout.php`, `app/Livewire/Cart.php` | MIT | 15 | |
|
||||
| 20 | | wp-buddypress-messages-rest-idor | https://github.com/buddypress/buddypress | access_control | CVE-2026-53673 / GHSA-j3j5-5m8v-7gvc | 2026-06-10 | `56b1fd3b4054` → `27d41e9749a8` | 14.5.0 | `src/bp-messages/classes/class-bp-messages-rest-controller.php::get_item_permissions_check` | GitHub: none [verify] | 79 | yes |
|
||||
| 21 | R | sharp-editor-iframe-srcdoc-xss | https://github.com/code16/sharp | output_encoding | CVE-2026-61823 / GHSA-qxg3-46rw-79j8 | 2026-09-25 | `677431493090` → `ec509a22c808` | 9.22.5 | file only: `src/Utils/Sanitization/FormatsSanitizedValue.php` | MIT | 109 | |
|
||||
| 22 | R | privatebin-jsonld-request-uri-injection | https://github.com/PrivateBin/PrivateBin | output_encoding | CVE-2026-55891 / GHSA-xrjc-c68j-hp7w | 2026-08-28 | `e0dd4c025c19` → `75f056dcda95` | 2.0.5 | `lib/Request.php::getRequestUri`, `lib/Controller.php::_jsonld` | GitHub: NOASSERTION [verify] | 16 | |
|
||||
| 23 | R | openstamanager-group-name-html-injection | https://github.com/devcode-it/openstamanager | output_encoding | CVE-2026-44701 / GHSA-crx4-7mmq-j74j | 2026-08-26 | `57a2734e2121` → `0b88ad196387` | 2.11-beta | `modules/utenti/edit.php::<global>` | GPL-3.0 | 43 | |
|
||||
| 24 | | symfony-ux-live-child-tag-xss | https://github.com/symfony/ux | output_encoding | CVE-2026-49210 / GHSA-38x5-rcv4-xf7x | 2026-06-19 | `a6e9bf808476` → `fbc5e9a1bda7` | 2.36.0 / 3.1.0 | `src/LiveComponent/src/Util/ChildComponentPartialRenderer.php::createHtml` | MIT | 59 | |
|
||||
| 25 | R | wp-podlove-generation-guard-poi | https://github.com/podlove/podlove-publisher | deserialization | CVE-2026-16099 / GHSA-q83h-6954-x259 | 2026-08-16 | `4eedbd667969` → `ea878175f261` | 4.5.4 (not stated in the advisory; first release after the fix) | `lib/modules/shownotes/rest_api.php::create_link_item`, `lib/image_cache/generation_guard.php::__destruct` | MIT | 33 | yes |
|
||||
| 26 | | wp-admin-columns-ids-to-collection-poi | https://github.com/codepress/admin-columns | deserialization | CVE-2026-7654 / GHSA-h6x4-229g-5f43 | 2026-06-06 | `ea6db9a02e01` → `c64e9aaecd20` | 7.0.19 (not stated; first release after the fix) | `classes/Formatter/IdsToCollection.php::get_ids_from_string` | GPL-3.0 | 37 | yes |
|
||||
|
||||
Full SHAs are in the JSON. Fix commits the advisory does not name (inferred from a release compare or file history): grav, backpack, facturascripts, pagible, paymenter, podlove, admin-columns.
|
||||
|
||||
### Per-candidate notes
|
||||
|
||||
#### grav-blueprint-bare-function-rce (injection; code injection (callable allow-list bypass → arbitrary PHP function))
|
||||
- Repo https://github.com/getgrav/grav (MIT, 34 MB, languages PHP 100%). Lane A. Advisory CVE-2026-75827 / GHSA-f8wv-xp27-6gq7, published 2026-09-17; affected <= 2.0.14; fixed 2.0.15.
|
||||
- Sink: system/src/Grav/Common/Data/Blueprint.php:455-458 `call_user_func_array($o, $params)` (bare-function dynamic-data provider), gated by `Blueprint::isSafeDynamicCall()` (Blueprint.php:514-536), whose bare-function branch uses only the denylist `Utils::isDangerousFunction()`; twin sink system/src/Grav/Framework/Flex/FlexDirectory.php:936-938 `call_user_func_array($function, $params)`. Source: a blueprint `data-*@` directive naming a bare PHP function (e.g. `error_log`) with attacker parameters, resolved by `Blueprint::init()`. Privilege: authenticated account that can edit pages/blueprint data (per the advisory's root-cause section; verify).
|
||||
- Fix: The advisory links no commit. Found in the release compare 2.0.14...2.0.15 (14 commits, each security fix labelled): `af31066e` '[security] Gate the blueprint bare-function data directive on a positive allowlist', 1 file (Blueprint.php +11-1). A later commit in the same release, `c51987f5` 'Trust blueprint dynamic-data providers by provenance, not by allowlist', reworks the same gate. Ancestry: parent; ancestor of tag 2.0.15.
|
||||
- Sites: file paths and function names quoted by the advisory. sink_evidence `call_user_func_array($o, $params)`.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `8661b3b56448` "Deduplicate repeated debugger deprecation traces" (172 lines, CHANGELOG.md, system/src/Grav/Common/Debugger.php, tests/unit/Grav/Common/DebuggerDeprecationsTest.php).
|
||||
- Caveats: guard bypass: the vulnerable pin already has a denylist check; the enclosing function of the call_user_func_array line is not named by this advisory (sink_evidence is used instead). Confidence: high.
|
||||
|
||||
#### backpack-stats-host-header-cmdi (injection; OS command injection)
|
||||
- Repo https://github.com/Laravel-Backpack/CRUD (MIT, 76 MB, languages PHP 80%, Blade 17%). Lane A. Advisory CVE-2026-54182 / GHSA-mrc5-3mm3-45c5, published 2026-08-20; affected 7.x < 7.0.36 (also 4.1.x < 4.1.72, 5.x < 5.6.2, 6.x < 6.8.13); fixed 7.0.36.
|
||||
- Sink: `Backpack\CRUD\Stats::makeCurlRequest` builds a shell command and passes it to `exec()`. Source: HTTP `Host` header, on every production request through `BackpackServiceProvider::boot()` (1-in-100 random gate). Privilege: unauthenticated.
|
||||
- Fix: The advisory links only release tags. The compare 7.0.35...7.0.36 contains exactly one commit, `282ba2a0` 'refactor stats to use guzzle only' (src/Stats.php +6-31), matching the advisory's 'makeCurlRequest was replaced with the Guzzle-based path'. Ancestry: parent; the fixed commit is tag 7.0.36.
|
||||
- Sites: class and method named by the advisory; the path src/Stats.php follows from the PSR-4 class name and is confirmed only by the fix commit's file list.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `608652b1020b` "fix color display (#5992)" (8 lines, src/resources/views/crud/columns/color.blade.php).
|
||||
- Caveats: needs exec() and curl available, and a Host header that the web server passes through (the advisory rates AC:H). Confidence: high.
|
||||
|
||||
#### winter-filter-numberrange-sqli (injection; SQL injection)
|
||||
- Repo https://github.com/wintercms/winter (MIT, 68 MB, languages PHP 70%, JavaScript 19%, Less 8%). Lane A. Advisory CVE-2026-32593 / GHSA-m7jc-g4rc-jmvh, published 2026-08-12; affected <= 1.2.12; fixed 1.2.13.
|
||||
- Sink: `Backend\Widgets\Filter` builds SQL for a `numberrange` scope configured with a `conditions` key. Source: the filter widget's AJAX handler (`numbersFromAjax`, per the advisory title). Privilege: authenticated backend user with access to an affected list view.
|
||||
- Fix: Advisory workaround names commit `50713de9` 'Use parameter binding for filter scope values in conditions statements' (modules/backend/widgets/Filter.php +64-24, plus a test). Ancestry: parent; ancestor of tag v1.2.13, not of v1.2.12.
|
||||
- Sites: class `Backend\Widgets\Filter` and `numbersFromAjax` named by the advisory; path confirmed by the fix commit.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `2a342f94ae7c` "Improved Ukrainian translation (#1467)" (46 lines, modules/backend/lang/uk/lang.php).
|
||||
- Caveats: no built-in view uses the vulnerable scope/config combination: exploitable only with a plugin that registers a numberrange scope with `conditions` (the sink is in core, the configuration is not). Confidence: medium.
|
||||
|
||||
#### mantisbt-tokenizer-eval-rce (injection; code injection (eval))
|
||||
- Repo https://github.com/mantisbt/mantisbt (GitHub: none [verify], 89 MB, languages PHP 82%, CSS 10%). Lane A. Advisory CVE-2026-49273 / GHSA-v84x-qvhg-f36r, published 2026-07-15; affected >= 1.3.0, <= 2.28.3; fixed 2.28.4.
|
||||
- Sink: `Tokenizer` (reached through `ConfigParser`) calls `eval()` with a `return;` prefix; PHP hoists class declarations past the return. Source: non-string configuration value submitted to adm_config_set.php (Manage Configuration). Privilege: administrator.
|
||||
- Fix: Advisory patch link `78c0af63` 'Remove use of eval() in Tokenizer class' (core/classes/Tokenizer.class.php +2-6). Ancestry: parent; ancestor of tag release-2.28.4.
|
||||
- Sites: the advisory names adm_config_set.php, ConfigParser, Tokenizer and eval(); the Tokenizer file path and the enclosing function are not named (path from the fix commit). sink_evidence `eval(`.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `c867fcc0c119` "Web UI: use project category commands" (58 lines, manage_proj_cat_add.php, manage_proj_cat_copy.php, manage_proj_cat_delete.php…).
|
||||
- Caveats: function-level site must be resolved at freeze by reading the file (the advisory names only the class). Confidence: high.
|
||||
|
||||
#### gitlist-commandline-cmdi (injection; OS command / argument injection)
|
||||
- Repo https://github.com/klaussilveira/gitlist (BSD-3-Clause, 6 MB, languages PHP 87%, Twig 9%). Lane B. Advisory CVE-2026-82668 / GHSA-v64j-pgc3-qgc9, published 2026-08-31; affected 2.0.0; fixed 3.0.0-beta.
|
||||
- Sink: `getDefaultBranch` of src/SCM/System/Git/CommandLine.php (git command line). Source: repository content/arguments reaching the git command line (not stated precisely). Privilege: not stated ('remotely').
|
||||
- Fix: Advisory names patch `88cf2866` 'Prevent textconv. Fixes #947.' (src/SCM/System/Git/CommandLine.php +2-2, plus a test). Ancestry: parent; ancestor of tag 3.0.0-beta.
|
||||
- Sites: file and function quoted by the advisory (VulDB-generated text).
|
||||
- Benign: fixed pin as fixed-side control; no API proposal passed the freeze.py filter within 60 commits after the fix; let freeze.py propose one from the clone, or pick by hand (a small non-security PHP change on the default branch that does not touch the sink file).
|
||||
- Caveats: VulDB auto-generated description; the fix message ('Prevent textconv') suggests git config/attribute abuse rather than shell metacharacters, and the patched lines may not be in getDefaultBranch -- verify at freeze. Confidence: medium.
|
||||
|
||||
#### qloapps-customermessage-bo-query-sqli (injection; SQL injection)
|
||||
- Repo https://github.com/Qloapps/QloApps (OSL-3.0, 129 MB, languages PHP 80%, Smarty 9%, JavaScript 7%). Lane B. Advisory CVE-2026-75497 / GHSA-5w6r-2mxc-fq3v, published 2026-08-25; affected not stated ('Fixed in 123c97c'); fixed commit 123c97c.
|
||||
- Sink: database query in CustomerMessage.php built from the `bo_query` parameter. Source: `bo_query` request parameter. Privilege: administrator.
|
||||
- Fix: Advisory names `123c97c1` 'Fixed: validate request parameters before database query' (PR #1783): classes/CustomerMessage.php +1-1, classes/Address.php +8-8 (twin advisory GHSA-mj48-jjc7-6wqr) and three hotel-module classes. Ancestry: parent.
|
||||
- Sites: the advisory names 'CustomerMessage.php' (basename) and the parameter; the classes/ path is from the fix commit; no function named.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `54c12b4efe41` "fixed issues" (106 lines, classes/order/Order.php, classes/order/OrderPayment.php, modules/hotelreservationsystem/classes/HotelBookingDetail.php…).
|
||||
- Caveats: one fix covers two advisories (Address.php is GHSA-mj48); repo 129 MB, near the size cap; function-level site to be resolved at freeze. Confidence: medium.
|
||||
|
||||
#### roundcube-markasjunk-cmd-learn-rce (injection; OS command injection)
|
||||
- Repo https://github.com/roundcube/roundcubemail (GPL-3.0, 67 MB, languages PHP 90%, JavaScript 7%). Lane B. Advisory CVE-2026-74997 / GHSA-q5rh-mmp4-ppgq, published 2026-08-17; affected < 1.6.18; 1.7.x < 1.7.3; fixed 1.6.18 / 1.7.3.
|
||||
- Sink: the markasjunk plugin's cmd_learn driver runs a shell command with placeholder replacement values. Source: crafted placeholder replacement values (message/user data) when a message is marked as junk. Privilege: authenticated webmail user.
|
||||
- Fix: Advisory links `14044f84` (master) and `b8f90e28` (1.6 line), both 'Fix RCE via cmd_learn driver of markasjunk plugin' (plugins/markasjunk/drivers/cmd_learn.php +14-10), and a follow-up warning fix `495d2116`. The 1.6-line pin is used because it is an ancestor of tag 1.6.18. Ancestry: parent; ancestor of tag 1.6.18.
|
||||
- Sites: the advisory names 'the cmd_learn driver of the markasjunk plugin'; the file path follows from that name and is confirmed by the fix commit; no function named.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `747f544708ed` "Simplify" (6 lines, tests/Framework/TnefDecoderTest.php); tests-only change, on master (the fixed pin is on the 1.6 line).
|
||||
- Caveats: only instances that enable the markasjunk plugin with the cmd_learn driver. Confidence: medium.
|
||||
|
||||
#### maccms10-template-blacklist-rce (injection; code injection (template → PHP, blacklist bypass))
|
||||
- Repo https://github.com/magicblack/maccms10 (GitHub: NOASSERTION [verify], 29 MB, languages PHP 66%, HTML 23%, JavaScript 9%). Lane B. Advisory CVE-2026-71232 / GHSA-cv48-22qx-mpmx, published 2026-08-05; affected before commit 71ad3bb (2026-06-22); fixed commit 71ad3bb.
|
||||
- Sink: ThinkPHP `{if condition=...}` compiled into raw PHP; the admin template editor only blocks a function blacklist. Source: template content saved through the admin template editor. Privilege: administrator.
|
||||
- Fix: Advisory names `71ad3bb2` (application/admin/controller/Template.php +1-1: adds the missing functions to the blacklist regex). Ancestry: parent.
|
||||
- Sites: file path quoted by the advisory; no function named.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `668ba418e50e` "fix #1356 修正注册API校验器解析错误及缺参报错" (166 lines, application/api/controller/User.php, application/common/model/User.php, tests/regression/user_register_validate.php…); review: subject says it fixes a register-API validator.
|
||||
- Caveats: guard bypass (incomplete blacklist); the fix is still a blacklist; licence not declared on GitHub. Confidence: medium.
|
||||
|
||||
#### kirby-realpath-sibling-traversal (path; path traversal (prefix containment check))
|
||||
- Repo https://github.com/getkirby/kirby (GitHub: NOASSERTION [verify], 66 MB, languages PHP 77%, Vue 15%). Lane A. Advisory CVE-2026-75592 / GHSA-6j4c-mgqr-qv76, published 2026-09-02; affected < 4.9.5; 5.0.0 - 5.5.1; fixed 4.9.5 / 5.5.2.
|
||||
- Sink: `Dir::realpath()` / `F::realpath()` containment check accepts a sibling directory sharing the prefix (site vs site2). Source: media handler request path for not-yet-generated thumbnails. Privilege: unauthenticated.
|
||||
- Fix: Advisory links `2b6fab95` 'fix: Ensure containment in ::realpath()' (4.x line: src/Filesystem/Dir.php, src/Filesystem/F.php) and `e0dca5f7` (5.x line, which also carries the Media::thumb fix of GHSA-9vx2-j98c-p72w). The 4.x pin keeps the case to this advisory alone. Ancestry: parent; ancestor of tag 4.9.5 (not of 5.5.2).
|
||||
- Sites: `Dir::realpath()` and `F::realpath()` named by the advisory; paths confirmed by the fix commit.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `3989964ac1cb` "fix: Stuck unsaved changes for untouched fields (#8355)" (80 lines, config/fields/object.php, config/fields/toggle.php, src/Form/Field.php…); on the default branch (5.x); the fixed pin is on the 4.x line.
|
||||
- Caveats: only deployments whose index root has a sibling directory with the same name prefix; Kirby is under a proprietary licence (GitHub NOASSERTION); must be cleared before publishing. Confidence: medium.
|
||||
|
||||
#### mineadmin-appstore-identifier-traversal (path; path traversal)
|
||||
- Repo https://github.com/mineadmin/MineAdmin (Apache-2.0, 53 MB, languages PHP 99%). Lane A. Advisory CVE-2026-55224 / GHSA-59xm-4m8c-g3xj, published 2026-08-18; affected < 3.2.0-alpha.2; fixed 3.2.0-alpha.2.
|
||||
- Sink: `BASE_PATH . '/plugin/' . $params['identifier']` in download()/install()/unInstall(), then `Plugin::install($params['identifier'])`. Source: `identifier` request parameter of the app-store plugin endpoints. Privilege: authenticated admin with app-store access (verify).
|
||||
- Fix: Advisory links `ca41902a` 'fix(app-store): 修复app-store几处安全风险。 (#728)' (Service.php +81-9, IndexController.php +4-0). Ancestry: parent; the fixed commit is tag v3.2.0-alpha.2.
|
||||
- Sites: file path and methods quoted by the advisory.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `c2f97b772c10` "feat(install): 添加前端项目下载与依赖安装功能 (#735)" (193 lines, app/Command/InstallCommand.php).
|
||||
- Caveats: Hyperf framework (not Laravel/Symfony). Confidence: high.
|
||||
|
||||
#### facturascripts-uploadedfile-move-traversal (path; path traversal (arbitrary file write))
|
||||
- Repo https://github.com/NeoRazorX/facturascripts (LGPL-3.0, 60 MB, languages PHP 90%, Twig 9%). Lane A. Advisory GHSA-hgjx-r89m-m7v4, published 2026-07-14; affected >= 2025, <= 2026.2; fixed not stated.
|
||||
- Sink: `UploadedFile::move($destiny, $destinyName)` concatenates the client filename into `move_uploaded_file()`. Source: `getClientOriginalName()` of an uploaded file. Privilege: authenticated user.
|
||||
- Fix: The advisory links no commit and states no patched version. File history of Core/UploadedFile.php has `d3770be6` 'Fix uploaded file path traversal' (2026-05-19; UploadedFile.php +38-4 plus tests). Inferred, not confirmed by the advisory. Ancestry: parent.
|
||||
- Sites: file and method quoted by the advisory.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `add96d2bbee5` "test: add API smoke test" (192 lines, Test/Core/Controller/ApiSmokeTest.php); tests-only change.
|
||||
- Caveats: fix commit inferred from file history; the advisory's affected range ends at 2026.2 while the inferred fix is dated 2026-05-19 -- confirm at freeze that the fixed pin is really unaffected. Confidence: medium.
|
||||
|
||||
#### wp-geo-my-wp-info-window-lfi (path; local file inclusion)
|
||||
- Repo https://github.com/Fitoussi/geo-my-wp (GitHub: none [verify], 9 MB, languages PHP 47%, JavaScript 44%, CSS 9%). Lane B. Advisory CVE-2026-85200 / GHSA-6hw9-7gx5-mw88, published 2026-09-12; affected <= 4.5.5.3; fixed 4.5.5.4.
|
||||
- Sink: include of a file chosen through `gmw_posts_locator_ajax_info_window_loader`. Source: AJAX request parameters (unauthenticated). Privilege: unauthenticated.
|
||||
- Fix: Advisory links `5a768bf1` 'Release 4.5.5.4' (5 files: includes/gmw-functions.php +93-6, the posts/members locator functions files, version bumps). A release commit, but small and on-topic. Ancestry: parent.
|
||||
- Sites: function named by the advisory text; files from the advisory's own trac reference links (gmw-functions.php#L1945/L1987, gmw-posts-locator-functions.php#L964/L975); the function-to-file pairing is an assumption to confirm at freeze.
|
||||
- Benign: fixed pin as fixed-side control; no API proposal passed the freeze.py filter within 60 commits after the fix; let freeze.py propose one from the clone, or pick by hand (a small non-security PHP change on the default branch that does not touch the sink file).
|
||||
- Caveats: WordPress plugin; GitHub shows no licence (plugin header GPL presumed, verify); release commit (small). Confidence: medium.
|
||||
|
||||
#### elfinder-fsock-dns-rebinding-ssrf (untrusted_destination; SSRF (DNS rebinding guard bypass))
|
||||
- Repo https://github.com/Studio-42/elFinder (GitHub: NOASSERTION [verify], 43 MB, languages JavaScript 65%, PHP 29%). Lane A. Advisory CVE-2026-81889 / GHSA-8x3q-jpjh-qh5c, published 2026-08-31; affected <= 2.1.69; fixed 2.1.70.
|
||||
- Sink: `fsock_get_contents()` fallback reconnects by hostname after the URL validation resolved it (TOCTOU). Source: URL submitted for server-side upload. Privilege: depends on the integrating connector.
|
||||
- Fix: Advisory links `6d997386` 'Merge commit from fork' (php/elFinder.class.php +113-12, i18n) and the build commit `191372c1`. `6d997386` has a single parent `dd73e702` (the fix of the sibling ZIP-extraction advisory). Ancestry: parent; ancestor of tag 2.1.70-src (tag 2.1.70 is a build branch with no common history).
|
||||
- Sites: function named by the advisory, file linked by the advisory.
|
||||
- Benign: fixed pin as fixed-side control; no API proposal passed the freeze.py filter within 60 commits after the fix; let freeze.py propose one from the clone, or pick by hand (a small non-security PHP change on the default branch that does not touch the sink file).
|
||||
- Caveats: only when PHP lacks cURL; guard bypass (DNS rebinding); repository is 65% JavaScript; licence NOASSERTION on GitHub (verify). Confidence: medium.
|
||||
|
||||
#### easyappointments-caldav-ssrf (untrusted_destination; SSRF)
|
||||
- Repo https://github.com/alextselegidis/easyappointments (GPL-3.0, 49 MB, languages PHP 89%, JavaScript 9%). Lane A. Advisory CVE-2026-52840 / GHSA-pm5p-7w5h-jm5q, published 2026-07-29; affected <= 1.5.2; fixed 1.6.0.
|
||||
- Sink: `$this->caldav_sync->test_connection($caldav_url, ...)` → Guzzle REPORT request. Source: `caldav_url` request parameter of Caldav::connect_to_server. Privilege: authenticated backend user (admin, provider or secretary).
|
||||
- Fix: Advisory links four linear commits of 2026-05-26: `4abb1054` 'fix: harden caldav url validation against ssrf' (the validation), then `6b34b78c`, `2da2baed`, `6eb9336a` (opt-in private-host toggle and its relocation). The first carries the fix and is used as the fixed pin; `6eb9336a` is the end of the chain if the maintainer prefers it. Ancestry: parent of 4abb1054; ancestor of tag 1.6.0.
|
||||
- Sites: controller path and method quoted by the advisory; the library path follows from `$this->caldav_sync` and the fix commit.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `1b51132026c7` "Minor translation change." (2 lines, application/language/german/translations_lang.php).
|
||||
- Caveats: fix spread over four linear commits (v2 allowed up to three; the first commit alone is the fix). Confidence: high.
|
||||
|
||||
#### koel-podcast-enclosure-ipv6-ssrf (untrusted_destination; SSRF (guard bypass via IPv6 transition addresses))
|
||||
- Repo https://github.com/koel/koel (MIT, 29 MB, languages PHP 55%, TypeScript 27%, Vue 18%). Lane A. Advisory CVE-2026-54494 / GHSA-rjg7-r26h-cfp2, published 2026-07-15; affected <= 9.7.0; fixed 9.7.1.
|
||||
- Sink: `EpisodePlayable::createForEpisode()` → `Http::sink($file)->get($url)`, guarded only by `Network::isPublicHost()`. Source: `<enclosure url>` of an attacker-hosted RSS feed subscribed by any user. Privilege: authenticated user.
|
||||
- Fix: Advisory links `5f6ce2ce` 'fix: close remaining SSRF advisories (DNS rebinding, IPv6 transition, bail) (#2549)', 28 files; one commit that closes several SSRF advisories (same family). Ancestry: parent; ancestor of tag v9.7.1.
|
||||
- Sites: app/Helpers/Network.php and isPublicHost quoted by the advisory; EpisodePlayable path from the class name `App\Values\Podcast\EpisodePlayable`.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `eafb61deaad0` "test: serve a real response to the radio stream connector (#2671)" (178 lines, tests/Integration/Services/Radio/RadioStreamConnectorTest.php, tests/Unit/Services/Radio/RadioStreamConnectorTest.php); tests-only change.
|
||||
- Caveats: large multi-advisory fix commit (28 files), same family; exploitation needs NAT64/6to4 routing. Confidence: medium.
|
||||
|
||||
#### pagible-admin-proxy-dns-rebinding-ssrf (untrusted_destination; SSRF (DNS rebinding guard bypass))
|
||||
- Repo https://github.com/aimeos/pagible (MIT, 73 MB, languages PHP 57%, JavaScript 16%, CSS 12%, Vue 11%). Lane A. Advisory CVE-2026-49262 / GHSA-mmj8-wcvw-6789, published 2026-06-26; affected < 0.10.4; fixed 0.10.4.
|
||||
- Sink: `AdminController::proxy` → Guzzle request after `\Aimeos\Cms\Utils::isValidUrl($url)`. Source: `url` query parameter of `/cmsproxy`. Privilege: authenticated (proxy token).
|
||||
- Fix: The advisory links no commit. The compare 0.10.3...0.10.4 has three commits; `09a8205d` 'Fixed SSRF via DNS rebinding in admin proxy' (admin/src/Controllers/AdminController.php +21-7, core/src/Utils.php +22-4). Ancestry: parent; ancestor of tag 0.10.4.
|
||||
- Sites: class::method names quoted by the advisory; paths from the fix commit.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `bc5a10a515bd` "Fixed admin test" (2 lines, admin/tests/AdminControllerTest.php); tests-only change.
|
||||
- Caveats: guard bypass (DNS rebinding). Confidence: medium.
|
||||
|
||||
#### wp-graphql-updatepost-publish-bypass (access_control; missing object-level capability check)
|
||||
- Repo https://github.com/wp-graphql/wp-graphql (GPL-3.0, 120 MB, languages PHP 68%, JavaScript 26%). Lane A. Advisory CVE-2026-88974 / GHSA-5mmc-8pc9-wggg, published 2026-09-23; affected < 2.22.2 (confirmed 2.19.0); fixed 2.22.2.
|
||||
- Sink: `updatePost` mutation checks only `$post_type_object->cap->edit_posts`. Source: GraphQL `updatePost` mutation input (status PUBLISH, published post id). Privilege: Contributor.
|
||||
- Fix: Advisory links `55441663` (PR #4270) whose subject is 'do not invoke bare callable-string field definitions in Model'; it also changes plugins/wp-graphql/src/Mutation/PostObjectUpdate.php (+27-0) -- two fixes in one commit. Ancestry: parent; ancestor of tag wp-graphql/v2.22.2.
|
||||
- Sites: the advisory names src/Mutation/PostObjectUpdate.php; the repository is a monorepo (plugins/wp-graphql/ prefix from the fix commit); no function named.
|
||||
- Benign: fixed pin as fixed-side control; no API proposal passed the freeze.py filter within 60 commits after the fix; let freeze.py propose one from the clone, or pick by hand (a small non-security PHP change on the default branch that does not touch the sink file).
|
||||
- Caveats: fix commit also carries an unrelated Model.php change; function-level site to resolve at freeze. Confidence: medium.
|
||||
|
||||
#### sylius-mollie-thankyou-order-idor (access_control; IDOR (missing ownership check))
|
||||
- Repo https://github.com/Sylius/MolliePlugin (MIT, 9 MB, languages PHP 90%). Lane A. Advisory CVE-2026-68501 / GHSA-x83g-979r-f5fh, published 2026-07-31; affected < 2.2.8; 3.0.0-3.2.3; 3.3.0; fixed 2.2.8 / 3.2.4 / 3.3.1.
|
||||
- Sink: `findOneBy(['id' => $orderId])` without ownership/session check, returning the order token (302 Location) and QR code. Source: `orderId` of GET /{_locale}/thank-you and /{_locale}/get-code. Privilege: unauthenticated.
|
||||
- Fix: Advisory links three merge commits (one per branch). `01316b3a` (PR #354, 3.3 line) is used: PageRedirectController.php +17-6, QrCodeAction.php +9-11, PaymentWebhookController.php (the sibling webhook advisory GHSA-rc52-c4hv-w89p). Ancestry: first parent of a merge; ancestor of tag v3.3.1.
|
||||
- Sites: Class::method names quoted by the advisory; paths from the fix commit.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `ad215c9c0fc3` "Refactor `PaymentMethodLogoUploader` to use `FilesystemAdapterInterface` instead of `Gaufrette\Filesystem`." (15 lines, config/services/uploader.xml, src/Uploader/PaymentMethodLogoUploader.php).
|
||||
- Caveats: merge commit; one fix covers two advisories (webhook forgery GHSA-rc52 is the other). Confidence: high.
|
||||
|
||||
#### paymenter-checkout-config-override (access_control; unvalidated URL-writable properties override provisioning parameters)
|
||||
- Repo https://github.com/Paymenter/Paymenter (MIT, 15 MB, languages PHP 87%, Blade 12%). Lane A. Advisory CVE-2026-47198 / GHSA-5q4q-834j-g8g4, published 2026-06-30; affected < 1.5.1; fixed 1.5.1.
|
||||
- Sink: `$checkoutConfig` (`#[Url]`, alias `config`) keys stored by Cart.php into service properties, later passed to createServer(). Source: `config[...]` query parameters of the checkout page. Privilege: authenticated customer.
|
||||
- Fix: The advisory links no commit. The compare v1.5.0...v1.5.1 has 17 commits; `75719ae4` 'Merge commit from fork' changes only app/Livewire/Products/Checkout.php (+7-1). Ancestry: parent; ancestor of tag v1.5.1.
|
||||
- Sites: file paths quoted by the advisory; no function named.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `e60567981acf` "fix: reCAPTCHA v3 not working after Livewire navigation" (24 lines, themes/default/views/components/captcha/recaptcha-v3.blade.php); review: subject mentions reCAPTCHA (anti-abuse).
|
||||
- Caveats: mass-assignment shape (CWE-20/639); classed access_control by its CWE; fix commit inferred from the release compare. Confidence: medium.
|
||||
|
||||
#### wp-buddypress-messages-rest-idor (access_control; IDOR in REST permission callback)
|
||||
- Repo https://github.com/buddypress/buddypress (GitHub: none [verify], 79 MB, languages PHP 76%, CSS 14%, SCSS 6%). Lane A. Advisory CVE-2026-53673 / GHSA-j3j5-5m8v-7gvc, published 2026-06-10; affected < 14.5.0 (confirmed 14.4.0); fixed 14.5.0.
|
||||
- Sink: `get_item_permissions_check` validates the supplied user_id instead of the logged-in user; reused by update/delete. Source: `user_id` parameter of the messages REST API. Privilege: authenticated (any user).
|
||||
- Fix: Advisory links `27d41e97` 'Improve security of messages endpoint.' (class-bp-messages-rest-controller.php +113-42, tests). The commit is dated 2026-07-07, after the advisory; confirm at freeze that the release carrying it is 14.5.0. Ancestry: parent.
|
||||
- Sites: method named by the advisory; file path from the fix commit.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `ea5cb85f4d8a` "PHPDoc: Improve parameter descriptions." (132 lines, src/bp-activity/bp-activity-embeds.php, src/bp-activity/bp-activity-filters.php, src/bp-activity/bp-activity-functions.php…).
|
||||
- Caveats: the GitHub repository is the project's own mirror of its development SVN; no GitHub licence metadata (GPL-2.0+ per project, verify). Confidence: medium.
|
||||
|
||||
#### sharp-editor-iframe-srcdoc-xss (output_encoding; stored XSS (sanitizer allow-list))
|
||||
- Repo https://github.com/code16/sharp (MIT, 109 MB, languages PHP 66%, Vue 23%, TypeScript 9%). Lane A. Advisory CVE-2026-61823 / GHSA-qxg3-46rw-79j8, published 2026-09-25; affected < 9.22.5; fixed 9.22.5.
|
||||
- Sink: Symfony HtmlSanitizer config allows the iframe `srcdoc` attribute; the value is rendered as live HTML. Source: Editor field content saved by a user with edit permission. Privilege: authenticated editor.
|
||||
- Fix: Advisory links `ec509a22` 'Improve sanitization' (src/Utils/Sanitization/FormatsSanitizedValue.php +4-2, test). The same commit fixes the sibling GHSA-vj3q-vp3g-j9c8. Ancestry: parent; ancestor of tag v9.22.5.
|
||||
- Sites: file path quoted by the advisory; no function named. sink_evidence `srcdoc`.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `7c601fffe930` "Update version to v9.22.5" (2 lines, src/SharpInternalServiceProvider.php).
|
||||
- Caveats: the flaw is a sanitizer allow-list entry, not a missing escape call. Confidence: medium.
|
||||
|
||||
#### privatebin-jsonld-request-uri-injection (output_encoding; reflected JSON injection (CWE-116))
|
||||
- Repo https://github.com/PrivateBin/PrivateBin (GitHub: NOASSERTION [verify], 16 MB, languages PHP 46%, JavaScript 38%, CSS 16%). Lane A. Advisory CVE-2026-55891 / GHSA-xrjc-c68j-hp7w, published 2026-08-28; affected <= 2.0.4; fixed 2.0.5.
|
||||
- Sink: `Controller::_jsonld()` splices `$_urlBase` into a .jsonld template with `str_replace()`. Source: `$_SERVER['REQUEST_URI']` via `Request::getRequestUri()` (FILTER_SANITIZE_URL keeps quotes and angle brackets). Privilege: unauthenticated.
|
||||
- Fix: Advisory links `75f056dc` 'Merge commit from fork' (first-parent diff: lib/Request.php +2-2, lib/Configuration.php, CHANGELOG, test). Its first parent `e0dd4c02` is the fix of the sibling attachment-XSS advisory GHSA-f2xf-7x3g-4272. Ancestry: first parent of a merge; ancestor of tag 2.0.5.
|
||||
- Sites: Class::method names quoted by the advisory; paths lib/*.php from the fix commit.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `48ae4ed5d3d4` "fix: quote administration storage path" (54 lines, bin/administration, tst/AdministrationEmptyDirsTest.php); review: 'quote administration storage path' may be a shell-quoting fix.
|
||||
- Caveats: the fix changes Request.php only; Controller::_jsonld is a declared site outside the diff; response type is JSON-LD, not HTML; licence NOASSERTION on GitHub (zlib-style per project, verify). Confidence: high.
|
||||
|
||||
#### openstamanager-group-name-html-injection (output_encoding; stored HTML injection)
|
||||
- Repo https://github.com/devcode-it/openstamanager (GPL-3.0, 43 MB, languages PHP 80%, JavaScript 17%). Lane A. Advisory CVE-2026-44701 / GHSA-crx4-7mmq-j74j, published 2026-08-26; affected <= 2.10.1; fixed 2.11-beta.
|
||||
- Sink: `'<span class="text-primary">'.$group->getTranslation('title').'</span>'` in modules/utenti/edit.php. Source: group name saved when creating a user group. Privilege: authenticated user able to create groups.
|
||||
- Fix: Advisory links `0b88ad19` 'fix: HTML Injection in modules/utenti/edit.php' (edit.php +2-2, ajax/select.php, src/AJAX.php, src/Modules.php) and a broader input-filter refactor `7c9eafe9`. The first is used. Ancestry: parent; ancestor of tag v2.11-beta.
|
||||
- Sites: file and code quoted by the advisory. sink_evidence `$group->getTranslation('title')`.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `570eff68c504` "fix: definizione variabile" (2 lines, modules/fatture/src/Gestori/Scadenze.php).
|
||||
- Caveats: second advisory commit (7c9eafe9, global GET/POST filtering) is not in the pinned pair. Confidence: high.
|
||||
|
||||
#### symfony-ux-live-child-tag-xss (output_encoding; XSS (unescaped tag name))
|
||||
- Repo https://github.com/symfony/ux (MIT, 59 MB, languages PHP 61%, TypeScript 22%, JavaScript 10%, Twig 7%). Lane A. Advisory CVE-2026-49210 / GHSA-38x5-rcv4-xf7x, published 2026-06-19; affected ux-live-component >= 2.8.0, < 2.36.0; >= 3.0.0, < 3.1.0; fixed 2.36.0 / 3.1.0.
|
||||
- Sink: `ChildComponentPartialRenderer::createHtml()` interpolates `$childTag` into HTML. Source: client JSON `children[id].tag` parsed by `LiveComponentSubscriber` and passed through `InterceptChildComponentRenderSubscriber`. Privilege: unauthenticated (Accept header gate; CORS or same-origin needed).
|
||||
- Fix: Advisory links `fbc5e9a1` '[LiveComponent] Reject malicious child component tags' (ChildComponentPartialRenderer.php +7-0, test). Ancestry: parent; ancestor of tag v2.36.0.
|
||||
- Sites: FQCN::method quoted by the advisory; monorepo path from the fix commit.
|
||||
- Benign: fixed pin as fixed-side control; no API proposal passed the freeze.py filter within 60 commits after the fix; let freeze.py propose one from the clone, or pick by hand (a small non-security PHP change on the default branch that does not touch the sink file).
|
||||
- Caveats: the advisory calls it primarily defense in depth (custom Accept header cannot be sent cross-origin without preflight); component framework in a monorepo (UX packages). Confidence: medium.
|
||||
|
||||
#### wp-podlove-generation-guard-poi (deserialization; PHP object injection → arbitrary file deletion (in-plugin POP chain))
|
||||
- Repo https://github.com/podlove/podlove-publisher (MIT, 33 MB, languages PHP 63%, JavaScript 18%, Vue 10%, TypeScript 6%). Lane C. Advisory CVE-2026-16099 / GHSA-q83h-6954-x259, published 2026-08-16; affected <= 4.5.3; fixed 4.5.4 (not stated in the advisory; first release after the fix).
|
||||
- Sink: unserialize() of model data (lib/model/base.php#L552) instantiating `Podlove\ImageCache\GenerationGuard`, whose `__destruct()` calls `wp_delete_file()`. Source: shownotes REST input handled by `create_link_item` (contributor+). Privilege: Contributor.
|
||||
- Fix: The advisory links wp.org trac only. The GitHub history of the referenced files has three consecutive commits of 2026-07-26: `d49c4d7f` 'fix: constrain image lock deletion' (generation_guard.php), `b59d7962` 'fix: harden shownotes REST input' (rest_api.php +173-20), `ea878175` 'fix: limit model unserialization' (lib/model/base.php). The pair spans all three (vulnerable = parent of d49c4d7f). Ancestry: linear chain of 3 commits (4eedbd66 → d49c4d7f → b59d7962 → ea878175); ancestor of tag 4.5.4.
|
||||
- Sites: create_link_item and GenerationGuard::__destruct named by the advisory; files from the advisory's own trac reference links.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `50da8db0f5a8` "fix PHP warning" (2 lines, lib/modules/contributors/contributor_list_table.php).
|
||||
- Caveats: fix inferred from file history (three commits); the advisory text describes the impact (file deletion) under CWE-502. Confidence: medium.
|
||||
|
||||
#### wp-admin-columns-ids-to-collection-poi (deserialization; PHP object injection (bundled POP chain))
|
||||
- Repo https://github.com/codepress/admin-columns (GPL-3.0, 37 MB, languages PHP 69%, Svelte 10%, SCSS 7%, JavaScript 6%, TypeScript 6%). Lane C. Advisory CVE-2026-7654 / GHSA-h6x4-229g-5f43, published 2026-06-06; affected <= 7.0.18; fixed 7.0.19 (not stated; first release after the fix).
|
||||
- Sink: `unserialize()` without `allowed_classes` in `IdsToCollection::get_ids_from_string()`. Source: post meta values set by a Contributor. Privilege: Contributor.
|
||||
- Fix: The advisory links wp.org trac only. The compare 7.0.18...7.0.19 has one commit touching the named file: `c64e9aae` 'Update IdsToCollection.php' (+1-1). An identical change `25e276e0` sits on the 7.1 development line. Inferred; the commit message does not say what changed. Ancestry: parent; ancestor of tag 7.0.19.
|
||||
- Sites: Class::method named by the advisory; files from its trac reference links. sink_evidence `unserialize(`.
|
||||
- Benign: fixed pin as fixed-side control; proposed benign change `e998321863c4` "Todo ans PHP code fixer" (40 lines, classes/ColumnCollection.php, classes/DI/Container.php, classes/Exception/IteratorException.php…).
|
||||
- Caveats: fix inferred (one-line change, uninformative message) -- confirm at freeze that it adds allowed_classes. Confidence: medium.
|
||||
|
||||
## Recommended balanced selection (16)
|
||||
|
||||
**Rule** (applied mechanically to the 26, never by how easy a case looks): per-family targets injection 5, output_encoding 3, path 3, untrusted_destination 2, access_control 2, deserialization 1. Within a family, candidates whose fix commit the advisory names come first, then newest first by publication; take the first *n*. The WordPress cap is checked afterwards, not used to choose.
|
||||
|
||||
- **injection** (5): **gitlist-commandline-cmdi** (2026-08-31), **qloapps-customermessage-bo-query-sqli** (2026-08-25), **roundcube-markasjunk-cmd-learn-rce** (2026-08-17), **winter-filter-numberrange-sqli** (2026-08-12), **maccms10-template-blacklist-rce** (2026-08-05), mantisbt-tokenizer-eval-rce (2026-07-15), grav-blueprint-bare-function-rce (2026-09-17, fix inferred), backpack-stats-host-header-cmdi (2026-08-20, fix inferred)
|
||||
- **output_encoding** (3): **sharp-editor-iframe-srcdoc-xss** (2026-09-25), **privatebin-jsonld-request-uri-injection** (2026-08-28), **openstamanager-group-name-html-injection** (2026-08-26), symfony-ux-live-child-tag-xss (2026-06-19)
|
||||
- **path** (3): **wp-geo-my-wp-info-window-lfi** (2026-09-12), **kirby-realpath-sibling-traversal** (2026-09-02), **mineadmin-appstore-identifier-traversal** (2026-08-18), facturascripts-uploadedfile-move-traversal (2026-07-14, fix inferred)
|
||||
- **untrusted_destination** (2): **elfinder-fsock-dns-rebinding-ssrf** (2026-08-31), **easyappointments-caldav-ssrf** (2026-07-29), koel-podcast-enclosure-ipv6-ssrf (2026-07-15), pagible-admin-proxy-dns-rebinding-ssrf (2026-06-26, fix inferred)
|
||||
- **access_control** (2): **wp-graphql-updatepost-publish-bypass** (2026-09-23), **sylius-mollie-thankyou-order-idor** (2026-07-31), wp-buddypress-messages-rest-idor (2026-06-10), paymenter-checkout-config-override (2026-06-30, fix inferred)
|
||||
- **deserialization** (1): **wp-podlove-generation-guard-poi** (2026-08-16, fix inferred), wp-admin-columns-ids-to-collection-poi (2026-06-06, fix inferred)
|
||||
|
||||
Selected (bold above): winter-filter-numberrange-sqli, gitlist-commandline-cmdi, qloapps-customermessage-bo-query-sqli, roundcube-markasjunk-cmd-learn-rce, maccms10-template-blacklist-rce, kirby-realpath-sibling-traversal, mineadmin-appstore-identifier-traversal, wp-geo-my-wp-info-window-lfi, elfinder-fsock-dns-rebinding-ssrf, easyappointments-caldav-ssrf, wp-graphql-updatepost-publish-bypass, sylius-mollie-thankyou-order-idor, sharp-editor-iframe-srcdoc-xss, privatebin-jsonld-request-uri-injection, openstamanager-group-name-html-injection, wp-podlove-generation-guard-poi.
|
||||
Mix: injection 5 (31%; SQL 2, OS command 2, code 1), output_encoding 3, path 3, untrusted_destination 2, access_control 2, deserialization 1; WordPress 3 of 16. Benign controls: the fixed pin for every case; a proposed benign change exists for 12 of 16 (gitlist, geo-my-wp, elfinder and wp-graphql need one chosen at freeze).
|
||||
|
||||
The deserialization pick has an inferred fix (both deserialization candidates do); it is the only family where the named-fix rule could not be met.
|
||||
|
||||
## Open decisions for the maintainer
|
||||
|
||||
- **Protocol coverage.** Protocol v2 scores only `injection`, `untrusted_destination` and `config_secrets`. v3 as proposed also needs scoring rules for `path`, `output_encoding`, `access_control` and `deserialization` (including what counts as a match for an absence bug such as a missing check, and the output_encoding limits stated in families.toml). Decide the family set before the population is reserved; dropping families changes the selection rule's targets, not the candidates.
|
||||
- **File-only sites.** Seven candidates have no advisory-named function (mantisbt, qloapps, roundcube, maccms10, wp-graphql, paymenter, sharp); several others name a second file without a function (`site_files`). Protocol v2 credits `path::function`; for these the function has to be resolved at freeze by reading the file, or the protocol must credit a file-level site. Where the advisory quotes code, `sink_evidence` carries it for freeze.py.
|
||||
- **Sites derived from class names.** Several advisories name `Class::method` but not the file; the path follows from PSR-4 or the plugin layout and is confirmed only by the fix commit's file list (marked in `site_derivation`).
|
||||
- **Licences to clear before publishing:** Kirby (proprietary Kirby licence, not OSI), MacCMS10 (no licence metadata), elFinder and PrivateBin (NOASSERTION; project files state BSD-3 and zlib-style respectively, verify), MantisBT, BuddyPress and GEO my WP (no GitHub licence metadata; GPL per project, verify).
|
||||
- **Inferred fixes:** grav, backpack, pagible and paymenter come from release compares with a single on-topic commit; facturascripts, podlove and admin-columns from file history (admin-columns' commit message is only 'Update IdsToCollection.php'). Freeze must confirm each diff changes the sink as the advisory describes.
|
||||
- **Multi-advisory fixes:** koel (28 files, several SSRF advisories), sylius mollie (merge also fixes the webhook advisory), sharp, privatebin, qloapps and wp-graphql (commit also changes Model.php).
|
||||
- **Non-default configuration required:** winter (plugin-registered scope), roundcube (markasjunk cmd_learn), elfinder (no cURL), kirby (sibling directory), backpack (exec/curl and Host passthrough), symfony-ux (CORS or same-origin).
|
||||
- **Branch pins:** kirby uses the 4.x-line fix (the 5.x fix also carries another advisory); roundcube uses the 1.6-line fix; sylius mollie uses the 3.3-line merge. Benign proposals come from the default branch, which may not contain these pins.
|
||||
- **Prior mention.** `research-2026-09-26.md` lists some of these advisory ids by name in its 'passed over without examination' note (Grav, elFinder, Backpack CRUD, Winter, Easy!Appointments, Koel). They were never examined and their URLs are not in the tree, so they pass the exclusion rule; the maintainer may still prefer to drop them.
|
||||
- **Alternates** if a candidate fails at freeze (first in line for its family, newest first): injection — Krayin CRM leads DataGrid SQLi (GHSA-8879-fh4f-q3qm), FrontAccounting SQLi ×3 (GHSA-w5j4-x499-pfwp …), LMS SQLi/command injection (GHSA-qrw9-xj5g-qxw8, GHSA-765c-7qf8-cw9w), Dockwatch command injection (GHSA-cgfv-42v7-7f53), MantisBT history_order SQLi (GHSA-mw6p-33vw-46cc, same repository as the MantisBT candidate); path — GEOFlow ImageLibraryController (GHSA-gf7h-mv4h-gg7c), rConfig download_export (GHSA-qrrp-6pvj-xfvf); untrusted_destination — CakePHP Authentication open redirect (GHSA-hhpq-7wg4-36jm), Koillection SSRF (GHSA-gmxh-hjfv-qc2w), Cotonti open redirects (GHSA-g8jm-3grw-8wh2 …); deserialization — Cypht logout back_query (GHSA-46j6-w3hw-4r9x; fix `e4aa2f34` known, but the advisory names no file or function). None of the alternates was fully verified.
|
||||
|
||||
## What constrained the search
|
||||
|
||||
- **Deserialization is nearly absent with a usable fix.** Lane A had one eligible record (October CMS, whose public repository only receives bulk 'Sync upstream' commits). Lanes B and C yielded two, both WordPress, both with fixes inferred from file history; most WordPress object-injection records name no function or have no development repository.
|
||||
- **SQL injection with a fix diff is scarce outside excluded or oversized repositories.** Most 2026 PHP SQLi advisories are on repositories already used (phpMyFAQ, YesWiki, froxlor, pimcore studio), over the size cap (Pimcore, TYPO3, Mautic, LimeSurvey, CodeIgniter4, Dolibarr, LibreNMS), in framework query builders (CakePHP), or in WordPress plugins with no GitHub repository. Only two SQLi candidates qualified in quota order; more exist as alternates.
|
||||
- **The size cap is the largest single filter** (161 rejections): Craft CMS, Snipe-IT (32 advisories), TYPO3, Pimcore, Shopware, Sylius core, Mautic, Filament, AVideo, LibreNMS, MISP, Dolibarr, OpenEMR, ILIAS, Joomla.
|
||||
- **Naming the site.** 46 advisories name no file or function (Statamic, Silverstripe and most WordPress records).
|
||||
- **Kimai** shipped the fixes for nine of this window's advisories inside one 'Release 2.58' squash, which removes a whole application from the pool.
|
||||
|
||||
## Rejection log
|
||||
|
||||
One line per examined advisory, newest first within each lane. `taken` rows name the candidate. Automatic exclusions by metadata are listed first within lane A.
|
||||
|
||||
### Lane A: reviewed composer advisories
|
||||
|
||||
- GHSA-r6hr-vr92-vv28 / CVE-2026-65954 (phpcsstandards/phpcsutils, injection, 2026-09-29): developer tool / CLI, not a web application. _PHPCSUtils: Remote code execution via eval() in AbstractArrayDeclarationSniff::getActualAr_
|
||||
- GHSA-jh5r-qr3c-85q8 / CVE-2026-102279 (laravel/framework, output_encoding, 2026-09-29): DOM XSS in the debug page's JavaScript tooltip (Tippy `allowHTML`); the sink is client-side JavaScript, and the advisory names no PHP file or function. _Laravel: XSS in Debug Page Information_
|
||||
- GHSA-v65j-hff3-753c / CVE-2026-57440 (starcitizenwiki/mediawiki-extensions-embedvideo, output_encoding, 2026-09-25): excluded by name. _Mediawiki EmbedVideo Extension has stored XSS via malformed src url with $wgEmbedVideoRequ_
|
||||
- GHSA-qxg3-46rw-79j8 / CVE-2026-61823 (code16/sharp, output_encoding, 2026-09-25): **taken** as sharp-editor-iframe-srcdoc-xss. _code16 Sharp vulnerable to stored XSS via iframe srcdoc Attribute_
|
||||
- GHSA-vj3q-vp3g-j9c8 / CVE-2026-61825 (code16/sharp, output_encoding, 2026-09-25): one case per repository (repository taken as sharp-editor-iframe-srcdoc-xss). _code16/sharp has a stored XSS via data-html-content Sanitizer Bypass_
|
||||
- GHSA-87mg-5grr-rhwh / CVE-2026-57232 (contao/contao, untrusted_destination, 2026-09-24): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Contao: Server-Side Request Forgery (SSRF) via Unvalidated RSS Feed URL in Feed Reader Mod_
|
||||
- GHSA-rw77-vq4g-x3hp / CVE-2026-56738 (thorsten/phpmyfaq, injection, 2026-09-24): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ has SQL Injection in `StopWords::add()` — Unescaped Stop Word Insertion_
|
||||
- GHSA-pgwp-vc7q-cvj3 / CVE-2026-56736 (thorsten/phpmyfaq, output_encoding, 2026-09-24): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ has Stored XSS in Admin FAQ Editor via HTML Entity Bypass in Frontend FAQ Submiss_
|
||||
- GHSA-396x-xmvh-p563 / CVE-2026-63498 (grokability/snipe-it, output_encoding, 2026-09-24): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Stored XSS via Inline XML Rendering in the Uploaded Files API_
|
||||
- GHSA-p9h3-gvpq-5539 / CVE-2026-62368 (grokability/snipe-it, output_encoding, 2026-09-24): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Stored XSS via Custom Field name in asset-list column headers_
|
||||
- GHSA-9rg8-2wvr-fgjh / CVE-2026-76089 (verbb/formie, access_control, 2026-09-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Formie: Missing authorization on sent notification resend modal exposes submission PII_
|
||||
- GHSA-584p-f93j-wpgc / CVE-2026-76087 (verbb/formie, access_control, 2026-09-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Formie: Unauthenticated users can overwrite incomplete submissions via submit action_
|
||||
- GHSA-v3f3-cmj4-cvj9 / CVE-2026-76086 (verbb/formie, access_control/untrusted_destination, 2026-09-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Formie: Integration form-settings action allows SSRF and exfiltration of stored integratio_
|
||||
- GHSA-jg26-q8hg-3pq4 / CVE-2026-92692 (sulu/sulu, injection, 2026-09-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Sulu: JCR-SQL2 injection via `categories` query parameter (unauthenticated)_
|
||||
- GHSA-gxrg-x694-283w / CVE-2026-73858 (solspace/craft-freeform, injection, 2026-09-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Solspace Freeform: Limited Twig template injection via submitted field values_
|
||||
- GHSA-w998-qmw9-mf4m / CVE-2026-63002 (redaxo/core, output_encoding, 2026-09-23): repo over ~150 MB (500 MB per GitHub API). _REDAXO: Stored XSS in Mediapool Sync Page via Unescaped Filesystem Filenames_
|
||||
- GHSA-mf2p-wjp4-99pq / CVE-2026-63001 (redaxo/core, output_encoding, 2026-09-23): repo over ~150 MB (500 MB per GitHub API). _REDAXO: Stored XSS via Unescaped Media Manager Type Name in `mediaIsInUse()`_
|
||||
- GHSA-5mmc-8pc9-wggg / CVE-2026-88974 (wp-graphql/wp-graphql, access_control, 2026-09-23): **taken** as wp-graphql-updatepost-publish-bypass. _WPGraphQL: Contributor can publish and modify posts without the required capabilities via _
|
||||
- GHSA-jr78-w6w5-m8f8 (semanticmediawiki/semanticmediawiki, access_control, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki'a missing authorization in the smwtask API module allows unauthenticate_
|
||||
- GHSA-9rcc-pmj8-ffhr (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki's Special:FacetedSearch cstate hidden inputs enable reflected XSS (resi_
|
||||
- GHSA-cx86-7xwp-w9wf / CVE-2026-77616 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki affected by reflected XSS in `Special:Ask` via a forged cursor paginati_
|
||||
- GHSA-q5fm-9mx6-44f4 / CVE-2026-77610 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki has a query debug output XSS (`DebugFormatter`)_
|
||||
- GHSA-hw3m-8j5x-94ff / CVE-2026-77609 (semanticmediawiki/semanticmediawiki, untrusted_destination, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki has an open redirect in Special:URIResolver_
|
||||
- GHSA-7xv3-gf2g-498h / CVE-2026-77607 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki affected by Special:Ask table `sep` parameter reflected XSS_
|
||||
- GHSA-59xw-qv23-j3rc / CVE-2026-77608 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki has reflected XSS in `Special:SearchByProperty` (`property` and `value`_
|
||||
- GHSA-3jp5-3h47-28qf / CVE-2026-77606 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki has reflected XSS in Special:Ask plain table headers_
|
||||
- GHSA-hg8h-557g-q8pp / CVE-2025-61682 (semanticmediawiki/semanticmediawiki, output_encoding, 2026-09-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Semantic MediaWiki vulnerable to stored XSS through wikitext via improper use of non-reser_
|
||||
- GHSA-g4c3-4g96-6g4m / CVE-2026-45140 (chamilo/chamilo-lms, injection/path, 2026-09-17): repo over ~150 MB (1625 MB per GitHub API). _Chamilo LMS CStudio upload flow allows unauthenticated remote code execution_
|
||||
- GHSA-r94f-hx44-8jqf / CVE-2026-72819 (getgrav/grav, injection, 2026-09-17): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) (only the attacker's `shell.php`). _Grav CMS vulnerable to remote code execution via .zip file upload_
|
||||
- GHSA-xhfv-7758-r9hx / CVE-2026-75837 (getgrav/grav, access_control, 2026-09-17): the fix changes only a YAML blueprint (`system/blueprints/user/group.yaml`); no PHP line changes, so the fixed pin cannot be told apart in PHP. _Grav: Missing admin.super guard on core group blueprint access field allows admin.users op_
|
||||
- GHSA-q2j8-x8hf-63ch / CVE-2026-75834 (getgrav/grav, output_encoding, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Single invalid UTF-8 byte disables every rule in Security::detectXss(), bypassing th_
|
||||
- GHSA-f8wv-xp27-6gq7 / CVE-2026-75827 (getgrav/grav, injection, 2026-09-17): **taken** as grav-blueprint-bare-function-rce. _Grav: Blueprint dynamic-data bare-function branch is denylist-gated and omits error_log, g_
|
||||
- GHSA-vfmf-q6x9-cw96 / CVE-2026-75828 (getgrav/grav, output_encoding, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: detectXss() misses an event-handler attribute after an unpaired quote in an unquoted_
|
||||
- GHSA-4v9q-p283-qc2m / CVE-2026-74907 (getgrav/grav, path, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Unauthenticated Path Traversal via Missing Directory-Boundary Check in `plugin-asset_
|
||||
- GHSA-jq29-c7v8-rg55 / CVE-2026-72695 (getgrav/grav, path, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Path Traversal in MediaUploadTrait::deleteFile() Allows Arbitrary File Deletion_
|
||||
- GHSA-vjqc-q4mp-2rvf / CVE-2026-79752 (cakephp/cakephp, injection, 2026-09-17): framework query-builder API (`FunctionsBuilder`); vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository. _CakePHP: Multiple methods in FunctionsBuilder vulnerable to SQL injection_
|
||||
- GHSA-47ch-6w46-6xm7 / CVE-2026-72697 (getgrav/grav, path, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: media_directory() Twig function allows filesystem path traversal and file content di_
|
||||
- GHSA-6qw9-4vv5-jr97 / CVE-2026-75831 (getgrav/grav, output_encoding, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Stored XSS via Markdown audio/video media <source> URL_
|
||||
- GHSA-269c-h76q-8cxw / CVE-2026-72832 (getgrav/grav, output_encoding, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Stored XSS via quoted-attribute bypass in detectXss_
|
||||
- GHSA-w3f4-8pj2-599w / CVE-2026-69089 (getgrav/grav, path, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Path Traversal in ImageMedium::watermark() — arbitrary file disclosure via publicly-_
|
||||
- GHSA-7pgq-cr25-xvc8 / CVE-2026-69088 (getgrav/grav, injection, 2026-09-17): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Incomplete callable validation in blueprint dynamic fields allows arbitrary static m_
|
||||
- GHSA-ffmg-hfvg-jhg9 / CVE-2026-58657 (getgrav/grav, output_encoding, 2026-09-16): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Stored CSS injection via Markdown image resize() bypasses prior media style sanitize_
|
||||
- GHSA-2c4f-86xc-cr74 / CVE-2026-61453 (getgrav/grav, injection/output_encoding, 2026-09-16): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: XSS Blueprint Validation Bypass via Twig String Concatenation_
|
||||
- GHSA-2ff2-mx52-q8wp / CVE-2026-49400 (octobercms/october, deserialization, 2026-09-14): no fix diff in the public repository: `modules/backend/traits/SessionMaker.php` last changed in a bulk 'Sync upstream (4.2)' commit (2026-05-20), and there is no v4.2.23 tag. _October CMS: PHP Object Injection via Backend Widget Session Storage_
|
||||
- GHSA-xv9m-fm3w-8w5x / CVE-2026-46696 (octobercms/october, access_control, 2026-09-14): no fix diff in the public repository (same bulk 'Sync upstream' history, no v4.2.23 tag); advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text). _October CMS: Safe Mode Sandbox Bypass via Session Store and Forwarded Builder Calls_
|
||||
- GHSA-2cg9-97gq-9mqp / CVE-2026-56825 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopper: Missing authorization on product removal actions in CollectionProducts component_
|
||||
- GHSA-99h5-jhh7-v3r3 / CVE-2026-56830 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopper: Media sub-form store() still lacks authorization (Incomplete fix for GHSA-h4mp-g9_
|
||||
- GHSA-g3f9-g5vj-p62f / CVE-2026-56829 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopper: Unauthorized inventory stock manipulation via unlocked variant property in Varian_
|
||||
- GHSA-j328-xmgp-j4q3 / CVE-2026-56828 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopper: privilege escalation via improper Livewire admin component authorization_
|
||||
- GHSA-f7h9-qv4x-9x57 / CVE-2026-56826 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopping privilege escalation through missing authorization in Settings components_
|
||||
- GHSA-243p-f3cv-c5wh / CVE-2026-56827 (shopperlabs/shopper, access_control, 2026-09-11): repo over ~150 MB (151 MB per GitHub API). _Shopper: Authorization bypass in Filament bulk actions allows browse-only staff to mass-de_
|
||||
- GHSA-23rh-xw42-fq82 / CVE-2026-55416 (pimcore/pimcore, injection, 2026-09-10): repo over ~150 MB (534 MB per GitHub API). _Pimcore: SQL Injection in Custom Reports via Malicious Report Configuration_
|
||||
- GHSA-65fr-j4p9-vc33 / CVE-2026-81525 (mongodb/mongo-php-library, injection, 2026-09-08): general-purpose library: the attacker source lies in the calling application, not in the repo. _mongodb: Reject "." and NUL bytes in database and collection names_
|
||||
- GHSA-rvx4-ffvw-m9q3 / CVE-2026-84361 (composer/composer, injection, 2026-09-08): developer tool / CLI, not a web application. _Composer arbitrary command execution via a malicious package's Perforce source URL_
|
||||
- GHSA-c7r6-vx3h-w5g2 / CVE-2026-84374 (spartnernl/laravel-excel, path, 2026-09-08): general-purpose library: the attacker source lies in the calling application, not in the repo. _Laravel Excel writes exports outside the configured filesystem disk when given a caller-co_
|
||||
- GHSA-fxf7-vhh8-7vpq / CVE-2026-77635 (cakephp/cakephp, injection, 2026-09-08): framework query-builder API (`FunctionsBuilder::jsonValue()`); vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository. _CakePHP: FunctionsBuilder::jsonValue() vulerable to SQL injection with PostgresDriver_
|
||||
- GHSA-pp4x-ccxq-6r33 / CVE-2026-82396 (sulu/sulu, output_encoding, 2026-09-02): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Sulu: Stored XSS via media download inline-disposition override_
|
||||
- GHSA-65cv-w493-7vhq / CVE-2026-82394 (sulu/sulu, access_control, 2026-09-02): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Sulu: Fix authorization bypass when creating preview links_
|
||||
- GHSA-h6cx-gjxx-v25c / CVE-2026-82395 (sulu/sulu, access_control, 2026-09-02): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Sulu: Media move/update authorization bypass (IDOR)_
|
||||
- GHSA-6j4c-mgqr-qv76 / CVE-2026-75592 (getkirby/kirby, path, 2026-09-02): **taken** as kirby-realpath-sibling-traversal. _Kirby: Access to image files outside of the site root via path traversal in the media hand_
|
||||
- GHSA-fj2p-qj2f-74v5 / CVE-2026-64850 (getgrav/grav, injection, 2026-09-02): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Remote code execution via unrestricted callable in Blueprint::dynamicData()_
|
||||
- GHSA-g2fm-8hr4-j82h / CVE-2026-81892 (easycorp/easyadminbundle, access_control, 2026-09-02): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) (only the public API `Action::linkToRoute()`; the dispatcher that swaps controllers is described, not named). _EasyAdmin custom-action dispatcher bypasses access_control on other routes_
|
||||
- GHSA-g3hc-697w-wm82 / CVE-2026-81887 (livewire/livewire, output_encoding, 2026-09-02): DOM-based XSS in Livewire's client-side JavaScript; not a PHP sink. _Livewire DOM-based cross-site scripting during client-side state handling_
|
||||
- GHSA-f8fg-pg57-v4j8 (thephpleague/commonmark, output_encoding, 2026-09-01): general-purpose library: the attacker source lies in the calling application, not in the repo. _league/commonmark XSS: `on*` event-handler filter in `AttributesExtension` bypassed with a_
|
||||
- GHSA-cq55-c7wv-pxmq / CVE-2026-62993 (smarty-php/smarty, untrusted_destination, 2026-09-01): general-purpose library: the attacker source lies in the calling application, not in the repo. _Smarty: SSRF via redirect bypass of trusted_uri using {fetch}_
|
||||
- GHSA-67mx-6wf2-92xp / CVE-2026-71415 (getkirby/kirby, access_control, 2026-08-31): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: File upload permissions are not checked during processing of chunk data_
|
||||
- GHSA-9vx2-j98c-p72w / CVE-2026-75594 (getkirby/kirby, path, 2026-08-31): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: Access to image files and limited access to JSON files outside of the site root via_
|
||||
- GHSA-8x3q-jpjh-qh5c / CVE-2026-81889 (studio-42/elfinder, untrusted_destination, 2026-08-31): **taken** as elfinder-fsock-dns-rebinding-ssrf. _elFinder: SSRF protection bypass via DNS rebinding in the `fsock_get_contents()` fallback_
|
||||
- GHSA-9x44-4gxf-8c25 / CVE-2026-55634 (pimcore/pimcore, injection, 2026-08-28): repo over ~150 MB (534 MB per GitHub API). _Pimcore Vulnerable to Remote Code Execution via DataObject Class-Definition Field Name_
|
||||
- GHSA-w23p-wrp7-ch38 / CVE-2026-55220 (pimcore/pimcore, deserialization, 2026-08-28): repo over ~150 MB (534 MB per GitHub API). _Pimcore Hotspotimage getDataFromResource() unrestricted Serialize::unserialize over object_
|
||||
- GHSA-f97c-ph8j-8vff / CVE-2026-55212 (pimcore/pimcore, access_control, 2026-08-28): repo over ~150 MB (534 MB per GitHub API). _Pimcore: Insufficient Permission Check on Class Definition Creation Endpoint Allows Privil_
|
||||
- GHSA-79cw-hfcc-7mw9 / CVE-2026-55208 (pimcore/pimcore, injection, 2026-08-28): repo over ~150 MB (534 MB per GitHub API). _Pimcore: SQL Injection via Column Name in DateFilter allows authenticated user to extract _
|
||||
- GHSA-575r-357h-fhch / CVE-2026-55516 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT vulnerable to cross-company asset maintenance re-parenting via API update_
|
||||
- GHSA-35cr-9hqq-p2mg / CVE-2026-55515 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Cross-company deletion of pending checkout acceptances via unscoped report endpo_
|
||||
- GHSA-w7qw-5wfv-gwx9 / CVE-2026-55481 (grokability/snipe-it, output_encoding, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has CSS Injection via `header_color` Setting_
|
||||
- GHSA-8frh-vhgh-64cf / CVE-2026-55479 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has incorrect permission for legacy license checkin API _
|
||||
- GHSA-crv3-j83j-f3r6 / CVE-2026-55478 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has missing object-level authorization in Kits API_
|
||||
- GHSA-53jc-27pc-x8r8 / CVE-2026-55476 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT Vulnerable to Unauthorized Asset Request Cancellation via Unguarded cancel_by_adm_
|
||||
- GHSA-5wx7-xq8j-v4qm / CVE-2026-55475 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT's import created_by can be overwritten_
|
||||
- GHSA-c6f4-wj38-m3g3 / CVE-2026-55474 (grokability/snipe-it, path, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT vulnerable to directory traversal in displaySig_
|
||||
- GHSA-8w8c-8mx9-52cw / CVE-2026-55472 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT's API Location Creation Bypasses FMCS Parent-Child Company Boundary Validation_
|
||||
- GHSA-xr9m-gphc-9p63 / CVE-2026-55469 (grokability/snipe-it, path, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has a path traversal vulnerability via CSV import `image` field_
|
||||
- GHSA-jhph-5q74-pmfx / CVE-2026-55466 (grokability/snipe-it, output_encoding, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT vulnerable to stored XSS via inline-served attachment_
|
||||
- GHSA-r52f-r9v5-66xr / CVE-2026-55464 (grokability/snipe-it, output_encoding, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT vulnerable to stored XSS via Markdown custom field _
|
||||
- GHSA-fc33-6w3q-538h / CVE-2026-55462 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has an authorization bypass on print inventory page_
|
||||
- GHSA-wg2f-x2c2-c4rp / CVE-2026-55461 (grokability/snipe-it, untrusted_destination, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has an Open Redirect After User Edit_
|
||||
- GHSA-vgx7-c78r-69w9 / CVE-2026-55460 (grokability/snipe-it, access_control, 2026-08-28): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has an authorization bypass on bulk editing users_
|
||||
- GHSA-m4g4-86qc-v8w7 / CVE-2026-55779 (silverstripe/silverstripe-versioned, output_encoding, 2026-08-28): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) (only the `ArchiveAdmin` screen; the fix is in `src/RestoreAction.php`). _silverstripe/versioned has XSS in archive admin restore_
|
||||
- GHSA-xrjc-c68j-hp7w / CVE-2026-55891 (privatebin/privatebin, output_encoding, 2026-08-28): **taken** as privatebin-jsonld-request-uri-injection. _PrivateBin has reflected JSON injection in backend responses via unescaped REQUEST_URI_
|
||||
- GHSA-f2xf-7x3g-4272 / CVE-2026-55696 (privatebin/privatebin, output_encoding, 2026-08-28): one case per repository (repository taken as privatebin-jsonld-request-uri-injection). _PrivateBin has stored Cross-Side-Scripting (XSS) vulnerability in attachment download link_
|
||||
- GHSA-39mm-rwm3-29jp / CVE-2026-54718 (silverstripe/silverstripe-advancedworkflow, injection, 2026-08-27): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text); the fix only bumps a dependency in composer.json (the repair lives in another package). _silverstripe-advancedworkflow vulnerable to remote code execution via advanced workflow em_
|
||||
- GHSA-g8wr-r2v2-vqc6 / CVE-2026-54721 (silverstripe/silverstripe-userforms, injection, 2026-08-27): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text); the fix only bumps a dependency in composer.json (the repair lives in another package). _silverstripe/userforms vulnerable to remote code execution via userforms email subject_
|
||||
- GHSA-gvrw-qqp5-jgc5 / CVE-2026-54720 (silverstripe/silverstripe-framework, output_encoding, 2026-08-27): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) ('Insert media from web'). _Silverstripe Framework: Possible XSS attack through media embed_
|
||||
- GHSA-7w8c-qgxg-m7jx (librenms/librenms, output_encoding, 2026-08-26): repo over ~150 MB (369 MB per GitHub API). _LibreNMS — Stored XSS via SNMP/Syslog Data in Legacy Templates_
|
||||
- GHSA-crx4-7mmq-j74j / CVE-2026-44701 (devcode-it/openstamanager, output_encoding, 2026-08-26): **taken** as openstamanager-group-name-html-injection. _OpenSTAManager has HTML Injection in modules/utenti/edit.php_
|
||||
- GHSA-mf8r-wm2w-f8c5 (thorsten/phpmyfaq, access_control, 2026-08-25): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ public FAQ APIs expose inactive FAQ content_
|
||||
- GHSA-88g4-74f3-63x9 (thorsten/phpmyfaq, path, 2026-08-25): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ has Potential Authenticated Path Traversal in PDF Export_
|
||||
- GHSA-896w-cw95-xq7w (, path, 2026-08-25): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Path Traversal in MediaUploadTrait::deleteFile() Allows Arbitrar_
|
||||
- GHSA-rj4c-4q9x-543x (, path, 2026-08-25): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: media_directory() Twig function allows filesystem path traversal_
|
||||
- GHSA-5h77-88j3-r659 / CVE-2026-63135 (yourls/yourls, output_encoding, 2026-08-21): repo over ~150 MB (489 MB per GitHub API). _YOURLS has stored XSS in referrer statistics chart via crafted Referer header_
|
||||
- GHSA-hrwp-4hh9-c8r8 / CVE-2026-59989 (phalcon/cphalcon, injection, 2026-08-21): repo over ~150 MB (273 MB per GitHub API). _Phalcon Volt compiler `join` filter compile-time PHP code injection (SSTI leads to RCE)_
|
||||
- GHSA-8hgv-xc77-jmcr (getgrav/grav, output_encoding, 2026-08-21): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Page editors can inject arbitrary script into rendered pages via the Twig sandbox's _
|
||||
- GHSA-hq84-x37p-j6q5 (wintercms/winter, output_encoding, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Reflected XSS through the search query parameter in the backend Table widget_
|
||||
- GHSA-5cwr-5jxg-pcf6 (wintercms/winter, output_encoding, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Stored XSS through cached Brand Settings and Editor Settings custom styles_
|
||||
- GHSA-fm29-4mq3-phg6 (wintercms/winter, access_control, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: ImportExportController AJAX handlers bypass granular import/export permission gate_
|
||||
- GHSA-mpmw-f6h6-3g26 (wintercms/winter, access_control, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: My Account preview exposes another backend user's profile by record ID_
|
||||
- GHSA-7mpf-4465-7fc2 (wintercms/winter, output_encoding, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Stored XSS through Backend List widget image columns_
|
||||
- GHSA-2223-f22x-24cq (wintercms/winter, path, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Local File Inclusion through =include directives in JavaScript asset compilation_
|
||||
- GHSA-58fp-mcx6-7qf9 / CVE-2026-63179 (wintercms/winter, path, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Local File Inclusion through @import directives in LESS compilation of backend cus_
|
||||
- GHSA-42vx-43vc-x6pr / CVE-2026-57570 (laravel-backpack/crud, access_control, 2026-08-20): one case per repository (repository taken as backpack-stats-host-header-cmdi). _Laravel Backpack CRUD: HasMany/MorphMany relation fields allow cross-tenant record re-pare_
|
||||
- GHSA-3277-h8g9-qj5f / CVE-2026-54256 (wintercms/winter, access_control, 2026-08-20): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Authenticated IDOR in backend FileUpload widget allows cross-user access to attach_
|
||||
- GHSA-mrc5-3mm3-45c5 / CVE-2026-54182 (laravel-backpack/crud, injection/output_encoding, 2026-08-20): **taken** as backpack-stats-host-header-cmdi. _Laravel Backpack CRUD: OS command injection in Stats::makeCurlRequest via attacker-control_
|
||||
- GHSA-mmg4-322v-6jvc / CVE-2026-54181 (laravel-backpack/crud, output_encoding, 2026-08-20): one case per repository (repository taken as backpack-stats-host-header-cmdi). _Laravel Backpack CRUD: Stored XSS in the color column — the `@if($column['escaped'])` bran_
|
||||
- GHSA-vgmv-8xjc-6rch / CVE-2026-54180 (laravel-backpack/crud, access_control, 2026-08-20): one case per repository (repository taken as backpack-stats-host-header-cmdi). _Laravel Backpack CRUD: CRUD panel query scopes are not enforced on Update, Delete, and Reo_
|
||||
- GHSA-8hw4-7qjr-3wxg / CVE-2026-54179 (laravel-backpack/crud, output_encoding, 2026-08-20): one case per repository (repository taken as backpack-stats-host-header-cmdi). _Laravel Backpack CRUD: SingleBase64Image accepts any base64 payload behind a `data:image` _
|
||||
- GHSA-8xjm-wqrp-2f25 / CVE-2026-54178 (laravel-backpack/crud, access_control/path, 2026-08-20): one case per repository (repository taken as backpack-stats-host-header-cmdi). _Laravel Backpack CRUD: Arbitrary file deletion via attacker-controlled clear_<attr>[] in H_
|
||||
- GHSA-c8qc-wf67-342w / CVE-2026-61807 (grokability/snipe-it, output_encoding, 2026-08-19): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Stored DOM XSS via table selected-count IDs_
|
||||
- GHSA-r9r3-g9fp-3q4q / CVE-2026-55703 (grokability/snipe-it, access_control, 2026-08-19): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Maintenance Record Disclosure via Missing Authorization on GET_
|
||||
- GHSA-3hgv-jr5j-cg9x / CVE-2026-55694 (grokability/snipe-it, access_control, 2026-08-19): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Chained Information Disclosure and IDOR Leads to Full EULA File Takeover_
|
||||
- GHSA-c6w2-j4wq-mvwg / CVE-2026-55643 (grokability/snipe-it, access_control, 2026-08-19): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Tenant Isolation Bypass in FMCS Floater Mode_
|
||||
- GHSA-7gww-x7fh-jf9j (librenms/librenms, output_encoding/untrusted_destination, 2026-08-18): repo over ~150 MB (369 MB per GitHub API). _LibreNMS: SSRF-driven stored XSS via Oxidized API response fields in device showconfig pag_
|
||||
- GHSA-7cj5-v4pp-v632 (librenms/librenms, output_encoding, 2026-08-18): repo over ~150 MB (369 MB per GitHub API). _LibreNMS: Stored XSS via graph_descr admin config settings echoed without escaping to all _
|
||||
- GHSA-jf24-8g2h-2wg7 (librenms/librenms, injection, 2026-08-18): repo over ~150 MB (369 MB per GitHub API). _LibreNMS Vulnerable to Remote Code Execution via AboutController_
|
||||
- GHSA-w27m-rmmf-g5w4 / CVE-2026-54348 (froxlor/froxlor, injection, 2026-08-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Froxlor: Second-Order SQL Injection via `Admins.add` `ipaddress` Parameter Allows Full Dat_
|
||||
- GHSA-43gm-9rr3-cx7g / CVE-2026-54347 (froxlor/froxlor, output_encoding, 2026-08-18): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Froxlor: Stored XSS in DNS TXT Record Content Allows Customer-to-Admin Account Takeover_
|
||||
- GHSA-pcw8-m77r-2528 / CVE-2026-54133 (jmespath/jmespath.php, injection/output_encoding, 2026-08-18): general-purpose library: the attacker source lies in the calling application, not in the repo. _jmespath.php has CompilerRuntime code injection via unescaped function names_
|
||||
- GHSA-c9fv-cgmm-2wg7 / CVE-2026-55182 (librenms/librenms, injection, 2026-08-18): repo over ~150 MB (369 MB per GitHub API). _LibreNMS Vulnerable to Remote Code Execution by Signal Alert Transportation module_
|
||||
- GHSA-q8cg-5m48-5c25 (, output_encoding, 2026-08-18): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Stored XSS via Markdown audio/video media <source> URL_
|
||||
- GHSA-9pr6-8r9w-wvmj (, output_encoding, 2026-08-18): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: detectXss() misses an event-handler attribute after an unpaired _
|
||||
- GHSA-993v-76jg-67xr (, output_encoding, 2026-08-18): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Single invalid UTF-8 byte disables every rule in Security::detec_
|
||||
- GHSA-fwwx-3362-3947 (, injection, 2026-08-18): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Blueprint dynamic-data bare-function branch is denylist-gated an_
|
||||
- GHSA-2rhw-8953-48q3 (, path, 2026-08-18): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Unauthenticated Path Traversal via Missing Directory-Boundary Ch_
|
||||
- GHSA-59xm-4m8c-g3xj / CVE-2026-55224 (mineadmin/mineadmin, path, 2026-08-18): **taken** as mineadmin-appstore-identifier-traversal. _MineAdmin Vulnerable to Path Traversal via Unsanitized identifier in Plugin Install/Uninst_
|
||||
- GHSA-cgvr-f65r-pjv3 (, output_encoding, 2026-08-14): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Stored XSS via quoted-attribute bypass in detectXss_
|
||||
- GHSA-wvxr-6v52-gfmh (, injection, 2026-08-14): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Remote code execution via .zip file upload in Grav CMS_
|
||||
- GHSA-2mhj-fhvg-v428 / CVE-2026-55072 (pimcore/pimcore, injection, 2026-08-13): repo over ~150 MB (534 MB per GitHub API). _Pimcore: ClassDefinition UID regex missing end anchor allows SQL injection via Block.php u_
|
||||
- GHSA-6pvm-2vjj-rx4w / CVE-2026-47132 (thorsten/phpmyfaq, injection, 2026-08-12): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ: SQL LIKE Wildcard Injection in Chat User Search Allows Authenticated User Enumer_
|
||||
- GHSA-jmqm-f8q4-v7wx / CVE-2026-45694 (librenms/librenms, output_encoding, 2026-08-12): repo over ~150 MB (369 MB per GitHub API). _LibreNMS: Reflected XSS via Proxmox instance/vmid GET parameters injected into document.ti_
|
||||
- GHSA-j5jq-cr68-v2xx / CVE-2026-35445 (wintercms/winter, access_control, 2026-08-12): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Authenticated backend users can bypass Users controller permission checks_
|
||||
- GHSA-m7jc-g4rc-jmvh / CVE-2026-32593 (wintercms/winter, injection, 2026-08-12): **taken** as winter-filter-numberrange-sqli. _Winter: SQL Injection in Backend Filter Widget numberrange Scope via numbersFromAjax_
|
||||
- GHSA-vgp4-2fc4-qff2 / CVE-2026-32258 (wintercms/winter, output_encoding, 2026-08-12): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Stored XSS through Editor Settings custom styles_
|
||||
- GHSA-v7cf-8gh9-gxmj / CVE-2026-32257 (wintercms/winter, output_encoding, 2026-08-12): one case per repository (repository taken as winter-filter-numberrange-sqli). _Winter: Stored XSS through Brand Settings custom styles_
|
||||
- GHSA-4hc4-qjfx-wjf3 (, access_control, 2026-08-11): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: Authorization bypass: view-only Categories user can modify _
|
||||
- GHSA-9w6w-8x3c-hfqp (, path, 2026-08-11): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: Incorrect path validation could potentially lead to path tr_
|
||||
- GHSA-hhmc-q9hp-r662 / CVE-2026-63222 (codeigniter4/codeigniter4, path, 2026-08-07): repo over ~150 MB (387 MB per GitHub API). _CodeIgniter: Path traversal in UploadedFile::move() when using client-provided filenames_
|
||||
- GHSA-c9w5-rwh3-7pm9 / CVE-2026-63221 (codeigniter4/codeigniter4, injection, 2026-08-07): repo over ~150 MB (387 MB per GitHub API). _CodeIgniter: SQL injection in Query Builder deleteBatch() when used with where() condition_
|
||||
- GHSA-rjhh-76wf-8xmw / CVE-2026-62996 (smarty-php/smarty, path, 2026-08-07): general-purpose library: the attacker source lies in the calling application, not in the repo. _Smarty Security stream restriction bypass through stream: resource_
|
||||
- GHSA-f6wf-28g6-769x / CVE-2026-62992 (smarty-php/smarty, path, 2026-08-07): general-purpose library: the attacker source lies in the calling application, not in the repo. _Smarty: Symlink path traversal out of trusted directories_
|
||||
- GHSA-xxpx-f366-4xpq / CVE-2026-72785 (craftcms/cms, access_control, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Authorization bypass: view-only Categories user can modify category structure v_
|
||||
- GHSA-rvmm-v933-jgxq / CVE-2026-14794 (craftcms/cms, access_control, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Missing authorization check allows non-admin control panel users access to user_
|
||||
- GHSA-7hxc-f267-h5q7 / CVE-2026-72783 (craftcms/cms, path, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Incorrect path validation could potentially lead to path traversal_
|
||||
- GHSA-2rp4-x2j7-qmcc (craftcms/cms, output_encoding, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Stored XSS in the control panel via unescaped draft name_
|
||||
- GHSA-hmqg-cxww-wqhq / CVE-2026-67434 (phpcsstandards/php_codesniffer, injection, 2026-08-06): repo over ~150 MB (210 MB per GitHub API). _PHP_CodeSniffer gitblame report command injection via crafted filename_
|
||||
- GHSA-p8x7-9vfw-p7vc (craftcms/cms, access_control, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Arbitrary user password reset leading to administrator account takeover_
|
||||
- GHSA-9p7c-v5x3-rfx8 / CVE-2026-14793 (craftcms/cms, access_control, 2026-08-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Missing authorization check allows non-admin control panel users to reorder Glo_
|
||||
- GHSA-29pj-957v-52mc / CVE-2026-71478 (thephpleague/commonmark, output_encoding, 2026-08-06): general-purpose library: the attacker source lies in the calling application, not in the repo. _league/commonmark: AttributesExtension href/src unsafe-link filter bypass via embedded con_
|
||||
- GHSA-grm4-wm43-9jh5 / CVE-2026-55825 (contao/contao, path, 2026-08-06): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Contao: Possible path traversal in job download URIs_
|
||||
- GHSA-w3cp-g2pf-65wh / CVE-2026-54717 (silverstripe/silverstripe-cms, output_encoding, 2026-08-06): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) (breadcrumbs in page list view). _Silverstripe: XSS in breadcrumbs in page list view_
|
||||
- GHSA-vx89-p3j7-8xqc / CVE-2026-71435 (statamic/cms, output_encoding, 2026-08-06): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) ('automagic' form notification email). _Statamic: Stored Cross-Site Scripting in Automagic Form Notification Email Template_
|
||||
- GHSA-qh8c-7588-qfrv / CVE-2026-64662 (statamic/cms, access_control, 2026-08-06): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text). _Statamic: Missing authorization on navigation endpoint allows disclosure of restricted ent_
|
||||
- GHSA-225x-3jhx-wh4q / CVE-2026-64664 (statamic/cms, access_control, 2026-08-06): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text). _Statamic: Missing authorization on Control Panel endpoint allows disclosure of user existe_
|
||||
- GHSA-v5mv-p594-2x33 / CVE-2026-69246 (guzzle/guzzle, untrusted_destination, 2026-08-03): general-purpose library: the attacker source lies in the calling application, not in the repo. _Guzzle: Noncanonical host can bypass host-based checks_
|
||||
- GHSA-vj8j-973f-r65j (, injection, 2026-08-03): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Incomplete callable validation in blueprint dynamic fields allow_
|
||||
- GHSA-mmwh-j75q-gxp8 (, path, 2026-08-03): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Path Traversal in ImageMedium::watermark() — arbitrary file disc_
|
||||
- GHSA-x83g-979r-f5fh / CVE-2026-68501 (sylius/mollieplugin, access_control, 2026-07-31): **taken** as sylius-mollie-thankyou-order-idor. _Sylius Mollie Plugin has unauthenticated IDOR that leaks order token and customer PII_
|
||||
- GHSA-rc52-c4hv-w89p / CVE-2026-68500 (sylius/mollieplugin, access_control, 2026-07-31): one case per repository (repository taken as sylius-mollie-thankyou-order-idor). _Sylius Mollie Plugin vulnerable to payment status forgery via the payment webhook_
|
||||
- GHSA-996f-334j-67g7 / CVE-2026-52838 (alextselegidis/easyappointments, output_encoding, 2026-07-29): one case per repository (repository taken as easyappointments-caldav-ssrf). _Easy!Appointments disable_booking_message rendered as raw HTML on public booking page — St_
|
||||
- GHSA-8hm4-r66f-29wr / CVE-2026-52841 (alextselegidis/easyappointments, access_control, 2026-07-29): one case per repository (repository taken as easyappointments-caldav-ssrf). _Easy!Appointments: Authorization bypass in Google OAuth provider binding lets any backend _
|
||||
- GHSA-xgr6-pqjv-3pf8 / CVE-2026-52837 (alextselegidis/easyappointments, access_control, 2026-07-29): one case per repository (repository taken as easyappointments-caldav-ssrf). _Easy!Appointments has unauthenticated customer PII disclosure on booking reschedule page_
|
||||
- GHSA-w8xc-8g92-v77h / CVE-2026-52839 (alextselegidis/easyappointments, access_control, 2026-07-29): one case per repository (repository taken as easyappointments-caldav-ssrf). _Easy!Appointments appointments/store and appointments/update allow cross-provider appointm_
|
||||
- GHSA-pm5p-7w5h-jm5q / CVE-2026-52840 (alextselegidis/easyappointments, untrusted_destination, 2026-07-29): **taken** as easyappointments-caldav-ssrf. _Easy!Appointments has server-side request forgery in CalDAV connection test that exposes t_
|
||||
- GHSA-3735-5339-xfwx / CVE-2026-54588 (poweradmin/poweradmin, untrusted_destination, 2026-07-28): repo over ~150 MB (508 MB per GitHub API). _Poweradmin has Host Header Injection in OIDC redirect_uri, SAML ACS/SLO URL, and Logout Re_
|
||||
- GHSA-3pwp-g2mj-5p3v / CVE-2026-45293 (wordpress/wordpress-coding-standards, injection, 2026-07-28): developer tool / CLI, not a web application. _WordPress Coding Standards (WordPressCS) contains an arbitrary code execution vulnerabilit_
|
||||
- GHSA-rm67-g9ch-vxff (poweradmin/poweradmin, access_control, 2026-07-24): repo over ~150 MB (508 MB per GitHub API). _Poweradmin: Broken access control (IDOR): any zone owner can modify DNS records in zones t_
|
||||
- GHSA-h4hf-v6w5-897x (poweradmin/poweradmin, access_control, 2026-07-24): repo over ~150 MB (508 MB per GitHub API). _Poweradmin: API user-update endpoint leads to a non-admin reset any user's password and ta_
|
||||
- GHSA-g3hq-hphg-8fhh (pheditor/pheditor, injection, 2026-07-24): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Pheditor: Terminal command-allowlist bypass via argument injection leads to RCE — survivin_
|
||||
- GHSA-6hq5-7373-42rg / CVE-2026-59931 (phpoffice/phpspreadsheet, untrusted_destination, 2026-07-23): general-purpose library: the attacker source lies in the calling application, not in the repo. _PHPSpreadsheet: SSRF bypass via HTTP redirect in WEBSERVICE() domain whitelist_
|
||||
- GHSA-cx96-42px-69fm / CVE-2026-56722 (dompdf/dompdf, path, 2026-07-22): general-purpose library: the attacker source lies in the calling application, not in the repo. _Dompdf: Local file read due to improper file path validation in SVG images encoded as data_
|
||||
- GHSA-wvh6-f5jh-8gw4 / CVE-2026-55554 (dompdf/dompdf, path, 2026-07-22): general-purpose library: the attacker source lies in the calling application, not in the repo. _Dompdf: Chroot Validation Bypass_
|
||||
- GHSA-gjfg-22fp-rrxx / CVE-2026-59946 (composer/composer, path, 2026-07-20): developer tool / CLI, not a web application. _Composer: Path traversal in package bin field lets dependencies chmod arbitrary host files_
|
||||
- GHSA-499r-g7pc-vmp9 / CVE-2026-59948 (composer/composer, path, 2026-07-20): developer tool / CLI, not a web application. _Composer: Arbitrary file write outside vendor via malicious transitive package name_
|
||||
- GHSA-cvpc-hccg-wmw4 (verbb/formie, access_control, 2026-07-17): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Formie: Missing authorization in administrative settings allows low-privileged CP users to_
|
||||
- GHSA-wg4w-wr5q-6vjc / CVE-2026-55578 (pheditor/pheditor, injection, 2026-07-16): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Pheditor: Incomplete command sanitization in terminal feature allows RCE via pipe operator_
|
||||
- GHSA-9643-6xjp-vx57 / CVE-2026-54540 (pheditor/pheditor, injection, 2026-07-16): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Pheditor has an authenticated terminal command whitelist bypass_
|
||||
- GHSA-373m-p57p-8665 (, output_encoding, 2026-07-15): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: XSS Blueprint Validation Bypass via Twig String Concatenation_
|
||||
- GHSA-h2wf-967x-gxvw / CVE-2026-62944 (mantisbt/mantisbt, output_encoding, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: Stored XSS in print_all_bug_page_word.php_
|
||||
- GHSA-3v2j-6fw9-f57c / CVE-2026-52882 (mantisbt/mantisbt, access_control, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: REST and SOAP API Issue Update Accepts Unreleased Product Versions From Updaters_
|
||||
- GHSA-vcrw-4xvv-jh49 / CVE-2026-52881 (mantisbt/mantisbt, output_encoding, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: Reflected XSS in admin/install.php via unescaped printf _
|
||||
- GHSA-77x8-3v3h-hrhv / CVE-2026-52847 (mantisbt/mantisbt, output_encoding, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: Reflected XSS in admin/install.php_
|
||||
- GHSA-rjg7-r26h-cfp2 / CVE-2026-54494 (koel/koel, untrusted_destination, 2026-07-15): **taken** as koel-podcast-enclosure-ipv6-ssrf. _Koel: Full-read SSRF via podcast enclosure URL: isPublicHost() filter_var guard does not r_
|
||||
- GHSA-jr4p-4xjh-fwvw / CVE-2026-50552 (koel/koel, untrusted_destination, 2026-07-15): one case per repository (repository taken as koel-podcast-enclosure-ipv6-ssrf). _Koel: Server-Side Request Forgery (SSRF) in radio station creation due to missing validati_
|
||||
- GHSA-6qvr-wjmv-v8mm / CVE-2026-54491 (koel/koel, untrusted_destination, 2026-07-15): one case per repository (repository taken as koel-podcast-enclosure-ipv6-ssrf). _Koel: Incomplete fix for CVE-2026-47260 — systemic SSRF in podcast & radio fetch paths_
|
||||
- GHSA-8q6q-m837-fv64 (koel/koel, untrusted_destination, 2026-07-15): one case per repository (repository taken as koel-podcast-enclosure-ipv6-ssrf). _ Koel has SSRF through Authenticated Subsonic podcast feed URLs_
|
||||
- GHSA-6p96-cfg5-4vhp / CVE-2026-54493 (koel/koel, untrusted_destination, 2026-07-15): one case per repository (repository taken as koel-podcast-enclosure-ipv6-ssrf). _Koel: Authenticated Full-Read SSRF via Subsonic Internet Radio Stations_
|
||||
- GHSA-w79m-f3jx-779v / CVE-2026-54492 (koel/koel, untrusted_destination, 2026-07-15): one case per repository (repository taken as koel-podcast-enclosure-ipv6-ssrf). _Koel: Authenticated Blind SSRF via Subsonic Podcast Channel Creation_
|
||||
- GHSA-m7ph-9558-mrx3 / CVE-2026-49280 (mantisbt/mantisbt, access_control, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: REST API unauthorized Issue status change_
|
||||
- GHSA-v84x-qvhg-f36r / CVE-2026-49273 (mantisbt/mantisbt, injection, 2026-07-15): **taken** as mantisbt-tokenizer-eval-rce. _MantisBT: Remote Code Execution via eval() Class Hoisting in adm_config_set.php_
|
||||
- GHSA-c2xg-qjqw-2v98 / CVE-2026-47156 (mantisbt/mantisbt, access_control, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: SOAP API Authentication Bypass with Privilege Escalation to Administrator_
|
||||
- GHSA-mw6p-33vw-46cc / CVE-2026-47142 (mantisbt/mantisbt, injection, 2026-07-15): one case per repository (repository taken as mantisbt-tokenizer-eval-rce). _MantisBT: SQL Injection via history_order Configuration Value_
|
||||
- GHSA-hgjx-r89m-m7v4 (neorazorx/facturascripts, path, 2026-07-14): **taken** as facturascripts-uploadedfile-move-traversal. _FacturaScripts: Path traversal in UploadedFile::move() via getClientOriginalName() — arbit_
|
||||
- GHSA-8559-gwj3-q37r / CVE-2026-54087 (easycorp/easyadminbundle, output_encoding, 2026-07-14): vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository (uploads must be stored under the public web root); the flaw is a missing default upload restriction, with no rendering sink in the repository. _EasyAdmin: Stored Cross-Site Scripting (XSS) via uploaded files served inline in FileField_
|
||||
- GHSA-3x7p-v8hj-xh5m / CVE-2026-45710 (neorazorx/facturascripts, output_encoding, 2026-07-14): one case per repository (repository taken as facturascripts-uploadedfile-move-traversal). _FacturaScripts: Stored XSS in WidgetVariante and WidgetSubcuenta modal lists via HTML-attr_
|
||||
- GHSA-cv65-7cg8-r623 / CVE-2026-45693 (neorazorx/facturascripts, path, 2026-07-14): one case per repository (repository taken as facturascripts-uploadedfile-move-traversal). _FacturaScripts: Unauthenticated Path Traversal in Static File Controllers Reads Private My_
|
||||
- GHSA-5qmh-x653-g8qj / CVE-2026-45262 (neorazorx/facturascripts, injection/untrusted_destination, 2026-07-14): no fixed version stated (`<= 2026.1`, patched: none) and no identifiable fix: the only candidate (`14ee211c`, 'centralizar parseFieldFilter con validación anti-SQLi') was reverted the same day (`7392b489`). _FacturaScripts: Authenticated SQL injection in the FacturaScripts REST API filter paramete_
|
||||
- GHSA-rw46-qg69-vg6h / CVE-2026-52828 (kimai/kimai, access_control, 2026-07-14): no fix commit referenced; per the v2 research, Kimai's fixes for this release shipped in the single 'Release 2.58' commit (`31a8f887`, 120 files across 2.57.0...2.58.0), mixed with unrelated changes. _Kimai: ExportTemplate CRUD Missing Authorization Check Allows Unauthorized TEAMLEAD Access_
|
||||
- GHSA-2xgg-2x8h-8xw4 / CVE-2026-52826 (kimai/kimai, access_control, 2026-07-14): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai: Improper Authorization in Project, Customer, and Activity Rate Edit Endpoints Allow_
|
||||
- GHSA-xv4r-4885-gwpg / CVE-2026-52825 (kimai/kimai, access_control, 2026-07-14): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai has Improper Authorization in Team Member and Team Activity Assignment APIs Which Al_
|
||||
- GHSA-c6w6-57jj-62vh / CVE-2026-52822 (kimai/kimai, access_control, 2026-07-14): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Improper Authorization in Kimai Timesheet Restart and Duplicate Allows New Timesheets Afte_
|
||||
- GHSA-3q6q-26vg-v97x / CVE-2026-52821 (kimai/kimai, access_control, 2026-07-14): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai: Improper Authorization Through Activity Creation with Preset Project Allows Creatio_
|
||||
- GHSA-xv8g-76mx-2rxc / CVE-2026-49970 (plank/laravel-mediable, path, 2026-07-13): general-purpose library: the attacker source lies in the calling application, not in the repo. _Laravel-Mediable: path traversal vulnerability in the File::sanitizePath()_
|
||||
- GHSA-4chg-4752-w88r / CVE-2026-55372 (nukeviet/nukeviet, untrusted_destination, 2026-07-13): repo over ~150 MB (190 MB per GitHub API). _NukeViet: Pre-authentication SSRF via X-Forwarded-Host_
|
||||
- GHSA-c9xg-64p9-f2jj / CVE-2026-54065 (nukeviet/nukeviet, path, 2026-07-13): repo over ~150 MB (190 MB per GitHub API). _NukeViet: Path Traversal to Arbitrary File Deletion in Edit Comment Function_
|
||||
- GHSA-465g-4q99-5x86 / CVE-2026-54064 (nukeviet/nukeviet, output_encoding, 2026-07-13): repo over ~150 MB (190 MB per GitHub API). _NukeViet: Multiple Anti-XSS Filter Bypasses Leading to Stored XSS in News Module_
|
||||
- GHSA-w2w5-w2pw-r929 / CVE-2026-49259 (nukeviet/nukeviet, output_encoding, 2026-07-13): repo over ~150 MB (190 MB per GitHub API). _NukeViet: Improper Neutralization of Input During Web Page Generation ('Cross-site Scripti_
|
||||
- GHSA-mxpf-qgg6-v3ff / CVE-2026-48118 (nukeviet/nukeviet, output_encoding, 2026-07-13): repo over ~150 MB (190 MB per GitHub API). _NukeViet: Unauthenticated Reflected XSS in Comment Module_
|
||||
- GHSA-vrr2-g9gh-c3jc / CVE-2026-52820 (kimai/kimai, access_control, 2026-07-13): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai: Timesheet PATCH/POST allows assigning to project outside user's team via query_buil_
|
||||
- GHSA-4m8q-55qv-9pwp / CVE-2026-52819 (kimai/kimai, access_control, 2026-07-13): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai: Teamlead authorization bypass in GET /api/timesheets allows reading other users' ti_
|
||||
- GHSA-qv4m-m73m-8hj7 (notrinos/notrinoserp, path, 2026-07-10): no fixed version stated and no identifiable fix commit (the history of `hrm/manage/employees.php` shows no matching change); the weakness is also mainly unrestricted upload (CWE-434). _NotrinosERP: Authenticated arbitrary file upload leads to remote code execution via HRM em_
|
||||
- GHSA-pj8j-p4g4-4vw8 / CVE-2026-49865 (kimai/kimai, untrusted_destination, 2026-07-10): no fix commit referenced; the fix is in Kimai's 'Release 2.58' squash (see above). _Kimai has Server-Side Request Forgery in Invoice PDF Rendering via Markdown Image URLs_
|
||||
- GHSA-pjhx-3c3w-9v23 / CVE-2026-49858 (api-platform/core, access_control, 2026-07-10): a per-process cache-key bug in the serializer; exposure depends on application-declared `#[ApiProperty(security: ...)]` predicates, and no attacker-controlled value reaches the flaw inside the repository. _API Platform Core vulnerable to cross-user attribute leak in JSON:API and HAL item normali_
|
||||
- GHSA-mr9r-h354-966r / CVE-2026-53639 (sylius/sylius, access_control, 2026-07-09): repo over ~150 MB (355 MB per GitHub API). _Sylius: IDOR on Shop Payment Request API endpoints_
|
||||
- GHSA-6955-hrm5-c4qp / CVE-2026-53638 (sylius/sylius, access_control, 2026-07-09): repo over ~150 MB (355 MB per GitHub API). _Sylius: Channel-based payment method restriction bypass on shop account orders API endpoin_
|
||||
- GHSA-px5m-h76g-p7p8 / CVE-2026-52778 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki has Unsafe eval() in its Formula Calculato, Leading to Remote Code Execution & Den_
|
||||
- GHSA-9369-69wj-7m2f / CVE-2026-52777 (yeswiki/yeswiki, deserialization, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki Vulnerable to Authenticated PHP Object Injection in BazarImportAction via unserial_
|
||||
- GHSA-4pf7-cc4r-g63h / CVE-2026-52775 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki has Authenticated SQL Injection via ReactionManager _
|
||||
- GHSA-r5xw-gcgw-hwp5 / CVE-2026-52774 (yeswiki/yeswiki, output_encoding, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki Vulnerable to Reflected XSS via Unescaped `id` Parameter in Bazar Widget HTML Attr_
|
||||
- GHSA-35f3-pg38-486f / CVE-2026-52773 (yeswiki/yeswiki, output_encoding, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki Vulnerable to Reflected XSS via Unescaped Archived-Revision `time` Parameter in `h_
|
||||
- GHSA-xc7j-3g8q-9vh4 / CVE-2026-52772 (yeswiki/yeswiki, output_encoding, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki has stored XSS in Bazar form-field templates via unescaped field.label / field.hin_
|
||||
- GHSA-8f2v-2qhj-gfwg / CVE-2026-52771 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki: Second-Order SQL Injection in Page Delete API via Unescaped Page Tag (`ApiControl_
|
||||
- GHSA-qg78-vmvc-fhjw / CVE-2026-52770 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki: SQL Injection possible through public Bazar entry-listing APIs via numeric `query_
|
||||
- GHSA-vw42-752g-5mrp / CVE-2026-52769 (yeswiki/yeswiki, untrusted_destination, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki has Unauthenticated Server-Side Request Forgery via ActivityPub `Signature.keyId`_
|
||||
- GHSA-6x7x-gcmf-7r8x / CVE-2026-52766 (yeswiki/yeswiki, access_control, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki vulnerable to unauthenticated arbitrary page deletion via `{{erasespamedcomments}}_
|
||||
- GHSA-89v6-j5x6-cmj3 / CVE-2026-52763 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki: SQL injection via the `recentchanges` action `period` argument leads to arbitrary_
|
||||
- GHSA-65p8-9433-jpcp / CVE-2026-52762 (yeswiki/yeswiki, injection, 2026-07-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _YesWiki: Authenticated (Admin) Server-Side Template Injection to Remote Code Execution via_
|
||||
- GHSA-w9mx-xmg4-gc4r / CVE-2026-53932 (stefanzweifel/laravel-backup-restore, injection, 2026-07-09): developer tool / CLI, not a web application. _laravel-backup-restore has an OS Command Injection during database restore_
|
||||
- GHSA-86vw-x4ww-x467 / CVE-2026-56382 (craftcms/cms, injection, 2026-07-09): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: RCE via missing cleanseConfig in FieldsController::actionRenderCardPreview_
|
||||
- GHSA-c43v-4cr8-6mvp / CVE-2026-56394 (craftcms/cms, path, 2026-07-09): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS has authenticated path traversal in `assets/icon`, allowing local `.svg` file re_
|
||||
- GHSA-4wj4-79rr-pvff (, output_encoding, 2026-07-08): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav: Stored CSS injection via Markdown image resize() bypasses prior _
|
||||
- GHSA-vmwx-m75v-qvch / CVE-2026-53634 (code16/sharp, access_control, 2026-07-08): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) ('create and store endpoints of the Quick Creation Command feature'). _Sharp Missing Authorization Check in Quick Creation Command Endpoints_
|
||||
- GHSA-c8m7-r2jv-rw63 / CVE-2026-45016 (egroupware/egroupware, path, 2026-07-07): repo over ~150 MB (188 MB per GitHub API). _EGroupware Vulnerable to Local File Inclusion via file:// URI in Mail Compose_
|
||||
- GHSA-8737-2x9g-xjj7 / CVE-2026-40187 (egroupware/egroupware, injection, 2026-07-07): repo over ~150 MB (188 MB per GitHub API). _EGroupware has Authenticated RCE via Malicious eTemplate Upload_
|
||||
- GHSA-h9qx-v5xp-ph8p / CVE-2026-27823 (egroupware/egroupware, access_control/injection/path, 2026-07-07): repo over ~150 MB (188 MB per GitHub API). _EGroupware has a Remote Code Execution Vulnerability_
|
||||
- GHSA-f74w-488g-8x5r / CVE-2026-55794 (craftcms/cms, injection, 2026-07-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Potential authenticated Remote Code Execution via referrer redirect_
|
||||
- GHSA-xrqc-p465-2xvg / CVE-2026-55793 (craftcms/cms, output_encoding, 2026-07-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Stored XSS via Structure entry title in table view_
|
||||
- GHSA-24x4-j6x9-rfw5 / CVE-2026-55790 (craftcms/cms, output_encoding, 2026-07-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: DOM XSS via GitHub issue title in CraftSupport widget_
|
||||
- GHSA-x76w-8c62-48mg / CVE-2026-56384 (craftcms/cms, access_control, 2026-07-06): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Authenticated "assets/preview-thumb" discloses signed fallback transform previe_
|
||||
- GHSA-565m-g33j-jq96 / CVE-2026-52889 (verbb/formie, injection, 2026-07-06): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Formie Hidden field defaults vulnerable to Server-Side Template Injection_
|
||||
- GHSA-rqjw-r5g4-x8qm (, access_control, 2026-07-06): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: Missing authorization check allows non-admin control panel _
|
||||
- GHSA-3w32-23wj-rxg3 / CVE-2026-50282 (craftcms/cms, access_control, 2026-07-02): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS Vulnerable to Unauthorized Deletion of Destination Folders During Forced Moves_
|
||||
- GHSA-5hvg-w58j-545m / CVE-2026-9811 (mautic/mautic, output_encoding, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic has Stored Cross-Site Scripting (XSS) in Project Option Selector_
|
||||
- GHSA-7h65-whp7-rgqf / CVE-2026-9809 (mautic/mautic, output_encoding, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic has Stored Cross-Site Scripting (XSS) in Projects Component_
|
||||
- GHSA-2jrw-c95w-h43g / CVE-2026-9808 (mautic/mautic, access_control, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic has an Authorization Bypass in API v2 Endpoints_
|
||||
- GHSA-6r9h-4h75-7q4x / CVE-2026-9559 (mautic/mautic, path, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic vulnerable to Path Traversal via Campaign Import_
|
||||
- GHSA-9fx4-7cmj-47vg / CVE-2026-9558 (mautic/mautic, injection, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic has Server-Side Template Injection (SSTI) in Theme Templates_
|
||||
- GHSA-jmv8-8j9j-rcpc / CVE-2026-9557 (mautic/mautic, untrusted_destination, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic Focus component Vulnerable to SSRF_
|
||||
- GHSA-fcmw-wx57-9p75 / CVE-2026-4776 (mautic/mautic, injection, 2026-07-02): repo over ~150 MB (308 MB per GitHub API). _Mautic has SQL Injection in API Contact Filtering_
|
||||
- GHSA-q4rm-m6xh-5pv7 (froxlor/froxlor, access_control, 2026-07-02): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Froxlor customer can create MySQL databases on disallowed servers via Mysqls.add API_
|
||||
- GHSA-7h62-6v23-v8fm / CVE-2026-50284 (craftcms/cms, access_control, 2026-07-02): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Missing peer-permission check in `AssetsController::actionDeleteFolder` allows _
|
||||
- GHSA-qh45-9g5p-m2v4 / CVE-2026-50283 (craftcms/cms, access_control, 2026-07-02): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Unauthorized Deletion of Source Assets During File Replacement_
|
||||
- GHSA-43cq-c2gq-pfpw / CVE-2026-50280 (craftcms/cms, access_control, 2026-07-02): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Authorization bypass in `entries/move-to-section` via missing target-section sa_
|
||||
- GHSA-qq2c-2q8j-jh27 / CVE-2026-50279 (craftcms/cms, access_control, 2026-07-02): repo over ~150 MB (1048 MB per GitHub API). _Craft CMS: Authorship spoofing in `entries/save-entry` via pre-check/post-mutation authori_
|
||||
- GHSA-4h7g-5542-v3fc / CVE-2026-52854 (professionalwiki/maps, output_encoding, 2026-07-02): excluded by name. _mediawiki/maps has stored XSS through the overlays parameter in the display_map parser fun_
|
||||
- GHSA-j5mc-p8qg-39j7 (kimai/kimai, access_control, 2026-07-02): same as the other Kimai rows: fix only inside the 'Release 2.58' squash. _Kimai Favorite Timesheet Add and Remove Endpoints Allows Cross-User Bookmark Manipulation_
|
||||
- GHSA-529h-vh3j-85hq / CVE-2026-49981 (twigphp/twig, access_control, 2026-07-01): general-purpose library: the attacker source lies in the calling application, not in the repo. _Twig: Sandbox filter, tag and function allow-list bypass when sandbox state changes betwee_
|
||||
- GHSA-hwmc-r6mf-jh83 (spatie/schema-org, output_encoding, 2026-07-01): general-purpose library: the attacker source lies in the calling application, not in the repo. _Schema.org has cross-site scripting (XSS) via script break-out in toScript() output_
|
||||
- GHSA-2wwr-9x6f-88gp (easycorp/easyadminbundle, output_encoding/path, 2026-07-01): vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository (the advisory says the bundle's own first-party usage is not affected). _EasyAdminBundle has path traversal and reflected XSS in Flag and Icon Twig components_
|
||||
- GHSA-h8vq-8gpg-mhcg / CVE-2026-48808 (twigphp/twig, access_control, 2026-06-30): general-purpose library: the attacker source lies in the calling application, not in the repo. _Twig: Sandbox property allowlist bypass via the `column` filter under `SourcePolicyInterfa_
|
||||
- GHSA-8x9c-rmqh-456c / CVE-2026-48807 (twigphp/twig, access_control, 2026-06-30): general-purpose library: the attacker source lies in the calling application, not in the repo. _Twig: Sandbox `__toString()` policy bypass via `Traversable` in `join` and `replace` filte_
|
||||
- GHSA-5v5v-ww74-355v / CVE-2026-48806 (twigphp/twig, access_control, 2026-06-30): general-purpose library: the attacker source lies in the calling application, not in the repo. _Twig: Sandbox `__toString()` policy bypass via dynamic mapping keys_
|
||||
- GHSA-5q4q-834j-g8g4 / CVE-2026-47198 (paymenter/paymenter, access_control, 2026-06-30): **taken** as paymenter-checkout-config-override. _Paymenter has URL parameter injection that bypasses paid plan limits at checkout_
|
||||
- GHSA-7vfx-4246-jcfh (solidinvoice/solidinvoice, access_control, 2026-06-26): repo over ~150 MB (151 MB per GitHub API). _SolidInvoice: IDOR in LiveComponent allows same-company cross-user access to API tokens an_
|
||||
- GHSA-x8g9-h984-pc36 / CVE-2026-49359 (pontedilana/php-weasyprint, untrusted_destination, 2026-06-26): general-purpose library: the attacker source lies in the calling application, not in the repo. _PhpWeasyPrint vulnerable to SSRF and local file disclosure via the attachment option_
|
||||
- GHSA-5g9f-cwwg-4p8g / CVE-2026-49358 (pontedilana/php-weasyprint, path, 2026-06-26): general-purpose library: the attacker source lies in the calling application, not in the repo. _PhpWeasyPrint vulnerable to arbitrary file deletion at shutdown via public $temporaryFiles_
|
||||
- GHSA-2fmj-p74r-3wjm / CVE-2026-49286 (pontedilana/php-weasyprint, deserialization, 2026-06-26): general-purpose library: the attacker source lies in the calling application, not in the repo. _PhpWeasyPrint vulnerable to PHAR deserialization via output filename (CVE-2023-28115 case-_
|
||||
- GHSA-f5gc-qxf8-mh9g / CVE-2026-49260 (pontedilana/php-weasyprint, injection, 2026-06-26): general-purpose library: the attacker source lies in the calling application, not in the repo. _php-weasyprint: shell command injection via configurable WeasyPrint binary path due to inv_
|
||||
- GHSA-7mqq-4v55-88gh / CVE-2026-54244 (statamic/cms, access_control, 2026-06-26): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text). _Statamic CMS's incorrect authorization lets view-only users submit Live Preview content re_
|
||||
- GHSA-v5c4-wcpj-x73m / CVE-2026-54242 (statamic/cms, untrusted_destination, 2026-06-26): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text) (Glide image proxy). _Statamic Vulnerable to Server-Side Request Forgery via Glide (DNS rebinding)_
|
||||
- GHSA-2497-6pwj-pwg7 / CVE-2026-49288 (statamic/cms, access_control, 2026-06-26): advisory names no vulnerable file or function (a `sites` entry cannot be declared from its text). _Statamic CMS: Missing authorization on Control Panel fieldtype endpoints allows disclosure_
|
||||
- GHSA-mmj8-wcvw-6789 / CVE-2026-49262 (aimeos/pagible, untrusted_destination, 2026-06-26): **taken** as pagible-admin-proxy-dns-rebinding-ssrf. _Aimeos Pagible CMS vulnerable to Server Side Request Forgery (SSRF) via DNS rebinding in a_
|
||||
- GHSA-wpvj-hjcr-h3p2 / CVE-2026-48820 (cakephp/cakephp, path, 2026-06-26): framework `View::element()`; vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository (the advisory: 'if developers are not using user-supplied data in element names, no action is required'). _CakePHP: View::element() is missing a path containment check_
|
||||
- GHSA-6mmj-jhqj-6c6q / CVE-2026-55542 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT's S3 signature image retrieval lacks authorization before temporary URL _
|
||||
- GHSA-x667-r589-43m7 / CVE-2026-55519 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has Improper Authorization in File Deletion (IDOR)_
|
||||
- GHSA-hf68-g98v-wp9g / CVE-2026-55483 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT Vulnerable to Privilege Escalation via Missing admin Permission Check in User Cre_
|
||||
- GHSA-33g4-646g-qwmm / CVE-2026-55482 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has Multi-Tenancy Bypass via Bulk Asset Update_
|
||||
- GHSA-6x4j-8954-5hxm / CVE-2026-50550 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT has a 2FA reset privilege bypass_
|
||||
- GHSA-p68w-rgmg-3c2v / CVE-2026-49976 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT Vulnerable to User Account Escalation via CSV Import_
|
||||
- GHSA-8c6h-7g6x-m5x4 / CVE-2026-49205 (thorsten/phpmyfaq, access_control, 2026-06-23): repo in used list (already referenced under benchmarks/src/tests/.kiro). _phpMyFAQ: Missing userHasPermission() in 4 API write endpoints (CVE-2026-24421 Incomplete _
|
||||
- GHSA-6f75-x745-xcpr / CVE-2026-48507 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT: Bulk editing users allowed `ldap_import` and `activated_in` bulk editing users_
|
||||
- GHSA-44wp-g8f4-f4v5 / CVE-2026-48500 (filamentphp/filament, access_control, 2026-06-23): repo over ~150 MB (582 MB per GitHub API). _Filament: Unauthenticated temporary file upload on auth pages_
|
||||
- GHSA-52fw-7fw2-fmv5 / CVE-2026-48493 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT Vulnerable to Privilege Escalation for self via API Permissions Assignment_
|
||||
- GHSA-f3c5-6cw8-fg57 / CVE-2026-48492 (grokability/snipe-it, access_control, 2026-06-23): repo over ~150 MB (324 MB per GitHub API). _Snipe-IT's selectlist visibility is too permissive_
|
||||
- GHSA-3fc8-8hp6-6jr4 / CVE-2026-48167 (filamentphp/filament, output_encoding, 2026-06-23): repo over ~150 MB (582 MB per GitHub API). _Filament: Unvalidated ImageColumn and ImageEntry values can be used for XSS_
|
||||
- GHSA-7cqp-7cfv-6c3q (wwbn/avideo, output_encoding, 2026-06-23): repo over ~150 MB (664 MB per GitHub API). _AVideo Meet plugin: anonymous-to-admin stored XSS via unescaped participant User-Agent in _
|
||||
- GHSA-wc3f-xc32-435f / CVE-2026-55173 (wwbn/avideo, injection, 2026-06-23): repo over ~150 MB (664 MB per GitHub API). _AVideo has an incomplete fix of CVE-2026-33482: sanitizeFFmpegCommand still allows a singl_
|
||||
- GHSA-32fw-h446-j4hh (, untrusted_destination, 2026-06-23): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Grav is Vulnerable to XXE via SVG Upload_
|
||||
- GHSA-53h4-8rc4-f539 / CVE-2026-48157 (slimphp/slim, output_encoding, 2026-06-23): vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository (only apps that pass request data to `HttpException::setTitle()`/`setDescription()`). _Slim has Reflected XSS in the HtmlErrorRenderer_
|
||||
- GHSA-8j8m-p79x-g4jm / CVE-2026-33684 (wwbn/avideo, access_control, 2026-06-22): repo over ~150 MB (664 MB per GitHub API). _AVideo's Privilege Escalation via Unguarded Permission Parameters in signUp API Allows Sel_
|
||||
- GHSA-x93q-x9pc-w5hw / CVE-2026-44585 (paymenter/paymenter, access_control, 2026-06-22): one case per repository (repository taken as paymenter-checkout-config-override). _Paymenter has broken object level authorization via service reference manipulation on tick_
|
||||
- GHSA-f95g-vm94-46c3 (, output_encoding, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS Vulnerable to Stored XSS in Settings Names and Field Options_
|
||||
- GHSA-jcmp-jxh2-4jc3 (, path, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS has authenticated path traversal in `assets/icon`, allowing _
|
||||
- GHSA-f4h3-qhg5-j6mq (, access_control, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: Authorized asset "preview file" requests bypass allows user_
|
||||
- GHSA-9r7j-7jhg-4f4c (, output_encoding, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS Vulnerable to Stored XSS via User Group Name in User Permiss_
|
||||
- GHSA-5w9j-w5p8-r4p7 (, output_encoding, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS has Stored XSS in Table Field in its "Row Heading" Column Ty_
|
||||
- GHSA-xj2c-g5xp-4p47 (, access_control, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: Authenticated "assets/preview-thumb" discloses signed fallb_
|
||||
- GHSA-pmm4-v8f6-4vpp (, injection, 2026-06-21): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: Craft CMS: RCE via missing cleanseConfig in FieldsController::actionRe_
|
||||
- GHSA-xj9w-cgqg-q897 (wwbn/avideo, access_control, 2026-06-20): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: AVideo has Unauthenticated PGP Message Decryption via Public Endpoint_
|
||||
- GHSA-rg7q-4223-phjw (, access_control, 2026-06-20): duplicate advisory (GitHub marks it "Duplicate Advisory"). _Duplicate Advisory: AVideo: Unauthenticated Access to Payment Log DataTables Endpoints Exp_
|
||||
- GHSA-5c7p-g73q-rpg5 / CVE-2026-55692 (starcitizenwiki/mediawiki-extensions-embedvideo, output_encoding, 2026-06-19): excluded by name. _StarCitizenWiki Extension Embed Video: Stored XSS via malformed src url with $wgEmbedVideo_
|
||||
- GHSA-7h5p-637f-jfr7 / CVE-2026-55691 (starcitizenwiki/mediawiki-extensions-embedvideo, output_encoding, 2026-06-19): excluded by name. _StarCitizenWiki Extension Embed Video: Stored XSS via unsanitized class passed to template_
|
||||
- GHSA-c29q-5xm7-5p62 / CVE-2026-55690 (starcitizenwiki/mediawiki-extensions-embedvideo, output_encoding, 2026-06-19): excluded by name. _StarCitizenWiki Extension Embed Video: Stored XSS via unsanitized service name in exceptio_
|
||||
- GHSA-vm85-hxw5-5432 / CVE-2026-55766 (guzzle/psr7, output_encoding, 2026-06-19): general-purpose library: the attacker source lies in the calling application, not in the repo. _guzzlehttp/psr7: CRLF Injection in HTTP Start-Line Serialization_
|
||||
- GHSA-9qfv-wgh2-m6p8 / CVE-2026-55374 (jleehr/canto-saas-api, untrusted_destination, 2026-06-19): general-purpose library: the attacker source lies in the calling application, not in the repo. _canto-saas-api: Authenticated API requests can be redirected via unencoded path variables_
|
||||
- GHSA-p9xj-fpr2-jf2q / CVE-2026-55878 (symfony/ux, path, 2026-06-19): developer tool: the `ux:install` console command, not a web request path. _symfony/ux-toolkit: Path Traversal Allows Arbitrary File Write and Read via Crafted Recipe_
|
||||
- GHSA-6v8j-33hc-mv84 / CVE-2026-55877 (symfony/ux, output_encoding, 2026-06-19): the source is icon files or a configured Iconify endpoint, not a request; no fix commit referenced. _symfony/ux-icons: XSS via unsanitized SVG content in local files and Iconify on-demand res_
|
||||
- GHSA-mwqm-4fw3-cjvr / CVE-2026-49216 (symfony/ux, output_encoding, 2026-06-19): the sink is the Stimulus (JavaScript) controller's template literal, not PHP. _symfony/ux-autocomplete: XSS via unescaped AJAX response data_
|
||||
- GHSA-38x5-rcv4-xf7x / CVE-2026-49210 (symfony/ux, output_encoding, 2026-06-19): **taken** as symfony-ux-live-child-tag-xss. _symfony/ux-live-component: XSS via attacker-controlled child component tag_
|
||||
- GHSA-r3w8-2c5r-h9j9 / CVE-2026-54005 (getkirby/kirby, access_control, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: `pages.access` permission is not checked in the `site/find` REST API route_
|
||||
- GHSA-89cp-7p28-jffg / CVE-2026-54004 (getkirby/kirby, access_control, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: Access to files of top-level drafts is not protected by permissions_
|
||||
- GHSA-wr9h-4r83-f4v6 / CVE-2026-54002 (getkirby/kirby, output_encoding, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: Cross-site scripting (XSS) from incomplete HTML/XML sanitization in `Dom::sanitize(_
|
||||
- GHSA-4v4h-m2qq-ppgw / CVE-2026-50188 (getkirby/kirby, output_encoding, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: Request header injection in `Http\Remote`_
|
||||
- GHSA-rhj6-r49h-5932 / CVE-2026-49276 (getkirby/kirby, output_encoding, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: Self cross-site scripting (self-XSS) in the writer field_
|
||||
- GHSA-23q2-54qv-rq5x / CVE-2026-49274 (getkirby/kirby, access_control, 2026-06-18): one case per repository (repository taken as kirby-realpath-sibling-traversal). _Kirby: `pages.access` permission is not checked in the pages picker for parent pages_
|
||||
- GHSA-pmf8-g7c8-7v54 / CVE-2026-55890 (getgrav/grav, output_encoding, 2026-06-18): one case per repository (repository taken as grav-blueprint-bare-function-rce). _Grav: Stored CSS injection via Markdown image ?style=… reaches MediaObjectTrait::style() —_
|
||||
- GHSA-7p36-fq2r-4h7r / CVE-2026-11407 (pimcore/pimcore, injection, 2026-06-17): repo over ~150 MB (534 MB per GitHub API). _Pimcore CMS Twig Sandbox Bypass via SecurityPolicy checkMethodAllowed_
|
||||
- GHSA-m9cv-24rx-8mv7 / CVE-2026-55409 (filamentphp/filament, output_encoding, 2026-06-17): repo over ~150 MB (582 MB per GitHub API). _Filament: Disabled RichEditor field state can be used for XSS_
|
||||
- GHSA-m557-wrgg-6rp4 / CVE-2026-55599 (phpseclib/phpseclib, untrusted_destination, 2026-06-16): general-purpose library: the attacker source lies in the calling application, not in the repo. _phpseclib: X.509 certificate validation sends attacker-controlled outbound requests (serve_
|
||||
- GHSA-9mmg-q95p-gp67 / CVE-2026-50886 (firefly-iii/firefly-iii, access_control, 2026-06-15): repo over ~150 MB (313 MB per GitHub API). _Project Firefly III has incorrect access control in the webhook management component_
|
||||
- GHSA-h5x3-xfc9-m39h / CVE-2026-48784 (symfony/symfony, untrusted_destination, 2026-06-15): repo over ~150 MB (339 MB per GitHub API). _Symfony: UrlGenerator Dot-Segment Encoding Skips Every Other Chained `../` or `./` → Gener_
|
||||
- GHSA-38cx-cq6f-5755 / CVE-2026-48736 (symfony/symfony, untrusted_destination, 2026-06-15): repo over ~150 MB (339 MB per GitHub API). _Symfony: IpUtils::PRIVATE_SUBNETS Omits IPv6 Transition Forms (6to4, NAT64, Teredo, IPv4-c_
|
||||
- GHSA-6h46-9jf5-q59x / CVE-2026-48489 (symfony/symfony, access_control, 2026-06-15): repo over ~150 MB (339 MB per GitHub API). _Symfony: Security Firewall Bypass via failure_forward Subrequest: Unauthenticated Access t_
|
||||
- GHSA-x5qj-865h-mgvm / CVE-2026-48761 (symfony/symfony, output_encoding, 2026-06-15): repo over ~150 MB (339 MB per GitHub API). _Symfony: HtmlSanitizer UrlAttributeSanitizer Misses URL Attributes_
|
||||
- GHSA-vv4x-qcpq-wgrg / CVE-2026-12198 (microweber/microweber, path, 2026-06-15): repo over ~150 MB (1185 MB per GitHub API). _Microweber vulnerable to Path Traversal_
|
||||
- GHSA-pjpj-v387-x4vq / CVE-2026-11607 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in its Form Framework_
|
||||
- GHSA-f34x-rx2w-7pm3 / CVE-2026-47349 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in the Recycler Module_
|
||||
- GHSA-3p42-w5ch-gg42 / CVE-2026-47347 (typo3/typo3, untrusted_destination, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has an Open Redirect Vulnerability via Core Utilities_
|
||||
- GHSA-3v8v-4wg6-r7qh / CVE-2026-47343 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS: Destructive Actions on File Mount Folders_
|
||||
- GHSA-p5j5-4j3q-8mq8 / CVE-2026-47345 (typo3/html-sanitizer, output_encoding, 2026-06-12): general-purpose library: the attacker source lies in the calling application, not in the repo. _TYPO3 HTML Sanitizer allows Cross-site Scripting_
|
||||
- GHSA-jh32-v29g-68pq / CVE-2026-49741 (typo3/typo3, access_control/injection, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Privilege Escalation & SQL Injection in its Form Framework_
|
||||
- GHSA-qcmw-6rm2-5x78 / CVE-2026-47350 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in its DataHandler_
|
||||
- GHSA-hwvq-2w67-rvxp / CVE-2026-47346 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in its Form Framework_
|
||||
- GHSA-chm7-4vch-h8vr / CVE-2026-49742 (typo3/typo3, path, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in its Media Module_
|
||||
- GHSA-c78m-c52x-jgwp / CVE-2026-49740 (typo3/typo3, deserialization, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Insecure Deserialization via Core API_
|
||||
- GHSA-jf56-v8jc-jcc5 / CVE-2026-49738 (typo3/typo3, path, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in its File Abstraction Layer_
|
||||
- GHSA-2j54-93q2-3hjq / CVE-2026-47352 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Broken Access Control in Backend API_
|
||||
- GHSA-q93m-25xv-94hh / CVE-2026-47351 (typo3/typo3, access_control, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS: Broken Access Control in Media Module_
|
||||
- GHSA-cg75-qfg2-w9hj / CVE-2026-47348 (typo3/typo3, output_encoding, 2026-06-12): repo over ~150 MB (644 MB per GitHub API). _TYPO3 CMS has Cross-Site Scripting in Indexed Search_
|
||||
- GHSA-jvf5-rxvv-3mcg / CVE-2026-47344 (typo3/html-sanitizer, output_encoding, 2026-06-12): general-purpose library: the attacker source lies in the calling application, not in the repo. _TYPO3 HTML Sanitizer allows Cross-site Scripting_
|
||||
- GHSA-6jq6-x4cx-qvcm (firefly-iii/firefly-iii, output_encoding, 2026-06-12): repo over ~150 MB (313 MB per GitHub API). _Firefly II has Stored XSS in Audit Log Entry view via piggy bank name (ale.twig)_
|
||||
- GHSA-7q3w-xqjw-g3cr / CVE-2026-48067 (filamentphp/filament, access_control, 2026-06-11): repo over ~150 MB (582 MB per GitHub API). _Filament has inconsistent scope enforcement for its AttachAction and AssociateAction Selec_
|
||||
- GHSA-34xg-wgjx-8xph / CVE-2026-48998 (guzzle/psr7, untrusted_destination, 2026-06-11): general-purpose library: the attacker source lies in the calling application, not in the repo. _guzzlehttp/psr7 has Host Confusion via Authority Reinterpretation_
|
||||
- GHSA-hq7v-mx3g-29hw / CVE-2026-49214 (guzzle/psr7, output_encoding, 2026-06-11): general-purpose library: the attacker source lies in the calling application, not in the repo. _guzzlehttp/psr7 has CRLF Injection via URI Host Component_
|
||||
- GHSA-q8r6-5hfw-5jff / CVE-2026-53723 (guzzle/guzzle-services, injection, 2026-06-11): general-purpose library (`guzzlehttp/guzzle-services` XML serialization); vulnerable only when the host application passes its own data or configuration to the API; no attacker source reaches the flaw inside the repository. _guzzlehttp/guzzle-services' XML Request Serialization Vulnerable to XML Injection via CDAT_
|
||||
- GHSA-g82f-9pw7-773w / CVE-2026-10721 (concretecms/concretecms, deserialization, 2026-06-10): repo over ~150 MB (303 MB per GitHub API). _Concrete CMS: PHP Object Injection via unserialize() calls_
|
||||
- GHSA-wmjr-58rf-xgrc / CVE-2026-53675 (buddypress/buddypress, access_control, 2026-06-10): one case per repository (repository taken as wp-buddypress-messages-rest-idor). _BuddyPress: Any authenticated attacker can enumerate another user's complete friend list v_
|
||||
- GHSA-j3j5-5m8v-7gvc / CVE-2026-53673 (buddypress/buddypress, access_control, 2026-06-10): **taken** as wp-buddypress-messages-rest-idor. _BuddyPress: Authenticated attackers can access arbitrary private message threads via user__
|
||||
- GHSA-jvc5-6g7q-c843 / CVE-2026-48030 (pheditor/pheditor, injection, 2026-06-09): repo in used list (already referenced under benchmarks/src/tests/.kiro). _Pheditor: OS Command Injection in terminal handler via unsanitized 'dir' parameter_
|
||||
- GHSA-pr6f-87hf-hx24 / CVE-2026-50636 (limesurvey/limesurvey, injection, 2026-06-09): repo over ~150 MB (1457 MB per GitHub API). _LimeSurvey has a SQL Injection issue_
|
||||
- GHSA-87m4-826x-3crx / CVE-2026-45034 (phpoffice/phpspreadsheet, deserialization, 2026-06-08): general-purpose library: the attacker source lies in the calling application, not in the repo. _PHPSpreadsheet has a patch bypass for CVE-2026-34084 _
|
||||
- GHSA-2g2g-8p8h-fgwm / CVE-2026-47730 (twigphp/twig, output_encoding, 2026-06-05): general-purpose library: the attacker source lies in the calling application, not in the repo. _Twig: XSS in profiler HtmlDumper via unescaped template and profile names_
|
||||
- GHSA-c3qp-2ggw-xjg7 / CVE-2026-47744 (shopperlabs/shopper, access_control, 2026-06-05): repo over ~150 MB (151 MB per GitHub API). _Shopper: Authorization bypass and RBAC privilege escalation in team settings_
|
||||
- GHSA-hr9v-r8r2-hg7j / CVE-2026-47743 (shopperlabs/shopper, access_control/output_encoding, 2026-06-05): repo over ~150 MB (151 MB per GitHub API). _Shopper: Multiple data integrity and disclosure issues in admin Livewire components_
|
||||
- GHSA-fxqw-97cc-7g5c / CVE-2026-47745 (shopperlabs/shopper, access_control, 2026-06-05): repo over ~150 MB (151 MB per GitHub API). _Shopper: Missing per-action authorization on PaymentMethods, Currencies and Carriers admin_
|
||||
- GHSA-h4mp-g9c6-xwph / CVE-2026-47742 (shopperlabs/shopper, access_control, 2026-06-05): repo over ~150 MB (151 MB per GitHub API). _Shopper: Missing authorization on Product admin Livewire sub-form components_
|
||||
- GHSA-vg35-5wq7-3x7w / CVE-2026-47761 (tinymce/tinymce, output_encoding, 2026-06-05): repo over ~150 MB (152 MB per GitHub API). _TinyMCE Cross-Site Scripting (XSS) vulnerability using media plugin `data-mce-object` inje_
|
||||
- GHSA-v98h-vmpc-fpqv / CVE-2026-47762 (tinymce/tinymce, output_encoding, 2026-06-05): repo over ~150 MB (152 MB per GitHub API). _TinyMCE Cross-Site Scripting (XSS) vulnerability through `mce:protected` comments_
|
||||
- GHSA-q742-qvgc-gc2f / CVE-2026-47759 (tinymce/tinymce, output_encoding, 2026-06-05): repo over ~150 MB (152 MB per GitHub API). _TinyMCE Cross-Site Scripting (XSS) vulnerability using through data-mce- prefixed src, hre_
|
||||
- GHSA-mh5m-5hw4-5c69 / CVE-2026-47760 (tinymce/tinymce, output_encoding, 2026-06-05): repo over ~150 MB (152 MB per GitHub API). _TinyMCE Cross-Site Scripting (XSS) vulnerability using sanitization bypass through nested _
|
||||
- GHSA-gq96-5pfx-f4vc / CVE-2026-48013 (shopware/shopware, untrusted_destination, 2026-06-04): repo over ~150 MB (521 MB per GitHub API). _Shopware: SSRF in Media External-Link Endpoint Bypasses IP Validation_
|
||||
- GHSA-xvhc-gm7j-mhmc / CVE-2026-48015 (shopware/shopware, output_encoding, 2026-06-04): repo over ~150 MB (521 MB per GitHub API). _Shopware: Stored XSS via SVG file upload — no SVG sanitization_
|
||||
- GHSA-f8q6-3g5w-jjr6 / CVE-2026-48014 (shopware/shopware, access_control, 2026-06-04): repo over ~150 MB (521 MB per GitHub API). _Shopware: Admin API ACL Bypass in Order State Transition Endpoints_
|
||||
- GHSA-4x3x-869w-xx3m / CVE-2026-48012 (shopware/shopware, untrusted_destination, 2026-06-04): repo over ~150 MB (521 MB per GitHub API). _Shopware SSO referer trust leading to an arbitrary redirect target_
|
||||
- GHSA-gv8p-48fr-4fxg / CVE-2026-48008 (shopware/shopware, access_control, 2026-06-04): repo over ~150 MB (521 MB per GitHub API). _Shopware: Privilege Escalation via Sync API Integration Admin Flag Bypass_
|
||||
- GHSA-8whc-2wmv-ww35 / CVE-2026-54458 (wwbn/avideo, output_encoding, 2026-06-04): repo over ~150 MB (664 MB per GitHub API). _WWBN AVideo: Unauthenticated Stored DOM Cross-Site Scripting via Per-Client Metadata Broad_
|
||||
- GHSA-66q5-cj5g-wrfx / CVE-2026-50183 (wwbn/avideo, output_encoding, 2026-06-04): repo over ~150 MB (664 MB per GitHub API). _WWBN AVideo: Stored XSS via Hostile YouTube Video Title in AVideo YouTubeAPI Gallery Secti_
|
||||
- GHSA-hgjh-6wj8-gcgf / CVE-2026-50182 (wwbn/avideo, output_encoding, 2026-06-04): repo over ~150 MB (664 MB per GitHub API). _WWBN AVideo: Unauthenticated Reflected XSS via $_GET['search'] in AVideo YouTubeAPI Galler_
|
||||
- GHSA-2fhx-q92v-5fhv / CVE-2026-49279 (wwbn/avideo, output_encoding, 2026-06-04): repo over ~150 MB (664 MB per GitHub API). _WWBN AVideo: Stored XSS via autoEvalCodeOnHTML Bypass in MessageSQLite WebSocket Handler (_
|
||||
- GHSA-c8h8-vq34-9fw2 / CVE-2026-47694 (wwbn/avideo, output_encoding, 2026-06-04): repo over ~150 MB (664 MB per GitHub API). _WWBN AVideo: Stored XSS via unescaped Gallery category description_
|
||||
- GHSA-52pr-7vmf-2w7x / CVE-2026-7888 (concretecms/concretecms, deserialization, 2026-06-03): repo over ~150 MB (303 MB per GitHub API). _Concrete CMS is vulnerable to PHP Object Injection via unserialize() calls in the Workflow_
|
||||
- GHSA-m8xx-3x29-84h8 / CVE-2022-31114 (laravel-backpack/crud, output_encoding, 2026-06-03): one case per repository (repository taken as backpack-stats-host-header-cmdi). _backpack/crud is vulnerable to Cross-Site Scripting (XSS)_
|
||||
|
||||
Passed over (not examined; family quota already met): GHSA-7wwh-xcc3-9fcg (paymenter/paymenter, untrusted_destination, 2026-06-22); GHSA-86hp-hf3j-3m8r (cotonti/cotonti, output_encoding, 2026-06-18); GHSA-hhpq-7wg4-36jm (cakephp/authentication, untrusted_destination, 2026-06-17); GHSA-crmm-hgp2-wgrp (laravel/framework, output_encoding, 2026-06-17); GHSA-gmxh-hjfv-qc2w (benjaminjonard/koillection, untrusted_destination, 2026-06-15); GHSA-p85r-x2wj-mxqj (shlinkio/shlink, untrusted_destination, 2026-06-15); GHSA-wrcg-234w-hfhq (intelliants/subrion, output_encoding, 2026-06-15).
|
||||
|
||||
### Lane B: unreviewed advisories with a GitHub fix reference on a PHP repository
|
||||
|
||||
- GHSA-84f7-xff5-hx38 / CVE-2022-51019 (akaunting/akaunting, injection, 2026-09-29): republished old vulnerability (CVE-2022-51019, fixed in Akaunting 2.1.31 in 2022), not a 2026 advisory; also advisory names no vulnerable file or function (only 'the alias parameter' of module installation). _Akaunting before 2.1.31 contains an OS command injection vulnerability in the module insta_
|
||||
- GHSA-rv7j-2hq3-r6fq / CVE-2024-58386 (zoneminder/zoneminder, path, 2026-09-29): repo over ~150 MB (161 MB per GitHub API). _ZoneMinder versions 1.37.0 before 1.38.0 contain a path traversal vulnerability in the fil_
|
||||
- GHSA-gxmj-vr4j-8j5c / CVE-2026-100884 (krayin/laravel-crm, path, 2026-09-28): the named location is not declarable: the advisory (VulDB text) puts 'the function Storage::download' in `packages/Webkul/Admin/src/Config/acl.php`, a configuration file; the weakness described is an IDOR (CWE-99 'resource identifiers'), not a path. _A vulnerability has been found in Krayin laravel-crm up to 2.2.5. The impacted element is _
|
||||
- GHSA-7q7h-5g76-8cq4 / CVE-2026-100520 (crivion/laranode, path, 2026-09-26): advisory names no vulnerable file or function (only the `POST /filemanager/upload-file` endpoint and its `path` parameter). _Laranode versions before 1.2.1 contain a path traversal vulnerability in the POST /fileman_
|
||||
- GHSA-vm72-96wq-2ppv / CVE-2026-96744 (mongodb/laravel-mongodb, injection, 2026-09-24): general-purpose library (`mongodb/laravel-mongodb` cache lock); the attacker source lies in the calling application. _Improper neutralization of special elements in data query logic in the cache lock implemen_
|
||||
- GHSA-c7hm-qgxh-6vgg / CVE-2026-95701 (misp/misp, path, 2026-09-22): repo over ~150 MB (201 MB). _In MISP, the __statisticsOrgs method in UsersController.php used the organization name dir_
|
||||
- GHSA-24qr-3pqh-xghm / CVE-2026-95698 (misp/misp, path, 2026-09-22): repo over ~150 MB (201 MB). _The findOrgImage method in MISP's OrgImgHelper constructs a filesystem path by concatenati_
|
||||
- GHSA-f496-75v4-rv5q / CVE-2026-95693 (misp/misp, path, 2026-09-22): repo over ~150 MB (201 MB). _In MISP, the EventReport::uploadPicture method in processed a caller-supplied tmp_name fie_
|
||||
- GHSA-qjpg-gmh7-3ch6 / CVE-2026-95667 (misp/misp, path, 2026-09-22): repo over ~150 MB (201 MB). _The MISP installer scripts (for Debian 12, Debian 13, Ubuntu 24.04, and RHEL 9.4) create a_
|
||||
- GHSA-2jvm-986q-589g / CVE-2026-94401 (misp/misp, path, 2026-09-21): repo over ~150 MB (201 MB). _MISP has a file-handling vulnerability that could let certain authenticated users make the_
|
||||
- GHSA-7c6f-jc48-6hfq / CVE-2026-94106 (jamesheinrich/getid3, injection, 2026-09-20): general-purpose library (getID3 media parser); the attacker source lies in the calling application. _getID3 before 1.9.26 contains an OS command injection vulnerability in shell-out handlers _
|
||||
- GHSA-pjph-5h96-886c / CVE-2026-93988 (qloapps/qloapps, path, 2026-09-20): one case per repository (repository taken as qloapps-customermessage-bo-query-sqli). _QloApps through 1.7.0 contains a path traversal vulnerability in the getEmailHTML action o_
|
||||
- GHSA-3v3g-3pf9-fgcf / CVE-2026-93872 (cotonti/cotonti, deserialization, 2026-09-18): no fix diff: the referenced PR #1897 is not merged, and `plugins/comments/controllers/actions/EditAction.php` has no later commit. _Cotonti 1.0.0 passes the base64-decoded cb parameter to unserialize() without allowed_clas_
|
||||
- GHSA-q246-734p-8872 / CVE-2026-93533 (spatie/scotty, injection, 2026-09-18): developer CLI (`scotty` SSH task runner, `DoctorCommand`), not a web application; the fix PR 'awaits acceptance'. _A vulnerability was determined in spatie Scotty up to 1.4.4. This impacts the function..._
|
||||
- GHSA-qqx6-v32j-728q / CVE-2026-89036 (appwrite/appwrite, injection, 2026-09-17): repo over ~150 MB (532 MB). _Appwrite before 2.0.0 contains an argument injection vulnerability that allows authenticat_
|
||||
- GHSA-2vrr-cgr4-gfmm / CVE-2026-76834 (b2evolution/b2evolution, deserialization, 2026-09-17): repo over ~150 MB (182 MB). _b2evolution CMS versions 6.7.8 through 7.2.5 contain an incomplete fix for CVE-2016-8901 w_
|
||||
- GHSA-6wrp-rq46-2frr / CVE-2026-92970 (hubzero/hubzero-cms, path, 2026-09-17): repo over ~150 MB (359 MB). _HUBzero CMS through 2.2.32 contains a path traversal vulnerability in project file upload._
|
||||
- GHSA-xv9c-9hwm-jp8g / CVE-2026-91939 (cotonti/cotonti, deserialization, 2026-09-15): no fix diff: the referenced PR #1889 is not merged. _Cotonti 1.0.0 Comments plugin passes the ci GET parameter to unserialize() without..._
|
||||
- GHSA-63h9-9gmf-c466 / CVE-2020-15875 (librenms/librenms, injection, 2026-09-13): repo over ~150 MB (369 MB). _An issue was discovered in LibreNMS 1.65. A remote authenticated attacker with normal priv_
|
||||
- GHSA-6hw9-7gx5-mw88 / CVE-2026-85200 (fitoussi/geo-my-wp, path, 2026-09-12): **taken** as wp-geo-my-wp-info-window-lfi. _The GEO my WP plugin for WordPress is vulnerable to Local File Inclusion in all versions u_
|
||||
- GHSA-hh3c-hgv7-gg2r / CVE-2026-79362 (woltlab/wcf, injection, 2026-09-11): repo over ~150 MB (203 MB). _Certain Woltlab products are affected by RCE via Cache Poisoning. WCF >= 6.1.0 until < 6.1_
|
||||
- GHSA-w3w8-r2xc-3rp3 / CVE-2024-58381 (pmmp/pocketmine-mp, deserialization, 2026-09-09): game server (Minecraft Bedrock), not a web application; old vulnerability (CVE-2024-58381); the impact is denial of service. _PocketMine-MP before 5.11.1 contains a denial of service vulnerability in LoginPacket JSON_
|
||||
- GHSA-cw46-936q-fjcf / CVE-2026-57777 (woocommerce/woocommerce, injection, 2026-09-04): repo over ~150 MB (932 MB). _Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection')..._
|
||||
- GHSA-xf4v-vq77-2rgv / CVE-2026-84694 (coollabsio/coolify, injection, 2026-09-02): advisory names no vulnerable file or function (only 'environment variable key names in Docker commands'). _Coolify before 4.2.0 fails to properly escape environment variable key names in Docker com_
|
||||
- GHSA-46j6-w3hw-4r9x / CVE-2026-71981 (cypht-org/cypht, deserialization, 2026-09-01): advisory names no vulnerable file or function (only 'the back_query GET parameter of the logout handler'); the fix is known (`e4aa2f34`, merge of PR #2073) — the nearest alternate for deserialization if the naming rule is relaxed. _Cypht before 2.12.2 contains a PHP object injection vulnerability that allows authenticate_
|
||||
- GHSA-v64j-pgc3-qgc9 / CVE-2026-82668 (klaussilveira/gitlist, injection, 2026-08-31): **taken** as gitlist-commandline-cmdi. _A security vulnerability has been detected in klaussilveira GitList 2.0.0. Affected by thi_
|
||||
- GHSA-j8xc-7c27-7wj4 / CVE-2026-81728 (dolibarr/dolibarr, injection, 2026-08-27): repo over ~150 MB (1705 MB). _Dolibarr before 24.0.0 contains a SQL injection in its CSV and XLSX import wizard. The wiz_
|
||||
- GHSA-xf89-whcw-pq6g / CVE-2020-15878 (librenms/librenms, injection, 2026-08-26): repo over ~150 MB (369 MB). _An issue was discovered in LibreNMS 1.65. A remote authenticated attacker with normal priv_
|
||||
- GHSA-2658-wj4j-m387 / CVE-2020-15876 (librenms/librenms, injection, 2026-08-26): repo over ~150 MB (369 MB). _An issue was discovered in LibreNMS 1.65. A remote authenticated attacker with normal priv_
|
||||
- GHSA-6h76-35rh-gxm9 / CVE-2026-80428 (ilias-elearning/ilias, deserialization, 2026-08-26): repo over ~150 MB (3341 MB). _ILIAS deserialises stored session data for an unauthenticated caller. The Shibboleth back-_
|
||||
- GHSA-4rvx-xf2v-6r64 / CVE-2020-15874 (librenms/librenms, injection, 2026-08-26): repo over ~150 MB (369 MB). _An issue was discovered in LibreNMS 1.65. A remote authenticated attacker with normal priv_
|
||||
- GHSA-5w6r-2mxc-fq3v / CVE-2026-75497 (qloapps/qloapps, injection, 2026-08-25): **taken** as qloapps-customermessage-bo-query-sqli. _Webkul QloApps does not validate request parameters before a database query. A remote,..._
|
||||
- GHSA-mj48-jjc7-6wqr / CVE-2026-75498 (qloapps/qloapps, injection, 2026-08-25): one case per repository (repository taken as qloapps-customermessage-bo-query-sqli). _Webkul QloApps does not validate request parameters before a database query. A remote,..._
|
||||
- GHSA-8x6f-m45x-3rjp / CVE-2026-71905 (volmarg/personal-management-system, injection/path, 2026-08-24): corrupt advisory record: the description is about DrayTek VigorAP routers, not this repository. _Multiple DrayTek VigorAP models contain a command injection vulnerability in the ExportSet_
|
||||
- GHSA-2c9p-fq48-6545 / CVE-2026-75007 (roundcube/roundcubemail, injection, 2026-08-17): advisory names no vulnerable file or function (only 'the LDAP search filter ... %u/%fu/%d substitution'). _In Roundcube Webmail before 1.6.18 and 1.7.x before 1.7.3, the LDAP search filter was subj_
|
||||
- GHSA-m462-3x7r-5r7f / CVE-2026-75002 (roundcube/roundcubemail, injection, 2026-08-17): advisory names no vulnerable file or function (only 'mail search and LITERAL+ byte-count desynchronization'). _In Roundcube Webmail before 1.6.18 and 1.7.x before 1.7.3, mail search and LITERAL+ byte-c_
|
||||
- GHSA-q5rh-mmp4-ppgq / CVE-2026-74997 (roundcube/roundcubemail, injection, 2026-08-17): **taken** as roundcube-markasjunk-cmd-learn-rce. _In Roundcube Webmail before 1.6.18 and 1.7.x before 1.7.3, the cmd_learn driver of the mar_
|
||||
- GHSA-4x4q-mx3g-g3m2 / CVE-2026-73680 (cockpit-hq/cockpit, injection, 2026-08-14): advisory names no vulnerable file or function in the repository (only the library call `Process::fromShellCommandline()` and 'the FFmpeg integration'). _Cockpit CMS 2.14.0 and prior contains a command injection vulnerability in the FFmpeg inte_
|
||||
- GHSA-w54w-xf28-wf79 / CVE-2026-71248 (harsh21patel/inventory-management-system-php, injection, 2026-08-05): personal/student project (`harsh21patel/inventory-management-system-php`), not a maintained application. _Inventory-Management-System-PHP's login.php constructs its authentication query via direct_
|
||||
- GHSA-cv48-22qx-mpmx / CVE-2026-71232 (magicblack/maccms10, injection, 2026-08-05): **taken** as maccms10-template-blacklist-rce. _MacCMS10's admin template editor (application/admin/controller/Template.php) blocks danger_
|
||||
|
||||
Passed over (not examined; family quota already met): GHSA-5528-3vrx-6443 (octobercms/october, untrusted_destination, 2026-09-28); GHSA-g8jm-3grw-8wh2 (cotonti/cotonti, untrusted_destination, 2026-09-26); GHSA-gmhw-pxmg-j22w (jamesheinrich/getid3, untrusted_destination, 2026-09-20); GHSA-gp9j-h9gm-p3q2 (cotonti/cotonti, untrusted_destination, 2026-09-18); GHSA-5p43-vwpq-wm4q (cotonti/cotonti, untrusted_destination, 2026-09-18); GHSA-w83g-wg62-w3qm (laradashboard/laradashboard, untrusted_destination, 2026-09-09); GHSA-pmmj-hr44-h52v (ilias-elearning/ilias, path, 2026-08-31); GHSA-wvjm-58fp-5q8g (yaojingang/geoflow, untrusted_destination, 2026-08-31); GHSA-gf7h-mv4h-gg7c (yaojingang/geoflow, path, 2026-08-31); GHSA-jgch-2jr2-87vj (dolibarr/dolibarr, path, 2026-08-27); GHSA-g95v-m725-f6xc (volmarg/personal-management-system, path, 2026-08-27); GHSA-gqr6-9g7c-7g7r (wintercms/winter, path, 2026-08-25); GHSA-j9wc-mrp3-532h (hieventsdev/hi.events, untrusted_destination, 2026-08-24); GHSA-7994-gr8j-h7xp (openemr/openemr, path, 2026-08-19); GHSA-vf28-qp34-hhm8 (roundcube/roundcubemail, untrusted_destination, 2026-08-17); GHSA-qf4h-jxxc-wvr9 (getgrav/grav, path, 2026-08-14); GHSA-qrrp-6pvj-xfvf (rconfig/rconfig, path, 2026-08-12); GHSA-f3gp-9xr4-qmvc (joomla/joomla-cms, path, 2026-08-12); GHSA-7gmf-8rwv-m7jc (ayecode/geodirectory, path, 2026-08-11); GHSA-8879-fh4f-q3qm (krayin/laravel-crm, injection, 2026-08-03); GHSA-pw6j-6h33-fv96 (ayecode/invoicing, path, 2026-08-01); GHSA-rfxf-9pcf-q3rq (leantime/leantime, untrusted_destination, 2026-07-30); GHSA-pjx9-wh57-j26p (leantime/leantime, untrusted_destination, 2026-07-30); GHSA-frjf-677q-65jf (misp/misp, untrusted_destination, 2026-07-28); GHSA-hc3p-4h64-j8wr (getgrav/grav-plugin-api, path, 2026-07-23); GHSA-9wvj-rhhh-8292 (simplemachines/smf, untrusted_destination, 2026-07-14); GHSA-5q79-cfqm-5cxh (roundcube/roundcubemail, untrusted_destination, 2026-07-14); GHSA-cfx5-q5qm-x936 (cockpit-hq/cockpit, path, 2026-07-14); GHSA-9fmc-j924-ghh2 (plank/laravel-mediable, untrusted_destination, 2026-07-13); GHSA-j69f-3cmp-jrjj (ayecode/userswp, path, 2026-07-09); GHSA-7793-hmg3-gh57 (librebooking/librebooking, path, 2026-07-09); GHSA-4crx-2rrv-xxcf (horde/vfs, injection, 2026-07-08); GHSA-qfr5-75q5-g2c7 (kirilkirkov/ecommerce-codeigniter-bootstrap, path, 2026-07-04); GHSA-jf5m-wm8f-5pq9 (kirilkirkov/ecommerce-codeigniter-bootstrap, untrusted_destination, 2026-07-04); GHSA-r5xq-x4r6-7g6x (kirilkirkov/ecommerce-codeigniter-bootstrap, path, 2026-07-04); GHSA-233w-5rfj-8hmj (myvesta/vesta, injection, 2026-07-04); GHSA-cr5v-q482-32fc (phpipam/phpipam, path, 2026-07-04); GHSA-cgfv-42v7-7f53 (notifiarr/dockwatch, injection, 2026-07-02); GHSA-mv7w-qwqh-w2g9 (horde/imp, path, 2026-07-01); GHSA-2852-jv28-2mv8 (dolibarr/dolibarr, injection, 2026-06-30); GHSA-w5j4-x499-pfwp (frontaccountingerp/fa, injection, 2026-06-29); GHSA-8ccm-j5hq-9jhr (frontaccountingerp/fa, injection, 2026-06-29); GHSA-53v3-ffpm-f9hw (frontaccountingerp/fa, injection, 2026-06-29); GHSA-9fmm-h59x-wr53 (frontaccountingerp/fa, path, 2026-06-29); GHSA-7v9f-64q7-x2jg (misp/misp, injection, 2026-06-22); GHSA-qrw9-xj5g-qxw8 (chilek/lms, injection, 2026-06-18); GHSA-765c-7qf8-cw9w (chilek/lms, injection, 2026-06-18); GHSA-c4wp-7485-699w (misp/misp, path, 2026-06-12); GHSA-83v9-496w-54wx (wpusermanager/wp-user-manager, path, 2026-06-06); GHSA-9gjj-gmcx-6fhj (misp/misp, untrusted_destination, 2026-06-04); GHSA-rx96-8w86-5hhh (misp/misp, untrusted_destination, 2026-06-04); GHSA-93rv-mf2v-vv2r (php-censor/php-censor, injection, 2026-06-01).
|
||||
|
||||
### Lane C: deserialization (WordPress and other PHP products without a GitHub fix reference)
|
||||
|
||||
- GHSA-ggx5-r53p-pf97 / CVE-2026-86841 (WordPress plugin, deserialization, 2026-09-27): advisory names no vulnerable function or file. _The Online Scheduling and Appointment Booking System WordPress plugin before 28.3 does no_
|
||||
- GHSA-j2rf-55r9-c5ph / CVE-2026-80513 (WordPress plugin, deserialization, 2026-09-24): advisory names no vulnerable function or file (only 'a user-supplied profile field value'). _The wpForo Forum WordPress plugin before 3.1.6 does not restrict which classes may be inst_
|
||||
- GHSA-jvgp-rj42-qjww / CVE-2026-91827 (WordPress plugin, deserialization, 2026-09-22): advisory names no vulnerable function or file; Ninja Forms has no public development repository. _The Ninja Forms WordPress plugin 3.15.3 does not prevent user-submitted form field values _
|
||||
- GHSA-wq6q-gx9m-7669 / CVE-2026-19658 (WordPress plugin, deserialization, 2026-09-22): advisory names no vulnerable function or file. _The Give Tributes plugin for WordPress is vulnerable to PHP Object Injection in all versio_
|
||||
- GHSA-gjf4-862p-rg96 (Softaculous Virtualizor, deserialization, 2026-09-22): proprietary product, no public repository.
|
||||
- GHSA-mwgw-rr28-m856 (CuteNews 2.1.2, deserialization, 2026-09-21): no fix diff (the advisory links only the master blob of core/core.php and a disclosure repository; cutenews/cutenews-2.0 has no commit since 2026-06).
|
||||
- GHSA-3wpx-h7q2-mv8g / CVE-2026-85017 (WordPress plugin, deserialization, 2026-09-20): advisory names no vulnerable function or file (an unnamed AJAX action). _The Unlimited Elements For Elementor WordPress plugin before 2.0.20 does not perform a cap_
|
||||
- GHSA-mqfm-qx5g-96vc / CVE-2026-17086 (WordPress plugin, deserialization, 2026-09-18): advisory names no vulnerable function or file. _The ShortPixel Image Optimizer – Optimize Images, Convert WebP & AVIF plugin for WordPress_
|
||||
- GHSA-cg5c-mg8j-jqc2 / CVE-2026-78175 (WordPress plugin, deserialization, 2026-09-12): no single identifiable fix diff: in themeum/tutor the withdraw-field fix was committed (`0f0b2b20`), reverted (`0f730b86`), re-applied as an empty commit (`1da79c03`) and followed by `642c8f5c`, all on a branch merged with several unrelated security fixes. _The Tutor LMS – eLearning and online course solution plugin for WordPress is vulnerable to_
|
||||
- GHSA-4q25-cq4p-f83q / CVE-2026-78006 (WordPress plugin, deserialization, 2026-09-12): repo over ~150 MB (the-events-calendar/the-events-calendar, 153 MB per GitHub API). _The The Events Calendar plugin for WordPress is vulnerable to Remote Code Execution in all_
|
||||
- GHSA-w3rx-h7m8-2ggf / CVE-2026-84099 (WordPress plugin, deserialization, 2026-09-12): advisory names no vulnerable function or file (an unnamed bundled add-on). _The wpstorecart WordPress plugin through 5.0.7 does not prevent direct, unauthenticated ac_
|
||||
- GHSA-mv63-gmf8-5x76 / CVE-2026-82845 (WordPress plugin, deserialization, 2026-09-12): advisory names no vulnerable function or file. _The Masteriyo LMS WordPress plugin before 3.4.1 does not prevent user-supplied values hel_
|
||||
- GHSA-3237-mv84-xmm9 / CVE-2026-82925 (WordPress plugin, deserialization, 2026-09-10): advisory names no vulnerable function or file. _The Site Reviews WordPress plugin before 8.3.0 does not prevent request data from being de_
|
||||
- GHSA-3xv5-w7r3-9wj8 (FileRun, deserialization, 2026-09-10): proprietary product, no public repository.
|
||||
- GHSA-7jv8-f84m-pq5j / CVE-2026-11363 (WordPress plugin, deserialization, 2026-09-09): advisory names no vulnerable function or file; no public development repository. _The Ninja Forms – The Contact Form Builder That Grows With You plugin for WordPress is vul_
|
||||
- GHSA-2phq-cm26-22pm (MaxSite CMS, deserialization, 2026-09-09): no fix diff (maxsite/cms has no fix commit after the 2026-09-08 disclosure).
|
||||
- GHSA-mmpp-99gg-2rq8 / CVE-2026-16502 (WordPress plugin, deserialization, 2026-09-08): advisory names no vulnerable function or file. _The Live Composer – Free WordPress Website Builder plugin for WordPress is vulnerable to P_
|
||||
- GHSA-v38w-pf7m-c3pr / CVE-2026-10196 (WordPress plugin, deserialization, 2026-09-05): function named (`handle_form_submission`) but no public development repository (GitHub has only an SVN mirror, common-repository/mail-mint, last push 2024). _The Mail Mint – Email Marketing, Newsletter, Email Automation & WooCommerce Emails plugin _
|
||||
- GHSA-fq25-6gpx-5vxf / CVE-2026-19887 (WordPress plugin, deserialization, 2026-09-05): function named (`usces_action_acting_transaction`) but no development repository (only SVN mirrors of usc-e-shop). _The Welcart e-Commerce plugin for WordPress is vulnerable to PHP Object Injection in all v_
|
||||
- GHSA-xw87-h6v2-j87m / CVE-2023-3360 (WordPress plugin, deserialization, 2026-09-02): republished old vulnerability (CVE-2023-3360). _The Weaver Show Posts WordPress plugin before 1.8.1 unserialises the content of an importe_
|
||||
- GHSA-f6hp-2qj5-48vv / CVE-2026-19116 (WordPress plugin, deserialization, 2026-09-02): advisory names no vulnerable function or file. _The User Frontend WordPress plugin before 4.3.11 does not prevent user-supplied field val_
|
||||
- GHSA-w8mx-gm67-wrqq / CVE-2026-76547 (WordPress plugin, deserialization, 2026-08-29): advisory names no vulnerable function or file. _The User Profile Builder WordPress plugin before 4.0.1 does not validate the type of data_
|
||||
- GHSA-j5mw-m2j3-5747 / CVE-2026-14558 (WordPress plugin, deserialization, 2026-08-28): advisory names no vulnerable function or file. _The User Frontend WordPress plugin before 4.3.10 does not properly validate field type de_
|
||||
- GHSA-rm66-h7v3-77rx / CVE-2026-78572 (WordPress plugin, deserialization, 2026-08-25): advisory names no vulnerable function or file. _The Kalles Addons plugin for WordPress is vulnerable to PHP Object Injection in all versio_
|
||||
- GHSA-89vc-p8p8-f8q8 / CVE-2026-32563 (WordPress plugin, deserialization, 2026-08-25): advisory names no vulnerable function or file (Patchstack one-line record). _Subscriber PHP Object Injection in ACPT (Pro) - Custom Post Types Plugin for WordPress <= _
|
||||
- GHSA-mj2c-4qfw-rhhc (TYPO3 extension (unnamed), deserialization, 2026-08-25): advisory names neither the extension's repository nor a function.
|
||||
- GHSA-r2vh-j53r-xpr5 (TYPO3 extension (Apache Solr for TYPO3), deserialization, 2026-08-25): no function or file named (only content object types), no fix link.
|
||||
- GHSA-fgg6-3rw5-h3j8 / CVE-2026-0551 (WordPress plugin, deserialization, 2026-08-23): advisory names no vulnerable function or file (only the 'post_protection_roles' parameter). _The PPWP – Password Protect Pages plugin for WordPress is vulnerable to PHP Object Injecti_
|
||||
- GHSA-hpc2-mqfw-fq4q / CVE-2026-4703 (WordPress plugin, deserialization, 2026-08-22): advisory names no vulnerable function or file. _The WS Form LITE – Drag & Drop Contact Form Builder plugin for WordPress is vulnerable to _
|
||||
- GHSA-p6x4-h8m9-9mr2 / CVE-2024-13784 (WordPress plugin, deserialization, 2026-08-16): republished old vulnerability (CVE-2024-13784); advisory names no vulnerable function or file. _The Contact Form, Survey, Quiz & Popup Form Builder – ARForms plugin for WordPress is vuln_
|
||||
- GHSA-v2rq-f773-w46v / CVE-2026-10035 (WordPress plugin, deserialization, 2026-08-16): function named (`wvrbbp_set_to_serialized_values()`) but no development repository (only the SVN mirror common-repository/weaver-for-bbpress). _The Turnkey bbPress by WeaverTheme plugin for WordPress is vulnerable to PHP Object Inject_
|
||||
- GHSA-q83h-6954-x259 / CVE-2026-16099 (WordPress plugin, deserialization, 2026-08-16): **taken** as wp-podlove-generation-guard-poi. _The Podlove Podcast Publisher plugin for WordPress is vulnerable to arbitrary file deletio_
|
||||
- GHSA-jxc3-pfg8-hv86 / CVE-2026-16267 (WordPress plugin, deserialization, 2026-08-08): advisory names no vulnerable function or file. _The Newsletters WordPress plugin before 4.16 does not restrict the classes allowed when un_
|
||||
- GHSA-m7vw-4232-p79x / CVE-2026-16258 (WordPress plugin, deserialization, 2026-08-07): advisory names no vulnerable function or file. _The Ajax Search Lite WordPress plugin before 4.14.5 does not prevent the deserialization _
|
||||
- GHSA-rm37-v7mc-6795 (Cotonti, deserialization, 2026-08-05): third Cotonti comments deserialization record; no merged fix (see lane B).
|
||||
- GHSA-jm66-348j-qvv2 (MaxSite CMS, deserialization, 2026-08-04): no fix diff.
|
||||
- GHSA-5953-j744-chvm / CVE-2026-16297 (WordPress plugin, deserialization, 2026-08-03): advisory names no vulnerable function or file. _The Clearfy Cache WordPress plugin before 2.4.3 does not restrict the classes allowed whe_
|
||||
- GHSA-w2rh-rmh8-jmwq / CVE-2025-15672 (WordPress plugin, deserialization, 2026-08-03): republished old vulnerability (CVE-2025-15672); advisory names no vulnerable function or file. _The ChamaWP WordPress plugin before 1.0.13 does not properly validate user input before p_
|
||||
- GHSA-xh9j-gpmv-5c2v / CVE-2026-16062 (WordPress plugin, deserialization, 2026-08-02): advisory names no vulnerable function or file. _The Event Booking Manager for WooCommerce WordPress plugin before 5.3.7 does not prevent _
|
||||
- GHSA-rpvq-6gf9-58vg / CVE-2026-12720 (WordPress plugin, deserialization, 2026-07-31): advisory names no vulnerable function or file; themeum/kirki has no 6.x code (v2 research). _The Kirki WordPress plugin before 6.0.13 does not restrict which classes may be instantia_
|
||||
- GHSA-95g3-wxff-cpfp / CVE-2026-1360 (WordPress plugin, deserialization, 2026-07-30): function named (`bp_unserialize_profile_field()`), but no fix commit is identifiable in buddypress/buddypress (history of `src/bp-xprofile/bp-xprofile-functions.php` from 2026-06 shows only coding-standard and PHPDoc commits); the repo is taken anyway by wp-buddypress-messages-rest-idor. _The BuddyPress plugin for WordPress is vulnerable to Deserialization of Untrusted Data in _
|
||||
- GHSA-4f62-78r3-84fq (Joomla extension Aimy Captcha-Less Form Guard, deserialization, 2026-07-29): commercial extension, no public repository.
|
||||
- GHSA-2hhm-rw5g-m2vq / CVE-2026-15962 (WordPress plugin, deserialization, 2026-07-26): advisory names no vulnerable function or file. _The Fluent Forms Pro Add On Pack plugin for WordPress is vulnerable to PHP Object Injectio_
|
||||
- GHSA-m5q4-xwgp-qwg2 / CVE-2026-15008 (WordPress plugin, deserialization, 2026-07-16): fix only inside a whole-release commit (UncannyOwl/Uncanny-Automator `ec5fde67` 'v7.4.0'). _The Uncanny Automator – Easy Automation, Integration, Webhooks & Workflow Builder Plugin p_
|
||||
- GHSA-cxqr-jhpf-63mm / CVE-2026-12583 (WordPress plugin, deserialization, 2026-07-14): advisory names no vulnerable function or file. _The Newsletters WordPress plugin before 4.15 does not prevent deserialization of untrusted_
|
||||
- GHSA-ww5c-p3qw-8xxf / CVE-2026-12240 (WordPress plugin, deserialization, 2026-06-30): no development repository on GitHub (only unrelated or stale repositories named export-user-data). _The Export User Data plugin for WordPress is vulnerable to arbitrary file deletion due to _
|
||||
- GHSA-46p4-p677-cfjc / CVE-2025-69130 (WordPress plugin, deserialization, 2026-06-17): advisory names no vulnerable function or file (Patchstack one-line record, a commercial theme). _Subscriber PHP Object Injection in Entrepreneur - Booking for Small Businesses WordPress T_
|
||||
- GHSA-v8h4-8wcw-7gjp / CVE-2026-12115 (WordPress plugin, deserialization, 2026-06-17): advisory names no vulnerable function or file. _The Counter Box – Add Countdowns, Timers & Dynamic Counters to WordPress plugin for WordPr_
|
||||
- GHSA-r622-c48h-h6qq / CVE-2026-8365 (WordPress plugin, deserialization, 2026-06-09): function named (`blocksy_sanitize_post_meta_options()`) but no public development repository for the Blocksy theme was found. _The Blocksy theme for WordPress is vulnerable to PHP Object Injection leading to Remote Co_
|
||||
- GHSA-4jj4-xwhh-x5m2 / CVE-2026-7566 (WordPress plugin, deserialization, 2026-06-06): excluded by name (LearnPress add-on); advisory names no vulnerable function or file. _The LearnPress – Backup & Migration Tool plugin for WordPress is vulnerable to PHP Object _
|
||||
- GHSA-h6x4-229g-5f43 / CVE-2026-7654 (WordPress plugin, deserialization, 2026-06-06): **taken** as wp-admin-columns-ids-to-collection-poi. _The Admin Columns plugin for WordPress is vulnerable to PHP Object Injection leading to Re_
|
||||
|
||||
@@ -0,0 +1,581 @@
|
||||
{
|
||||
"schema_version": 1,
|
||||
"experiment": "independent-population-v1-evaluation",
|
||||
"protocol": "protocol-v1.md",
|
||||
"analyzer_commit": "ebcc2cf7fd847aa6ff7a5d08b6d9dce340f5ed4f",
|
||||
"started": "2026-09-24T16:45:45+02:00",
|
||||
"population": "population-v1.toml",
|
||||
"summary": {
|
||||
"recall": [
|
||||
1,
|
||||
11
|
||||
],
|
||||
"recall_wilson95": [
|
||||
0.016,
|
||||
0.377
|
||||
],
|
||||
"precision": [
|
||||
1,
|
||||
12
|
||||
],
|
||||
"precision_wilson95": [
|
||||
0.015,
|
||||
0.354
|
||||
],
|
||||
"fixed_side_alerts": 4,
|
||||
"benign_alerts": 1,
|
||||
"cases_with_zero_completed_questions_at_vulnerable_pin": [
|
||||
"phpmyfaq-captcha-ua-sqli",
|
||||
"yeswiki-bazar-form-sqli",
|
||||
"signalk-remote-conn-ssrf",
|
||||
"saltcorn-sync-sqli",
|
||||
"mongo-express-bson-rce",
|
||||
"mockoon-admin-api-cors",
|
||||
"alerta-query-sqli",
|
||||
"wger-next-open-redirect"
|
||||
]
|
||||
},
|
||||
"m4_gates": {
|
||||
"precision>=0.95": false,
|
||||
"recall>=0.90": false,
|
||||
"completion>=0.95": false,
|
||||
"zero_fixed_or_benign_alerts": false
|
||||
},
|
||||
"reading": "The first measurement on untouched code fails every M4 gate. One of eleven targets is detected (Signal K's SSRF), eleven of twelve reported findings are false, and four fixed-side and one benign false alerts are raised. The dominant cause is not precision but coverage: at the vulnerable pin eight of eleven cases completed no question at all (Saltcorn asked none), so most targets were never decided. That is the first thing to investigate; nothing here may inform a change before this record is committed.",
|
||||
"cases": [
|
||||
{
|
||||
"id": "wp-ultimate-member-sqli",
|
||||
"family": "injection",
|
||||
"language": "php",
|
||||
"detected": false,
|
||||
"detected_sites": [],
|
||||
"unanswered": false,
|
||||
"instrument_errors": {},
|
||||
"fixed_side_alerts": [],
|
||||
"benign_alerts": [],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
76,
|
||||
303
|
||||
],
|
||||
"fixed": [
|
||||
76,
|
||||
303
|
||||
],
|
||||
"benign_base": [
|
||||
71,
|
||||
229
|
||||
],
|
||||
"benign_tip": [
|
||||
71,
|
||||
229
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": 215.8,
|
||||
"fixed": 218.0,
|
||||
"benign_base": 199.3,
|
||||
"benign_tip": 202.8
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wp-learnpress-sqli",
|
||||
"family": "injection",
|
||||
"language": "php",
|
||||
"detected": false,
|
||||
"detected_sites": [],
|
||||
"unanswered": false,
|
||||
"instrument_errors": {},
|
||||
"fixed_side_alerts": [],
|
||||
"benign_alerts": [],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
4815,
|
||||
5000
|
||||
],
|
||||
"fixed": [
|
||||
4815,
|
||||
5000
|
||||
],
|
||||
"benign_base": [
|
||||
4815,
|
||||
5000
|
||||
],
|
||||
"benign_tip": [
|
||||
4815,
|
||||
5000
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": 414.2,
|
||||
"fixed": 470.5,
|
||||
"benign_base": 418.1,
|
||||
"benign_tip": 467.7
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "phpmyfaq-captcha-ua-sqli",
|
||||
"family": "injection",
|
||||
"language": "php",
|
||||
"detected": false,
|
||||
"detected_sites": [],
|
||||
"unanswered": true,
|
||||
"instrument_errors": {},
|
||||
"fixed_side_alerts": [],
|
||||
"benign_alerts": [],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
0,
|
||||
5000
|
||||
],
|
||||
"fixed": [
|
||||
0,
|
||||
5000
|
||||
],
|
||||
"benign_base": [
|
||||
0,
|
||||
5000
|
||||
],
|
||||
"benign_tip": [
|
||||
0,
|
||||
5000
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": 437.7,
|
||||
"fixed": 426.2,
|
||||
"benign_base": 416.4,
|
||||
"benign_tip": 425.7
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "yeswiki-bazar-form-sqli",
|
||||
"family": "injection",
|
||||
"language": "php",
|
||||
"detected": false,
|
||||
"detected_sites": [],
|
||||
"unanswered": true,
|
||||
"instrument_errors": {},
|
||||
"fixed_side_alerts": [],
|
||||
"benign_alerts": [],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
0,
|
||||
1128
|
||||
],
|
||||
"fixed": [
|
||||
0,
|
||||
1128
|
||||
],
|
||||
"benign_base": [
|
||||
0,
|
||||
1128
|
||||
],
|
||||
"benign_tip": [
|
||||
0,
|
||||
1128
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": 334.9,
|
||||
"fixed": 306.9,
|
||||
"benign_base": 304.3,
|
||||
"benign_tip": 305.3
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "signalk-remote-conn-ssrf",
|
||||
"family": "untrusted_destination",
|
||||
"language": "typescript",
|
||||
"detected": true,
|
||||
"detected_sites": [
|
||||
"src/serverroutes.ts:2506:<lambda>1"
|
||||
],
|
||||
"unanswered": false,
|
||||
"instrument_errors": {},
|
||||
"fixed_side_alerts": [],
|
||||
"benign_alerts": [
|
||||
"src/serverroutes.ts:2577:<lambda>1"
|
||||
],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
0,
|
||||
525
|
||||
],
|
||||
"fixed": [
|
||||
0,
|
||||
529
|
||||
],
|
||||
"benign_base": [
|
||||
0,
|
||||
533
|
||||
],
|
||||
"benign_tip": [
|
||||
0,
|
||||
533
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": 313.3,
|
||||
"fixed": 319.5,
|
||||
"benign_base": 334.3,
|
||||
"benign_tip": 315.4
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "saltcorn-sync-sqli",
|
||||
"family": "injection",
|
||||
"language": "javascript",
|
||||
"detected": false,
|
||||
"detected_sites": [],
|
||||
"unanswered": true,
|
||||
"instrument_errors": {
|
||||
"vulnerable": "no question was asked",
|
||||
"fixed": "no question was asked",
|
||||
"benign_base": "no question was asked",
|
||||
"benign_tip": "no question was asked"
|
||||
},
|
||||
"fixed_side_alerts": [],
|
||||
"benign_alerts": [],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
null,
|
||||
0
|
||||
],
|
||||
"fixed": [
|
||||
null,
|
||||
0
|
||||
],
|
||||
"benign_base": [
|
||||
null,
|
||||
0
|
||||
],
|
||||
"benign_tip": [
|
||||
null,
|
||||
0
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": null,
|
||||
"fixed": null,
|
||||
"benign_base": null,
|
||||
"benign_tip": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "mongo-express-bson-rce",
|
||||
"family": "injection",
|
||||
"language": "javascript",
|
||||
"detected": false,
|
||||
"detected_sites": [],
|
||||
"unanswered": true,
|
||||
"instrument_errors": {},
|
||||
"fixed_side_alerts": [
|
||||
"lib/bson.js:32:toString",
|
||||
"lib/bson.js:8:toBSON"
|
||||
],
|
||||
"benign_alerts": [],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
0,
|
||||
40
|
||||
],
|
||||
"fixed": [
|
||||
0,
|
||||
39
|
||||
],
|
||||
"benign_base": [
|
||||
0,
|
||||
39
|
||||
],
|
||||
"benign_tip": [
|
||||
0,
|
||||
39
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": 47.9,
|
||||
"fixed": 48.5,
|
||||
"benign_base": 47.5,
|
||||
"benign_tip": 50.3
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "mockoon-admin-api-cors",
|
||||
"family": "config_secrets",
|
||||
"language": "typescript",
|
||||
"detected": false,
|
||||
"detected_sites": [],
|
||||
"unanswered": true,
|
||||
"instrument_errors": {},
|
||||
"fixed_side_alerts": [],
|
||||
"benign_alerts": [],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
0,
|
||||
450
|
||||
],
|
||||
"fixed": [
|
||||
0,
|
||||
453
|
||||
],
|
||||
"benign_base": [
|
||||
0,
|
||||
453
|
||||
],
|
||||
"benign_tip": [
|
||||
0,
|
||||
453
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": 150.9,
|
||||
"fixed": 158.5,
|
||||
"benign_base": 146.4,
|
||||
"benign_tip": 144.7
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "alerta-query-sqli",
|
||||
"family": "injection",
|
||||
"language": "python",
|
||||
"detected": false,
|
||||
"detected_sites": [],
|
||||
"unanswered": true,
|
||||
"instrument_errors": {},
|
||||
"fixed_side_alerts": [],
|
||||
"benign_alerts": [],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
0,
|
||||
261
|
||||
],
|
||||
"fixed": [
|
||||
0,
|
||||
261
|
||||
],
|
||||
"benign_base": [
|
||||
0,
|
||||
261
|
||||
],
|
||||
"benign_tip": [
|
||||
0,
|
||||
261
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": 114.8,
|
||||
"fixed": 111.7,
|
||||
"benign_base": 109.9,
|
||||
"benign_tip": 112.9
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "wger-next-open-redirect",
|
||||
"family": "untrusted_destination",
|
||||
"language": "python",
|
||||
"detected": false,
|
||||
"detected_sites": [],
|
||||
"unanswered": true,
|
||||
"instrument_errors": {},
|
||||
"fixed_side_alerts": [
|
||||
"wger/core/views/languages.py:131:use_browser_language",
|
||||
"wger/core/views/user.py:211:trainer_login"
|
||||
],
|
||||
"benign_alerts": [],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
0,
|
||||
683
|
||||
],
|
||||
"fixed": [
|
||||
0,
|
||||
683
|
||||
],
|
||||
"benign_base": [
|
||||
0,
|
||||
683
|
||||
],
|
||||
"benign_tip": [
|
||||
0,
|
||||
683
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": 156.5,
|
||||
"fixed": 160.0,
|
||||
"benign_base": 160.8,
|
||||
"benign_tip": 164.7
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "opencve-notification-ssrf",
|
||||
"family": "untrusted_destination",
|
||||
"language": "python",
|
||||
"detected": false,
|
||||
"detected_sites": [],
|
||||
"unanswered": false,
|
||||
"instrument_errors": {},
|
||||
"fixed_side_alerts": [],
|
||||
"benign_alerts": [],
|
||||
"completion": {
|
||||
"vulnerable": [
|
||||
14,
|
||||
27
|
||||
],
|
||||
"fixed": [
|
||||
13,
|
||||
26
|
||||
],
|
||||
"benign_base": [
|
||||
13,
|
||||
26
|
||||
],
|
||||
"benign_tip": [
|
||||
13,
|
||||
26
|
||||
]
|
||||
},
|
||||
"seconds": {
|
||||
"vulnerable": 129.7,
|
||||
"fixed": 132.4,
|
||||
"benign_base": 144.3,
|
||||
"benign_tip": 139.4
|
||||
}
|
||||
}
|
||||
],
|
||||
"precision_review": [
|
||||
{
|
||||
"case": "signalk-remote-conn-ssrf",
|
||||
"site": "src/index.ts:916:<lambda>6",
|
||||
"family": "untrusted_destination",
|
||||
"rung": "model_entailed",
|
||||
"witness": "req.headers.host?.split(':')[0] -> res.redirect(`https://${host}:${redirectPort}${req.path}`) in src/index.ts (line 916, 5 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "FP",
|
||||
"fp_class": "not_attacker",
|
||||
"reason": "HTTP->HTTPS redirect built from the request's own Host header, which a browser sets to the real target"
|
||||
},
|
||||
{
|
||||
"case": "signalk-remote-conn-ssrf",
|
||||
"site": "src/oidc/oidc-auth.ts:303:<lambda>2",
|
||||
"family": "untrusted_destination",
|
||||
"rung": "model_entailed",
|
||||
"witness": "req.headers[\"x-forwarded-proto\"] -> res.redirect(authUrl) in src/oidc/oidc-auth.ts (line 303, 11 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "FP",
|
||||
"fp_class": "guarded",
|
||||
"reason": "authorization URL from provider config; the request redirect is validated by isSafeRelativeUrl"
|
||||
},
|
||||
{
|
||||
"case": "signalk-remote-conn-ssrf",
|
||||
"site": "src/oidc/oidc-auth.ts:490:<lambda>6",
|
||||
"family": "untrusted_destination",
|
||||
"rung": "model_entailed",
|
||||
"witness": "req.query.redirect -> res.redirect(postLogoutRedirect) in src/oidc/oidc-auth.ts (line 490, 22 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "FP",
|
||||
"fp_class": "guarded",
|
||||
"reason": "postLogoutRedirect = isSafeRelativeUrl(req.query.redirect) ? ... : '/'"
|
||||
},
|
||||
{
|
||||
"case": "signalk-remote-conn-ssrf",
|
||||
"site": "src/oidc/oidc-auth.ts:501:<lambda>6",
|
||||
"family": "untrusted_destination",
|
||||
"rung": "model_entailed",
|
||||
"witness": "req.query.redirect -> res.redirect(postLogoutRedirect) in src/oidc/oidc-auth.ts (line 501, 22 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "FP",
|
||||
"fp_class": "guarded",
|
||||
"reason": "same isSafeRelativeUrl guard"
|
||||
},
|
||||
{
|
||||
"case": "signalk-remote-conn-ssrf",
|
||||
"site": "src/oidc/oidc-auth.ts:508:<lambda>6",
|
||||
"family": "untrusted_destination",
|
||||
"rung": "model_entailed",
|
||||
"witness": "req.query.redirect -> res.redirect(postLogoutRedirect) in src/oidc/oidc-auth.ts (line 508, 22 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "FP",
|
||||
"fp_class": "guarded",
|
||||
"reason": "same isSafeRelativeUrl guard"
|
||||
},
|
||||
{
|
||||
"case": "signalk-remote-conn-ssrf",
|
||||
"site": "src/serverroutes.ts:2506:<lambda>1",
|
||||
"family": "untrusted_destination",
|
||||
"rung": "model_entailed",
|
||||
"witness": "req.body -> protocol.request(options, (response) => {\n let data = ''\n response.on('data', (chunk: string) => {\n data += chunk\n })\n response.on('end', () => {\n resolve({ status: r in src/serverroutes.ts (line 2506, 16 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "TP",
|
||||
"fp_class": "",
|
||||
"reason": "the declared CVE-2026-55591 sink: req.body host/port to protocol.request, unauthenticated by default"
|
||||
},
|
||||
{
|
||||
"case": "signalk-remote-conn-ssrf",
|
||||
"site": "src/serverroutes.ts:457:<lambda>24",
|
||||
"family": "untrusted_destination",
|
||||
"rung": "model_entailed",
|
||||
"witness": "(_tmp_69 = (req.headers?.host || '')\n .split(':')[0]\n .split('.')[0]).toLowerCase -> res.redirect(app.config.settings.landingPage || landingPage) in src/serverroutes.ts (line 457, 26 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "FP",
|
||||
"fp_class": "not_attacker",
|
||||
"reason": "redirect target is the configured landing page; the Host header only selects nothing"
|
||||
},
|
||||
{
|
||||
"case": "signalk-remote-conn-ssrf",
|
||||
"site": "src/tokensecurity.ts:677:<lambda>5",
|
||||
"family": "untrusted_destination",
|
||||
"rung": "model_entailed",
|
||||
"witness": "req.body.destination -> res.redirect(getSafeDestination(req.body.destination)) in src/tokensecurity.ts (line 677, 12 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "FP",
|
||||
"fp_class": "guarded",
|
||||
"reason": "getSafeDestination allows only relative paths and rejects //"
|
||||
},
|
||||
{
|
||||
"case": "yeswiki-bazar-form-sqli",
|
||||
"site": "includes/entities/ConfigurationFile.php:69:load",
|
||||
"family": "injection",
|
||||
"rung": "model_entailed",
|
||||
"witness": "$this->_file -> eval($content) in includes/entities/ConfigurationFile.php (line 69, 5 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "FP",
|
||||
"fp_class": "not_attacker",
|
||||
"reason": "eval of the site's own configuration file"
|
||||
},
|
||||
{
|
||||
"case": "yeswiki-bazar-form-sqli",
|
||||
"site": "includes/services/DbService.php:46:initSqlConnection",
|
||||
"family": "injection",
|
||||
"rung": "model_entailed",
|
||||
"witness": "$this->params -> mysqli_query($this->link, \"SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci\") in includes/services/DbService.php (line 46, 9 steps)",
|
||||
"reached_from": 2,
|
||||
"verdict": "FP",
|
||||
"fp_class": "constant_query",
|
||||
"reason": "the SQL text is a constant SET NAMES; taint reaches only the connection handle"
|
||||
},
|
||||
{
|
||||
"case": "yeswiki-bazar-form-sqli",
|
||||
"site": "setup/install.php:113:<global>",
|
||||
"family": "injection",
|
||||
"rung": "model_entailed",
|
||||
"witness": "$_POST[\"config\"] -> mysqli_query($dblink, \"SET NAMES utf8mb4 COLLATE utf8mb4_general_ci\") in setup/install.php (line 113, 19 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "FP",
|
||||
"fp_class": "constant_query",
|
||||
"reason": "constant SET NAMES; taint reaches only the connection handle"
|
||||
},
|
||||
{
|
||||
"case": "yeswiki-bazar-form-sqli",
|
||||
"site": "setup/install.php:54:<global>",
|
||||
"family": "injection",
|
||||
"rung": "model_entailed",
|
||||
"witness": "$_POST[\"config\"] -> mysqli_query($dblink, \"CREATE DATABASE \" . $config[\"mysql_database\"]) in setup/install.php (line 54, 16 steps)",
|
||||
"reached_from": 1,
|
||||
"verdict": "FP",
|
||||
"fp_class": "no_boundary",
|
||||
"reason": "installer: whoever submits it must already supply the database password; borderline, counted false"
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user