Files
Open_Ultra_SAST/benchmarks/measurements/2026-09-30-php-engine-alerts/froxlor-admin-ip-second-order-sqli-alerts-summary.json
T
Marius CiepluchandClaude Opus 5.5 45d02ecf20 feat(plane): honest coverage in alerts; PHP alerts from the engine on the host
Quick mode has rules only for c/cpp, groovy, java, javascript/typescript and
python, so every PHP case's "0 alerts" read as clean. The `alerts` task now
records per language of the files read whether quick mode covers it
(derived from the bundled ruleset, not a list): coverage quick | none, with
file counts per pin, and `uncovered`. `remember` turns that into `coverage`
memory rows (a new kind; the Run passes the alerts summary.json as input),
loop-measure reports coverage per language, and improve/memory.py's M1/M2
skip a repository+pin whose coverage for the rule's language is only
`none` (counted under `coverage_none` in the proposal's excluded).

`ousast plane alerts-engine <Run.yaml> --source <frozen export>`
(plane/engine_alerts.py) does the alerts task on the host for cases whose
files include a language the engine covers but quick mode does not (PHP):
exports both pins from the case cache, runs the quick scan and the
openultrasast:dev engine container (finding_dump.py, --network none,
--memory 3g, one at a time, per-pin deadline), writes engine findings in
the alerts.jsonl row shape (rule_id engine:<family>, source engine,
in_fix_range from case.json) with a summary (files read, questions,
completed, seconds, degradations per pin), and marks the task done via
reconciler.mark_done. No result, no files read, or questions == 0 fails
loudly and leaves the run untouched. Refuses a git checkout as source.

Runner image rebuilt and re-pinned (sha256:ebb40a9a...); validation-46
regenerated (digest only). Live, recorded in
benchmarks/measurements/2026-09-30-php-engine-alerts/: froxlor 366 files,
130/130 questions per pin, 6 alerts per pin, 0 in the fix range; wp-gopay
19 files, 101/101 questions, 0 alerts. The loop: 32 run rows (coverage 8,
php=engine), 0 proposals, gate no_proposals, $0.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 13:44:32 +02:00

97 lines
1.7 KiB
JSON

{
"alerts": {
"fixed": 6,
"vulnerable": 6
},
"alerts_engine": {
"fixed": 6,
"vulnerable": 6
},
"calls": 0,
"coverage": {
"javascript": {
"coverage": "quick",
"files": {
"fixed": 18,
"vulnerable": 18
}
},
"php": {
"coverage": "engine",
"files": {
"fixed": 348,
"vulnerable": 348
}
}
},
"deadline": 1800.0,
"engine": {
"fixed": {
"bytes": 4256572,
"completed": 130,
"degradations": [
"cross_partition_semantics_unresolved"
],
"exit": 0,
"files": 366,
"findings": 6,
"questions": 130,
"seconds": 383.2
},
"vulnerable": {
"bytes": 4256094,
"completed": 130,
"degradations": [
"cross_partition_semantics_unresolved"
],
"exit": 0,
"files": 366,
"findings": 6,
"questions": 130,
"seconds": 398.2
}
},
"engine_languages": [
"php"
],
"family": "injection",
"host": true,
"image": "sha256:e6f8dc6cc7547f7118bd058687de676e0fcf741799d1d3839b3e8648ce190fbb",
"in_fix_range": {
"fixed": 0,
"vulnerable": 0
},
"model": null,
"pins": {
"fixed": "a1eaca5a1601c8a30e00814a4fc73ad0c185f89e",
"vulnerable": "9bf03de643a2edee4642bfc6c94db1f05253da02"
},
"quick_languages": [
"c",
"cpp",
"groovy",
"java",
"javascript",
"python",
"typescript"
],
"read": {
"fixed": {
"bytes": 4256572,
"files": 366
},
"vulnerable": {
"bytes": 4256094,
"files": 366
}
},
"seconds": 789.1,
"source": "engine",
"status": "done",
"uncovered": [],
"units_done": 2,
"units_total": 2,
"usage": {},
"usd": 0
}