feat(detection-core): measured per-class budget replacing the static 120s gate

This commit is contained in:
Marius Ciepluch
2026-10-30 16:11:37 +01:00
parent 0b56ad5514
commit 92fb372498
7 changed files with 240 additions and 4 deletions
@@ -0,0 +1,35 @@
# Measured budget derivation (detection-core task 2.2, 2026-10-30)
The static 120s gate was replaced by per-workload-class budgets derived from
measurement. This is measurement only: no detector, threshold, or prompt was fitted
on these numbers (pool measures, never trains).
## Result
| class | n | max | over static 120s | p90 | p95 | p99 |
|---|---|---|---|---|---|---|
| dev-pool (four-arm cacfon records) | 483 | 240.0s | 20 | 61.4s | **88.7s** | 210.0s |
| admission-pilot (264 records) | 264 | 7.7s | 0 | 2.6s | **3.2s** | 3.8s |
(derivation.json carries the exact numbers; p95 shown in bold as the recommended
class budget.)
## Reading
One static constant cannot serve both classes: 120s over-fails the dev pool (20/483
overruns, max 240s) while being ~37x the pilot's p95 (3.2s). Per-class measured budgets
fix both directions. The static 120s remains the provisional default only until a
consumer adopts the derived class budget; adopting it starts a new baseline under the
fixed measurement schema (Req 12.2).
## Method
Complexity features (regions = changed files, calls = resolution-bucket total,
changed_bytes = instrumented changed bytes) and seconds come from the existing
record sets; budget = linear-interpolated percentile of measured seconds per class,
with n, percentile, and hardware disclosed in `derivation.json`. Hardware:
local dev host, records of 2026-10-30.
Sources: `~/ousast-results/prover/ac-four-arm-1/cacfon/` (483),
`~/ousast-results/prover/ac-admission-1/{vulnerable,fixed,ordinary}/` (264).
Script: `/tmp/opencode/derive_budget_2.py`.
@@ -0,0 +1,46 @@
{
"derivations": {
"admission-pilot:p90": {
"budget_seconds": 2.608,
"hardware": "local dev host, 2026-10-30 measurement records",
"n": 264,
"percentile": 0.9
},
"admission-pilot:p95": {
"budget_seconds": 3.216,
"hardware": "local dev host, 2026-10-30 measurement records",
"n": 264,
"percentile": 0.95
},
"admission-pilot:p99": {
"budget_seconds": 3.82,
"hardware": "local dev host, 2026-10-30 measurement records",
"n": 264,
"percentile": 0.99
},
"dev-pool:p90": {
"budget_seconds": 61.352,
"hardware": "local dev host, 2026-10-30 measurement records",
"n": 483,
"percentile": 0.9
},
"dev-pool:p95": {
"budget_seconds": 88.693,
"hardware": "local dev host, 2026-10-30 measurement records",
"n": 483,
"percentile": 0.95
},
"dev-pool:p99": {
"budget_seconds": 209.997,
"hardware": "local dev host, 2026-10-30 measurement records",
"n": 483,
"percentile": 0.99
}
},
"note": "measurement only; no detector trained on these numbers",
"sources": {
"admission-pilot": "~/ousast-results/prover/ac-admission-1/{vulnerable,fixed,ordinary} (264 records, 2026-10-30 admission)",
"dev-pool": "~/ousast-results/prover/ac-four-arm-1/cacfon (483 records, 2026-10-30 four-arm)"
},
"static_gate_for_comparison_seconds": 120.0
}
+7 -1
View File
@@ -165,13 +165,19 @@ def _prove_one(
_instrument_reads(ctx, root, change, instrument)
result = prove_change(root, change, ctx, judgment_mode=True)
instrument["hook_bytes_read"] = sum(instrument["file_bytes"].values())
funnel = stage_funnel(result)
record.update(
findings=list(result.findings),
traces=list(result.traces),
coverage=asdict(result.coverage),
degradations=list(result.degradations),
stage_funnel=stage_funnel(result),
stage_funnel=funnel,
)
record["complexity"] = {
"regions": len(change.get("files", ())),
"calls": sum(funnel.get("resolution", {}).values()),
"changed_bytes": sum(instrument.get("changed_bytes", {}).values()),
}
record["push"]["admission"]["defects"] = [
{**finding, "defect_id": finding["id"], "source": "prover", "locations": [[finding["path"], finding["line"]]]}
for finding in result.findings