mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 20:49:28 +00:00
Marius CiepluchandClaude Opus 5
eaf9e3e370
Requirements approved in session. All 36 criteria trace to a component. The shape: a deterministic pass enumerates candidate sites from the semantic IR, shaped by family -- call sites for injection and path, operations with their guards for access control, bindings and literals for configuration. The model answers one bounded typed question per candidate with no tools and no choice of what to look at. The predicates that decide whether a finding counts are applied while it is being made, with one retry carrying the violation, and they come from a single definition that the scorer also uses, so the two cannot drift. Determinism is a design property, not a hope: candidates are ordered by (path, line, name) and batched at a fixed size, so two runs make an identical sequence of model calls. That is the direct answer to 120 runs producing 120 distinct trajectories. Phase 1 is the candidate ceiling, offline, no model call, measured on the real-project slices as well as the teaching one. If the ceiling on a deciding slice comes in below that slice's current detection rate, the design stops there having spent nothing. Building that in as the first phase is deliberate: the ruleset variant of this design would have capped recall at 12.1% against a 64.9% baseline, and only a measurement caught it. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
27 lines
1.4 KiB
JSON
27 lines
1.4 KiB
JSON
{
|
|
"feature_name": "constrained-detector",
|
|
"created_at": "2026-09-06T14:30:00Z",
|
|
"updated_at": "2026-09-06T16:30:00Z",
|
|
"language": "en",
|
|
"phase": "design",
|
|
"approvals": {
|
|
"requirements": {
|
|
"generated": true,
|
|
"approved": true
|
|
},
|
|
"design": {
|
|
"generated": true,
|
|
"approved": false
|
|
},
|
|
"tasks": {
|
|
"generated": false,
|
|
"approved": false
|
|
}
|
|
},
|
|
"ready_for_implementation": false,
|
|
"depends_on": [
|
|
"learning-harness"
|
|
],
|
|
"approval_note": "2026-09-06 brief drafted at the maintainer's direction ('I'd rather spend time to design the exact DSPy style interface and evolutionary / statistical prompt optimization than fixing piece by piece'). Not approved; requirements not generated.; 2026-09-06 renamed from detector-optimizer to constrained-detector after the trajectory measurement: the brief's own conclusion is that the optimiser is not the next thing to build, and the spec should be named for what it delivers; 2026-09-06 requirements generated at the maintainer's direction, with the maintainer's emphasis recorded as Requirement 3 ('don't forget the emphasis on the real world bugs from the larger projects to avoid overfitting. we otherwise benchmark for cases that are not relevant'); 2026-09-06 requirements approved by the maintainer in session ('approve the requirements and run the design phase')"
|
|
}
|