mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-19 21:19:28 +00:00
docs: plain-English pass over the active specs
Ran the humanizer rules over 79 prose files in the active specs (brief, design, requirements, research and narrative docs). Removed em dashes, significance inflation, forced rule-of-three, bold-label list items and filler. Skipped tasks.md checklists and the retired specs (harnessx-*, openrouter-sast-harness, unseen-repo-evaluation). Every number, path, command, hash and identifier was preserved: checked each file against HEAD for drift in numbers, inline code and code fences, zero drift across all 79 files. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
b08f372022
commit
39a1873e5b
@@ -69,7 +69,7 @@ family semantics remain with their existing specs rather than being reimplemente
|
||||
- No promise of all-language coverage; capabilities are admitted per language/framework/family.
|
||||
- Initial work must establish feasibility and noise before investing in an LLM ranker.
|
||||
|
||||
## Cross-language amendment — 2026-09-12
|
||||
## Cross-language amendment, 2026-09-12
|
||||
|
||||
Keep PMPro as the primary worked optimization example, and require a JavaScript/Express/Node API
|
||||
transfer experiment before claiming the core improvement generalizes. Abstract evidence, ranking,
|
||||
|
||||
@@ -24,7 +24,7 @@ an irrelevant old defect, or an unactionable witness can still produce noise.
|
||||
- Do not build a second ranker/scope heuristic, an incremental CPG engine, a daemon, an automatic fixer,
|
||||
or a mandatory model service. Do not scan intermediate pushed history in this feature.
|
||||
|
||||
## What this hook is, and is not — 2026-09-21
|
||||
## What this hook is, and is not, 2026-09-21
|
||||
|
||||
A general-purpose pre-commit hook, not an application-specific one. It may be measured on WordPress
|
||||
plugins and Node applications, and it may not become a tool that only works on them. The practical
|
||||
@@ -63,8 +63,8 @@ The fourth and fifth cost the declared v0.1 target its entire admissibility: a W
|
||||
PHP plus admin JavaScript, so the product could detect defects on its own target and never admit
|
||||
one. Neither passing subject could reveal it, because both hold one language.
|
||||
|
||||
**In review:** every new degradation reason must state the scope it reaches — one question, one file,
|
||||
one partition, or the whole read — and a reason that demotes an outcome must justify reaching that
|
||||
**In review:** every new degradation reason must state the scope it reaches (one question, one file,
|
||||
one partition, or the whole read), and a reason that demotes an outcome must justify reaching that
|
||||
far. A gap that costs coverage without threatening correctness is a declared exclusion: it stays
|
||||
published and it does not demote.
|
||||
|
||||
@@ -462,7 +462,7 @@ decision in the terminal; do not fabricate a successful cache entry or hang whil
|
||||
separate from a future C/C++ positive control. See `evaluation.md` for selection and labeling gates.
|
||||
|
||||
Measure the accepted experimental acceptance profile in requirements on declared hardware and fixed workloads.
|
||||
Warm means prior reusable work from the preceding state, not only repeating the identical tip.
|
||||
Warm means prior reusable work from the preceding state. Repeating the identical tip alone is insufficient.
|
||||
Report identical-tip hits separately from a one-function change and a dependency/configuration
|
||||
change. Include first installs, cold graphs, growing repository size, multi-ref pushes and incomplete
|
||||
outcomes. Freeze the applicable population before ranker selection; deferred questions count against
|
||||
|
||||
@@ -68,7 +68,7 @@ real capability; task 8.3 still owns actual eligibility. Task 4.2 brings progres
|
||||
|
||||
The agent service rejected the task 4.3 dispatch with `agent thread limit reached`.
|
||||
The [kiro-impl skill](../../../.agents/skills/kiro-impl/SKILL.md) explicitly provides:
|
||||
“If multi-agent is not available, fall back to manual mode execution for all tasks.”
|
||||
"If multi-agent is not available, fall back to manual mode execution for all tasks."
|
||||
Implementation, review and completion checks therefore ran in the main context.
|
||||
This is a manual review record, not an independent-review claim.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Implementation verification: group 5
|
||||
|
||||
Group 5 was authorized by the maintainer's “approved” after group 4 completion.
|
||||
Group 5 was authorized by the maintainer's "approved" after group 4 completion.
|
||||
Tasks 5.1–5.2 are implemented and verified (16/27 total); the measured feasibility decision is NO-GO.
|
||||
Real capability eligibility and rollout readiness remain unproven.
|
||||
|
||||
@@ -8,7 +8,7 @@ Real capability eligibility and rollout readiness remain unproven.
|
||||
|
||||
Fresh implementer dispatch for 5.1 returned `agent thread limit reached`.
|
||||
The [Kiro implementation skill](../../../.agents/skills/kiro-impl/SKILL.md) explicitly
|
||||
states: “If multi-agent is not available, fall back to manual mode execution for all tasks.”
|
||||
states: "If multi-agent is not available, fall back to manual mode execution for all tasks."
|
||||
Its manual review rule permits review in the main context when a fresh reviewer is unavailable.
|
||||
This run therefore uses manual implementation and adversarial review, not independent review.
|
||||
|
||||
|
||||
@@ -300,15 +300,15 @@ and its [bundle](../../../benchmarks/measurements/2026-09-16-nodegoat-m1b-explai
|
||||
Three evaluation findings were rendered on the vulnerable revision, one per eval site. For
|
||||
line 32:
|
||||
|
||||
- **Location** app/routes/contributions.js:32, family injection, novelty new with
|
||||
- Location: app/routes/contributions.js:32, family injection, novelty new with
|
||||
`operation_absent_from_comparable_base`.
|
||||
- **Witness** `eval(req.body.preTax) -> const preTax = eval(req.body.preTax)`.
|
||||
- **Change attribution** `base:app/routes/contributions.js:32-34`.
|
||||
- **Provenance** `file_scope`: the question named no function, the engine reported `<lambda>2`.
|
||||
- **Consequence** "A request field reaches const preTax = eval(req.body.preTax), where it is
|
||||
- Witness: `eval(req.body.preTax) -> const preTax = eval(req.body.preTax)`.
|
||||
- Change attribution: `base:app/routes/contributions.js:32-34`.
|
||||
- Provenance: `file_scope`: the question named no function, the engine reported `<lambda>2`.
|
||||
- Consequence: "A request field reaches const preTax = eval(req.body.preTax), where it is
|
||||
evaluated as JavaScript, so an attacker controlling that field can execute arbitrary code in
|
||||
the server process."
|
||||
- **Repair** "Replace the evaluation at const preTax = eval(req.body.preTax) with a numeric
|
||||
- Repair: "Replace the evaluation at const preTax = eval(req.body.preTax) with a numeric
|
||||
conversion such as Number(...) or parseInt(...), then validate the result before use."
|
||||
|
||||
Both sentences come from the declaration's templates through the grounded-template path, not from
|
||||
|
||||
@@ -64,9 +64,9 @@ code the precision estimate will move, against a bar of 95% with zero false aler
|
||||
benign sides. Expect the first M4 evaluation to fail and budget for a second. That is the strongest
|
||||
argument for decision 2.
|
||||
|
||||
## M1 — One real security change works end to end
|
||||
## M1: One real security change works end to end
|
||||
|
||||
Status: **NEXT — not verified**. Dependencies: existing immutable replay and scan machinery.
|
||||
Status: **NEXT, not verified**. Dependencies: existing immutable replay and scan machinery.
|
||||
M1 is split into M1a (diagnose with vetoes recorded, not applied) and M1b (repair only
|
||||
the vetoes M1a showed to matter). M1 is verified only when M1b passes.
|
||||
|
||||
@@ -80,7 +80,7 @@ a global completion veto, line ambiguity inherited by every question, and query
|
||||
differing from witness identity. Requiring all four repaired before the first visible
|
||||
result is a full production pass, not one detection.
|
||||
|
||||
### M1a — Show the detection with every veto recorded
|
||||
### M1a: Show the detection with every veto recorded
|
||||
|
||||
Status: **MET 2026-09-16**, inside its one-week box. Evidence:
|
||||
[M1a record](m1a-recorded-vetoes.md) and
|
||||
@@ -112,9 +112,9 @@ source and census receipts. **Kill criterion:** if the fixed twin is not quiet u
|
||||
recorded-veto run, M1 has a detector problem, and M1b does not start until that is
|
||||
diagnosed and recorded.
|
||||
|
||||
### M1b — Repair only the vetoes M1a showed to matter
|
||||
### M1b: Repair only the vetoes M1a showed to matter
|
||||
|
||||
Status: **NEXT — not verified**. M1a supplied the veto list: per-family change context,
|
||||
Status: **NEXT, not verified**. M1a supplied the veto list: per-family change context,
|
||||
question-owned vendor relevance, operation correspondence for an operation on a changed line,
|
||||
file-wide correspondence ambiguity scope, and query-to-operation provenance. The owner
|
||||
diagnosis expected the first and the last two; operation correspondence is the addition M1a
|
||||
@@ -144,7 +144,7 @@ Owners: contributor-scan for query/operation/context evidence; pre-push-safety-n
|
||||
comparison, admission, the recorded-veto flag and reporting. Requirements: 1.1–1.4,
|
||||
2.1–2.4, 3.1–3.5, 6.3, 7.1–7.3.
|
||||
|
||||
## M2 — The same useful analysis fits the push budget
|
||||
## M2: The same useful analysis fits the push budget
|
||||
|
||||
Status: **MEASURED AND UNMET, 2026-09-17.** Two attacks on engine startup were measured and
|
||||
neither reaches the gate. A transaction-owned session is implemented and proven equivalent for
|
||||
@@ -179,7 +179,7 @@ completion at least 95%, under the predeclared profile. A fast census alone does
|
||||
Owner: contributor-scan runtime, with pre-push-safety-net deadline/cache integration.
|
||||
Requirements: 4.1–4.4, 6.2–6.5, 7.3.
|
||||
|
||||
## M3 — The shared design transfers across languages
|
||||
## M3: The shared design transfers across languages
|
||||
|
||||
Status: **PARTIAL, 2026-09-17. Not met.** Python is established and PHP is blocked by runtime.
|
||||
|
||||
@@ -218,7 +218,7 @@ remain a separate unsupported abstraction; adding a new engine is not a prerequi
|
||||
Owners: contributor-scan shared evidence and framework adapters; pre-push-safety-net evaluation.
|
||||
Requirements: 6.2–6.4, 8.1–8.4.
|
||||
|
||||
## M4 — Independent examples establish useful detection and low noise
|
||||
## M4: Independent examples establish useful detection and low noise
|
||||
|
||||
Status: **PENDING M3**. Population preparation may proceed earlier, but evaluation follows
|
||||
the frozen candidate. Reserve untouched cases before examining their scan results.
|
||||
@@ -241,7 +241,7 @@ do not require every possible language or family to become eligible for a narrow
|
||||
Owner: pre-push-safety-net task 8.3, consuming verified contributor-scan semantics.
|
||||
Requirements: 6.1–6.5, 8.1–8.4 and the accepted experimental acceptance profile.
|
||||
|
||||
## M5 — Enable a narrow advisory rollout
|
||||
## M5: Enable a narrow advisory rollout
|
||||
|
||||
Status: **PENDING M4**.
|
||||
|
||||
|
||||
@@ -196,6 +196,6 @@ failures without lowering the bar after inspecting holdout results.
|
||||
- >= 95% observed actionable-alert precision with counts and confidence interval; zero known false
|
||||
alerts on the fixed/benign regression suite. Small samples do not establish broad precision.
|
||||
- >= 90% recall on the predeclared supported security-change suite; unsupported classes remain in
|
||||
coverage reporting, not reclassified after a miss. Publish per-capability results, not only an aggregate.
|
||||
coverage reporting, not reclassified after a miss. Publish per-capability results; an aggregate alone is insufficient.
|
||||
- No capability is admitted with zero positive controls, zero reviewed emitted alerts, or no untouched
|
||||
evaluation repository. Failed profiles remain experimental and silent in the normal hook.
|
||||
|
||||
@@ -89,7 +89,7 @@ feasibility gate. [Official CPG slicing documentation](https://docs.joern.io/cpg
|
||||
2. **Separate claim from interruption.** Existing rungs describe analysis evidence. Hook admission
|
||||
additionally requires evaluated capability, change attribution and actionability. This meets the
|
||||
user's low-noise requirement without pretending an LLM confidence score proves a bug.
|
||||
3. **Compare evidence, not just finding lists.** A missing base finding can mean truncated analysis.
|
||||
3. **Compare evidence as well as finding lists.** A missing base finding can mean truncated analysis.
|
||||
Targeted comparable base checks are required for novelty, including removed-guard cases.
|
||||
4. **Adopt Git and the existing engine.** Build only the push transaction, cache and admission adapters.
|
||||
Do not add an IR, database or service merely to describe future scale.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Pre-push safety net status — v1.2.0-alpha.1
|
||||
# Pre-push safety net status, v1.2.0-alpha.1
|
||||
|
||||
Updated 2026-09-16. Phase: **implementation**. Requirements, design and tasks are
|
||||
approved. **28/29 executable tasks complete; task 8.3 remains blocked.**
|
||||
@@ -141,7 +141,7 @@ Four of seven declared families establish findings on the two development subjec
|
||||
injection, access_control, config_secrets and untrusted_destination. Output encoding, path,
|
||||
prototype and deserialization have established nothing anywhere. Every question completed on
|
||||
both subjects, 299 of 299 on NodeGoat and 35 of 35 on VAmPI, so the silences are answers
|
||||
rather than failures — but three of them are diagnosed gaps rather than clean code:
|
||||
rather than failures, but three of them are diagnosed gaps rather than clean code:
|
||||
NodeGoat's documented SSRF is invisible because `needle.get` is not in the sink vocabulary,
|
||||
its XSS is invisible because output encoding declares only DOM sinks, and its two
|
||||
access-control defects raise no obligation because the modelled data operations live in DAO
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
# Packaged acceptance validation — group 8
|
||||
# Packaged acceptance validation, group 8
|
||||
|
||||
## Validation Report
|
||||
- DECISION: NO-GO
|
||||
- FEATURE_COMPLETE: false
|
||||
- BLOCKED_TASKS: 8.3 — independent reviewed populations and verified VAmPI transitive evidence are missing; useful completion and runtime gates fail.
|
||||
- BLOCKED_TASKS: 8.3, independent reviewed populations and verified VAmPI transitive evidence are missing; useful completion and runtime gates fail.
|
||||
- OWNERSHIP: UPSTREAM for engine/census/context prerequisites; LOCAL for evaluation population and capability qualification.
|
||||
- UPSTREAM_SPEC: contributor-scan; see `pre-push-runtime-followup.md` there.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user