mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
Marius CiepluchandClaude Opus 5.5
af81bccc70
ruleset/frameworks.toml lists 19 framework and library ids (wordpress, flask, django, express,
spring, jakarta-servlet, react, vue and 11 libraries), each with its languages, dependency
package names and a symbol lexicon used only by the lint and for stratification.
Every framework/library entry of section 2 is tagged `framework`/`library`, and mixed entries
are split so a tag covers a whole entry (split pieces keep the entry's id and flags, in place):
semantic facts 38 tagged entries (wordpress 6, django 6, flask 4, express 4, jakarta-servlet 3,
pyyaml 2, one each for spring, sqlalchemy, requests, httpx, lodash, node-http-clients,
escape-html, sanitize-html, dompurify, esapi, commons-text, react, vue); the dispatch fact
behind the hook-as-source path is wordpress; the PHP layout row is renamed php_layout and stays
untagged (a product filter). Quick rules: 9 tagged (wordpress 6, flask, pyyaml,
node-http-clients); php-unserialize is split into php-unserialize and php-maybe-unserialize;
the lint also caught WordPress escapers in exclusions and axios/yaml.load in mixed rules, so
those keep their ids as tagged rules and gain a `fallback_for` language-level rule (6) that is
active only when the prior is off.
load_facts and load_ruleset take priors ("all" by default -- today's scan --, "off", or a set
of ids). tests/test_semantic_priors.py: the lint (no untagged entry contains a lexicon symbol
of its language), "off" removes exactly the tagged entries and activates the fallbacks, split
entries keep their language-level calls. Rule and layout counts in three existing tests follow the split and the rename.
Unchanged defaults: test_benchmark_output_is_byte_identical_to_golden_baseline passes, and the
quick findings of all 15 benchmark manifests (full StaticFinding records) are byte-identical
before and after. priors=off on the same manifests: 61 -> 57 findings; the 4 lost are js-ssrf
(axios), php-wpdb-sql-composition, python-ssti and python yaml.load; 5 PHP and 1 Python finding
move to their fallback rule at the same location. Semantic calls with priors off: php 75/92,
python 32/77, javascript 42/70, java 24/41, c 27/27 (the engine-side effect needs Joern and was
not measured).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
3.4 KiB
3.4 KiB
Implementation Plan
Order follows the design's commit sequence. Every task is a commit gated on the full suite's own exit code,
ruff check, ruff format --check and mypy. No reported number changes until task 9 adopts a model; without an
adopted model every path is today's. Tasks 5, 7 and 10 spend model money or host hours: each starts only after
the maintainer confirms its budget.
-
1. Schema and features
learn/schema.py(closed, typed allow-list; no identities; missing != zero),learn/features.py(host builder), new memoryKINDS,plane/tasks/features.pyand therememberinput; tests enforcing the allow-list and the exclusion of label-carrying fields.- Requirements: 1.1, 1.2, 1.3, 7.2
-
2. Framework tags
ruleset/frameworks.toml; tag and split every framework/library entry listed in design section 2;priors=in the semantic and quick loaders (defaultall, so today's scan is unchanged -- the golden benchmark test stays byte-identical); a lint test that no untagged framework entry remains.- Requirements: 8.2, 8.4
-
3. Role inference
learn/roles.py(deterministic wrapper inference),plane/tasks/roles.py(model roles, themodel_sinks.pyapproach), the vocabulary overlay for the engine; fixture tests with no model calls.- Requirements: 8.1
-
4. Labels
learn/sources.toml(fail-closed allowed sources, v3 excluded unread),learn/labels.py,ousast learn labels, repository-group normalisation and CVE merge, theassumed_benignsource mined from non-security commits (separate, never merged); a first label snapshot committed as counts only.- Requirements: 2.1, 2.2, 2.3, 7.1
-
5. Harvest (needs budget go-ahead)
- A plane Run of verify a/b on the verify-family pairs and development cases (~$6, ceiling $10), model roles on the candidate files (ceiling $10), engine features for the pair corpus on the host (frozen source, one container at a time, background).
- Requirements: 1.1
-
6. Train, calibrate, evaluate
learn/{folds,train,calibrate,evaluate}.py(repository-grouped, leave-one-source-out, leave-one-framework-out, nested tuning), the model blob API inplane/memory.py,ousast learn train|evaluate|curve, the first evaluation report with priors off underbenchmarks/measurements/<date>-decision-engine-v1/.- Requirements: 3.1-3.5, 5.1, 5.2, 7.3, 7.4, 8.3
-
7. Experiments (exp-001 needs budget go-ahead)
learn/experiments.py,register,experiment-outcome, Run annotation checks,OUSAST_VERIFY_CALLERS; exp-001 (known-callers line on/off, ~$7.7 expected, ceiling $15) registered, run and recorded -- an inconclusive result is pre-registered as likely.- Requirements: 4.1-4.4
-
8. Integration
- The
decidetask, thescandecision stage anddropped_by_engine.json,pre-pushadmission with the engine (structural admission reasons unchanged), report fields, the fallback line, the opt-in adaptation layer. - Requirements: 6.1-6.3, 7.3, 7.5
- The
-
9. Adopt
ousast learn adopt, packaged models for the families that pass, the priors-on/off experiment per framework with a recorded decision.- Requirements: 3.2, 8.2
-
10. The one-time v3 check
- Commit
prediction-v3.jsonfirst; then v3 is run under protocol v3 andresult-v3-engine.jsonis committed. Nothing between the two commits may touch the engine, features, priors or thresholds. - Requirements: 5.3
- Commit