marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-10-16 03:49:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
d1896cb8e73be258f9ec706a84d55fc8aea18c98
Open_Ultra_SAST/.kiro/specs/learned-decision-engine/tasks.md
T
Marius CiepluchandClaude Opus 5.5 d1896cb8e7 feat(learn): candidate feature schema and builders (decision-engine task 1) 
learn/schema.py is the closed, typed allow-list of candidate features: 66 features over
11 instruments (language one-hot, quick rules with 23 mechanism buckets and a prior count,
engine, repo-facts, source span, entry-point distance, roles, delta, verify, agree, model
sinks), two profiles (static 57; plane adds verify, agree and model sinks for 66), schema_version and a feature-set digest.
No repository, path, file, function, rule or commit identity is a feature; site_match,
in_fix_range, fixed-pin alerts, case.json fields and costs are refused. Missing is not zero:
each record carries per-instrument state (ran | none | failed | not_applicable), and every
feature of an instrument that did not run is null.

learn/features.py builds records on the host (build_for_scan, whole repository or base..head
delta); plane/tasks/features.py builds them per case in a Run from the delivered artifacts,
dropping site_match and in_fix_range on read and ignoring fixed-pin alerts. The generator
adds a model-free `features` entry per case before `remember`, which stores the records as
`features` rows (re-validated). memory KINDS gains features, label, decision, experiment,
arm_outcome and experiment_result. The committed validation-46 manifests are regenerated
(entries appended; the first Run's entries are unchanged). The runner image must be rebuilt
and re-pinned before a Run executes the new task.

Proof on real artifacts (scratch store, not committed): the recorded validation-46 run gives
75 feature records over 15 cases (43 verify candidates plus vulnerable-pin alerts), each
case's vulnerable pin exported and read (0.5k-9k files); remember stored 75 features rows
beside 43 verdict, 84 alert, 61 unit_cost, 15 facts and 8 coverage rows.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-30 18:38:04 +02:00

3.4 KiB
Raw Blame History

Implementation Plan

Order follows the design's commit sequence. Every task is a commit gated on the full suite's own exit code, ruff check, ruff format --check and mypy. No reported number changes until task 9 adopts a model; without an adopted model every path is today's. Tasks 5, 7 and 10 spend model money or host hours: each starts only after the maintainer confirms its budget.

  • 1. Schema and features

    • learn/schema.py (closed, typed allow-list; no identities; missing != zero), learn/features.py (host builder), new memory KINDS, plane/tasks/features.py and the remember input; tests enforcing the allow-list and the exclusion of label-carrying fields.
    • Requirements: 1.1, 1.2, 1.3, 7.2

  • 2. Framework tags

    • ruleset/frameworks.toml; tag and split every framework/library entry listed in design section 2; priors= in the semantic and quick loaders (default all, so today's scan is unchanged -- the golden benchmark test stays byte-identical); a lint test that no untagged framework entry remains.
    • Requirements: 8.2, 8.4

  • 3. Role inference

    • learn/roles.py (deterministic wrapper inference), plane/tasks/roles.py (model roles, the model_sinks.py approach), the vocabulary overlay for the engine; fixture tests with no model calls.
    • Requirements: 8.1

  • 4. Labels

    • learn/sources.toml (fail-closed allowed sources, v3 excluded unread), learn/labels.py, ousast learn labels, repository-group normalisation and CVE merge, the assumed_benign source mined from non-security commits (separate, never merged); a first label snapshot committed as counts only.
    • Requirements: 2.1, 2.2, 2.3, 7.1

  • 5. Harvest (needs budget go-ahead)

    • A plane Run of verify a/b on the verify-family pairs and development cases (~$6, ceiling $10), model roles on the candidate files (ceiling $10), engine features for the pair corpus on the host (frozen source, one container at a time, background).
    • Requirements: 1.1

  • 6. Train, calibrate, evaluate

    • learn/{folds,train,calibrate,evaluate}.py (repository-grouped, leave-one-source-out, leave-one-framework-out, nested tuning), the model blob API in plane/memory.py, ousast learn train|evaluate|curve, the first evaluation report with priors off under benchmarks/measurements/<date>-decision-engine-v1/.
    • Requirements: 3.1-3.5, 5.1, 5.2, 7.3, 7.4, 8.3

  • 7. Experiments (exp-001 needs budget go-ahead)

    • learn/experiments.py, register, experiment-outcome, Run annotation checks, OUSAST_VERIFY_CALLERS; exp-001 (known-callers line on/off, ~$7.7 expected, ceiling $15) registered, run and recorded -- an inconclusive result is pre-registered as likely.
    • Requirements: 4.1-4.4

  • 8. Integration

    • The decide task, the scan decision stage and dropped_by_engine.json, pre-push admission with the engine (structural admission reasons unchanged), report fields, the fallback line, the opt-in adaptation layer.
    • Requirements: 6.1-6.3, 7.3, 7.5

  • 9. Adopt

    • ousast learn adopt, packaged models for the families that pass, the priors-on/off experiment per framework with a recorded decision.
    • Requirements: 3.2, 8.2

  • 10. The one-time v3 check

    • Commit prediction-v3.json first; then v3 is run under protocol v3 and result-v3-engine.json is committed. Nothing between the two commits may touch the engine, features, priors or thresholds.
    • Requirements: 5.3
Reference in New Issue View Git Blame Copy Permalink