mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-18 12:39:28 +00:00
feat(learn): candidate feature schema and builders (decision-engine task 1)
learn/schema.py is the closed, typed allow-list of candidate features: 66 features over 11 instruments (language one-hot, quick rules with 23 mechanism buckets and a prior count, engine, repo-facts, source span, entry-point distance, roles, delta, verify, agree, model sinks), two profiles (static 57; plane adds verify, agree and model sinks for 66), schema_version and a feature-set digest. No repository, path, file, function, rule or commit identity is a feature; site_match, in_fix_range, fixed-pin alerts, case.json fields and costs are refused. Missing is not zero: each record carries per-instrument state (ran | none | failed | not_applicable), and every feature of an instrument that did not run is null. learn/features.py builds records on the host (build_for_scan, whole repository or base..head delta); plane/tasks/features.py builds them per case in a Run from the delivered artifacts, dropping site_match and in_fix_range on read and ignoring fixed-pin alerts. The generator adds a model-free `features` entry per case before `remember`, which stores the records as `features` rows (re-validated). memory KINDS gains features, label, decision, experiment, arm_outcome and experiment_result. The committed validation-46 manifests are regenerated (entries appended; the first Run's entries are unchanged). The runner image must be rebuilt and re-pinned before a Run executes the new task. Proof on real artifacts (scratch store, not committed): the recorded validation-46 run gives 75 feature records over 15 cases (43 verify candidates plus vulnerable-pin alerts), each case's vulnerable pin exported and read (0.5k-9k files); remember stored 75 features rows beside 43 verdict, 84 alert, 61 unit_cost, 15 facts and 8 coverage rows. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
edcf145b00
commit
d1896cb8e7
@@ -1,8 +1,9 @@
|
||||
"""The plane's memory store (harnessx-removal Requirement 6.1/6.4, design section 4).
|
||||
|
||||
One store, keyed by repository and pin, holds what plane runs learned: repository facts by content hash, and rows
|
||||
of six kinds (``facts``, ``verdict``, ``unit_cost``, ``alert``, ``coverage``, ``proposal_outcome``) that ``remember``
|
||||
(:mod:`.tasks.remember`) derives from a run's delivered artifacts. Layout, identical for both backends::
|
||||
of the kinds in ``KINDS`` (``facts``, ``verdict``, ``unit_cost``, ``alert``, ``coverage``, ``features``, ...) that
|
||||
``remember`` (:mod:`.tasks.remember`) derives from a run's delivered artifacts, plus the loop's and the decision
|
||||
engine's own. Layout, identical for both backends::
|
||||
|
||||
index.jsonl one row per ingested (run, task): sha256 of its rows, row count
|
||||
facts/<sha256>.json facts.json by content (identical facts stored once)
|
||||
@@ -49,7 +50,12 @@ MEMORY_KEY_ANNOTATION = "openultrasast.io/memory-key"
|
||||
POPULATION_ANNOTATION = "openultrasast.io/population"
|
||||
SPLIT_ANNOTATION = "openultrasast.io/split"
|
||||
SNAPSHOT_ANNOTATION = "openultrasast.io/memory-snapshot" # the loop's guard: {"catalog", "manifest", "populations"}
|
||||
KINDS = ("facts", "verdict", "unit_cost", "alert", "coverage", "proposal_outcome")
|
||||
# The decision engine's kinds (learned-decision-engine, Data Models): `features` from the `features` task through
|
||||
# `remember`; `label`, `decision`, `experiment`, `arm_outcome` and `experiment_result` for the later tasks.
|
||||
KINDS = (
|
||||
"facts", "verdict", "unit_cost", "alert", "coverage", "proposal_outcome",
|
||||
"features", "label", "decision", "experiment", "arm_outcome", "experiment_result",
|
||||
) # fmt: skip
|
||||
ROW_FIELDS = ("id", "kind", "repo", "pin", "run", "task", "population", "split", "image")
|
||||
TAG_FIELDS = ("repo", "pin", "kind", "family", "run", "population", "split")
|
||||
MIN_FREE_BYTES = 1 << 30
|
||||
|
||||
Reference in New Issue
Block a user