feat(learn): candidate feature schema and builders (decision-engine task 1)

learn/schema.py is the closed, typed allow-list of candidate features: 66 features over
11 instruments (language one-hot, quick rules with 23 mechanism buckets and a prior count,
engine, repo-facts, source span, entry-point distance, roles, delta, verify, agree, model
sinks), two profiles (static 57; plane adds verify, agree and model sinks for 66), schema_version and a feature-set digest.
No repository, path, file, function, rule or commit identity is a feature; site_match,
in_fix_range, fixed-pin alerts, case.json fields and costs are refused. Missing is not zero:
each record carries per-instrument state (ran | none | failed | not_applicable), and every
feature of an instrument that did not run is null.

learn/features.py builds records on the host (build_for_scan, whole repository or base..head
delta); plane/tasks/features.py builds them per case in a Run from the delivered artifacts,
dropping site_match and in_fix_range on read and ignoring fixed-pin alerts. The generator
adds a model-free `features` entry per case before `remember`, which stores the records as
`features` rows (re-validated). memory KINDS gains features, label, decision, experiment,
arm_outcome and experiment_result. The committed validation-46 manifests are regenerated
(entries appended; the first Run's entries are unchanged). The runner image must be rebuilt
and re-pinned before a Run executes the new task.

Proof on real artifacts (scratch store, not committed): the recorded validation-46 run gives
75 feature records over 15 cases (43 verify candidates plus vulnerable-pin alerts), each
case's vulnerable pin exported and read (0.5k-9k files); remember stored 75 features rows
beside 43 verdict, 84 alert, 61 unit_cost, 15 facts and 8 coverage rows.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-30 18:38:04 +02:00
co-authored by Claude Opus 5.5
parent edcf145b00
commit d1896cb8e7
17 changed files with 2129 additions and 31 deletions
+9 -3
View File
@@ -1,8 +1,9 @@
"""The plane's memory store (harnessx-removal Requirement 6.1/6.4, design section 4).
One store, keyed by repository and pin, holds what plane runs learned: repository facts by content hash, and rows
of six kinds (``facts``, ``verdict``, ``unit_cost``, ``alert``, ``coverage``, ``proposal_outcome``) that ``remember``
(:mod:`.tasks.remember`) derives from a run's delivered artifacts. Layout, identical for both backends::
of the kinds in ``KINDS`` (``facts``, ``verdict``, ``unit_cost``, ``alert``, ``coverage``, ``features``, ...) that
``remember`` (:mod:`.tasks.remember`) derives from a run's delivered artifacts, plus the loop's and the decision
engine's own. Layout, identical for both backends::
index.jsonl one row per ingested (run, task): sha256 of its rows, row count
facts/<sha256>.json facts.json by content (identical facts stored once)
@@ -49,7 +50,12 @@ MEMORY_KEY_ANNOTATION = "openultrasast.io/memory-key"
POPULATION_ANNOTATION = "openultrasast.io/population"
SPLIT_ANNOTATION = "openultrasast.io/split"
SNAPSHOT_ANNOTATION = "openultrasast.io/memory-snapshot" # the loop's guard: {"catalog", "manifest", "populations"}
KINDS = ("facts", "verdict", "unit_cost", "alert", "coverage", "proposal_outcome")
# The decision engine's kinds (learned-decision-engine, Data Models): `features` from the `features` task through
# `remember`; `label`, `decision`, `experiment`, `arm_outcome` and `experiment_result` for the later tasks.
KINDS = (
"facts", "verdict", "unit_cost", "alert", "coverage", "proposal_outcome",
"features", "label", "decision", "experiment", "arm_outcome", "experiment_result",
) # fmt: skip
ROW_FIELDS = ("id", "kind", "repo", "pin", "run", "task", "population", "split", "image")
TAG_FIELDS = ("repo", "pin", "kind", "family", "run", "population", "split")
MIN_FREE_BYTES = 1 << 30