ops/ax/up.sh reads kube_context, registry, pools.default and the images path
with `python -m openultrasast.plane.profile --print`, writes the runner digest to
ops/k8s/profiles/kind-images.json instead of ~/.cache/.../runner-image, renders
ops/ax/workerpool.yaml.tmpl from the profile's pool (POOL_NAME/REPLICAS/CPU/
MEMORY, WORKER_IMAGE), and keeps an existing pool and an ax CLI newer than its
checkout instead of rebuilding them (ko and go install would refill several GB
of Go cache on this disk). smoke-run.sh takes the runner from `--print
images.runner` and runs the checkout's code with OUSAST_PYTHON. repin_templates
reads the JSON images file (runner key; the one-line pin still reads). `--print
images` returns the file path (the first live rerun fell over this). Templates
and the generated validation-46 Tasks re-pinned to the rebuilt image (128
lines, one digest).
Live: up.sh reran idempotently on kind in 38 s (every step kept except the
runner build); smoke-run.sh done in 24 s on the new image, facts.json 451 B,
summary.json 238 B delivered (record: benchmarks/measurements/2026-10-02-k8s-01-profile/).
Gates: pytest 2010 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0; mkdocs build --strict 0.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
tests/test_plane_literals.py scans src/, plane/, ops/ (minus ops/ax/up.sh and
ops/k8s/profiles/) and docs/ for the kind context, the kind registry, the kind
network's gateway, a loopback address with a port and port-forward; router.py
keeps the last two, a line tagged ax-tunnel or loopback is allowed, and a
committed template's digest-pinned image is accepted only while it names the
kind profile's registry (checked by a third test). A self-check reinserts each
literal into a fixture and sees it caught. The runner's health URL and the Joern
session's query URL are tagged loopback; docs/deployment.md and ops/ax/README.md
stop naming this host and describe the kube context and the registry as the
profile's (what 1.2 made configurable).
Gates: pytest 2009 passed, 42 skipped, exit 0; ruff check 0; ruff format 0;
mypy 0; mkdocs build --strict 0.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
doctor(profile) checks the profile's kube context, the two namespaces behind a
reachable context only, the images file's runner reference in the profile's
registry (a local registry over its v2 API, a remote one via docker manifest
inspect) and opens the memory store; its first line reports every configured
address (Req 1.2). Egress(executable, context), open_router(context, url=...),
receiver_cluster_ip(context) and Receiver(..., context) take the profile's
context and build none; generate.increment/render and harvest take the profile's
atespace. `ousast plane doctor|run|workspaces|harvest --profile`. The test fakes
(kubectl-ate, kubectl, ax) record the context they received and the tests assert
it is the profile's. grep "kind-" src/openultrasast/plane: no match.
Gates: pytest 2006 passed, 42 skipped, exit 0; ruff check 0; ruff format 0; mypy 0.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
ops/k8s/profiles/kind.toml and k3s.toml hold the values that differ per cluster
(exec, kube_context, registry, images, router_url, ax_server, memory, atespace,
snapshots_bucket, image_pull_secret, pools); kind-images.json carries the digest
pin ops/ax/up.sh wrote. load_profile() reads OUSAST_PLANE_PROFILE (name or path,
default kind), applies OUSAST_<FIELD> and OUSAST_POOL_<NAME>_<FIELD>, and refuses
an unknown key, a credential-shaped key or value, an exec outside local|remote
and an images file without digest pins. `python -m openultrasast.plane.profile
--print FIELD` serves shell scripts. The module-audit manifest is regenerated
for the new module (counts 147/5, no orphan).
Gates: pytest full suite 2002 passed, 1 failed (the module-audit manifest, fixed
by regenerating it; tests/test_module_audit.py re-run exit 0), ruff check 0,
ruff format 0, mypy 0.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
openultrasast-scan: evidence ladder as verification.py defines it, with the levels nothing produces today; --mode prerequisites, --fail-on, recorded degradations, run artifacts, ousast mcp; retired [evidence] threshold language removed. openultrasast-triage: reasons verified against calibration.py; deterministic fusion; adjudications as labels, plane verdicts as features, rule changes through improve or a registered experiment; fix-audit merged as a fix-and-re-scan section. kiro: spec-quick and spec-batch removed; validate-gap and validate-design merged into spec-design (--gap, --review; the review gate stays); steering-custom merged into steering; core steering names overview.md, roadmap.md, safety-net.md; instrument rule in debug and verify-completion; repo gates named in validate-impl. AGENTS.md, README skills paragraph and tests/test_agent_skills.py (.agents/skills, two product skills, drift tests) updated.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Legacy inventory 2, item R3. The three keys loaded into ResolvedConfig and nothing read them: no
`.network`, `.workspace_readonly` or `.top_k` access exists in src outside config.py. The sandbox
runner always passes `--network none` and `--read-only` and rejects host networking by argv
inspection, so the two sandbox keys were knobs that turned nothing; no stage read top_k, and
`[complexity] max_hunter_hotspots` is the one complexity key the scan reads. They join RETIRED_KEYS
with the same 2026-10-02 mechanism as the other unread keys (RetiredConfigError naming the reason).
The fields and their loaders are removed; memory_mb, timeout_seconds, pids_limit (read by
regress/recipes.py and sandbox/runner.py) and max_hunter_hotspots stay. test_config.py covers the three
retirements and the override test now exercises max_hunter_hotspots. RELEASE_NOTES Unreleased carries
the line.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Legacy inventory 2, item R2. tests/test_harnessx_removal_equality.py loaded
benchmarks/archive/harnessx_removal_equality.py by path and checked that its comparison sees a
difference. The spec is implementation-complete and the proof is the committed pair of records
(2026-09-30-harnessx-removal-{baseline,equality}); the archive README says these scripts are
reference only and a record is re-run at the commit it names, not with today's package. A unit test
that keeps the archived copy importable contradicted that and would break the next time cli or
learn.cli move. The archive README row now says the test was dropped. test_removed_plane_references.py
(the Req 2.4 name search) is a different test and stays.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Legacy inventory 2, item R1. The vector half of index.py (VectorIndex, VectorRecord, RetrievalHit,
RetrievalPackage, the EmbeddingClient protocol, index_reuse_key, build/write/read/query_vector_index,
build_retrieval_package and their helpers) had no importer in src; `ousast index` wrote index/chunks.json
that nothing read, and was the only reader of `[embeddings]`. index.py keeps CodeChunk and
chunk_text_namespace, which skills.py imports; chunk_file/build_code_chunks go with their only caller.
`[embeddings] model` and `[embeddings] store` join RETIRED_KEYS (RetiredConfigError naming the reason),
the same way the [variants] keys were retired; OPENROUTER_EMBEDDING_MODEL is no longer read, so the
README and .env.example drop it (`ousast learn memory embed --model` names the model). test_index.py
now tests the chunker only; test_config.py covers the two retired keys and no longer asserts the
removed field. The module audit record is regenerated (counts unchanged: 151 modules, 0 orphaned;
the index module's importer list shrinks to skills). RELEASE_NOTES Unreleased carries the line.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Maintainer decision 2026-10-02 (no symlink). AGENTS.md, README, the two spec
docs, the kiro-spec-batch skill and the provenance ignore list are repointed;
the OpenCode plugin package.json stays under .opencode.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
An experiment's register, run, result and arm_outcome rows share one object,
and RustFS's S3 Select infers one schema per object: exp-002's result row
carried `units` as an object beside the run row's integer (and `looks` as
objects beside the register row's fractions), and after `analyse --record` no
row of the experiment could be read (JSONParsingError). The result row now
writes `unit_counts` and `looks_taken`; a test asserts one JSON shape per
column across the four row kinds. The live object was repaired by dropping
the one row (drop_rows) and recording the analysis again.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Static-profile feature records carry the nine plane-only features (agree.*,
ms.*, verify.*) as null in x; after cb85542 the 937 live example rows loaded,
but the renderer's validate_x refused every one of them ("features outside the
static allow-list"), so no program could prepare a prompt (found while probing
exp-002 model-free). profile_x drops a null feature outside the row's profile
and keeps a valued one, so a real leak still fails. Test added.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Static-profile feature records carry the plane-only instruments as null; every
one of the 937 live example rows was unreadable (TypeError), which blocked
ousast learn evaluate and freezing exp-002 units.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The brace matcher now finds a declaration head in more forms -- name(...) with
any return type, qualifiers, generics or a parameter list spread over many
lines (a paren-aware scan replaces the 8-line lookahead; a prototype's ';' is
not a body, a ')' closing an outer parenthesis marks a call), JS object-literal
methods and property/chained assignments of function expressions and arrows,
$ as an identifier character, Perl 'sub name {', C# expression bodies -- and a
file given as 'other'/'unknown' is sniffed for Perl and Ruby so their def/sub
forms are matched by the right pattern instead of a {} block inside them. The
memory build hands the matcher the label's language where the record's closed
vocabulary language ('other' for Go, Ruby, C#) resolves nothing. One test per
form. Fixed-side negatives still require the fixed side to declare the
function; the 94 vibe-py positives-only groups stay counted.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
learn/experiments.py: YAML manifests under plane/experiments/<id>.yaml,
digest-checked at register, run and analyse time; `experiment`,
`arm_outcome` and `experiment_result` rows; paired arms in a seeded
repository order with a per-unit coin; arm A replay-only at $0; canary
replicates under a fresh cache salt; repository-cluster paired bootstrap,
exact McNemar, two O'Brien-Fleming looks, adopt/reject/equivalent/
inconclusive. `ousast learn experiment register|units|run|analyse`.
Program: `ProgramSpec.sampling = retrieval_ensemble` (k samples at
temperature 0 over k disjoint retrieved example sets; default unchanged);
`Caller.salt`; `retrieve(exclude=)`. verify: OUSAST_VERIFY_CALLERS=off.
Manifests exp-001-known-callers (plane arms, units pending) and
exp-002-retrieval-ensemble (slice-2 incumbents, ceiling $4) -- not run.
18 scripted tests; module audit regenerated (151 modules).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Decision-engine task 6.7 (partial). After the advisory-fix harvest (memory 724 -> 904 static examples),
one compile per evaluable family and an outer grouped 5-fold evaluation of that family's own candidates,
input profile v1, priors off, k = 5, cross-fitted Platt, nested operating points (repository-cluster
bootstrap 95%):
family n AUC(s) ADVISORY R / P 'vulnerable' P / R ECE canary
injection 110 0.77 [0.67, 0.88] 0.88 / 0.57 0.84 / 0.36 0.093 0.77
untrusted_dest. 62 0.85 [0.76, 0.94] 0.84 / 0.73 0.96 / 0.66 0.071 0.87
path 35 0.89 [0.79, 0.97] 0.86 / 0.69 0.89 / 0.76 0.119 0.90
output_encoding 44 0.91 [0.82, 0.97] 0.86 / 0.81 0.94 / 0.55 0.045 0.90
deserialization 46 0.90 [0.82, 0.97] 0.83 / 0.76 0.93 / 0.83 0.095 0.93
access_control 37 0.96 [0.90, 1.00] 0.91 / 0.91 1.00 / 0.61 0.076 0.83
BLOCK offered for none: calibration fails everywhere except output_encoding, where no fold reaches a
Wilson lower bound of 0.95. Within paired groups (a function and its own fix) AUC is lower, 0.72-0.93:
every single-label group in the new families holds positives only. Injection learning-curve point,
paired (same program, candidates, folds): memory 724 -> 904, AUC 0.76 -> 0.77 (no injection example
was added). $2.88 metered DeepSeek (ceiling $8; balance 17.73 -> 16.73), $0.0009 embeddings.
Instrument fix: compiled demonstrations now follow the near-duplicate rule (cos >= 0.98) that retrieval
and the pre-call assertion apply; the assertion had stopped the untrusted_destination evaluation.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The plane's memory store no longer names one vendor. `MinioStore`/`SdkClient`/
`minio_settings`/`MINIO_KEYS` become `S3Store`/`S3Client`/`s3_settings`/`S3_KEYS`;
the URL scheme is `s3://<bucket>[/<prefix>]` (`s3://` alone uses `S3_BUCKET`); the
settings are `S3_ENDPOINT` (a full URL), `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`,
optional `AWS_SESSION_TOKEN` and `S3_REGION`. The client is boto3 (extra `s3`,
imported lazily): SigV4, path-style addressing, SDK checksum trailers off, paginated
listing, versions and delete markers listed and deleted by version (`purge`, used by
the contract tests' teardown so the bucket keeps nothing of a run).
Behaviour is unchanged: the store never configures its bucket and verifies
versioning, the `runs/` expiry rule, S3 Select and tag reads at startup; no local
fallback; every row carries every queryable field with "" placeholders. The admin
hints read `--endpoint-url "$S3_ENDPOINT"`. The fake client still reproduces RustFS's
two Select failures (unbound leading-row field, null-typed column).
Docs: rustfs.md stays the tested server's setup page (policy JSON unchanged, env
names updated, any S3-compatible store with versioning, lifecycle, tags and Select
works); memory.md, ops/ax/README.md, README.md, steering and the two current spec
designs use the S3 naming. Retired specs and committed measurements are untouched.
Real server (RustFS, OUSAST_MEMORY_TEST_S3=1): test_plane_memory + test_learn_compile
75 passed, 1 skipped. Store copy of the live local store to a temporary prefix: every
count matched (776 row objects, 2982 rows, 22 facts, 788 excerpts, 2601 responses,
2 programs, 1793 index entries), 4190 versions purged afterwards.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The 97 advisory-fix pairs (194 sides) harvested for the decision engine: verify a/b + agree on the 34
untrusted_destination sides and model roles on every side as plane Runs on ax ($0.38 metered, task
budgets $1.49, ceiling $5; DeepSeek balance 17.88 -> 17.73), engine features on the host (frozen
7b17eac, one container, --memory 3g: 54 ran, 58 asked no question, 82 no language model), feature
records for all 194 sides, static memory rebuilt with the merged harvest units: 724 -> 904 examples.
No unit was skipped for the 20 KB inline limit.
Instrument fix: SideReader read only the first pair-kind source, so the first rebuild left all 194
advisory sides `unread` (0 examples). It now reads every pair-kind source; test added.
Leak audit: the static profile flags only engine signals and facts.function_lines (withheld by v1);
the plane profile newly flags verify.flag_b and ms.operation (plane-only, not in any v1 prompt).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Material for MkDocs site (mkdocs.yml, optional extra `docs`) with Mermaid via
the superfences custom fence. New pages: overview, scanning (modes, quick
rules, Joern engine, priors, pre-push delta), the plane on ax (one task as a
sequence, golden snapshot, Run DAGs), memory (row kinds, write/read paths,
FileStore vs S3, verify at startup), memory and detection, and RustFS setup
(admin steps, complete agent policy, .env, refusal messages, Select limit).
The ops READMEs are included at build time; evaluation gains the population
lifecycle and the fold layout. tests/test_docs_site.py runs the strict build
when mkdocs is importable and checks Mermaid blocks and the RustFS page.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
A1-A3 of the legacy inventory, moved with git mv to benchmarks/archive/:
the stage-2 verifier chain (verify_batched, score_batched, verify_sinks,
model_sinks, sink_candidates), the v2 tool hunter (hunt.py) and the
harnessx-removal equality proof. The plane's verify/agree/roles tasks
replaced the first six; the harnessx-removal spec is closed.
- The six independent scripts keep HERE = benchmarks/independent (population,
evaluate.py, results) and add their own directory for archived siblings.
- harnessx_removal_equality's REPO root is one level deeper.
- tests/test_plane_scoring_parity.py loads score_batched from the archive
(ran here, not skipped); test_harnessx_removal_equality points at the new path.
- benchmarks/archive/README.md: what each script was, which committed
records cite it, the commit that last ran it. paths.md maps old -> new;
committed records are not rewritten.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
RustFS's S3 Select infers an object's schema from its first 1000 rows: a where
field absent there is unbound (EvaluatorBindingDoesNotExist), and a column null
in all of them but set later fails the whole object (JSONParsingError, measured:
999 leading nulls answer, 1000 fail). plane/memory.py now declares per kind the
fields a where may name (QUERY_FIELDS: facts candidates_digest, verdict
candidate/final, example profile, plus the ROW_FIELDS every row has) and writes
their union on every row at write time as strings, "" where one does not apply
(null measured to break). rows() refuses an unknown kind, an undeclared where
field and where = null before any read. `ousast plane memory-normalise
[--store URL] [--dry-run]` rewrites pre-rule rows once, idempotently, with
counts. The fake object client mimics both RustFS failures.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Nothing in src reads config.dynamic, config.evidence or config.variants, and
[models] patcher was only echoed into harness.json. They now go through the
existing RETIRED_KEYS mechanism and fail with RetiredConfigError naming why
("retired 2026-10-02 because nothing ever read it: ..."); RETIRED_KEYS now
carries the full reason per key so the HarnessX entries keep their wording.
- DynamicConfig, EvidenceConfig, VariantsConfig and their loaders removed;
ModelConfig.patcher removed. resolved_config.json loses those sections.
- harness.json keeps its shape: model_roles.patcher is always null.
- RELEASE_NOTES "Unreleased" lists the keys and the output shape change.
- Delete .opencode/skills/openultrasast-kiro-impl: it pointed agents at the
retired openrouter-sast-harness tasks.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Maintainer decisions 2026-10-02: "it has the featurs. fallback not desired" and
"Admin sets it once; agent verifies".
- MinioStore never configures its bucket: configure()/enable_versioning()/
expire() are gone; the client protocol only reads versioning and lifecycle.
- verify_bucket() runs when a MinioStore is opened (once per process per
store): versioning Enabled, an enabled lifecycle rule expiring <prefix>/runs/
(none expiring the whole store), S3 Select answering a probe, object tags
readable. Every missing piece is listed in one MemoryStoreError with the
aws-cli admin command or the permission to grant.
- No S3 Select fallback: a server without Select is refused at open, and a
failing Select on one object raises (naming RustFS's leading-row schema
inference when the field is unbound) instead of filtering locally.
- MINIO_REGION is passed through when set.
- Tests: fakes for versioning off/unreadable, lifecycle missing/disabled/
tag-narrowed/whole-store, Select missing, tags unreadable, and the pass case;
the real-server contract uses OUSAST_MEMORY_TEST_BUCKET (else MINIO_BUCKET)
and never writes bucket configuration.
- ops/ax/README.md: RustFS support, the admin one-time setup, the agent policy.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The reachability-first audit (previous commit) found these five with no
import path from any entry point:
- model/judge.py: replaced by model/pipeline.py, which never imported it.
- model/execution.py: the MGD Req 10 execution seam, never built or called.
- model/calibrate.py: no CLI, no importer; learn/calibrate.py is the live one.
- stage_processors.py + slot_contract.py: kept only so test_gate could pin
them; the scan runs findings.quick_scan_findings directly.
Their tests go with them, test_gate's zero-dependency list drops the two
entries, the audit lists drop their rows, and the manifest is regenerated:
150 modules, 145 load_bearing, 5 standalone, 0 orphaned.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
model/audit.py classified anything named in SPEC_OWNED or STANDALONE before
looking at the import graph, so five modules nothing imports were reported
load_bearing and the committed manifest said 0 orphans.
- Reachability from the gates, the CLI and every module with a __main__ block
(python -m, the plane runner) now decides the class; the lists only supply
reason text, and an unreachable listed module is orphaned with its listing.
- `from .pkg import sub` now records the submodule edge (plane.engine_alerts
was only reachable by its SPEC_OWNED entry).
- `python -m openultrasast.model.audit` regenerates the manifest.
- Manifest regenerated: 145 load_bearing, 5 standalone, 5 orphaned
(model.calibrate, model.execution, model.judge, slot_contract,
stage_processors). The zero-orphan test is strict-xfail until they go.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
6.2: the 724 static examples' 616 distinct excerpts embedded with openai/text-embedding-3-small via
OpenRouter, 198,819 tokens, $0.0040 metered (ceiling $1).
6.6, family injection only, memory = every family (at most 2 contrast examples per prompt), input
profile v1: smoke fixed k = 5 (repeat prefix-cache hit share 0.96); one compile (program c1237a49,
proposed instruction 5, balanced Brier 0.128 on 13 C_val candidates vs 0.199 baseline); outer grouped
5-fold, 110 out-of-repository candidates in 29 groups: ADVISORY recall 0.90 [0.82, 0.97], precision
0.54 [0.44, 0.65], AUC 0.74; BLOCK not offered (ECE 0.064 > 0.05; no fold reached a Wilson lower bound
of 0.95); canary re-ask agreement 0.83. $0.91 metered at list price; DeepSeek balance $18.20 -> $17.90.
Instrument fixes found on the way, each with a test:
- memory build: pair sides join the record stored under their harvest unit pin (--units), and only the
built profile's records are read (static and plane shared a key; 85 -> 724 examples);
- retrieval: stage 1 per side (family / contrast) and at most 2 contrast examples, never padded;
- program: the JSON answer contract is part of the signature, appended to proposed instructions
(DeepSeek JSON mode refused a prompt without "json");
- compile: candidate_families for a one-family compile; compile.toml k = 5 from the smoke rule.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
The leak audit found facts.function_lines separating fixed from vulnerable pair
sides (a fix guard lengthens the function: 0.80 untrusted_destination, 0.75
config_secrets). Input profiles may now name single features; v1 withholds the
engine and the function length from prompts, retrieval and the proposer summary.
A withheld feature keeps its instrument block in the distance (state still counts).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Maintainer decision "Neutralise both for v1".
1. Phantom candidates. A pair side's candidates are now only the functions that
side declares (plane/harvest.py `declares`, the excerpt builder's
function_bounds); a labelled function absent from a side is listed under
`absent_on_side` and never asked, and a side declaring none of its verify
family's functions is no verify unit. The recorded harvest units held 58
phantom verify candidates (53 fixed, 5 vulnerable sides). build_features.py
skips label rows whose side does not declare the function and drops their
earlier rows (MemoryStore.drop_rows); rebuilt the pair records from the
recorded signals, no model call: 126 label rows absent_on_side (117 fixed,
9 vulnerable) -- access_control 13, config_secrets 4, deserialization 20,
injection 28, memory 4, output_encoding 16, path 15, untrusted_destination
26 -- so 252 feature rows dropped (1712 -> 1460); no other row changed.
2. Engine withheld. Input profiles in learn/schema.py (`v1` withholds the
engine instrument, `full` withholds nothing); ProgramSpec.inputs defaults to
v1 and compile.toml records it. Neither the engine's state nor its values
reach the candidate, examples, demonstrations or the instruction proposer's
error summary, and the Gower retrieval distance leaves the engine block out.
Stored records are unchanged. design.md Risks records the decision and that
the engine returns only through a pre-registered A/B on assumed-benign and
benign-control negatives.
3. `ousast learn audit-leaks` (learn/leaks.py): per feature and per instrument
state, separation = (pairs the signal alone puts on the right side - on the
wrong side) / pairs, over all pairs and per family; flagged at >= 0.20 over
>= 10 pairs. Counts under
benchmarks/measurements/2026-10-01-decision-engine-leak-audit/ (before and
after the rebuild). After: facts.function_lines (0.80 untrusted_destination,
0.75 config_secrets, 0.26 injection; higher on fixed), verify.turns_mean
0.67 config_secrets, eng.* 0.35-0.40 output_encoding, verify.flag_a/votes/
site_offset and agree.final 0.20-0.30 on 10-12 pair families. Before the
rebuild the phantom fixed sides drove state.source/state.roles/ms.flagged
to 1.0 and state.engine to 0.95 on deserialization.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
`ousast plane harvest` (plane/harvest.py) writes two Runs over a label snapshot:
verify a/b + agree on the verify-family units (both sides of each labelled pair as
an opaque files-only Workspace, candidates outside the hunt's root, no declared
sites; v1 and dev-php pins with repo-facts first) and model roles on the candidate
files (pair excerpts grouped, one Task per population pin). Budgets sum under the
ceiling by construction. New roles task template; the generator re-pins it.
Spent by attribution: verify $2.54 (1961 calls, 150 units, 182 hunts per pass),
roles $1.40 (1034 calls, 799 files, 0 unclassified chunks); DeepSeek balance
19.63 -> 18.23. Engine on the pair corpus (benchmarks/learn/engine_pairs.py,
frozen e6ed785, one 3g container) is PARTIAL: 384 of 686 sides at commit, still
running; unfinished sides stay missing. benchmarks/learn/build_features.py built
1712 static+plane records into the memory store; counts-only record under
benchmarks/measurements/2026-10-01-decision-engine-harvest/.
source/roles parts are `none`, not `failed`, for a language without a declaration
pattern (C). ops/ax/README: Substrate golden-actor disk leak and the inline
Workspace limit (~20 KB).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
97 advisory-fix pairs, one repository each (309 advisories examined, 212
rejected with reasons in research-2026-10-01.md): a public fix commit
distinct from its parent, the advisory-named function declared on both
sides with a changed body. Pointer pairs: materialize.py reads one file
per side from a shallow blobless fetch into the pair cache and verifies
them with the label builder's matcher (verification.json, bytes read).
learn/sources.toml lists the catalog as the advisory-fixes source. Label
snapshot: negative repository groups untrusted_destination 8->25, path
3->21, output_encoding 1->22, deserialization 0->21, access_control 0->20;
all five reach the per-family floor. Five pairs on reserved repositories
were dropped after the reserved-population guard flagged them.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
A pair now gives a fixed-side negative only when the fixed side's source declares the
labelled function (the excerpt builder's matcher; a name on a code line where the matcher
resolves neither side), or when a deterministic rename at a distinct fix commit replaces it
(one new function, same parameter list, changed body: provenance fixed_side_moved). Anything
else is counted under pair_fixed_side.absent_fixed_side, not labelled.
108 pair labels lose their negative: 80 seeded pointer pairs whose fixed side is a guard
helper of the same commit, and 28 Real-Vuln-Benchmark traps whose fixed side is a sibling
handler of the same snapshot (4 of those handlers are labelled positives themselves). The
memory build no longer substitutes the catalog's fix_function on the fixed side. The pair
cache is an allowed label-source tree so pointer sides are read through the guard.
Snapshot: benchmarks/measurements/2026-10-01-decision-engine-labels (counts only).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
learn/compile.toml: the pre-registered spec -- balanced Brier score of the
raw score on C_val (cells by family x label, repositories capped at 5,
unsure = 0.5, ties within 0.005 to fewer prompt tokens), seed, the
compile split (25%, boot 60/40, compile-split-only memory), bootstrap
(150 candidates, confidence >= 0.7, 4 sets of 3), search (6 proposals +
baseline, 20 errors shown, 60 validation candidates), k = 1, the
retrieval settings, and the $3 / 1,200-call ceiling.
learn/compile.py: bootstrap few-shot keeps only right, confident
rationales that cite a line and name no repository, path or CVE/GHSA id;
seeded demo sets (positive, negative, any; families mixed) with
other-framework alternates; one proposal call; 7 instructions scored with
set 1, the best with sets 2-4. The artifact is canonical JSON stored as
programs/<program_id>.json, program_id = sha256 of the artifact without
its id; it records the spec sha, memory/compile-groups/folds digests and
the response-cache digest, with spend priced from cached usage, so a
rerun from the cache is byte-identical at $0 (tested on FileStore and
MinioStore through the fake client; real MinIO gated as today). A ceiling
ends the compile `unfinished` with no artifact. packageable() keeps only
permissively licensed demonstrations (MIT, BSD, Apache-2.0, ISC, zlib),
swapping in a permissive alternate or dropping, under a new program id.
learn/calibrate.py: Platt scaling on logit(s), pure Python Newton, per
family / pooled with family intercepts / insufficient data by the floors;
cross-fitted (fold j's map is fitted on the other folds only); the check
(5-bin reliability, slope and intercept with repository-bootstrap CIs,
ECE <= 0.05), ADVISORY only when it fails.
learn/evaluate.py: predictions under each fold's boundary, families below
the floor never asked; operating points read from the calibrated,
cross-fitted predictions of the other folds (BLOCK: Wilson lower bound
>= 0.95, never on a majority unsure; ADVISORY: recall >= 0.90);
repository-cluster bootstrap intervals; strata with "insufficient data";
>5% parse failures is an instrument failure; explanations keep only
signals the record holds; version mismatch names the fallback reason;
the memory-size learning curve.
`ousast learn compile|evaluate|curve`: paid runs need --budget-usd (exit
3 at the ceiling), --replay-only runs from the cache. Nothing was spent:
every test and the dry run used scripted clients.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
learn/program.py: the declared signature (code, signals, roles, family in;
verdict, family, confidence, rationale with cited lines out), the
baseline instruction, rendering as [system: instruction + demonstrations]
-- byte-identical for every candidate, so the provider's prefix cache can
hit -- then [user: retrieved examples with their labels, the candidate
without one]. Signals pass validate_x first (no EXCLUDED_FIELDS, missing
instruments in words). Answers are one JSON object; one that does not
parse or cites no line inside the excerpt is asked once more at
temperature 0 and otherwise becomes unsure/parse_failed, never
not_vulnerable. Classify(k): sample 1 at 0.0, the rest at 0.7; score s =
mean p_j; majority verdict, ties unsure; a majority unsure cannot BLOCK.
Caller caches each response as responses/<sha>.json keyed by (model,
Model parameters digest, messages, temperature, sample, json_object) and
replays at $0; spend is priced from the recorded usage once per distinct
request, so a replay reports the same usd. Program.decide asserts the
evaluation boundary on the rendered ids before any call.
MeteredClient.complete passes temperature/logprobs through (only when
set); DeepSeekChatClient sends temperature beside thinking=disabled.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
learn/retrieve.py: stage 1 is a per-instrument Gower distance over the
profile's allow-listed features with explicit missing (both ran: mean
feature distance; differing states 1; same non-ran state 0; null in one 1;
priors dropped with priors=off), n1 = 40 with seeded tie-breaks; stage 2
re-ranks by r = lam(1 - D) + (1 - lam)cos, signals_only without the
target's vector; the balance rule takes k_ret = 6 (<= k/2 per label, <= 2
per group, <= 2 other families; `short` recorded). eligible() is the only
way into a prompt: not the target's group, not an evaluated group, not the
held-out source or framework, and outside deployment not a near-duplicate
(cos >= 0.98). assert_boundary repeats the check on rendered ids;
demonstrations of a held-out framework swap to their alternates;
deployment_fold excludes a scanned repository that is a corpus group;
seed_memory never hands a case its own group.
learn/folds.py: compile split (25% of groups per primary family, 60/40
boot/val), outer grouped 5-fold stratified by family and label mix,
leave-one-source-out, leave-one-framework-out (insufficient data below 10
groups), assert_disjoint and a folds digest; all seeded.
Tests: test_learn_retrieve.py (hand-computed Gower, lam 0/0.5, balance,
near-duplicates, test_boundary over outer/LOSO/LOFO folds with
demonstrations), test_learn_folds.py.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
plane/models/openrouter-embedding.yaml: openai/text-embedding-3-small via
OpenRouter (embeddings only), input_per_m 0.02, cache/output 0, egress
openrouter.ai; `openrouter` joins the provider extensions.
plane/budget.py MeteredEmbeddingClient: prompt_tokens priced from the
Model manifest, usd/calls ceilings stop the next call, a priced call that
reports no tokens raises UnmeteredUsage (a silent zero is an instrument
failure). OpenRouterEmbeddingClient keeps one usage row per call
(parse_embedding_payload); its api key is out of repr.
learn/embeddings.py + `ousast learn memory embed`: excerpts embedded in
batches into embeddings/<model-slug>/<excerpt_sha>.json (float32 base64),
cached by content hash so a rerun makes no call; the declared dimension is
checked; --dry-run estimates tokens and cost without a call; a paid run
needs --budget-usd and exits 3 (unfinished) at its ceiling. Not run against
the provider: the store embedding waits for its budget go-ahead.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
learn/excerpt.py: the labelled function at its pin as numbered lines, bounded
(80 lines / 4,000 chars for candidates, 40 / 2,000 for memory examples),
centred on focus lines; advisory ids anywhere and comments with security
wording (labels.security_reason, multilingual) become <redacted>; owner,
owner/name, path and commit ids are replaced; a delta unit appends the
base->head diff (<= 40 lines). C/C++ and other brace languages without a
repo-facts declaration pattern use a brace matcher (346 c_cpp pair rows
had no excerpt without it).
learn/examples.py + `ousast learn memory build`: label + feature record +
excerpt -> `example` rows (no rationale), excerpts as content-addressed
blobs; refuses origin: user and sources outside sources.toml (exit 2);
conditional labels skipped; exit 2 when >10% of a source's recorded
candidates have no excerpt. A fixed side that was read but no longer
declares the labelled function is counted apart (absent_fixed_side): a
label defect, not an unread clone.
plane/memory.py: `example` in KINDS; put_blob/get_blob/blob_names on the
MemoryStore base, so FileStore and MinioStore share them (tested on both,
MinIO through the fake object client).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
learn/sources.toml is the fail-closed source list: every read goes through one guard
that refuses an unlisted path or an [[excluded]] file name before opening it, and a
source whose evaluation record is missing cannot label. Population v3 is named only
as excluded, by file name, and is never opened (tested with a v3-named fixture and
open/read_text tracked).
learn/labels.py and `ousast learn labels` apply the design's rules: v2 declared
sites and v1 fix-range functions positive, the same function on the fixed pin
negative only when the fix changed it, introduce/repair delta units, benign control
pushes as conditional negatives; pairs vulnerable/fixed (unscorable and title tier
out, title as a sensitivity arm); dev-php recipe known functions; recorded
adjudications with their reasons. v2 rows are labels marked design_informed, as the
design says. Repository groups: normalised owner/name, merged by shared CVE/GHSA,
shared fix commits and [[alias]] rows.
assumed_benign (maintainer decision): ordinary commits of the population
repositories, >= 90 days from any fix, no later security commit touching their
files, weight 0.5, conditional, reported separately and never merged with verified
negatives. The security filter is multilingual (CJK, Korean, Russian, European and
other languages) plus CVE/GHSA/CWE references and dependency bumps; widened after
reading accepted pushes (validation, passwords, guards).
First snapshot, counts only (benchmarks/measurements/2026-09-30-decision-engine-
labels/labels.json): verified pin labels in positive/negative repository groups
injection 44/41, untrusted_destination 35/34, deserialization 20/20 (per-family),
output_encoding 15/15, path 15/14, access_control 12/12 (pooled), config_secrets 5/5,
memory 5/5 (insufficient). assumed_benign: 969 pushes in 27 groups from 36,000
commits scanned.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
Wrapper inference (learn/roles.py, no model): a function whose parameter reaches a
call already in the role set becomes a derived sink (or a sanitizer when it returns
the cleaned value) of the same operation, at depth + 1; a function returning a
source's value becomes a derived source. The role set starts from language-level
entries only (priors off by default, optional), grows to depth 3, and reads
vendored dependency source when present, so a vendored database class reaching
mysqli makes `->query` a sink without a framework table. A name the repository
declares more than once becomes a role only when every definition derives it.
Model roles (plane/tasks/roles.py): the model_sinks.py classification moved into
the package and extended to sink/source/sanitizer/guard/dispatch, per file chunk,
metered and resumable, client injected; a failed chunk is unclassified, never
"no roles". vocabulary_overlay merges inferred and model roles over the
language-level facts in the shape taint_specs takes, origin in the entry id.
Features: the roles (static) and model_sinks (plane) instruments are filled.
On the recorded validation-46 facts (15 cases, 14,368 product files, 87 MB read):
1,999 wrapper roles (1,753 sinks, 157 sources, 89 sanitizers; depth 1/2/3 =
973/651/375), 14 of 15 cases with at least one; the WordPress gopay plugin has
none with priors off, which is what model roles are for.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
ruleset/frameworks.toml lists 19 framework and library ids (wordpress, flask, django, express,
spring, jakarta-servlet, react, vue and 11 libraries), each with its languages, dependency
package names and a symbol lexicon used only by the lint and for stratification.
Every framework/library entry of section 2 is tagged `framework`/`library`, and mixed entries
are split so a tag covers a whole entry (split pieces keep the entry's id and flags, in place):
semantic facts 38 tagged entries (wordpress 6, django 6, flask 4, express 4, jakarta-servlet 3,
pyyaml 2, one each for spring, sqlalchemy, requests, httpx, lodash, node-http-clients,
escape-html, sanitize-html, dompurify, esapi, commons-text, react, vue); the dispatch fact
behind the hook-as-source path is wordpress; the PHP layout row is renamed php_layout and stays
untagged (a product filter). Quick rules: 9 tagged (wordpress 6, flask, pyyaml,
node-http-clients); php-unserialize is split into php-unserialize and php-maybe-unserialize;
the lint also caught WordPress escapers in exclusions and axios/yaml.load in mixed rules, so
those keep their ids as tagged rules and gain a `fallback_for` language-level rule (6) that is
active only when the prior is off.
load_facts and load_ruleset take priors ("all" by default -- today's scan --, "off", or a set
of ids). tests/test_semantic_priors.py: the lint (no untagged entry contains a lexicon symbol
of its language), "off" removes exactly the tagged entries and activates the fallbacks, split
entries keep their language-level calls. Rule and layout counts in three existing tests follow the split and the rename.
Unchanged defaults: test_benchmark_output_is_byte_identical_to_golden_baseline passes, and the
quick findings of all 15 benchmark manifests (full StaticFinding records) are byte-identical
before and after. priors=off on the same manifests: 61 -> 57 findings; the 4 lost are js-ssrf
(axios), php-wpdb-sql-composition, python-ssti and python yaml.load; 5 PHP and 1 Python finding
move to their fallback rule at the same location. Semantic calls with priors off: php 75/92,
python 32/77, javascript 42/70, java 24/41, c 27/27 (the engine-side effect needs Joern and was
not measured).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
learn/schema.py is the closed, typed allow-list of candidate features: 66 features over
11 instruments (language one-hot, quick rules with 23 mechanism buckets and a prior count,
engine, repo-facts, source span, entry-point distance, roles, delta, verify, agree, model
sinks), two profiles (static 57; plane adds verify, agree and model sinks for 66), schema_version and a feature-set digest.
No repository, path, file, function, rule or commit identity is a feature; site_match,
in_fix_range, fixed-pin alerts, case.json fields and costs are refused. Missing is not zero:
each record carries per-instrument state (ran | none | failed | not_applicable), and every
feature of an instrument that did not run is null.
learn/features.py builds records on the host (build_for_scan, whole repository or base..head
delta); plane/tasks/features.py builds them per case in a Run from the delivered artifacts,
dropping site_match and in_fix_range on read and ignoring fixed-pin alerts. The generator
adds a model-free `features` entry per case before `remember`, which stores the records as
`features` rows (re-validated). memory KINDS gains features, label, decision, experiment,
arm_outcome and experiment_result. The committed validation-46 manifests are regenerated
(entries appended; the first Run's entries are unchanged). The runner image must be rebuilt
and re-pinned before a Run executes the new task.
Proof on real artifacts (scratch store, not committed): the recorded validation-46 run gives
75 feature records over 15 cases (43 verify candidates plus vulnerable-pin alerts), each
case's vulnerable pin exported and read (0.5k-9k files); remember stored 75 features rows
beside 43 verdict, 84 alert, 61 unit_cost, 15 facts and 8 coverage rows.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
