99 source modules -> 81. 832 tests -> 646. The whole learning/ package (16
modules, ~4,100 lines) and six semantic/ modules removed, with the three gates
byte-identical to 2026-09-06-gate-baseline.txt at every step.
The sequence is what made it safe: vocabularies ported first (1.1), importers
detached second (1.2-1.3), deletion third (1.4). Every import break during the
deletion was therefore a detachment miss in the importer and was fixed there --
no module was restored to satisfy one. Both misses were the same shape and were
fixed as a class: operations.py and shapes.py each imported trailing_name from
the deleted variants.py, so it moved to semantic/ir.py, its natural home, and
both importers changed in one edit.
Ported before deleting: the closed guard vocabulary (verbatim, nothing else
holds it) into model/specs.py, which derives TaintSpec and DominanceSpec from
the retained ruleset facts rather than copying them. Re-homed: families ->
model/taxonomy, candidates -> model/candidates, endpoint -> model/endpoint,
plus Region, labeled_family and labeled_spans. labeled_family no longer falls
back to the LLM classifier -- an unlabeled pair is honestly unknown.
Four deviations, each recorded in the spec notes with its reason. improve/
evolve.py and validator.py were stripped, not deleted: each mixes the mechanism
lever (gone) with the deterministic rule-status loop `ousast improve` still
runs, which has no LLM in it; a test now asserts the lever's absence by symbol
so it cannot creep back. The learning CLI group became `model candidates`,
because deleting it outright would have orphaned the retained enumerator by
removing its only entry point. thinking moved from [learning] to [models].
A gap named rather than papered over: no retained sink fact routes by CWE to
path, output_encoding, untrusted_destination or prototype -- the same hole the
entailment ceiling diagnosed. The test asserts the seeded set and the gap set
exactly, so task 4.2 must update it when it authors those sinks.
The audit found exactly one genuine orphan: vectorstore.py, a bakeoff harness
with no production caller and no CLI, alive only through its own test. The five
named subsystems each carry an explicit decision. Manifest committed, with a
test asserting the tree still matches it.
Group 2 (the Joern go/no-go) has not started.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Three defects, two of them the same shape as ones already rejected once:
- The meta-agent proposer still could not propose. `MetaAgent(inner_model=)`
takes a ModelConfig; it was handed a model name, a `# type: ignore` hid
that from mypy, and a bare `except Exception` turned the resulting
AttributeError into `meta_agent_failed` -- which reads exactly like the
model declining. The composition is now a seam a test can pin, and a
construction error is re-raised rather than reported as a quiet round.
- `ousast pairs --hunter` crashed in its documented form: the K floor landed
with `--k-runs` still defaulting to 1. The covering test always passed
`--k-runs 4`, which is why nothing noticed.
- 28 rows (12% of the corpus) resolved a span on the vulnerable side and
none on the fixed side, so every fixed-side finding read as outside the
labeled function and the pair scored as silence whatever the detector
said. `fix_function` now reaches the catalog and the scorer, and the
generator derives it from the fixed excerpt for the 25 trap rows that
never declared one. Measured after: 0 rows.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
The five byte-identical vibe-py twins now say so in the catalog instead of
only hashing to it, and a declared limit that names one of the unscorable
vocabulary's own reasons keeps that spelling, so `identical_twin` stays one
fact. Leave-one-out over vibe-py consequently runs on 30 rows, not 35.
The difficulty check measures three axes the corpus varies on: binding hops
from the identity source to the constrained use, registration distance, and
guard distance. Measured first, and it failed: the shared obligations
fixture reached (1, 0, 2) where the vibe-py absence rows reach (5, 2, 2).
Every fixture handler carried its own route decorator, so nothing in the
suite exercised a handler registered from another document and no binding
chain was longer than one hop -- the checker was passing on a simpler world
than the one it meets. `from_spec` closes that with the connexion shape
those rows really have. The verdict is committed beside the numbers.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Eight of nine group-3 tasks were rejected. Each defect was hiding a number,
and the numbers are why the loop was not learning:
- The fixed side was judged against the *vulnerable* side's function
ranges. 191 of 227 pairs with a resolvable labeled span (84%) have
different spans on the two sides, so a leak was recorded as silence,
always in the direction that flatters Youden.
- Round zero measured every family with the generalist. `family:unknown`
earns no credit against any real family, so every family but `unknown`
baselined at a structural zero and every noise floor with it.
- The cross-family sweep counted absolute failures instead of negative
flips. On a corpus where most pairs fail at round zero that rejects
every round for damage it did not do. Unscorable rows voted too.
- K runs was declared and never threaded; `pairs --k-runs` was parsed and
discarded, and its only test grepped `--help`.
- The meta-agent proposer was a stub, so `learning round` could never
propose anything. It now composes MetaAgent.evolve against a scratch
copy of the family directory, and `--proposals FILE` runs a recorded
sequence offline.
- `refused_holdout` was a vocabulary word no code path could produce, and
the covering test accepted either outcome.
- Cost was metered once, after everything had been paid for; a stage that
raised left no record at all.
- The corrected lever number was published beside the earlier one rather
than in place of it, and the test named for that property never checked.
- Refusals were `continue` statements: `Refusal.degradation()` existed and
nothing called it. A train-split identical twin was reported as
`split:train`, which is untrue.
Test repairs that each changed a number: two split tests measured an
`x = 1` fixture that teaches nothing whatever the rule says; two CLI
scoreboard tests asserted against all-zero output produced by a stub that
answers twice and puts its finding outside the labeled function.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 3.1. A teacher is a vendored pair, at a review tier a maintainer stands
behind, on the train split, and scorable. Every learning path now asks the same
module that one question, so the split cannot be enforced in one place and
forgotten in another. A refusal is data rather than an exception: the path
records which holdout pairs a candidate touched and moves on, so a leak is
visible in the run instead of silently accepted.
The exporter refuses to seed from a holdout pair, leave-one-out teaches from the
train split while still scoring every pair, and the mechanism lever refuses a
candidate a holdout pair taught, because such a shape can only recover the pair
that produced it.
Measured and committed: on the vibe-py holdout the lever's own number falls from
seven detected and a Youden of +0.059 to five detected and -0.059 once holdout
pairs stop teaching, and the candidate set shrinks from ten shapes to six. That
gap is the inflation this rule removes.
Two existing tests encoded the leak and now encode the truth. The documented
export-to-improve flow had a single holdout pair teaching itself, and now needs
a train pair to teach and a holdout pair to recover. The obligation export
claimed the corpus seeds at least one obligation shape, which was true only
because holdout pairs were teaching: every vibe-py absence pair that yields an
obligation lesson sits on the holdout side, so the honest number today is zero.
The derivation stays covered by fixtures; the corpus gap belongs to the absence
re-harvest and to a maintainer pass over the splits.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 1.4. The optional extra gains the TypeScript grammar wheel, which ships two
grammars: a file with JSX needs the JSX-aware one and plain TypeScript needs the
other, so the parser picks by file suffix while the parsed file keeps the
language it was given. The walker already treated TypeScript as JavaScript, so
nothing else changes.
Eighteen agent-vfc rows and every TypeScript absence row were counted as
unsupported before this and could not be scored at all. They now parse on both
sides. The core install stays zero-dependency and the extra-free suite still
skips these tests.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 4.2. For an obligation-labeled pair, leave-one-out runs the checker on both
sides with the held-out store's obligation shapes. The pair counts as detected
only when a finding inside the labeled function carries both the labeled
obligation and a known fix from a store record, so the checker's own facts-only
finding is never mistaken for the corpus teaching. Recall, silence and Youden
are reported per obligation kind beside the flow mechanisms.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 3.3. For every finding that carries a route the hunter answers one closed
question, is this route meant to be public, and the answer is recorded as
public, protected or unknown with its rationale. Nothing else changes: not the
label, not the evidence, not the evidence level, and the hunter never authors a
pattern or a policy. The prompt carries the handler's own lines only and is
redacted before it leaves the process. Without a client one degradation is
recorded and the findings are identical.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 2.5. An obligated operation that is reached and has no dominating
discharger is reported at suspicion, labeled by why the obligation exists:
a declared policy clause, the practice of sibling handlers, or, in the degraded
no-path mode, the operation fact alone. With path records and neither a clause
nor a sibling norm nothing justifies the obligation and nothing is reported.
A declared public route waives the path guard wherever it is required, not only
where siblings practice it (Req 4.4). A malformed store row is recorded as a
degradation naming the record instead of being skipped silently.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 2.2. Handlers on the same resource under the same module are siblings;
when at least min_siblings of them discharge an obligation and one does not,
that one is the anomaly and the discharging siblings are its evidence (Req 3.2).
A set with fewer members than the minimum is under-populated and never reported
(Req 3.3). A public route among authenticated siblings is a missing path guard
with the mapper's access classification as evidence (Req 3.4).
The sibling key is the module path without its suffix, so two same-named files
in different directories are never merged. Witness validity (an identity
constraint counts only when it binds the authenticated context) is one helper
shared by operations, siblings and the checker.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 2.4 (Req 5.1, 5.2). Decorator and router witnesses dominate the
handler; a constraint or a non-permissive value dominates the operation
it sits on; a guard call before the operation dominates by contract;
other statement witnesses need an early exit between them and the
operation; along a path record an earlier hop dominates. Guard dominance
replaces the implementation behind the same protocol later.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 2.3 (Req 4.1, 4.2). load_declared_policy: closed top-level,
resource and route fields; access public|authenticated|role; identity
source restricted to the facts' authenticated-context sources; comment-
stripped version hash; trailing-* route globs; unknown fields and values
rejected by name; the tool never writes the file.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 2.1 (Req 3.1, 5.4). find_operations, find_discharges (decorators,
guard and check calls, identity constraints with their provenance,
non-permissive settings) and covers; chained call sites deduped; the
resource token is the receiver head, the fact's resource argument, or a
SQL table name.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 1.3 (Req 2.1, 2.2, 2.4).
- seed.pair_lessons derives obligation shapes for absence-mechanism rows
and falls back to the sink shape when no obligation is learned, so the
flow store is unchanged; skip reasons name the accepted dischargers.
- append_from_pair accepts both shape families (guard carries the
discharger kind); search_tree ignores obligation rows;
obligation_mechanisms selects them.
- Measured on vibe-py: the two vampi pairs seed protected_read /
identity_constraint / authenticated_context.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 1.2 (Req 2.1, 2.3, 2.5).
- semantic/obligations/shapes.py: ObligationShape (family obligation,
text-free key), derive_obligation and explain_skip. The lesson is any
discharged operation of the target's walker-visible kind, fixed side
first and then the vulnerable function's own safe branch, since
Real-Vuln twins are flag-toggled identical functions or unrelated trap
functions; keyword constraints are read from the call's source line
(positional-only IR call sites); an identity constraint bound from the
request or a constant discharges nothing.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Review rounds 3-5 remediations (Req 5, 7).
- ousast improve resolves candidates from the target's calibration dir,
then the working directory's (where `mechanisms export` writes), and
admits into <target>/.openultrasast/calibration/mechanisms.jsonl,
where a scan of that target reads; --mechanism-candidates,
--mechanism-store, --no-mechanisms; --dry-run redirects the store to
scratch; accepted rounds print admit/retract <id>.
- run_improvement threads the store paths; RoundOutcome carries the
mechanism edits; reverted mechanism keys are blocked outright; a
no-gain admission is rejected as mechanism_no_gain; a retraction may
name an admitted record that left the candidate set.
- leave-one-out forwards variant-search degradations and pins the
containment and label rules; mechanisms export defaults to the
candidates file so it never re-admits a tombstone.
- design §Lever rewritten to the implemented gate; README and docs show
export -> improve -> scan.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 4.1 (Req 5.1, 5.2, 5.5).
- validator: MechanismEdit (admit | retract by record id) under the
`mechanisms` lever; validate_mechanism accepts only records from the
exporter candidate set with origin corpus, a closed guard and source
kinds, and identifier-only shape text; free-form shapes are rejected.
- apply_mechanism_edits appends the record (admit) or a retracted
tombstone (retract) so the log stays append-only; StoreSnapshot.restore
reverts byte for byte. MechanismStore.load folds tombstones away.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 3.1 (Req 4.1-4.4).
- semantic/loo.py: every scorable pair is held out once; a temporary
store is seeded from the other seeded/reviewed pairs (advisory and
title pairs are scored as targets, never teach; known_limit skipped);
both excerpts are searched for variants and scored with the pair rules
(detected inside the labeled function naming the labeled mechanism or
sink; silent when the fixed side yields no hit); found_by and leaked_by
name the records and the pairs that taught them; metrics per slice,
profile and mechanism; score_pair_with_store is shared with the lever.
- cli: `pairs --slice S --loo [--loo-out P]` writes loo.json; never a gate.
- First measurement: vibe-py 35 pairs, 11 detected, 21 silent, Youden
-0.086, 16 teaching pairs.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
- variant_search.search_tree skips a malformed store row with a
variants_store_row_invalid degradation instead of aborting the scan;
a hit on a demoted or flowless overlay record is never merged (test).
- seed.pair_lessons is the one shape derivation shared by the exporter
and leave-one-out; labels pass line and sink together; pairs without a
labeled function are reported as no_labeled_function.
- config parses [variants].max_mechanisms with the shared int helper.
- regress.candidate.hotspot_from_finding is the one zero-score hotspot
helper for forced inventory and variant findings.
- design synced (SearchResult, hits_to_findings summaries, candidate.py).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 2.2 (Req 3.1-3.3, 3.6).
- variants.match_shapes: call sites whose trailing callee name and arity
match a shape and whose source positions all carry a source of any
kind; constants and arity mismatches never match.
- semantic/variant_search.py: search_tree over a tree's parsed files with
the store's corpus shapes (bounded by max_mechanisms, language
degradations recorded); hits_to_findings merges a hit into an overlay
promote/coverage record at the same call site (mechanism_id, no
duplicate) and otherwise emits variant:<id>:<path>:<line> at suspicion
with the pair provenance in the rationale.
- OverlayRecord.mechanism_id (additive).
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 2.1 (Req 1.1, 1.2, 1.5, 2.5).
- semantic/seed.py: export_mechanisms seeds only seeded/reviewed,
non-known-limit, parsing pairs with cached excerpts; every skip carries
a reason (tier, known_limit, parse_failed, language unsupported with a
degradation, pointer not cached, no labeled sink call site); runs
offline over the vendored catalog.
- variants.derive_shape: labels that name only a function and a mechanism
resolve to the fact-sink call inside that function (CWE match and a
non-constant argument first); line and sink intersect when both exist;
container reads are subscript or .get( only.
- append_from_pair keeps the strongest review tier and accumulates
provenance tags when a shape is re-seeded.
- CLI: `ousast mechanisms export --slice S [--store P] [--json]`.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 1.2 (Req 1.3, 1.4).
- Mechanism gains origin, review_tier, pairs, shape, guard with defaults;
rows written before this spec load unchanged.
- MechanismStore.load folds the append-only log by id (latest row wins)
so a re-seeded shape carries every pair that taught it.
- append_from_pair writes origin=corpus rows with a deterministic id
derived from the shape key; corpus_mechanisms selects searchable rows.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Task 1.1 (Req 2.1-2.4). Spec approved by the maintainer in session.
- semantic/variants.py: Shape (language, sink trailing name, arity,
source positions and kinds, closed guard kind, mechanism id) with a
stable text-free key; derive_shape finds the labeled sink call site in
the vulnerable FileIR, traces argument identifiers through binds to a
fact source, a parameter or a container read, and classifies the guard
the fixed side added from the labeled function's added statements
against a closed set (none when unknown). No path, line or literal
enters a shape; a constant-only call teaches nothing.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Tasks 1.2, 2.1-2.5 and 6.2 (pair-side filters).
- pairs.py: coverage with a source counts as detection and as leak;
function-scoped matching by line-range containment from the vulnerable
side's parse (unresolvable labels are counted loss, never silent
misses); exact or alias sink matching; weak CWE-only labels never match;
side-by-side inventory/overlay/hunter scorers; per_profile,
per_mechanism, achievable, known_limit and loss payload; loader rejects
unknown mechanism/provenance/split and reserved or missing function
labels on harvested slices; candidate rulesets thread through for the
improve loop.
- overlay.py: OverlayRecord.function (additive) filled from the shared
naming helper in one parse; benchmark.py: ExpectedFinding.mechanism.
- cli pairs: --slice over every slice, --profile, --split, --hunter,
--hunter-model.
- catalogs migrated to the new schema (provenance, split, function,
mechanism, known_limit for owasp-java-hash); baseline recorded in the
spec notes.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Tasks 3.1, 3.2 (harvest library) and the Scorer-owned naming helper.
- semantic/functions.py: one source of function names for scorer, overlay
and catalog generator (declarator wins over parser name; JS/TS bare-param
arrows and undeclared methods are <anon>; C++ Class::method resolves to
its unqualified tail; Perl sub; statements never name).
- benchmarks/pairs/harvest.py: shared library with name / line_range /
hunk / enclosing modes, comment-blind masking (triple-quote aware),
upstream_start headers, code-safe quoted-only redaction.
- benchmarks/pairs/catalog_gen.py: derive function labels and sinks from
the declarator at the mapped sink location, CWE-family sink tokens,
--prune with refusal when nothing derived, fails loud without the
package; vfc/harvest.py and vfc/generate_catalog.py become shims.
- tests/test_pair_harvest.py replaces test_vfc_harvest.py.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
Add openultrasast[semantic] with official grammar wheels and one CST walker
into FileIR. quick stays extra-free. Host CLI is not overlay-ready. Default CI
stays extra-free; a semantic job runs pytest -m semantic.
Regex inventory stays the proposer. MAP attaches promote/demote/unadjudicated/coverage
from intra-file taint (tree-sitter when a grammar parses, Python ast as last resort).
Deep sandboxes promotions only, ordered by the existing heuristic ranker plus optional
OpenRouter mechanism recall. The SAST pair slice scores overlay outcomes; the stage-1
detection gate and local pair gate stay inventory-only.
