From b49c6610d669801fee84bbe0c816c5cba956a635 Mon Sep 17 00:00:00 2001
From: Marius Ciepluch <11855163+norandom@users.noreply.github.com>
Date: Sun, 6 Sep 2026 08:43:47 +0200
Subject: [PATCH] docs(learning-harness): technical design and research log

A learning package layered families -> classify -> scoring -> split ->
detectors and verifiers -> rounds -> publish, composing the existing seams:
the tool hunter gains prompt, context and tool parameters; the hunter path of
the pair scorer takes an injected family matcher so pairs never imports
learning; every learning path selects teachers through one train-only rule;
the regress verdict takes an optional canary oracle; the chat endpoint resolves
DeepSeek before OpenRouter with thinking disabled and reasoning replayed, while
embeddings keep their own endpoint; HarnessX proposes behind the seam with a
scripted proposer for offline runs. Fifty-nine acceptance criteria traced;
boundary sections, file structure plan and migration phases populated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
---
 .kiro/specs/learning-harness/design.md   | 641 +++++++++++++++++++++++
 .kiro/specs/learning-harness/research.md |  90 +++-
 .kiro/specs/learning-harness/spec.json   |  11 +-
 3 files changed, 735 insertions(+), 7 deletions(-)
 create mode 100644 .kiro/specs/learning-harness/design.md

diff --git a/.kiro/specs/learning-harness/design.md b/.kiro/specs/learning-harness/design.md
new file mode 100644
index 0000000..da3e362
--- /dev/null
+++ b/.kiro/specs/learning-harness/design.md
@@ -0,0 +1,641 @@
+# Design Document: learning-harness
+
+## Overview
+
+**Purpose**: Turn OpenUltraSAST from a hand-authored analyzer with a leaking improvement loop into a learning harness: an auto-classifier routes every pair, finding and code region to one of ten verifier-aligned families; one detector configuration per family is evolved one change per round from structured failure facts; a class-aware pair-wise scorer with unscorable denominators and a measured per-family noise floor decides acceptance; only verifier-confirmed claims are reported above suspicion; the split is enforced by construction.
+
+**Users**: the maintainer runs round zero, evolve rounds and the publish command; operators read scan findings that now carry a family, a detector version and a verifier outcome; reviewers work the classifier disagreement queue.
+
+**Impact**: the tool hunter becomes the primary detector and the static machinery becomes tools and verifiers. The pair scorer's hunter path, the exporter, leave-one-out, the mechanism lever and the improve journal change behavior; findings, catalogs and the manifest gain additive fields. Gates stay byte-identical.
+
+### Goals
+- One trustworthy number per family per slice, with its denominator, taxonomy version, noise floor and cost.
+- A round can only be accepted when it helps its target family on held-out pairs and hurts no other family beyond its measured budget.
+- Nothing above `suspicion` without a verifier; verifier code and gold labels outside every editable surface.
+- Round zero on the web families over vibe-py and agent-vfc before any learning.
+
+### Non-Goals
+- Browser-engine, interpreter or renderer bugs; crash oracles for C and C++ (the memory-safety family is measured, never gating).
+- New static shape families, new hand-written facts, evidence-ladder or disposition changes, merge gates on non-local slices, model fine-tuning.
+- A general router framework or a generic agent platform; the classifier is a function and a detector configuration is a directory.
+
+## Boundary Commitments
+
+### This Spec Owns
+- The family taxonomy file, its loader and its version (1.x).
+- Classification of pairs, findings and regions, and the classifier's measurement report and review queue (2.x).
+- Class-aware pair scoring: outcomes, directional bias, fixed-FPR view, K-run reliable-change test, negative flips (3.x).
+- Unscorable accounting and its reasons; the additive `unscorable` and `family` fields on catalog rows (4.x).
+- The single teacher-selection rule every learning path calls, and holdout refusal (5.x).
+- Per-family detector configurations, their loader, the family detector runner and finding tags (6.1–6.5); the chat-endpoint resolution independent of embeddings (6.6).
+- The verifier registry keyed by family, canary oracles, the second-judge step (7.x).
+- Round zero, the noise floor, evolve rounds, acceptance, attribution, the archive, the round journal, cost caps (8.x, 9.x).
+- The corpus repairs listed in 10.x and the publish command with committed artifacts (11.x).
+
+### Out of Boundary
+- Pair labels, tiers, splits and `known_limit` semantics beyond the two additive fields (`pair-corpus-honesty`).
+- Facts, walker, dominance, obligations checker internals; the mechanism store schema; the rule and policy levers' validators.
+- The sandbox runner, snippet safety rules and Docker images; this spec adds an oracle to the verdict, not a runner.
+- HarnessX internals; this spec composes `MetaAgent.evolve` and the journal behind `harness_ext`.
+- Harvesting new pairs; only re-harvesting existing absence recipes.
+
+### Allowed Dependencies
+- `pairs`, `benchmark`, `semantic.loo`, `semantic.seed`, `semantic.mechanisms`, `improve.evolve`, `improve.validator`, `tool_hunter`, `hunter_tools`, `provider.openrouter`, `regress.*`, `sandbox.*`, `redaction`, `harness_ext`, `config`.
+- Optional extras only through `harness_ext` (HarnessX) and `semantic.extra` (tree-sitter, now including TypeScript); every step degrades to a recorded reason without them.
+- Direction: `learning.families` → `learning.classify` → `learning.scoring` → `learning.split` → `learning.detectors` and `learning.verifiers` → `learning.rounds` → `learning.publish` and `cli`. `learning.*` imports the packages above; nothing outside `learning` imports `learning.rounds` or `learning.publish`. `pairs` and `benchmark` never import `learning`.
+
+### Revalidation Triggers
+- A change to the family list, a family's CWE set or its verifier (taxonomy version bump).
+- A change to the pair outcome vocabulary or the reliable-change test.
+- A change to the acceptance rule, the regression-budget definition or the journal schema.
+- A change to the chat-endpoint resolution order or the tool-hunter finding shape.
+- A change to the sandbox verdict contract (oracle callback) or snippet safety rules.
+
+## Architecture
+
+### Existing Architecture Analysis
+- `tool_hunter.run_tool_hunter` is a 4-step loop over three curated tools with findings parsed into `StaticFinding` at `suspicion` (`finding_id = tool-hunter:<path>:<line>`, tag `tool-hunter`). It refuses to emit findings when no tool was called. It has no temperature, budget or context knobs beyond `max_steps`.
+- `pairs._hunter_matches_expected` keys detection on text tokens and `_evaluate_hunter_pair` counts every fixed-side finding as a leak under `fix_policy = silent`; both are replaced on the hunter path only. The overlay and inventory paths and `pair_gate` are untouched.
+- `evaluate_profiles`, `evaluate_mechanism_profiles`, `propose_mechanism_edits`, `evaluate_loo` and `build_pair_signals` filter by tier and vendored status but not by split; `export_mechanisms` seeds from every case it is given.
+- `OpenRouterChatClient` and `OpenRouterEmbeddingClient` read the same `OPENROUTER_BASE_URL`; `complete_chat` always sends `temperature: 0` and drops `reasoning_content` from replies. DeepSeek requires thinking disabled for temperature to apply, `reasoning_content` replayed on tool-call turns, no `/v1` suffix, `json_object` only, no seed.
+- `regress.candidate` derives a verdict from the exit code; snippets are compile-only templates or the model's `proposed_snippet`; `check_snippet_safety` bans sockets, curl and workspace writes, which a canary file in `/scratch` respects.
+- `MechanismStore` is an append-only JSONL with tombstones and deterministic ids, the pattern reused for the archive and the journal.
+- HarnessX 0.1.0 provides `MetaAgent.evolve(current_config, trajectories_dir, output_dir)` with post-flight checks, `JournalEntrySpec(round, label, hypothesis_id, levers, predicted_affected, prose)`, `build_context` with precision discounted by unpredicted regressions, and `RLTask.task_type` routing.
+
+### Architecture Pattern & Boundary Map
+
+```mermaid
+graph TB
+    subgraph Corpus
+        Catalog[pair catalogs]
+        Loader[pairs loader with unscorable and family fields]
+    end
+    subgraph Learning
+        Families[families taxonomy]
+        Classify[classifier three tiers]
+        Scoring[class aware pair scorer]
+        Split[teachers and holdout refusal]
+        Detectors[family detector configs and runner]
+        Verifiers[verifier registry canary static none]
+        Rounds[round zero and evolve rounds]
+        Journal[round journal and archive]
+        Publish[publish command]
+    end
+    subgraph Existing
+        Hunter[tool hunter loop]
+        Chat[chat endpoint client]
+        Regress[regress verdicts and sandbox]
+        Levers[exporter loo mechanism lever]
+        Meta[HarnessX MetaAgent optional]
+    end
+    Catalog --> Loader --> Scoring
+    Families --> Classify --> Scoring
+    Families --> Detectors
+    Families --> Verifiers
+    Classify --> Detectors
+    Split --> Levers
+    Split --> Rounds
+    Detectors --> Hunter --> Chat
+    Verifiers --> Regress
+    Scoring --> Rounds
+    Detectors --> Rounds
+    Verifiers --> Rounds
+    Rounds --> Journal
+    Rounds --> Meta
+    Journal --> Publish
+    Scoring --> Publish
+```
+
+**Architecture Integration**:
+- Selected pattern: a `learning` package layered left to right (families → classify → scoring → split → detectors/verifiers → rounds → publish), composing existing seams rather than replacing them.
+- Domain boundaries: the corpus owns labels and rows; `learning` owns class assignment, scoring, teaching rules, detector configurations, verifier keys and rounds; `regress` owns execution; `harness_ext` owns the HarnessX seam.
+- Existing patterns preserved: append-only JSONL with tombstones for the archive and journal; `HarnessRuntime.run_stage` and degradations for every step; `ScriptedChatClient` for offline tests; `FakeSandboxRunner` for verifier tests.
+- New components rationale: each is one requirement area with one owner; the classifier and scorer are functions, the detector configuration is a directory, the journal is a file.
+- Steering compliance: core `dependencies = []`; recall first under a false-positive ceiling, now per family; gates byte-identical; no model-authored artifact retained without held-out validation.
+
+### Technology Stack
+
+| Layer | Choice / Version | Role in Feature | Notes |
+|-------|------------------|-----------------|-------|
+| Chat endpoint | DeepSeek platform API, OpenAI-compatible, `https://api.deepseek.com` (no `/v1`); models `deepseek-v4-flash` (default), `deepseek-v4-pro` (second judge, review queue) | Detectors, classifier model tier, proposer, judge | Thinking on by default: the client sends `thinking: {"type": "disabled"}` for detectors and the classifier so `temperature` applies; `reasoning_content` replayed on tool-call turns; `response_format json_object` with empty-content retry; `logprobs` requested by the classifier tier only. No seed: determinism is best-effort and absorbed by the noise floor. Pricing (peak, USD/1M): flash 0.44 in / 1.32 out, cache hit 0.014; pro 1.32 / 3.96. Concurrency 2500 (flash). |
+| Embedding endpoint | OpenRouter, `openai/text-embedding-3-small`, unchanged | Prove-order retrieval | Reads `OPENROUTER_*` only; DeepSeek has no embeddings endpoint. |
+| Evolve loop | HarnessX 0.1.0 (`meta_harness.agent.MetaAgent`, `meta_harness.journal`), optional extra | Proposal generation and attribution context | Behind `harness_ext.require_harnessx`; a scripted proposer stands in without it. |
+| Parsing | tree-sitter extra + `tree-sitter-typescript>=0.23` | TypeScript rows become scorable | Optional extra only. |
+| Sandbox | existing Docker CLI runner, images `python:3.12-alpine`, `node:22-alpine` | Canary verifiers | Oracle callback added to the verdict; safety rules unchanged. |
+| Storage | JSONL under `.openultrasast/learning/` and `benchmarks/measurements/` | Journal, archive, noise floors, published artifacts | Append-only, committed when published. |
+
+## File Structure Plan
+
+### Directory Structure
+```
+src/openultrasast/learning/
+├── __init__.py            # public surface: load_families, classify_*, score_pair_family, teachers, run_baseline, run_learning_round, publish
+├── families.py            # Family, FamilyTaxonomy, load_families(ruleset/families.toml), version, verifier kind per family
+├── classify.py            # Classification, classify_pair/finding/region (tiers), measure_classifier, review queue entries
+├── scoring.py             # FamilyOutcome, score_pair_family, FamilyMetrics, directional_bias, negative_flips, reliable_change, fixed_fpr_view, unscorable_reason
+├── split.py               # teachers(cases), refuse_if_holdout(candidate_pairs, cases) -> Refusal | None
+├── detectors.py           # FamilyConfig, load_family_configs, run_family_detector (whole-file context, curated tools), finding tags
+├── endpoint.py            # ChatEndpoint, resolve_chat_endpoint (DeepSeek first, OpenRouter fallback, scripted), DeepSeekChatClient adapter
+├── verifiers.py           # Verifier protocol, CanaryVerifier, StaticVerifier, NoVerifier, verifier_for(family), second_judge
+├── rounds.py              # run_baseline (round zero, noise floor), run_learning_round (propose, minibatch, holdout, sweep, accept/revert), Attribution
+├── proposer.py            # Proposer protocol, ScriptedProposer, HarnessXProposer (MetaAgent.evolve behind harness_ext), failure facts, rejected buffer
+├── journal.py             # RoundRecord, LearningJournal (append-only JSONL), Archive (family -> winning config per pair)
+└── publish.py             # publish(): regenerate numbers from one command, write roadmap table and measurement artifacts
+src/openultrasast/ruleset/families.toml          # closed, versioned taxonomy: id, cwes, mechanisms, verifier, description
+src/openultrasast/learning/configs/<family>/     # round-zero detector configuration per family (prompt.md, checklist.md, tools.toml, budget.toml, hard_negatives.jsonl, counterexamples.jsonl)
+tests/test_learning_families.py, test_learning_classify.py, test_learning_scoring.py, test_learning_split.py, test_learning_detectors.py, test_learning_endpoint.py, test_learning_verifiers.py, test_learning_rounds.py, test_learning_journal.py, test_learning_publish.py, test_fixture_difficulty.py
+```
+
+### Modified Files
+- `src/openultrasast/pairs.py` — `PairCase.unscorable: str | None` computed at load (identical twin via normalized body hash, `known_limit` passthrough); duplicate report across pairs before split; `split_by_repository` helper honoring existing `split` fields and reporting straddles; `_evaluate_hunter_pair` delegates matching and leaks to `learning.scoring` through an injected callable so `pairs` does not import `learning`; `build_pair_signals` gains a `split` filter.
+- `src/openultrasast/benchmark.py` — `ExpectedFinding.family: str | None = None` (additive; validated against the taxonomy when present).
+- `src/openultrasast/tool_hunter.py` — `run_tool_hunter` gains `system_prompt`, `context_files`, `tools`, `max_chars` and `tags` parameters with today's values as defaults; finding parsing accepts an optional `family` field; the reply loop replays `reasoning_content`.
+- `src/openultrasast/provider/openrouter.py` — `OpenRouterChatClient` gains `extra_body` (thinking, response_format, logprobs) and keeps `reasoning_content` on assistant messages; no behavior change for existing callers.
+- `src/openultrasast/config.py` — `[models]` gains `chat_base_url`, `chat_api_key_env`, `judge`; `[learning]` section (`families_path`, `configs_dir`, `k_runs = 5`, `round_cost_cap_usd`, `minibatch`, `cutoff_date`); `load_dotenv` unchanged, tests inject endpoints explicitly.
+- `src/openultrasast/semantic/seed.py`, `semantic/loo.py`, `improve/evolve.py` — teacher selection through `learning.split.teachers`; refusals recorded; `evaluate_loo` teaches from train only.
+- `src/openultrasast/regress/candidate.py`, `regress/verdict.py` — `run_regression` accepts an optional `oracle: Callable[[SandboxResult], bool]`; `verdict_from_result` uses the oracle when given, exit code otherwise.
+- `src/openultrasast/cli.py` — `ousast learning classify|score|baseline|round|publish`; `pairs --hunter` uses the family scorer and `--k-runs`; `[models].hunter` resolution through `learning.endpoint`.
+- `pyproject.toml` — `tree-sitter-typescript>=0.23` in the semantic extra; `src/openultrasast/semantic/extra.py` maps `typescript`/`tsx`.
+- `benchmarks/pairs/vibe-py/catalog.toml` and `recipes.toml` — the five identical twins gain `known_limit = "identical_twin"`; absence rows gain `family` and `mode = "handler_context"` (re-harvest is a maintainer step).
+- `benchmarks/pairs/mechanisms.toml` — unchanged; families reference mechanism ids.
+- `.kiro/steering/roadmap.md` — rewritten by `publish` from artifacts.
+
+## System Flows
+
+### Class-aware scoring of one pair (hunter path)
+
+```mermaid
+sequenceDiagram
+    participant S as Scorer
+    participant C as Classifier
+    participant D as FamilyDetectors
+    participant V as Verifier
+    S->>C: classify_pair(case)
+    C-->>S: families or unknown, tier
+    loop K runs
+        S->>D: run on vuln side and fixed side
+        D-->>S: findings with family tags
+    end
+    S->>S: detection = family match inside labeled function
+    S->>S: silence = no labeled family inside function on fixed side
+    S->>V: verify claims that a verifier exists for
+    V-->>S: suspicion, static corroboration, or proven
+    S-->>S: FamilyOutcome, negative flips, reliable change
+```
+
+Decisions: unscorable pairs short-circuit before any run; K runs share one classification; verifier outcomes never change detection or silence, they change the reported rung.
+
+### One evolve round
+
+```mermaid
+flowchart TB
+    Start[failure facts for target family] --> Propose[proposer drafts one structural change]
+    Propose --> Refuse{touches holdout or exceeds length cap}
+    Refuse -->|yes| Reject[reject and record]
+    Refuse -->|no| Mini[train minibatch of target family]
+    Mini --> MiniOk{no decrease}
+    MiniOk -->|no| Reject
+    MiniOk -->|yes| Hold[target family full holdout]
+    Hold --> HoldOk{train and holdout non negative and one strictly positive}
+    HoldOk -->|no| Reject
+    HoldOk -->|yes| Sweep[every other family holdout]
+    Sweep --> SweepOk{negative flips within each budget}
+    SweepOk -->|no| Reject
+    SweepOk -->|yes| Accept[accept freeze config bump version]
+    Accept --> Journal[journal hypothesis predicted attribution cost]
+    Reject --> Journal
+    Journal --> Buffer[rejected edit buffer updated]
+```
+
+Decisions: the proposer never sees holdout inputs, outputs or scores; cost is metered per stage and the round is reverted when the cap is crossed at any stage; ties reject.
+
+## Requirements Traceability
+
+| Requirement | Summary | Components | Interfaces | Flows |
+|---|---|---|---|---|
+| 1.1–1.4 | closed taxonomy, verifier per family, version, attributes not keys | Families | `load_families`, `FamilyTaxonomy` | — |
+| 2.1–2.5 | three-tier multi-label classifier, measured, review queue, whole corpus | Classifier | `classify_*`, `measure_classifier` | scoring |
+| 3.1–3.6 | family-in-function detection and silence, outcomes, bias, FPR, K runs, no text tokens | Scorer | `score_pair_family`, `FamilyMetrics` | scoring |
+| 4.1–4.5 | unscorable reasons, never delete, dedup, repository split, denominators shown | Loader (pairs), Scorer, Publish | `PairCase.unscorable`, `unscorable_reason`, `split_by_repository` | — |
+| 5.1–5.4 | train-only teachers, refusal, re-measure lever, proposer blind | Split, Levers, Proposer | `teachers`, `refuse_if_holdout` | round |
+| 6.1–6.6 | per-family config, frozen, routing plus generalist, whole file, curated tools, independent chat endpoint | Detectors, Endpoint | `FamilyConfig`, `run_family_detector`, `resolve_chat_endpoint` | scoring |
+| 7.1–7.6 | suspicion unless verified, canary oracles, static corroboration, none, second judge, sealed surface | Verifiers, Regress | `Verifier`, `verifier_for`, `second_judge`, `run_regression(oracle=)` | scoring |
+| 8.1–8.4 | clone, K = 5 noise floor, publish round zero, web families first | Rounds | `run_baseline`, `NoiseFloor` | — |
+| 9.1–9.9 | failure facts, one change, journal, staged evaluation, acceptance, attribution, archive, cost cap, retention | Rounds, Proposer, Journal | `run_learning_round`, `Proposer`, `RoundRecord`, `Archive` | round |
+| 10.1–10.5 | TypeScript, re-harvest, identical twins, fixture difficulty, post-cutoff slice | Extra, Corpus, Tests | `semantic.extra`, catalog fields, `test_fixture_difficulty` | — |
+| 11.1–11.5 | one command, artifacts committed, trajectories persisted, degrade with reason, gates identical | Publish, Journal, Runtime | `publish`, `LearningJournal` | — |
+
+## Components and Interfaces
+
+| Component | Layer | Intent | Req | Key Dependencies | Contracts |
+|---|---|---|---|---|---|
+| Families | data | closed taxonomy with verifier per family | 1.x | ruleset file (P0) | Service |
+| Classifier | logic | assign families with tier, measure itself | 2.x | Families (P0), Endpoint (P1) | Service, State |
+| Scorer | logic | class-aware pair outcomes and statistics | 3.x, 4.x | Families (P0), Detectors (P0), Verifiers (P1) | Service |
+| Split | logic | the one teacher rule and holdout refusal | 5.x | pairs (P0) | Service |
+| Detectors | logic | per-family configuration and runner | 6.1–6.5 | tool_hunter (P0), Endpoint (P0), Classifier (P0) | Service, State |
+| Endpoint | integration | independent chat endpoint resolution and DeepSeek adapter | 6.6 | provider.openrouter (P0), config (P0) | Service |
+| Verifiers | integration | family-keyed oracles and the second judge | 7.x | regress, sandbox (P0), obligations (P1) | Service |
+| Rounds | orchestration | round zero, noise floor, evolve rounds, acceptance | 8.x, 9.x | Scorer, Detectors, Proposer, Journal (P0) | Batch, State |
+| Proposer | integration | structural change proposals from failure facts | 9.1–9.2 | harness_ext (P1), Endpoint (P1) | Service |
+| Journal | data | round records, attribution, archive | 9.3, 9.6, 9.7, 11.3 | JSONL (P0) | State |
+| Publish | orchestration | numbers from one command, artifacts, roadmap | 11.x | Scorer, Journal (P0) | Batch |
+
+### Data layer
+
+#### Families
+
+| Field | Detail |
+|---|---|
+| Intent | Load and version the closed family taxonomy with its CWE set, mechanism set and verifier kind |
+| Requirements | 1.1, 1.2, 1.3, 1.4 |
+
+**Responsibilities & Constraints**
+- Exactly ten families as decided; ids are stable identifiers; the file carries `version`.
+- Verifier kind per family is one of `canary`, `static`, `none`; memory safety is `none`.
+- No component may route or score on a CWE or mechanism id; they are looked up only through a family.
+
+**Dependencies**: Inbound: Classifier, Scorer, Detectors, Verifiers (P0). External: `ruleset/families.toml` (P0).
+
+**Contracts**: Service [x]
+
+```python
+@dataclass(frozen=True)
+class Family:
+    id: str                      # e.g. "injection", "path", "deserialization", "access_control", "output_encoding", "ssrf", "prototype", "config_secrets", "memory", "unknown"
+    cwes: frozenset[str]
+    mechanisms: frozenset[str]
+    verifier: Literal["canary", "static", "none"]
+    description: str
+
+@dataclass(frozen=True)
+class FamilyTaxonomy:
+    version: str
+    families: tuple[Family, ...]
+    def by_id(self, family_id: str) -> Family: ...
+    def family_of_cwe(self, cwe: str) -> Family | None: ...
+    def family_of_mechanism(self, mechanism: str) -> Family | None: ...
+    def related(self, a: str, b: str) -> Literal["same", "parent_child", "lateral", "fabricated"]: ...
+
+def load_families(path: Path | None = None) -> FamilyTaxonomy   # raises FamiliesError naming the field on any deviation
+```
+- Invariant: `unknown` is always present and never has a verifier.
+
+#### Journal
+
+| Field | Detail |
+|---|---|
+| Intent | Append-only record of rounds with attribution, and the archive of which configuration wins which pair |
+| Requirements | 9.3, 9.6, 9.7, 11.3 |
+
+**Contracts**: State [x]
+
+```python
+@dataclass(frozen=True)
+class RoundRecord:
+    round: int
+    family: str
+    hypothesis: str
+    levers: tuple[str, ...]           # "tool" | "checklist" | "counterexample" | "memory" | "prompt"
+    predicted_affected: tuple[str, ...]
+    predicted_at_risk: tuple[str, ...]
+    outcome: Literal["accepted", "rejected", "reverted_cost", "refused_holdout"]
+    reason: str
+    target_train_delta: float
+    target_holdout_delta: float
+    sweep: dict[str, int]             # family -> negative flips
+    attribution: Attribution          # flipped_predicted, flipped_unpredicted, precision
+    cost_usd: float
+    taxonomy_version: str
+    config_version: str
+
+class LearningJournal:                 # .openultrasast/learning/journal.jsonl
+    def append(self, record: RoundRecord) -> None: ...
+    def rounds(self) -> tuple[RoundRecord, ...]: ...
+    def rejected_buffer(self, family: str) -> tuple[str, ...]: ...   # hypotheses rejected for this family
+
+class Archive:                          # .openultrasast/learning/archive.jsonl
+    def record(self, family: str, config_version: str, pair: str, outcome: FamilyOutcome) -> None: ...
+    def winners(self, family: str) -> dict[str, str]: ...             # pair -> config version that scores it best
+```
+- Persistence: JSONL, one record per line, never rewritten; the HarnessX journal is mirrored when the extra is present, never the source of truth.
+
+### Logic layer
+
+#### Classifier
+
+| Field | Detail |
+|---|---|
+| Intent | Assign zero or more families with the deciding tier; measure itself; queue disagreements |
+| Requirements | 2.1, 2.2, 2.3, 2.4, 2.5 |
+
+**Responsibilities & Constraints**
+- Tier 1 (deterministic): rule id, sink family from facts, obligation kind, mechanism id, labeled CWE; any hit decides without a model.
+- Tier 2 (model): `json_object` question with the taxonomy definitions, `thinking` disabled, `logprobs` requested; averaged log-probability under a threshold yields `unknown`.
+- Tier 3: `unknown`.
+- Never writes a label; disagreements go to `.openultrasast/learning/review-queue.jsonl`.
+
+**Dependencies**: Inbound: Scorer, Detectors (P0). Outbound: Families (P0), Endpoint (P1, degrades to tier 1 + unknown without a client).
+
+**Contracts**: Service [x] State [x]
+
+```python
+@dataclass(frozen=True)
+class Classification:
+    families: tuple[str, ...]          # () means unknown
+    tier: Literal["static", "model", "none"]
+    confidence: float | None
+
+def classify_pair(case: PairCase, taxonomy: FamilyTaxonomy, *, client: ChatClient | None) -> Classification
+def classify_finding(finding: StaticFinding, taxonomy: FamilyTaxonomy, *, client: ChatClient | None) -> Classification
+def classify_region(path: str, text: str, function: str | None, taxonomy: FamilyTaxonomy, *, facts, client: ChatClient | None) -> Classification
+
+@dataclass(frozen=True)
+class ClassifierReport:
+    taxonomy_version: str
+    agreement: float                   # hierarchical credit vs maintainer labels on reviewed pairs
+    confusion: dict[str, dict[str, int]]
+    random_router: float
+    oracle_router: float
+    per_tier: dict[str, int]
+    unknown: int
+    queued: int
+
+def measure_classifier(cases: Sequence[PairCase], taxonomy: FamilyTaxonomy, *, client: ChatClient | None) -> ClassifierReport
+```
+- Postcondition: every reviewed pair whose label disagrees with the classification appears once in the review queue with both answers and the tier.
+
+#### Scorer
+
+| Field | Detail |
+|---|---|
+| Intent | Score one pair per family from detector findings, aggregate per family and slice, decide reliable change |
+| Requirements | 3.1–3.6, 4.1, 4.5 |
+
+**Responsibilities & Constraints**
+- Detection: a finding whose `family:` tag relates `same` or `parent_child` to the labeled family and whose line lies in the labeled function; `fabricated` scores the maximum penalty; text is never inspected.
+- Silence: no `same`/`parent_child` finding of the labeled family inside the labeled function on the fixed side; other findings are counted in `other_findings`, never as leaks.
+- K runs per pair; reliable change against the last blessed round via a paired test on per-pair outcomes; negative flips per family.
+
+**Contracts**: Service [x]
+
+```python
+FamilyOutcome = Literal["pair_correct", "both_flagged", "both_silent", "reversed", "unscorable"]
+
+@dataclass(frozen=True)
+class PairFamilyScore:
+    pair: str
+    family: str
+    runs: tuple[FamilyOutcome, ...]    # length K
+    outcome: FamilyOutcome             # majority of runs
+    other_findings_vuln: int
+    other_findings_fixed: int
+    rung: Literal["suspicion", "static_corroboration", "proven"]
+    unscorable_reason: str | None
+
+@dataclass(frozen=True)
+class FamilyMetrics:
+    family: str
+    scorable: int
+    unscorable: dict[str, int]         # reason -> count
+    outcomes: dict[FamilyOutcome, int]
+    recall: float
+    silence: float
+    youden: float
+    directional_bias: float            # (both_flagged - both_silent) / scorable
+    fixed_fpr_recall: float            # recall at the family's FPR ceiling
+    negative_flips: int                # vs last blessed round
+    reliable_change: bool
+    taxonomy_version: str
+
+def score_pair_family(case: PairCase, family: str, vuln_runs: Sequence[Sequence[StaticFinding]], fixed_runs: Sequence[Sequence[StaticFinding]], *, ranges, taxonomy) -> PairFamilyScore
+def aggregate(scores: Sequence[PairFamilyScore], *, baseline: Sequence[PairFamilyScore] | None) -> dict[str, FamilyMetrics]
+def unscorable_reason(case: PairCase, *, parse_ok: bool, ranges) -> str | None   # identical_twin | unsupported_language | unresolved_label | missing_context | known_limit:<x>
+```
+
+#### Split
+
+| Field | Detail |
+|---|---|
+| Intent | The one rule that selects teachers, and the refusal for anything touching holdout |
+| Requirements | 5.1, 5.2, 5.3, 5.4 |
+
+**Contracts**: Service [x]
+
+```python
+def teachers(cases: Sequence[PairCase]) -> tuple[PairCase, ...]                 # vendored, gating tier, split == "train", scorable
+def refuse_if_holdout(taught_by: Iterable[str], cases: Sequence[PairCase]) -> Refusal | None   # Refusal(pairs=..., reason="holdout_pair")
+```
+- `export_mechanisms`, `evaluate_loo`, `evaluate_profiles`, `evaluate_mechanism_profiles`, `propose_mechanism_edits` and `build_pair_signals` obtain teachers only through `teachers`; refusals are recorded as degradations `{"stage": "learning", "reason": "holdout_pair_refused", "pairs": [...]}`.
+
+#### Detectors
+
+| Field | Detail |
+|---|---|
+| Intent | Load frozen per-family configurations and run the family detector with whole-file context and curated tools |
+| Requirements | 6.1, 6.2, 6.3, 6.4, 6.5 |
+
+**Responsibilities & Constraints**
+- A configuration is a directory `learning/configs/<family>/` with `prompt.md` (length cap enforced at load), `checklist.md`, `tools.toml` (subset of `read_file`, `grep_repo`, `find_refs`, `read_definition`, `entry_points`, `flows`, `obligations`), `budget.toml` (`max_steps`, `max_cost_usd`, `max_chars`), `hard_negatives.jsonl`, `counterexamples.jsonl`, `version`.
+- Round zero writes every family's directory from the current hunter prompt; a round rewrites only its target family and bumps that `version`.
+- The runner builds the user message from the whole file plus resolved definitions, runs `run_tool_hunter` with the family's tools and prompt, and tags findings `family:<id>`, `detector:<id>@<version>`; the generalist is the `unknown` family's configuration.
+
+**Contracts**: Service [x] State [x]
+
+```python
+@dataclass(frozen=True)
+class FamilyConfig:
+    family: str
+    version: str
+    prompt: str
+    checklist: str
+    tools: tuple[str, ...]
+    max_steps: int
+    max_cost_usd: float
+    max_chars: int
+    hard_negatives: tuple[dict[str, object], ...]
+    counterexamples: tuple[dict[str, object], ...]
+
+def load_family_configs(configs_dir: Path, taxonomy: FamilyTaxonomy) -> dict[str, FamilyConfig]   # raises ConfigError on cap or tool violations
+def run_family_detector(root: Path, region: Region, config: FamilyConfig, *, client: ChatClient, model: str) -> list[StaticFinding]
+```
+- Precondition: `region.families` from the classifier; postcondition: every finding carries exactly one `family:` tag and one `detector:` tag.
+
+#### Endpoint
+
+| Field | Detail |
+|---|---|
+| Intent | Resolve the chat endpoint independently of the embedding endpoint and adapt DeepSeek quirks |
+| Requirements | 6.6 |
+
+**Responsibilities & Constraints**
+- Resolution order: explicit injection (tests) → `OPENULTRASAST_HUNTER_CLIENT` scripted modes → `DEEPSEEK_API_KEY` with `DEEPSEEK_BASE_URL` (default `https://api.deepseek.com`) → `OPENROUTER_API_KEY` with `OPENROUTER_BASE_URL`. Embeddings keep reading `OPENROUTER_*` only.
+- `[models]`: `hunter` (default `deepseek-v4-flash`), `judge` (default `deepseek-v4-pro`), `chat_base_url`, `chat_api_key_env`.
+- Adapter: `thinking` disabled and `temperature 0` for detectors and classifier; `reasoning_content` replayed; `response_format json_object` with one retry on empty content; `logprobs` on request; usage fields normalized (`prompt_cache_hit_tokens`) into cost accounting.
+
+**Contracts**: Service [x]
+
+```python
+@dataclass(frozen=True)
+class ChatEndpoint:
+    base_url: str
+    api_key: str
+    provider: Literal["deepseek", "openrouter", "scripted"]
+    thinking: bool
+    price_in_per_m: float
+    price_out_per_m: float
+
+def resolve_chat_endpoint(config: ResolvedConfig, *, override: ChatClient | None = None) -> tuple[ChatClient, ChatEndpoint] | None
+class DeepSeekChatClient:   # implements ChatClient; wraps OpenRouterChatClient with extra_body and reasoning_content replay
+    def complete(self, *, model, messages, tools, timeout_seconds=60, json_object=False, logprobs=False) -> ChatResponse
+```
+
+#### Verifiers
+
+| Field | Detail |
+|---|---|
+| Intent | Confirm a claim through the family's oracle, or refuse to raise it |
+| Requirements | 7.1–7.6 |
+
+**Responsibilities & Constraints**
+- `CanaryVerifier` (injection, path, deserialization, and command execution inside injection): writes a snippet from a family template that imports the target function, plants a canary (row in an in-memory database, file under `/scratch`, object with a marker method) and prints `CANARY:<token>` only when the vulnerability fires; the oracle is the token in stdout, never the exit code. The snippet passes `check_snippet_safety` unchanged.
+- `StaticVerifier` (access control): the obligations checker's finding on the same function raises to `static_corroboration`.
+- `NoVerifier`: stays at `suspicion`.
+- `second_judge`: a `deepseek-v4-pro` call with the snippet, the oracle output and the claim, answering `json_object` `{"confirms": bool, "reason": str}`; a proven claim is published only when it confirms.
+- Templates, oracle code and gold labels live under `learning/verifiers/` and `benchmarks/`, both excluded from every proposer write root.
+
+**Contracts**: Service [x]
+
+```python
+class Verifier(Protocol):
+    kind: Literal["canary", "static", "none"]
+    def verify(self, claim: StaticFinding, *, root: Path, language: str, runner: SandboxRunner, limits: SandboxConfig) -> Verification
+
+@dataclass(frozen=True)
+class Verification:
+    rung: Literal["suspicion", "static_corroboration", "proven"]
+    oracle_output: str
+    reason: str
+
+def verifier_for(family: Family) -> Verifier
+def second_judge(verification: Verification, claim: StaticFinding, *, client: ChatClient, model: str) -> bool
+```
+- `regress.run_regression(..., oracle=Callable[[SandboxResult], bool] | None)`: when given, `TRIGGERABLE` iff the oracle returns true; exit-code behavior unchanged otherwise.
+
+### Orchestration layer
+
+#### Rounds
+
+| Field | Detail |
+|---|---|
+| Intent | Round zero with noise floors; evolve rounds with staged evaluation, acceptance, attribution and revert |
+| Requirements | 8.1–8.4, 9.1–9.9 |
+
+**Responsibilities & Constraints**
+- `run_baseline`: clone the current hunter prompt into every family directory, run K = 5 over scorable vendored pairs of the selected slices, compute per-family negative-flip rate between runs as `NoiseFloor`, persist `noise-floors.json`, publish round zero.
+- `run_learning_round`: failure facts for the target family → proposer → refusal checks (holdout, length cap, allowed levers) → minibatch → holdout → sweep → accept (freeze, bump version, archive) or revert (restore the directory snapshot) → journal → rejected buffer.
+- Acceptance: `train_delta >= 0 and holdout_delta >= 0 and (train_delta > 0 or holdout_delta > 0) and all(sweep[f] <= floor[f])`; ties reject.
+- Cost metered per stage from usage; crossing `round_cost_cap_usd` at any stage reverts and records `reverted_cost`.
+- The proposer receives no holdout material; scoring of holdout happens after the proposal is frozen.
+
+**Contracts**: Batch [x] State [x]
+
+```python
+@dataclass(frozen=True)
+class NoiseFloor:
+    family: str
+    k_runs: int
+    negative_flip_rate: float
+    budget_flips: int                  # ceil(rate * scorable pairs)
+
+def run_baseline(cases, *, slices, taxonomy, configs_dir, client, model, k_runs=5, out_dir) -> BaselineReport
+def run_learning_round(cases, *, family, taxonomy, configs_dir, proposer, client, model, journal, archive, floors, cost_cap_usd, minibatch) -> RoundRecord
+```
+- Idempotency: a round directory is written under `.openultrasast/learning/rounds/<n>/` with the snapshot, proposal, scores and trajectories; rerunning a round number refuses.
+
+#### Proposer
+
+| Field | Detail |
+|---|---|
+| Intent | Draft one structural change for one family from structured failure facts |
+| Requirements | 9.1, 9.2, 5.4 |
+
+**Contracts**: Service [x]
+
+```python
+@dataclass(frozen=True)
+class FailureFacts:
+    family: str
+    misses: tuple[MissFact, ...]       # pair, function, sink_or_check, line, verifier_outcome, runs
+    leaks: tuple[LeakFact, ...]
+    rejected: tuple[str, ...]          # hypotheses from the buffer
+    config: FamilyConfig
+
+@dataclass(frozen=True)
+class Proposal:
+    hypothesis: str
+    lever: Literal["tool", "checklist", "counterexample", "memory", "prompt"]
+    change: dict[str, object]          # file-level edit inside the family directory
+    predicted_affected: tuple[str, ...]
+    predicted_at_risk: tuple[str, ...]
+
+class Proposer(Protocol):
+    def propose(self, facts: FailureFacts) -> Proposal | None
+
+class ScriptedProposer: ...            # tests and offline runs
+class HarnessXProposer: ...            # MetaAgent.evolve with allowed_write_roots = the family directory only; degrades to None with a recorded reason without the extra
+```
+
+#### Publish
+
+| Field | Detail |
+|---|---|
+| Intent | Regenerate every published number from one command and commit its artifacts |
+| Requirements | 11.1, 11.2, 4.5 |
+
+**Contracts**: Batch [x]
+
+```python
+def publish(cases, *, taxonomy, journal, floors, out_dir=Path("benchmarks/measurements"), roadmap=Path(".kiro/steering/roadmap.md")) -> PublishReport
+```
+- Writes `<date>-<slice>-<family>.json` artifacts, a markdown table with scorable/unscorable counts, taxonomy version, noise floor and cost per point beside every number, and replaces the roadmap's measured section between markers.
+
+## Data Models
+
+### Domain Model
+- Aggregates: `FamilyTaxonomy` (versioned, immutable per version); `FamilyConfig` (per family, versioned, frozen between rounds); `RoundRecord` (append-only); `PairCase` (owned by the corpus, gains two additive fields).
+- Invariants: a finding carries exactly one `family:` tag; a pair has at most one `unscorable` reason; a round targets exactly one family; the archive maps a pair to one winning configuration per family.
+
+### Data Contracts
+- Catalog row additions: `[[pair.expected]] family = "<id>"` (validated against the taxonomy), `known_limit = "identical_twin"` for the five twins; `unscorable` is computed, never stored.
+- Manifest block `learning = {taxonomy_version, families: {id: FamilyMetrics}, classifier: ClassifierReport, round: n | null}`; findings carry tags `family:`, `detector:`, `verifier:<rung>`.
+- Journal and archive JSONL as above; `noise-floors.json`; `review-queue.jsonl` entries `{pair, label_family, classified, tier, confidence}`.
+
+## Error Handling
+
+- Taxonomy or configuration deviations fail fast at load with the field named (`FamiliesError`, `ConfigError`).
+- Missing extras degrade with a recorded reason: no HarnessX → `ScriptedProposer` or `proposer_unavailable`; no TypeScript grammar → rows `unsupported_language`; no sandbox → `NoVerifier` for canary families with `sandbox_unavailable`.
+- Endpoint errors: transient retries as today; empty `json_object` content retried once then recorded; a round whose endpoint fails mid-stage is reverted with `endpoint_failed`.
+- Holdout refusal, cost overrun and safety rejection are outcomes, not exceptions, and always land in the journal.
+- Monitoring: every stage runs under `HarnessRuntime.run_stage`; cost per stage and per round is written to the round directory.
+
+## Testing Strategy
+
+- Unit: taxonomy loader rejects an eleventh family and a missing verifier kind (1.1, 1.2); `related` returns `parent_child` for a CWE child and `fabricated` for an unknown id (3.1); classifier tier 1 decides from a rule id without a client and tier 2 falls to `unknown` under the threshold with a scripted client (2.1, 2.2); scorer counts detection only inside the labeled function and never from text (3.1, 3.6); silence ignores other-family findings (3.2); directional bias and negative flips on synthetic run sets (3.3, 3.5); `unscorable_reason` on an identical twin and an unsupported language (4.1); `teachers` excludes holdout and `refuse_if_holdout` names the pair (5.1, 5.2); configuration loader rejects a prompt over the cap and a tool outside the allowed set (6.1, 6.5); endpoint resolution prefers DeepSeek over OpenRouter and never touches the embedding variables (6.6); canary snippet passes safety and the oracle keys on the token (7.2); acceptance rule rejects ties and budget breaches (9.5); journal round-trips a record and the rejected buffer (9.3, 9.6).
+- Integration: `pairs --hunter` with a scripted client on the shared handler fixture yields family outcomes and the manifest block (3.x, 6.3); `export_mechanisms` and `evaluate_loo` teach from train only and record refusals (5.1, 5.3); `run_baseline` with a scripted client writes noise floors and round-zero artifacts (8.x); `run_learning_round` with `ScriptedProposer` accepts a helpful change, rejects a tie, reverts on cost and journals attribution (9.x); `run_regression(oracle=)` returns `TRIGGERABLE` on the token with `FakeSandboxRunner` (7.2).
+- Corpus: TypeScript rows parse under the extra (10.1); the five twins load as `known_limit:identical_twin` and are excluded from denominators (10.3, 4.1); `test_fixture_difficulty` fails on a fixture easier than its named rows (10.4); post-cutoff slice reported separately (10.5).
+- Gates: `gate`, `map_gate`, `pair_gate` stdout byte-identical to the pre-feature commit; extra-free suite green with semantic and harnessx tests skipping (11.5).
+- Live (maintainer, not CI): round zero on vibe-py and agent-vfc web families through DeepSeek, artifacts committed under `benchmarks/measurements/`.
+
+## Security Considerations
+- Every prompt and every persisted trajectory passes `redact_secrets`; verifier templates, oracle code and gold labels are outside the proposer's write roots; snippets keep today's safety bans; the sandbox stays network-off and workspace read-only; the second judge sees the snippet and oracle output, never the labels.
+
+## Performance & Cost
+- Round zero: K = 5 over ~250 vendored pairs, both sides, whole-file context ≈ 25M input tokens; at DeepSeek flash peak pricing under $15, off-peak half. A round: minibatch + holdout + sweep ≈ one baseline pass; `round_cost_cap_usd` default 10. Concurrency bounded by the runtime's existing budget controls.
+
+## Migration Strategy
+- Phase 1 (offline, no model): taxonomy, classifier tier 1, scorer, split, unscorable rows, endpoint resolution, TypeScript grammar, identical twins, fixture-difficulty check; existing numbers re-published with denominators.
+- Phase 2 (live): round zero on web families; noise floors; classifier tier 2 measured; review queue worked.
+- Phase 3: verifiers for the four canary families and the static verifier; second judge.
+- Phase 4: evolve rounds with `ScriptedProposer` first, then `HarnessXProposer`; obligations lever as the access-control family's first round.
+- Rollback: every round is a directory snapshot; `publish` regenerates from artifacts, so a bad round is reverted by restoring the family directory and re-publishing.
diff --git a/.kiro/specs/learning-harness/research.md b/.kiro/specs/learning-harness/research.md
index 82379e4..5f661d1 100644
--- a/.kiro/specs/learning-harness/research.md
+++ b/.kiro/specs/learning-harness/research.md
@@ -1,6 +1,92 @@
-# Research: learning-harness
+# Research & Design Decisions: learning-harness
 
-Web research conducted 2026-09-06 in four threads by research subagents; summaries are theirs, condensed, with primary sources. Speculation is marked. This file backs the "What others learned" section of `brief.md`.
+## Summary
+
+- **Feature**: `learning-harness`
+- **Discovery Scope**: Complex Integration (extension of pairs, tool hunter, provider, regress, improve; adoption of HarnessX meta-harness; corpus repairs)
+- **Key Findings**:
+  - The tool hunter already beats the overlay 12/14 vs 5/14 on the class-aware, scorable vibe-py holdout; the published 0/17 was the text-token scorer.
+  - Every learning path ignores the split (`evaluate_profiles`, `evaluate_mechanism_profiles`, `propose_mechanism_edits`, `evaluate_loo`, `build_pair_signals`, `export_mechanisms`); one teacher rule fixes all of them.
+  - The sandbox verdict is exit-code only and snippets are compile-only templates; a canary oracle is an additive callback that respects the existing safety bans.
+  - The chat and embedding clients share one base-URL variable; DeepSeek needs thinking disabled, `reasoning_content` replay, no `/v1`, `json_object` only, and offers no seed and no embeddings.
+  - HarnessX 0.1.0 already provides the round primitive (`MetaAgent.evolve`, journal with `predicted_affected` and precision discounted by unpredicted regressions); the tool supplies tasks, trajectories, the evaluator and routing.
+  - The field routes on coarse verifier-aligned families, gates reports on proof, and reports regression blindness in meta-agents; per-instance archives and no-regression acceptance rules are the published remedies.
+
+## Research Log
+
+### Codebase seams (2026-09-06, Explore subagent over `src/openultrasast`)
+- **Context**: design an extension without re-reading every module.
+- **Findings**: `run_tool_hunter(root, hotspots, *, client, model, max_steps)` with three curated tools and `suspicion` findings tagged `tool-hunter`; `ChatClient.complete(*, model, messages, tools, timeout_seconds)`; `OpenRouterChatClient.complete_chat` always sends `temperature: 0` and drops `reasoning_content`; `_hunter_matches_expected` is text-token based and every fixed-side finding leaks under `fix_policy = silent`; `run_regression(..., sandbox, sandbox_limits, images)` with `verdict_from_result` on exit code; `check_snippet_safety` bans sockets, curl and workspace writes; `MechanismStore` append-only JSONL with tombstones; `improve.run_round` journal has no hypothesis, attribution or cost fields and its regression check is per provenance profile; `load_dotenv` no-ops under pytest; `pyproject` semantic extra lacks TypeScript; `pair_gate` scores only vendored `local` pairs.
+- **Implications**: the design adds parameters and additive fields instead of replacing modules; the family scorer is injected into the hunter path so `pairs` never imports `learning`; the oracle is an optional callback on `run_regression`; endpoint resolution takes an explicit override for tests.
+
+### DeepSeek platform API (2026-09-06, api-docs.deepseek.com)
+- **Findings**: base `https://api.deepseek.com` (no `/v1` in current docs), Anthropic-shape endpoint at `/anthropic`; models `deepseek-v4-flash`, `deepseek-v4-pro`, `deepseek-v4-flash-vision-exp`; 1M context, 384K output; thinking on by default (`thinking: {"type": "disabled"}` to turn off; `temperature` ignored while thinking); tools with `tool_choice`, `reasoning_content` must be replayed on tool turns; `response_format json_object` only, occasional empty content; `logprobs`/`top_logprobs` documented, unverified in thinking mode; no `seed`, `n`, `logit_bias`; flash 0.44/1.32 USD per M in/out peak, cache hit 0.014, pro 1.32/3.96, off-peak half; concurrency 2500 flash / 500 pro; no embeddings endpoint.
+- **Implications**: detectors and classifier run with thinking disabled and temperature 0; determinism is best-effort and the K-run noise floor absorbs it; the classifier's model tier uses `logprobs` averaged and falls to `unknown`; embeddings stay on OpenRouter; cost accounting reads DeepSeek's cache-hit usage fields.
+
+### Measurement that triggered the spec (2026-09-06)
+- See `brief.md` Problem section and `benchmarks/measurements/2026-09-06-*`.
+
+## Architecture Pattern Evaluation
+
+| Option | Description | Strengths | Risks / Limitations | Notes |
+|--------|-------------|-----------|---------------------|-------|
+| Keep static machinery as the detector, add a class field | Minimal change | Cheap | Ceiling by construction; measured 5/14 vs 12/14 | Rejected |
+| One generalist hunter, evolved as a whole | Single config | Simple loop | Aggregate fitness hides per-class regressions; the maintainer's oscillation observation; regression blindness in the literature | Rejected |
+| Per-family detector set with auto-classifier, per-family attribution, verifier-gated reporting | This design | Matches AIxCC routing practice, GEPA per-instance archives, slice-based gating | Router error (63.8% in MoEVD); mitigated by multi-label routing plus a generalist and by measuring the router separately | **Selected** |
+| Fine-tune a model per class | Weights | Potentially strong | Out of scope; label noise 25–60% in public sets; not reproducible offline | Rejected |
+
+## Design Decisions
+
+### Decision: families are the only routing and scoring key
+- **Context**: 1.4, 3.6; text tokens and CWE ids were the proxy.
+- **Alternatives**: fine CWE routing; mechanism ids as key.
+- **Selected**: ten coarse families with CWE and mechanism as attributes; hierarchical relation for partial credit.
+- **Rationale**: fine CWE from code is unsolved (≤14.7% top-1) and coarse families are what worked in AIxCC and MoEVD.
+- **Trade-offs**: less granular reporting; compensated by attributes on findings.
+
+### Decision: the scorer is injected into the hunter path, not imported by `pairs`
+- **Context**: dependency direction; `pairs` is owned by `pair-corpus-honesty`.
+- **Selected**: `_evaluate_hunter_pair` accepts a matcher/leak callable; the CLI wires `learning.scoring`.
+- **Trade-offs**: one more parameter; keeps `pairs` free of `learning` and the overlay path unchanged.
+
+### Decision: canary oracle as an optional callback on `run_regression`
+- **Context**: 7.2; exit code cannot express "the vulnerability fired".
+- **Alternatives**: a parallel verifier runner; parsing stderr.
+- **Selected**: `oracle: Callable[[SandboxResult], bool] | None`; token in stdout.
+- **Trade-offs**: minimal change to regress; templates per family live in `learning/verifiers/`.
+
+### Decision: HarnessX proposer behind the seam, scripted proposer first
+- **Context**: 9.x with core `dependencies = []`.
+- **Selected**: `Proposer` protocol; `HarnessXProposer` composes `MetaAgent.evolve` with `allowed_write_roots` = the family directory; `ScriptedProposer` for tests and phase 4 start.
+- **Trade-offs**: two proposers; the loop is testable offline and the extra stays optional.
+
+### Decision: noise floor from K = 5 baseline runs is the regression budget
+- **Context**: 8.2, 9.5; no public numeric negative-flip budget exists.
+- **Selected**: per-family negative-flip rate between seed runs; `budget_flips = ceil(rate × scorable)`.
+- **Trade-offs**: a noisy family gets a loose budget; reported beside the number so it is visible.
+
+### Decision: DeepSeek adapter wraps the existing OpenRouter client
+- **Context**: 6.6; both clients share one base URL today.
+- **Selected**: `ChatEndpoint` resolution order DeepSeek → OpenRouter → scripted; adapter adds `extra_body` and `reasoning_content` replay; embeddings untouched.
+- **Trade-offs**: one adapter class; no new dependency.
+
+## Risks & Mitigations
+- Router error routes a region to the wrong family → multi-label routing plus the generalist; router measured separately against random and oracle.
+- No seed on DeepSeek → K runs and the reliable-change test; determinism recorded as best-effort.
+- Canary templates cannot express some families (XSS, SSRF) → those report at `suspicion` only, stated in the taxonomy.
+- Proposer confabulation → structured failure facts only; rejected buffer; length cap; structural levers preferred.
+- Cost drift → per-stage metering, per-round cap with revert, cost per point published.
+- Re-harvest needs network and licenses → maintainer step; excerpts re-redacted; failures leave the old excerpt and a recorded reason.
+
+## References
+- Codebase seams and DeepSeek facts: this file, sections above.
+- External literature: the four threads below.
+
+---
+
+## External research (2026-09-06)
+
+Web research conducted 2026-09-06 in four threads by research subagents; summaries are theirs, condensed, with primary sources. Speculation is marked. This section backs the "What others learned" section of `brief.md`.
 
 ## 1. Cyber reasoning systems (AIxCC finals 2025, Big Sleep, XBOW, Glasswing)
 
diff --git a/.kiro/specs/learning-harness/spec.json b/.kiro/specs/learning-harness/spec.json
index 36df784..83fde80 100644
--- a/.kiro/specs/learning-harness/spec.json
+++ b/.kiro/specs/learning-harness/spec.json
@@ -1,16 +1,16 @@
 {
   "feature_name": "learning-harness",
   "created_at": "2026-09-06T06:32:41Z",
-  "updated_at": "2026-09-06T06:34:14Z",
+  "updated_at": "2026-09-06T06:43:38Z",
   "language": "en",
-  "phase": "requirements-generated",
+  "phase": "design-generated",
   "approvals": {
     "requirements": {
       "generated": true,
-      "approved": false
+      "approved": true
     },
     "design": {
-      "generated": false,
+      "generated": true,
       "approved": false
     },
     "tasks": {
@@ -25,5 +25,6 @@
     "authorization-obligations",
     "tree-sitter-overlay-extra",
     "harnessx-self-improving-rulesets"
-  ]
+  ],
+  "approval_note": "2026-09-06 requirements approved by the maintainer in session ('approve the requirements and run the design phase')"
 }