marius/Open_Ultra_SAST
1
0
Fork 0
mirror of https://github.com/norandom/OpenUltraSAST.git synced 2026-11-13 17:58:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(detection-core): approve v2 implementation plan

Browse Source
This commit is contained in:
Marius Ciepluch
2026-10-31 12:35:42 +01:00
parent 2c594b2c27
commit 1eb9ef11b5
5 changed files with 3650 additions and 426 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.kiro/specs/detection-core/design.md
+2536 -163
View File
File diff suppressed because it is too large Load Diff
.kiro/specs/detection-core/requirements.md
+274 -112
View File
@@ -2,139 +2,301 @@
## Project Description (Input)
**Who has the problem:** the maintainer of OpenUltraSAST. Every detection arm built so far
(prover, hunter, push hook, proof arm) is corrected link by link, and corrections do not
compose — v1→v2 retunes transferred 0/17 on untouched cases, and the 2026-10-30 admission
run on the untouched pilot population returned NO-GO (0/12 pairs, in-family 0/3, 9/12
repositories with zero framework recognition).
**Who has the problem:** the maintainer of OpenUltraSAST. The intended detection loop is
to augment family-specific model programs with deterministic program-analysis evidence.
Today that loop cannot be extended safely because the shared core embeds authorization
vocabulary, semantic gaps surface only during repository admission, and a clean-looking
zero can still hide a failure before the model receives useful evidence.
**Current situation:** detection is specified as per-family, per-arm runners around an
implicit core. The ordered trace exists for one family (access-control-prover), learned
components fit pool shortcuts (deletion count separates reversed fixes at 0.90), and
misses concentrate in unmeasured links: entry recognition, call resolution (11.9%
complete), guard dominance, and the 120s pre-push budget.
**Current situation:** the instrument-verified 2026-10-31 authorization diagnosis showed
two different failures. Framework-mediated reachability stopped before protected
operations, and paths that survived still lacked the constraint semantics needed to
distinguish vulnerable and fixed code. These are useful limits of the deterministic
evidence producer, not a reason to require it to become a complete vulnerability prover.
The core must expose such limits early and give each vulnerability family a stable way to
interpret the evidence without translating its semantics into authorization terms.
**What should change:** one specified core object — the security trace — with a contract
every arm, fact pack, and consumer reuses: given a push (base, head) within a fixed budget,
output proven defect claims (each with its witness) plus explicit coverage of what was not
decided, on repositories never seen. Models are bounded to residuals; admission on an
untouched population gates the trace itself; no fitted component ever sees pool labels.
**What should change:** the detection core shall provide a family-neutral evidence and
verdict envelope. Deterministic analysis shall contribute typed transitions, effects,
provenance, and explicit completeness. Each vulnerability family shall define its security
object and decision vocabulary and may use a bounded model program to answer residual
questions over that evidence. A staged readiness contract shall stop a family at the first
failing semantic layer, before population-level admission is attempted.
## Boundary Context
- **In scope**: the family-neutral evidence, outcome, trace, coverage, provenance, budget,
staged-readiness, admission, measurement, fact-pack, model-handoff, and consumer
contracts; family conformance requirements; failure-visible semantic-stage accounting.
- **Out of scope**: vulnerability-family detection algorithms and prompt content;
framework-specific fact content; model optimization; analysis of vendored dependency
source; the pre-push transaction and rollout policy; proof execution; enabling a new
vulnerability family for users.
- **Adjacent expectations**: the existing deterministic analysis is an evidence producer,
not a required complete prover; family-specific DSPy programs may consume the evidence
packet under the model-handoff contract; proof-based blocking remains a separate
consumer. A parked arm remains parked unless a separately reviewed strategy creates a
new arm, baseline, and admission population.
## Requirements
### Requirement 1: Canonical trace object
The detection core shall define one canonical trace object per analyzed change as the
shared security object for all detection arms.
1. When a change is analyzed, the detection core shall produce a canonical trace carrying
entry identity, ordered hops with call-target completeness, operation kind with resource
and authority, witnesses with provenance and match relations, guard effects, completeness
reasons, a final judgment, and the input digests of the analysis.
2. The detection core shall serialize every trace deterministically, so that identical
inputs produce byte-identical traces.
### Requirement 1: Family-neutral evidence record
The detection core shall define one canonical evidence record for every examined security
question within an analyzed change.
1. When a security question is analyzed, the detection core shall produce a canonical
evidence record carrying the family identity, security-object identity, analyzed scope,
ordered semantic transitions, observations with provenance, completeness reasons,
family decision, neutral outcome, evidence basis, and input digests.
2. When a transition or observation is derived from product code or reviewed semantic
knowledge, the detection core shall record its semantic role, source location,
provenance, and completeness independently of every other transition or observation.
3. When a family supplies family-specific evidence fields, the detection core shall
preserve those fields under the declared family schema without requiring them to use
another family's vocabulary.
4. The detection core shall serialize every evidence record deterministically, so that
identical inputs produce byte-identical records.
### Requirement 2: Three-way judgment vocabulary
The detection core shall judge every analyzed operation as exactly one of unguarded,
discharged, or indeterminate.
1. The detection core shall never convert an indeterminate judgment into a clean
(discharged or unguarded-free) result.
2. When evidence for a judgment is incomplete, the detection core shall return
indeterminate with the named incompleteness reasons.
### Requirement 2: Neutral outcome and evidence basis
The detection core shall separate a family-specific decision from the neutral outcome and
the basis supporting that outcome.
1. When a family reaches a decision, the detection core shall record exactly one neutral
outcome of violation, satisfied, or indeterminate and shall retain the family's own
decision code separately.
2. When an outcome is satisfied, the detection core shall report the exact bounded scope
over which satisfaction was established and shall not present it as repository-wide
safety.
3. When evidence is incomplete for the family decision, the detection core shall return
indeterminate with every named incompleteness reason.
4. If a required semantic relation is unavailable or ambiguous, the detection core shall
prevent that missing relation from creating a violation or satisfied outcome.
5. When an outcome is reported, the detection core shall identify whether its basis is
deterministic analysis, model corroboration, or execution-backed proof.
### Requirement 3: Coverage accounting
The detection core shall account for every input it did or did not use, so a zero result is
never indistinguishable from an unread input.
1. When a change is analyzed, the detection core shall report entry-recognition coverage,
per-call resolution outcomes with exactly one outcome bucket per call, and named
reasons for every unchecked operation.
2. If any input file is read as zero bytes, the detection core shall fail the run as an
instrument failure rather than reporting a clean result.
### Requirement 3: Vulnerability-family conformance contract
The detection core shall require every vulnerability family to declare the meaning and
limits of the evidence it consumes and the decisions it produces.
1. When a family is registered, the detection core shall require its family and schema
versions, claimed ecosystems, security-object definition, accepted semantic roles,
decision vocabulary, neutral-outcome mapping, completeness conditions, and named
indeterminate reasons.
2. When a family claims a violation or satisfied outcome, the detection core shall require
the family contract to identify the minimum evidence and completeness conditions for
that outcome.
3. When a family receives evidence outside its declared ecosystem or schema scope, the
detection core shall report the question as unsupported rather than applying a
different family's interpretation.
4. If a family decision code, evidence field, or semantic role is not declared by the
family contract, the detection core shall refuse the result.
5. The detection core shall demonstrate conformance with at least two reference families
that use materially different security objects before the family-neutral contract is
declared ready for downstream family specifications.
### Requirement 4: Measured budget and change scoping
### Requirement 4: Typed semantic transitions and effects
The detection core shall preserve reviewed semantic knowledge needed to connect product
code across framework and dependency boundaries without analyzing vendored source.
1. When reviewed semantic knowledge supplies a registration, invocation, call, return,
value transformation, operation effect, or constraint effect, the detection core shall
record the typed relation and the supplying fact identity in the evidence record.
2. When a typed relation is complete only under stated conditions, the detection core shall
retain those conditions and shall mark the relation incomplete when they are not met.
3. If multiple reviewed relations conflict or match ambiguously, the detection core shall
report the ambiguity and shall not choose a complete relation silently.
4. When vendored source is excluded, the detection core shall require a reviewed semantic
relation for any dependency behavior needed by a determinate family decision.
### Requirement 5: Coverage and instrument accounting
The detection core shall account for every input and semantic stage it did or did not use,
so a zero result is never indistinguishable from unread or unmodeled input.
1. When a change is analyzed, the detection core shall report counts for inputs read,
security questions formed, semantic sites recognized, transitions materialized,
security objects constructed, constraints evaluated, determinate outcomes, and findings
emitted.
2. When product-code or fact-derived relations are considered, the detection core shall
assign each relation exactly one complete, incomplete, or unresolved outcome.
3. If a semantic site, transition, security object, or constraint cannot be modeled, the
detection core shall report the specific missing relation rather than interpreting its
absence as safe.
4. If any tracked input file is read as zero bytes or cannot be opened, the detection core
shall fail the run as an instrument failure rather than reporting a clean result.
5. When a stage produces zero outputs, the detection core shall identify the last stage
with verified nonzero input and the first stage that failed to produce its expected
output.
### Requirement 6: Measured budget and change scoping
The detection core shall derive its time budget from measurement, not from a fixed constant.
1. When a change is analyzed, the detection core shall measure and record reply time and
change complexity (changed regions, call count, changed size) for every change.
2. The detection core shall derive the per-change budget from the measured distribution
(sample count, percentile, reference workload and hardware disclosed) and shall publish
that derivation under the fixed measurement schema; the budget shall be configurable per
workload class, never a single static constant.
3. When a change exceeds the measured budget, the detection core shall fail the budget gate
naming the change, the elapsed time, and the budget derivation exceeded.
4. While a push is analyzed, the detection core shall limit analysis to regions affected by
the change and shall record the scope it used.
change complexity, including changed regions, transition count, and changed size.
2. When a workload-class budget is published, the detection core shall disclose the sample
count, percentile, reference workload, reference hardware, and derived budget.
3. When a change exceeds its measured workload-class budget, the detection core shall fail
the budget gate naming the change, elapsed time, and budget derivation exceeded.
4. While a push is analyzed, the detection core shall limit analysis to regions affected
by the change and shall record the scope it used.
### Requirement 5: Verdict contract
The detection core shall distinguish proven defect claims from bounded silence.
1. When the detection core reports a defect, the claim shall carry its witness evidence and
an exact location tied to the change.
2. When the detection core cannot decide a site, it shall report that site as undecided
with its coverage reason, and shall never present an undecided site as a clean result.
### Requirement 7: Verdict contract
The detection core shall distinguish supported defect claims, bounded satisfaction, and
bounded silence.
1. When the detection core reports a defect, the claim shall carry its evidence basis, an
exact location tied to the change, and the complete evidence record supporting the
claim.
2. When the detection core reports satisfaction, the claim shall carry the completed
security questions and the bounded scope over which they were decided.
3. When the detection core cannot decide a question, it shall report that question as
indeterminate with its coverage reasons and shall never present it as clean.
4. If a claim depends on an incomplete or unresolved semantic relation, the detection core
shall expose the affected question as bounded silence unless another complete path
independently supports the claim.
### Requirement 6: Provenance and digests
The detection core shall stamp every trace and verdict with the identities of its inputs.
1. The detection core shall record, for every analysis, the analyzer revision, fact-set
digest, acquisition-record digest, configuration digest, population digest, and
per-change input digest.
2. If any recorded digest cannot be re-derived from the named inputs, the detection core
shall refuse the result rather than accept it.
### Requirement 8: Staged readiness and promotion
The detection core shall require a vulnerability-family arm to pass each semantic readiness
stage before it can advance to broader evaluation.
1. When an arm begins development, the detection core shall require separate readiness
results for instrument validity, semantic recognition, transition construction,
security-object and constraint evaluation, vulnerable/fixed pair behavior, development
population behavior, fresh admission, and consumer integration.
2. When a readiness stage is evaluated, the detection core shall require a positive case,
a negative case, and an ambiguity or missing-evidence case appropriate to that stage.
3. If a readiness stage fails, the detection core shall stop promotion at that stage and
shall report the failing semantic layer and named correction lever.
4. If any prerequisite readiness result is missing, stale, or failed, the detection core
shall refuse to run a later population or integration gate as evidence of readiness.
5. When a semantic change invalidates an earlier readiness result, the detection core shall
mark every dependent later result stale.
6. When a family passes a stage, the detection core shall preserve the fixtures and results
as regression evidence without treating them as fresh admission evidence.
### Requirement 7: Admission gate contract
The detection core shall gate any generalization claim on an untouched population.
1. When an arm is proposed for pre-push use, the detection core shall require a reviewed
manifest naming the frozen analyzer, facts, acquisition, configuration, and an untouched
admission population before any admission label is opened.
### Requirement 9: Model-augmentation contract
The detection core shall support bounded family-specific model reasoning over deterministic
evidence without allowing model output to rewrite that evidence.
1. Where a family uses a model program, the detection core shall present the canonical
evidence record, its completeness state, the family contract, and the residual question
to be answered.
2. When a model program returns a family decision, the detection core shall preserve the
model response, program and configuration identities, evidence-record digest, and
declared decision mapping.
3. If deterministic evidence is incomplete, the detection core shall prevent a model
response from converting the affected question into a satisfied outcome or a
proof-backed violation.
4. When model output supports an advisory claim, the detection core shall label the claim
as model-corroborated and shall keep it distinct from deterministic and execution-backed
claims.
5. The detection core shall not allow a model to propose and independently adjudicate the
same claim within one evidence path.
6. The detection core shall reserve blocking verdicts for execution-backed proof with a
differential witness.
### Requirement 10: Provenance and digests
The detection core shall stamp every evidence record and verdict with the identities of its
inputs.
1. When an analysis is recorded, the detection core shall include the analyzer revision,
family-contract digest, fact-set digest, acquisition-record digest, configuration
digest, population digest, and per-change input digest.
2. When an evidence record uses reviewed semantic knowledge, the detection core shall
include the semantic pack and fact identities needed to reproduce each derived relation
or effect.
3. When a model program contributes a decision, the detection core shall include the model,
program, prompt, and configuration identities needed to reproduce the call.
4. If any recorded digest or semantic fact identity cannot be re-derived from the named
inputs, the detection core shall refuse the result rather than accept it.
### Requirement 11: Reviewed semantic fact packs as the extension point
The detection core shall admit family and ecosystem knowledge only through reviewed,
versioned semantic fact packs over the family-neutral contract.
1. When a semantic fact pack is admitted, the detection core shall validate its schema
version, claimed family and ecosystem scope, stable fact identities, declared semantic
roles, and review evidence.
2. When a semantic fact matches analyzed product code, the detection core shall permit the
consuming family to record the corresponding transition or effect only with the matching
evidence and fact provenance attached.
3. The detection core shall not allow a name-only match without the fact's required
structural evidence to create a complete semantic relation or determinate outcome.
4. If a semantic fact has an undeclared role, unknown field, invalid scope, missing review
evidence, or inconsistent digest, the detection core shall refuse the pack.
5. When admitted knowledge is insufficient to resolve a semantic relation, the detection
core shall preserve the partial evidence record and report the relation as incomplete or
unresolved.
6. The detection core shall report, per arm, the vulnerability families and ecosystems the
arm claims, so admission and rollout numbers are never read across unsupported scopes.
### Requirement 12: Admission gate contract
The detection core shall gate every generalization claim on completed readiness stages and
a reviewed untouched population whose denominator matches the arm's declared scope.
1. When an arm is proposed for consumer use, the detection core shall require a reviewed
manifest naming the frozen analyzer, family contract, model program where applicable,
facts, acquisition, configuration, claimed families, claimed ecosystems, denominator
policy, readiness results, and untouched admission population before any admission
label is opened.
2. The detection core shall consume admission manifests without producing, mutating, or
re-labeling them, and shall refuse manifests that are unreviewed, label-exposing,
re-labeling them and shall refuse manifests that are unreviewed, label-exposing,
acquisition-changed, or spent.
3. When admission evidence is scored, the detection core shall report paired
vulnerable/fixed behavior, ordinary false alarms, and every gate (budget, instrument,
pre-registered criteria) as an evidence-backed GO or NO-GO.
3. When an admission manifest is validated before label opening, the detection core shall
verify that every pre-registered success threshold is arithmetically reachable under the
declared family, ecosystem, and denominator policy.
4. If a threshold cannot be reached even when every in-scope case succeeds, the detection
core shall refuse the admission configuration rather than run or report a misleading
no-go result.
5. When a population contains cases outside an arm's declared scope, the detection core
shall score quality gates on the pre-registered in-scope denominator and shall report
whole-population coverage separately.
6. When admission evidence is scored, the detection core shall report paired
vulnerable/fixed behavior, ordinary false alarms, semantic-stage coverage, model-call
outcomes where applicable, and every budget, instrument, and quality gate as an
evidence-backed go or no-go result.
### Requirement 8: No fitting on evaluation labels
The detection core shall not allow any fitted component to train on evaluation labels.
1. The detection core shall prohibit tuned prompts, thresholds, or fitted models whose
training data includes pool or admission labels.
2. When a fitted component is proposed, the detection core shall require it to pass
reversed-pair probes (a fix and its reversed fix shall not be separable by the
component's decision features), per-repository holdout, and admission on an untouched
population before any claim is made.
### Requirement 13: Separation of development and evaluation
The detection core shall prevent evaluation labels from influencing an arm before its
generalization claim is scored.
1. The detection core shall prohibit prompts, examples, thresholds, fitted models, semantic
facts, or decision mappings whose preparation data includes admission labels.
2. When a fitted component is proposed, the detection core shall require reversed-pair
probes, repository-held-out development evaluation, and admission on an untouched
population before any generalization claim is made.
3. When semantic knowledge is acquired, the detection core shall require source,
documentation, or independently reviewed behavioral evidence that is isolated from
vulnerability, fix-location, and ordinary-change labels.
4. If evaluation labels influence semantic fact content, matching behavior, prompt
content, examples, thresholds, or promotion, the detection core shall mark the affected
population spent and shall refuse it for admission.
### Requirement 9: Bounded role for models
The detection core shall bound every model to residual questions over the trace.
1. Where a model judges a trace, the detection core shall present the deterministic trace
and require the judgment to be zero-shot over that trace.
2. The detection core shall not allow a model to propose-and-adjudicate the same claim, and
shall reserve blocking verdicts for demonstrated proofs with a differential witness.
### Requirement 10: Family fact packs as the extension point
The detection core shall admit new vulnerability families only as fact packs over the core
trace contract.
1. When a new family is added, the detection core shall require the family's entry,
operation, and guard knowledge as reviewed fact packs stamped into the trace's
provenance, without changing the trace schema or judgment vocabulary.
2. The detection core shall report, per arm, the vulnerability families the arm claims, so
admission and rollout numbers are never read across families the arm does not cover.
### Requirement 11: Consumer and re-parenting contract
The detection core shall be the single contract that existing detection specs reuse.
### Requirement 14: Consumer and re-parenting contract
The detection core shall be the single contract that existing and future detection specs
reuse.
1. The detection core shall define the interface through which the pre-push transaction
consumes traces and verdicts within its deadline.
2. The detection core shall define the interface through which proof-based blocking evidence
attaches to a trace verdict.
consumes evidence records and verdicts within its deadline.
2. The detection core shall define the interface through which proof-based blocking
evidence attaches to an existing claim.
3. When an existing arm's behavior is described, the detection core shall require that
description to be expressed over the core contract rather than as a separate runner
contract.
description to be expressed over the family-neutral contract rather than as a separate
runner contract.
4. When a consumer receives bounded silence caused by missing semantic knowledge, the
detection core shall require the consumer to preserve the named coverage reasons.
5. When a family specification is created, the detection core shall require it to identify
its family contract, readiness fixtures, development population, and admission boundary.
### Requirement 12: Fixed measurement schema
### Requirement 15: Fixed measurement schema
The detection core shall require one fixed measurement schema for every detection number.
1. When any detection result is reported, the detection core shall require population and
freeze digests, judgment vocabulary, every rate with its sample count and uncertainty
interval, coverage, instrument failures, and runtime.
2. When a measurement changes population or judgment vocabulary, the detection core shall
treat the result as a new baseline and shall not present it as a trend against the
previous baseline.
freeze digests, claimed family and ecosystem scope, denominator policy, family decision
vocabulary, neutral outcomes, evidence bases, every rate with its sample count and
uncertainty interval, coverage, instrument failures, model-call failures, and runtime.
2. When semantic relations participate in an arm, the detection core shall require the
report to include sites considered, sites recognized, transitions completed, security
objects constructed, constraints evaluated, determinate outcomes, and unresolved counts
by named reason.
3. When a measurement changes population, denominator policy, claimed scope, family
contract, fact schema, model program, or outcome mapping, the detection core shall treat
the result as a new baseline and shall not present it as a trend against the previous
baseline.
4. When a zero or clean-looking result is reported, the detection core shall require proof
that every tracked input was read and that every prerequisite semantic stage executed.
5. When a later-stage metric is reported, the detection core shall include the results and
digests of every prerequisite readiness stage.
## Out of Scope
- The pre-push hook transaction, rollout policy, and capability registry (pre-push-safety-net).
- New vulnerability families and their fact content (fact-pack specs).
- Fitted models of any kind in the core path.
- Engine and graph internals (contributor-scan).
- The pre-push hook transaction, rollout policy, and capability registry.
- Vulnerability-family algorithms, prompts, examples, and optimization programs.
- Framework- or repository-specific semantic fact content.
- Analysis or indexing of vendored dependency source.
- Proof execution and sandbox orchestration.
- Enabling a vulnerability family for advisory or blocking use.
- Analysis-engine, frontend, graph, and family-internal implementation details.
.kiro/specs/detection-core/research.md
+472
View File
@@ -51,3 +51,475 @@ in `arm_regression.py`), and the admission workflow (`benchmarks/valid/prover_ad
(dev 483 changes with seconds; pilot 264). Carry the 13.3/13.4 record sets as the fitting
data — measurement only, never training a detector on them.
- Reversed-pair probe harness for Req 8: fix/reversed-fix pairs exist in the pools.
---
## Gap analysis: family-neutral evidence and model augmentation (2026-10-31)
### Scope and approval state
- **Feature type:** complex integration over an existing contract package. The core exists,
but its public schema, tests, producer adapter, and historical admission schema embody the
authorization family that supplied the original implementation.
- **Requirements state:** generated but not approved. This analysis may inform review; design
generation remains gated on explicit requirements approval.
- **Discovery method:** integration-focused source inspection. No external dependency research
was needed: the neutral core can remain Python 3.11 typed dataclasses and canonical JSON,
while DSPy remains the existing optional `hunt` dependency (`>=3.4,<3.5`) behind an adapter.
- **Instrument check:** representative requirements, core, authorization-adapter, model-runner,
and project-metadata inputs were opened with nonzero byte counts before interpreting gaps.
### Current-state inventory
The existing `src/openultrasast/core/` package is a compact contract layer with reusable
machinery:
- `trace.py` supplies immutable records and deterministic JSON serialization.
- `verdict.py` supplies the authorization three-way judgment plus claim and bounded-silence
records.
- `coverage.py` supplies exact call-resolution buckets and zero-byte input refusal.
- `budget.py` supplies measured percentile budgets and a named budget gate.
- `provenance.py` supplies input-identity stamping and mismatch refusal.
- `factpack.py` supplies reviewed pack loading and content digests.
- `admission.py` supplies consume-only manifest validation and a fixed report schema.
- `probes.py` supplies reversed-pair and holdout/admission checks.
- `consumer.py` supplies a push request/verdict envelope.
Current integration is narrow. `semantic/obligations/trace.py` converts the authorization
trace into the core `TraceRecord`; `hunt/prover.py` converts authorization coverage;
`mapping/entry_packs.py` consumes core fact packs; benchmark admission modules consume part
of the admission contract. The production push and proof paths do not yet consume the core
end to end. `benchmarks/valid/judge_pass.py` is the closest existing DSPy residual-judgment
adapter, but it reads authorization findings directly rather than a family-neutral evidence
record.
### Requirement-to-asset map
| Requirements | Existing assets | Gap classification and impact |
|---|---|---|
| 1–2 Evidence record and outcomes | Deterministic serialization, frozen dataclasses | **Missing / breaking:** `TraceRecord` requires authorization fields (`entry`, operation, resource, authority, guards) and `Judgment` is exactly unguarded/discharged/indeterminate. Missing family/security-object identity, analyzed scope, generic transitions and observations, family decision, neutral outcome, evidence basis, and bounded satisfaction. |
| 3 Family contract | Family strings exist in findings and packs | **Missing:** no versioned contract, ecosystem scope, security-object schema, accepted semantic roles, decision mapping, completeness policy, or two-family conformance fixtures. |
| 4 Typed semantics | Authorization hops/guards and recognition packs | **Missing:** no generic registration, invocation, call, return, transformation, operation-effect, or constraint-effect relation with conditions, fact identity, ambiguity handling, and per-relation completeness. |
| 5 Coverage and instrument | Zero-byte refusal, call buckets, authorization funnel | **Partial:** missing generic question → recognition → transition → security-object → constraint → outcome → finding counters and last-success/first-zero diagnosis. Existing coverage cannot distinguish all newly required failure layers. |
| 6 Budget and scope | `Complexity`, `Budget`, percentile derivation; change-scoped runners | **Constraint:** reusable budget logic is not bound to the neutral run, `calls` is not a generic transition count, and current runners may still pass a static 120-second value. Scope identity is absent from the evidence result. |
| 7 Verdict | `ProvenClaim`, `BoundedSilence`, consumer validation | **Partial / semantic mismatch:** no bounded-satisfaction record, evidence-basis tag, evidence-record identity, completed-question inventory, or multi-path aggregation. `ProvenClaim` currently implies proof-grade evidence for every claim. |
| 8 Readiness | Separate development and admission runners | **Missing:** no ordered readiness stages, prerequisite digests, staleness propagation, positive/negative/ambiguity fixture contract, or refusal of later gates after an earlier failure. |
| 9 Model augmentation | Existing DSPy program, cached calls, parsing/failure records, zero-shot judge experiment | **Missing / contradictory test:** no core handoff/result contract; a core test explicitly asserts that no model path is public. Model programs consume code/features or authorization findings, not canonical evidence records. The core must not import optional DSPy. |
| 10 Provenance | Six current digests and mismatch refusal | **Partial:** missing family-contract, semantic-pack/fact, model, program, prompt, and model-configuration identities. |
| 11 Fact packs | Review attestation, content digest, Express entry pack | **Partial / validation gap:** facts remain `dict[str, Any]`; the typed object lacks schema version, ecosystem/language scope, closed roles, structural match requirements, relation conditions, and conflict detection. The embedded review digest authenticates the review note, not the fact content itself. |
| 12 Admission | Consume-only manifests, pair scoring, Wilson intervals, GO/NO-GO reports | **Partial / authorization-shaped:** missing readiness prerequisites, ecosystem and denominator policy, family/model identities, arithmetic feasibility, in-scope scoring, neutral outcomes, semantic-stage funnel, and model-call results. |
| 13 Evaluation separation | Reversed-pair probe and coarse holdout/admission check | **Partial:** no lineage proving prompts, examples, facts, mappings, and promotion excluded admission labels. The current probe treats any differing score as a shortcut and is not a semantic audit by itself. |
| 14 Consumers | Push wrapper and authorization native conversion | **Partial:** the request is not used by `consume_verdict`; no proof-attachment interface, family readiness declaration, or production pre-push consumption exists. |
| 15 Measurement | Fixed report keys and benchmark metric helpers | **Partial / schema migration:** missing neutral outcomes, evidence bases, family/model identities, the generic semantic funnel, model failures, readiness digests, and enforceable baseline-reset metadata. Existing keys encode the legacy admission vocabulary. |
### Compatibility and architecture constraints
1. `TraceRecord` and `Judgment` are public exports and are pinned exactly by contract tests.
Replacing them in place breaks the authorization adapter immediately.
2. Historical admission reports are immutable evidence. A neutral schema must have a new
version and baseline; historical artifacts must not be rewritten to pass a new validator.
3. The authorization trace has no per-hop completeness and conservatively maps every hop to
incomplete. Migration must preserve that behavior until the family producer supplies
stronger evidence.
4. The core package must remain dependency-light. DSPy execution belongs in a hunt-side or
family-side adapter; the core owns only typed request, result, and provenance contracts.
5. Vendored source remains excluded. Framework behavior needed for a determinate decision
must arrive through reviewed semantic relations or remain indeterminate.
6. `core/admission.py` currently imports forbidden-label markers from `tools.fact_init`, an
upward dependency from a foundational contract into acquisition tooling. The design must
reverse or remove this dependency.
7. Fact-pack strengthening affects `mapping/entry_packs.py` and the shipped Express pack;
versioned loading or migration is required.
8. Core-unit success alone cannot establish integration because the production push and
proof consumers are not yet wired to the contract.
### Implementation approach options
#### Option A: Extend the existing types in place
Add neutral fields and aliases while retaining authorization fields and judgment values.
- **Strengths:** smallest initial patch; current producers and tests need fewer immediate
changes.
- **Risks:** creates dual sources of truth, permanently leaks authorization vocabulary into
every family, and cannot convincingly satisfy two-family neutrality.
- **Effort / risk:** L / High.
#### Option B: Add an independent v2 core beside the current package
Build a separate neutral package and migrate consumers later.
- **Strengths:** clean schema and clear compatibility boundary.
- **Risks:** recreates the parallel-contract problem this spec exists to remove; downstream
specs can continue using v1 indefinitely; admission and measurement logic is duplicated.
- **Effort / risk:** XL / High.
#### Option C: Hybrid versioned migration
Add neutral typed primitives and family, readiness, and model contracts inside the existing
core boundary; retain reusable budget, provenance, and validation machinery; place explicit
authorization and DSPy adapters outside the neutral types; migrate consumers before removing
legacy exports.
- **Strengths:** preserves measured machinery without preserving the wrong abstraction;
maintains a dependency-light core; provides a bounded migration path and a natural place
for two unlike reference-family conformance fixtures.
- **Risks:** requires deliberate schema versioning, temporary compatibility adapters, and a
new measurement baseline.
- **Effort / risk:** XL / Medium-High.
- **Design-phase recommendation:** prefer this option, subject to requirements approval and
resolution of the carried research items below.
### Candidate responsibility and file boundaries
The gap analysis identifies responsibilities, not final design choices:
- `core/evidence.py` — family-neutral evidence record, typed transitions, observations, and
completeness.
- `core/family.py` — versioned family contract, declared schemas/roles, and decision mapping.
- `core/readiness.py` — stage results, prerequisites, invalidation, and promotion refusal.
- `core/model.py` — dependency-free model handoff/result and identities.
- `core/measurement.py` — neutral funnel and versioned measurement schema.
- Existing `verdict.py`, `coverage.py`, `factpack.py`, `provenance.py`, `admission.py`,
`consumer.py`, `probes.py`, `budget.py`, and `__init__.py` — extend reusable behavior and
phase out authorization-shaped exports.
- `semantic/obligations/trace.py` — authorization-family adapter rather than a core schema
definition.
- A hunt-side adapter — converts an evidence packet into the family DSPy input and converts
the model result into the core model-result contract without importing DSPy into core.
- Benchmark development/admission modules — produce readiness artifacts and score the neutral
schema while preserving historical report validation separately.
- Contract, migration, two-reference-family, model-handoff, readiness, and admission tests —
replace authorization-specific assumptions with explicit adapter conformance.
### Design research carried forward
1. Choose whether the two conformance families are synthetic authorization-plus-taint
fixtures or real adapters. They must prove different security-object shapes without
implying user-facing capability enablement.
2. Define the stable proof-attachment subset of demonstrated search results; proof execution
remains outside this spec.
3. Assign readiness-artifact persistence: the likely seam is that benchmark workflows
produce/store artifacts while the core validates prerequisites and staleness.
4. Define the wire migration and removal policy for legacy `TraceRecord`, `Judgment`, and
historical authorization report validation.
5. Pin the rule for model-corroborated outcomes: a model may answer only a family-declared
residual question. A missing relation declared necessary for the family outcome remains
indeterminate; model corroboration cannot fill it implicitly.
6. Decide whether budget workload-class assignment stays a runner responsibility or becomes
part of the neutral request contract; the core should validate, not infer, change scope.
### Gap conclusion
The existing package provides useful primitives but does not yet implement the amended
family-neutral contract. The dominant work is schema and lifecycle migration, not new graph
analysis. The authorization arm should remain a family adapter and regression source; it
must not define the new core vocabulary. Design generation should begin only after explicit
requirements approval.
---
## Design discovery and synthesis (2026-10-31)
### Summary
- **Feature:** detection-core
- **Discovery scope:** complex integration and public-contract migration.
- **Key findings:**
- The reusable assets are canonical serialization, immutable records, budget derivation,
digest verification, manifest refusal, and metric helpers. Authorization field names and
outcomes are not reusable core concepts.
- The smallest viable architecture is a versioned ports-and-adapters contract: the core
owns typed evidence and validation; deterministic analyzers, DSPy, proof execution,
benchmarks, and push policy remain outside it.
- Readiness must be an immutable prerequisite graph validated by the core and persisted by
benchmark workflows. This makes a failed semantic stage stop later population gates.
- Migration must be finite: v2 is the only write contract after consumers migrate; v1
remains a read-only historical decoder, not a second detection core.
### Research log
#### Existing integration and dependency direction
- **Context:** the amended requirements generalize an already implemented authorization
contract without reopening the parked arm.
- **Sources consulted:** `src/openultrasast/core/`, authorization trace and prover adapters,
benchmark development/admission runners, push consumers, search verifier records, current
tests, and project steering.
- **Findings:**
- Core production imports are concentrated, so a staged adapter migration is feasible.
- `core/admission.py` currently imports acquisition-tool policy, which inverts the desired
dependency direction.
- Production push and proof paths are not yet end-to-end core consumers; unit conformance
alone cannot establish integration readiness.
- **Implications:** core modules import only the standard library and other core modules.
Analyzer, model, benchmark, proof, and push adapters import core, never the reverse.
#### DSPy boundary
- **Context:** the original product direction augments DSPy programs with deterministic
analysis evidence, while the core installation must remain independent of the optional
model stack.
- **Sources consulted:** existing `hunt/program.py` and `judge_pass.py`; official DSPy
documentation for signatures and modules.
- **Findings:** DSPy signatures declare structured input/output behavior, while modules may
contain learnable prompt or model parameters. A stable core handoff can therefore be a
typed request/result contract, but reproducibility requires program, prompt, model, and
configuration identities rather than a model name alone.
- **Implications:** DSPy execution, caching, parsing, and redaction remain in a hunt-side
adapter. The core receives a `ModelResult` tied to an immutable evidence digest and a
family-declared residual question.
#### Canonical records and schema choice
- **Context:** evidence records, facts, readiness receipts, and admission identities require
deterministic bytes and closed family extensions.
- **Sources consulted:** Python 3.11 `json` documentation, RFC 8785, JSON Schema Draft
2020-12, and the existing core serializer.
- **Findings:** Python's sorted-key compact JSON is sufficient for repeatable records inside
the current Python boundary when non-finite numbers and unsupported objects are refused.
RFC 8785 is stricter and useful for cross-language signatures, which are not required here.
General JSON Schema validation would add a runtime dependency and permit a broader payload
language than the current families require.
- **Implications:** v2 defines a narrow canonical JSON profile and closed typed `FieldSpec`
values. Ordered transitions remain tuples; family payloads stay shallow and typed; complex
structure is represented by semantic transitions and observations. Cross-language signing
is a revalidation trigger rather than speculative scope.
#### Proof attachment boundary
- **Context:** execution-backed evidence may support blocking, but proof execution and
rollout policy are outside this spec.
- **Sources consulted:** `search/verify.py`, `search/coordinator.py`, search tests, and
`safety-net.md`.
- **Findings:** the verifier already produces a differential `demonstrated` outcome over
affected and safe sides. The core needs only the immutable result identity, matched family
and location, side digests, verifier revision, witness reference, and evidence-record
digest.
- **Implications:** the core validates and attaches a proof record; it does not run or
reinterpret the oracle. Execution-backed basis is necessary but does not by itself enable
downstream BLOCK policy.
#### Readiness and admission ownership
- **Context:** repository admission exposed semantic failures that should have stopped the
arm at recognition, transitions, or constraints.
- **Sources consulted:** development/admission runners, the 2026-10-31 handoff, and detection
quality steering.
- **Findings:** artifact production and storage already belong to benchmark workflows.
Contract validation, prerequisite freshness, denominator feasibility, pure scoring, and
report validation are reusable core responsibilities.
- **Implications:** benchmark workflows persist immutable `ReadinessResult` and case-result
artifacts. The core derives `passed`, `failed`, or `stale`, refuses later gates when
prerequisites do not pass, and never mutates historical artifacts.
### Architecture pattern evaluation
| Option | Description | Strengths | Risks and limitations | Decision |
|---|---|---|---|---|
| Extend v1 in place | Add neutral fields and aliases to authorization records | Small initial patch | Dual sources of truth and permanent family leakage | Rejected |
| Parallel v2 package | Build a second neutral package and migrate later | Clean initial schema | Recreates parallel contracts and duplicated admission | Rejected |
| Versioned ports and adapters | Add v2 neutral contracts to core, migrate through explicit adapters, retain v1 read-only decoding | Preserves reusable machinery and enforces boundaries | Requires controlled schema and baseline migration | Selected |
### Design decisions
#### Decision: split deterministic packet from decision
- **Context:** model and proof adapters must not rewrite program-analysis evidence.
- **Selected approach:** `EvidencePacket` contains immutable deterministic semantics;
`Decision` contains family code, neutral outcome, basis, and bounded scope;
`EvidenceRecord` binds them by packet digest.
- **Rationale:** derived decisions can change while the underlying evidence identity remains
stable and auditable.
- **Trade-off:** one additional aggregate type, offset by a clear mutation boundary.
#### Decision: family contracts are data, family reasoning stays outside core
- **Context:** the core must validate family vocabulary without implementing every
vulnerability abstraction.
- **Selected approach:** an immutable `FamilyContract` declares ecosystems, shallow typed
fields, roles, decisions, mappings, required evidence, completeness rules, residual
questions, and named indeterminate reasons. Family adapters construct evidence and apply
their algorithms.
- **Rationale:** the core can refuse undeclared roles, fields, decisions, and scopes without
importing family logic.
- **Trade-off:** adding a new field kind changes the core schema and triggers revalidation;
this is preferred to unrestricted `Any` payloads.
#### Decision: conformance uses two test-only semantic shapes
- **Context:** authorization alone cannot demonstrate neutrality, and the parked arm must
not be revived through the core spec.
- **Selected approach:** test-only obligation and taint-flow contracts each supply positive,
negative, and missing-evidence fixtures. They make no user-facing capability claim.
- **Rationale:** the pair exercises control/constraint and flow/sanitization shapes without
target vocabulary or admission reuse.
#### Decision: model corroboration answers declared residuals only
- **Context:** Requirements 2.4 and 9.3–9.4 must not allow a model to fill missing required
deterministic relations.
- **Selected approach:** each residual question declares deterministic prerequisites and
allowed family decisions. A missing required prerequisite forces indeterminate. A valid
model result may produce model-corroborated advisory evidence but never execution-backed
basis.
- **Rationale:** this restores DSPy augmentation without treating plausible text as missing
static evidence.
#### Decision: readiness persistence belongs to benchmarks
- **Context:** core-owned files or storage would duplicate run orchestration.
- **Selected approach:** the core defines and validates receipts and freshness; benchmark
workflows write and load them.
- **Rationale:** separates pure contract logic from population custody and run persistence.
#### Decision: versioned canonical JSON profile, not a new serializer dependency
- **Context:** identical input must produce byte-identical records and stable digests.
- **Selected approach:** UTF-8 JSON with sorted keys, compact separators, preserved sequence
order, no non-finite numbers, and explicit enum values. The schema version is part of every
digestible aggregate.
- **Rationale:** uses the existing Python runtime and meets the current boundary without
claiming RFC 8785 interoperability.
#### Decision: finite legacy migration
- **Context:** historical v1 evidence must remain readable without allowing two live write
contracts.
- **Selected approach:** add v2, migrate authorization and benchmark adapters, stop v1
writes, migrate downstream consumers, remove v1 public exports, and retain only a
read-only legacy decoder/validator.
- **Rationale:** preserves historical artifacts while restoring one canonical write path.
### Risks and mitigations
- **Permanent dual contracts:** milestone explicitly stops v1 writes and removes v1 exports
after an import census reaches zero.
- **Authorization vocabulary leaks into v2:** neutral-type review plus two unlike synthetic
family conformance suites.
- **Model fills semantic holes:** family-declared residual prerequisites and core-side result
validation.
- **Readiness becomes ceremonial:** later gates require current predecessor digests and
refuse stale, failed, or absent receipts.
- **Historical comparability is lost:** v1 stays immutable and v2 creates a new baseline
identity; cross-baseline trend comparison is refused.
- **Fact review attests prose but not content:** v2 review identity covers normalized facts,
schema, scope, and review evidence together.
- **Proof result silently enables blocking:** proof attachment changes evidence basis only;
rollout eligibility remains downstream.
- **Hidden training contamination:** the core validates declared lineage and population
digests; external review remains the trust boundary for undisclosed training data.
### References
- [DSPy signatures](https://dspy.ai/3.0.4/learn/programming/signatures/) — structured,
declarative model inputs and outputs.
- [DSPy modules](https://dspy.ai/3.0.4/learn/programming/modules/) — module composition and
learnable program state.
- [Python 3.11 JSON](https://docs.python.org/3.11/library/json.html) — sorted keys, compact
separators, and strict number handling.
- [RFC 8785](https://www.rfc-editor.org/rfc/rfc8785.html) — evaluated but not adopted for
the current single-runtime wire boundary.
- [JSON Schema Draft 2020-12](https://json-schema.org/draft/2020-12) — evaluated but not
adopted for the bounded family-field contract.
---
## Review-gate contract closure (2026-10-31)
### Scope of this pass
- **Context:** the first merge draft covered all requirement IDs but several late validators
received only digests or caller-produced summaries, so their stated revalidation behavior
was not implementable from the interface.
- **Sources consulted:** the approved detection-core requirements, complete project steering,
the merge draft, current core/benchmark seams already recorded above, and independent
architecture, risk, and traceability reviews.
- **Decision:** keep the requirements and architecture boundary unchanged; repair the
interfaces that must carry canonical objects and fix the label-custody order before task
generation.
### Decision: admission receives a closed validation context
- `FamilyCatalog` is now a real immutable value with re-derived contract and catalog
digests and exact family/version/ecosystem resolution.
- `AdmissionValidationContext` carries the actual catalog, fact packs, model declarations,
lineages, and review/custody/provenance/proof policies. Authorization and scoring both
re-derive its digest and validate the objects; a set digest is not treated as the set.
- The threshold set is a baseline identity. Its actual lineage targets the threshold-set
digest and is validated before labels open.
### Decision: excluded-dependency behavior is typed
- A family role declares either product-or-reviewed-fact provenance or
excluded-dependency-fact-only provenance.
- A dependency-fact-only relation cannot be complete from product-code evidence. Missing
reviewed knowledge is an explicit incomplete reason; only a revalidated pack, fact, and
structural match can make it complete.
- This closes the vendored-source boundary without importing dependency source or family
algorithms into core.
### Decision: lineage binds model, matcher, and threshold behavior
- Lineage manifests name a subject; individual uses name the exact influenced artifact and
resolve to current custody receipts.
- Model requests embed a frozen program declaration and the actual lineage used for prompt,
examples, fitted components, and decision mappings.
- Fact matches bind matcher revision, matcher-behavior digest, and matcher lineage.
Admission-label influence on matching behavior spends the population just as prompt,
fact, threshold, or promotion influence does.
### Decision: provenance validation must resolve bytes
- Named provenance inputs use a closed role vocabulary and a manifest digest.
- Validation receives a resolver and trust policy and checks bytes, SHA-256, length,
storage receipt, retention policy, trusted store, and size bound before accepting the
stamp. Unreadable input cannot validate as a clean identity-only result.
### Decision: spend before label release
- Authorization returns a label-release request committed to the exact current custody
head and authorization digest.
- The benchmark custodian compare-and-swap appends `AVAILABLE` to `SPENT` durably and only
then releases label bytes. Scoring accepts that exact current-chain receipt and returns a
report bound to it; it does not request a later spend.
- This removes the crash and concurrency window in which labels could be observed while the
population still appeared available.
### Decision: fitted-component authorization replays probes
- Authorization receives probe plus observation pairs and calls the canonical evaluator.
It does not accept caller-produced `passed` summaries.
- Derived probe digests must equal the fitted-component evidence, alongside disjoint
repository holdout, fitting lineage, and the bound admission report.
- Contract, review policy, and custody policy are resolved from the same context bound into
the admission report. The authorization API does not accept caller-selected alternatives.
### Decision: raw records share one validation composition seam
- Admission and push consumption previously had unequal validation context: admission could
re-open facts and lineage while a consumer could still receive a raw `EvidenceRecord`.
- `validation.py` now owns `EvidenceValidationContext`, `EvidenceValidationInput`, and the
composition validator over catalog, named provenance inputs, facts, model derivation,
lineage, custody, and resolver trust. Both admission and consumer APIs call that seam;
there is no constructible validated-token shortcut.
- Per-case fact/model tuples were removed in favor of the single validation envelope, so
context membership and derivation replay have one source of truth.
### Decision: admission review is independently attestable
- The manifest and label-blind census each carry a payload digest and independence
attestation. Their reviewer cannot equal the producer and must satisfy a context-bound
review policy; the conflict disclosure is resolver-validated.
- A review flag or unattached digest is insufficient for criteria 12.1–12.2 and fails before
feasibility or label release.
### Decision: lineage expectations bind the influenced artifact
- A matching manifest subject is not enough because an embedded receipt could name a
different influenced artifact.
- Callers provide typed expectations containing purpose, artifact digest, and allowed use
counts. Generic lineage validation checks every current-chain receipt against those
expectations. Fact content forbids all labelled ancestry; model, threshold, matcher, and
fitted-component validation protects the current evaluation population while allowing
separately spent development populations.
.kiro/specs/detection-core/spec.json
+1 -1
View File
@@ -1,7 +1,7 @@
{
"feature_name": "detection-core",
"created_at": "2026-10-30T13:55:30Z",
"updated_at": "2026-10-30T13:55:30Z",
"updated_at": "2026-10-31T10:59:35Z",
"language": "en",
"phase": "tasks-generated",
"approvals": {
.kiro/specs/detection-core/tasks.md
+367 -150
View File
@@ -1,162 +1,379 @@
# Implementation Plan
_Requirement IDs use N.M where N is the requirement number in requirements.md and M is its acceptance criterion._
## Merge record
| Previous v1 item | Disposition in this plan |
|---|---|
| Completed groups 1–7 and 8.1 | Retained as regression substrate; their former requirement mappings do not count toward v2 coverage |
| Previous task 8.2 | Permanently parked after admission NO-GO #3; no task below revives that arm |
| Previous tasks 8.3–8.4 | Superseded by v2 consumer and proof conformance work; no rollout or live capability enablement is included |
## Task constraints
- Complexity: McCabe at most 10 per touched function, at most 50 logical lines per function and 400 per new module.
- The core contract package owns contracts only; no analysis logic moves into it.
- Every migration step keeps existing behavior byte-identical; the 2026-10-30 admission suite is the regression gate.
- Instrument rule: prove nonzero input reads before interpreting any zero.
- Pool measures, never trains: no fitted component reads pool or admission labels.
- New and amended code keeps McCabe complexity at most 10, functions at most 50 logical lines, and new modules at most 400 logical lines.
- Core owns contracts and pure validation only; analyzers, model execution, proof execution, persistence, acquisition, and rollout remain with adjacent owners.
- Every zero-result test first proves nonzero readable input; an unread input is an instrument failure.
- Historical v1 artifacts remain byte-identical. V2 creates a new baseline and is never trended against v1.
- The parked authorization arm remains parked. Admission orchestration uses synthetic/reference inputs only and does not consume a live population.
- Existing target facts may be schema-migrated without expanding vocabulary or behavior.
- [x] 1. Foundation: core contract package skeleton
- [x] 1.1 (P) Canonical trace schema in the core package with deterministic serialization
- Define the trace record (entry, hops with call-target completeness, operation, witnesses, guards, completeness reasons, judgment, digests) as a frozen, family-agnostic contract.
- Done when a fixed trace serializes to byte-identical canonical output on repeat runs and a conformance test pins the schema.
- _Requirements: 1.1, 1.2_
- _Boundary: core trace contract_
- [x] 1.2 (P) Three-way verdict vocabulary and verdict contract
- Define unguarded/discharged/indeterminate, the proven-claim-with-witness vs bounded-silence verdict, and the rule that indeterminate never converts to clean.
- Done when unit tests cover all three judgments, the conversion refusal, and zero-shot-judge-over-trace as the only model role on verdicts.
- _Requirements: 2.1, 2.2, 5.1, 5.2, 9.1_
- _Boundary: core verdict contract_
- [x] 1.3 (P) Coverage accounting schema with instrument refusal
- Define entry-recognition coverage, exactly-one-bucket-per-call resolution outcomes, named unchecked reasons, and zero-byte/unreadable input as an instrument failure at core level.
- Done when a zero-byte read fails the run in a unit test and the bucket exclusivity is pinned.
- _Requirements: 3.1, 3.2_
- _Boundary: core coverage contract_
- [x] 1.4 (P) Provenance digest stamping with mismatch refusal
- Define the analyzer/fact/acquisition/configuration/population/input digest stamping and refusal when any digest cannot be re-derived.
- Done when a mismatched digest is refused in a unit test.
- _Requirements: 6.1, 6.2_
- _Boundary: core provenance contract_
- [ ] 1. Establish migration-safe foundations
- [ ] 1.1 Freeze the v1 compatibility boundary and enumerate active readers and writers
- Pin representative v1 trace and report bytes as immutable regression fixtures.
- Record every current import and artifact-writing path that still depends on the authorization-shaped contract.
- Preserve existing v1 behavior as migration input without treating it as v2 conformance.
- Done when fixture tests detect any byte change and the import/writer census identifies every cutover target.
- _Requirements: 14.3, 15.3_
- [ ] 1.2 Build the strict canonical codec and resolver-backed artifact boundary
- Produce deterministic UTF-8 JSON with full SHA-256 identities, stable mapping order, and retained sequence order.
- Refuse non-finite numbers, unsupported values, unknown schemas, duplicate logical identities, and malformed digests.
- Validate resolved bytes, length, storage receipt, retention identity, trusted store, and configured size limits.
- Provide deterministic in-memory resolver and receipt fixtures for later contract tests.
- Done when identical values produce byte-identical output and each content or receipt mismatch fails before parsing.
- _Requirements: 1.4, 10.4_
- [x] 2. Measured budget replaces the static gate
- [x] 2.1 Record per-change complexity and reply time
- Extend runner records with complexity features (changed regions, call count, changed bytes) alongside the already-recorded seconds.
- Done when dev-pool records carry all complexity fields and seconds.
- _Requirements: 4.1, 12.1_
- _Boundary: runner records_
- _Depends: 1.3_
- [x] 2.2 Derive per-workload-class budgets from measurement
- Fit complexity to reply time on the dev (483-change) and pilot (264-change) record sets as measurement only; publish per-class budgets with n, percentile, and hardware; make the budget configurable per class, demoting the static constant to provisional default.
- Done when the derivation table (class, n, percentile, hardware, budget) is published under the fixed measurement schema and a new-baseline note records the change.
- _Requirements: 4.2, 12.1, 12.2_
- _Boundary: core budget contract_
- _Depends: 2.1_
- [x] 2.3 Budget gate and scoping record
- Gate failures name the change, elapsed time, and the derivation exceeded; the analyzed scope (change regions) is recorded per run.
- Done when an over-budget fixture fails with all three fields and the scope is present in the record.
- _Requirements: 4.3, 4.4_
- _Boundary: core budget contract_
- _Depends: 2.2_
- [x] 3. Prover arm adopts the core contract
- [x] 3.1 Prover trace implements the core schema
- Re-export/implement the core trace contract in the access_control arm without behavior change.
- Done when a conformance test proves the prover's trace satisfies the core schema field-for-field.
- _Requirements: 1.1, 1.2, 2.1_
- _Boundary: prover arm_
- _Depends: 1.1, 1.2_
- [x] 3.2 Prover verdict and coverage over the core contract
- Emit verdicts and coverage through the core contracts; existing strict-lane behavior stays byte-identical.
- Done when the keyword regression and prover suites pass unchanged after adoption.
- _Requirements: 3.1, 5.1, 5.2_
- _Boundary: prover arm_
- _Depends: 3.1, 1.3_
- [x] 4. Generalized admission contract
- [x] 4.1 (P) Consume-only manifest validation generalized into the core
- Move the ordered refusal chain (malformed, producer, unreviewed, label-isolated, label-exposure, acquisition-changed, spent) into the core admission contract; the benchmark validator becomes a thin wrapper.
- Done when all refusal branches pass unit tests through the core contract.
- _Requirements: 7.1, 7.2_
- _Boundary: core admission contract_
- _Depends: 1.4_
- [x] 4.2 Paired scoring and GO/NO-GO report schema with family scope
- Generalize paired vulnerable/fixed scoring, ordinary false alarms, gates, and the report schema into the core; add the claimed-families field.
- Done when the report schema includes family scope and every rate carries n and Wilson 95.
- _Requirements: 7.3, 10.2, 12.1_
- _Boundary: core admission contract_
- _Depends: 1.2, 4.1_
- [x] 4.3 Admission suite reproduces the 2026-10-30 NO-GO
- Re-run the pilot admission through the core contract and reproduce 0/12 pairs, 0.83 flagged/100, 0 instrument failures, NO-GO exactly.
- Done when the reproduced report matches the committed 2026-10-30 report field-for-field.
- _Requirements: 7.3, 12.1_
- _Boundary: admission measurement_
- _Depends: 4.2, 3.2_
- [x] 5. Anti-shortcut probes for any fitted component
- [x] 5.1 (P) Reversed-pair probe harness
- Build the fix/reversed-fix probe; the deletion-count shape feature must fail it (control test).
- Done when the control probe fails a shape-only component and passes a relation-based one.
- _Requirements: 8.1, 8.2_
- _Boundary: core probes_
- [ ] 2. Implement canonical semantic, provenance, and evidence contracts
- [ ] 2.1 (P) Represent family-neutral semantic relations and observations
- Cover registration, invocation, call, return, value transformation, operation effect, and constraint effect.
- Preserve role, location, conditions, provenance, and independent complete, incomplete, or unresolved status.
- Refuse conflicting complete candidates by producing explicit ambiguity rather than selecting one.
- Done when contract tests exercise every relation kind and all three relation statuses with canonical identities.
- _Requirements: 1.2, 4.1, 4.2, 4.3, 5.2_
- _Boundary: Semantic Evidence_
- _Depends: 1.2_
- [x] 5.2 Fitted-component gate
- Require per-repository holdout and untouched-population admission before any fitted component makes a claim; BLOCK reserved for demonstrated proofs.
- Done when a fitted component without admission evidence is refused by the gate in a test.
- _Requirements: 8.2, 9.2_
- _Boundary: core probes + admission_
- _Depends: 4.1, 5.1_
- [ ] 2.2 (P) Validate complete input and decision provenance
- Stamp analyzer, family contract, fact set, acquisition, configuration, population, and change-input identities.
- Require the closed named-input role set and resolve every referenced input through the trusted artifact boundary.
- Keep model and proof identities outside the immutable deterministic packet.
- Done when every stamp field has a tamper test and unreadable or substituted named input is refused.
- _Requirements: 10.1, 10.4_
- _Boundary: Provenance and Lineage_
- _Depends: 1.2_
- [ ] 2.3 Enforce append-only population custody
- Construct and validate contiguous reviewed-custodian receipt chains against the complete current log.
- Require compare-and-swap requests to name the expected log head and derive the next sequence and predecessor.
- Make spent status monotonic and reject old prefixes, omitted receipts, broken chains, and later availability claims.
- Provide deterministic in-memory custody-port and custodian fixtures for lineage and admission tests.
- Done when concurrent requests have one valid successor and no fixture can restore a spent population.
- _Requirements: 12.2, 13.1, 13.4_
- [ ] 2.4 Bind lineage purposes to exact influenced artifacts
- Require each lineage use to name its purpose, influenced artifact, source population, label access, and current-chain receipt.
- Enforce required and optional use counts for prompts, examples, thresholds, fitted models, facts, matching behavior, mappings, and promotion.
- Distinguish fact ancestry that forbids all label access from development lineage that may use a different spent population.
- Refuse protected-population influence, redirected artifact identities, wrong purposes, and incomplete receipt history.
- Done when matching subjects cannot mask a wrong influenced artifact and contaminated populations are deterministically refused.
- _Requirements: 13.1, 13.3, 13.4_
- [ ] 2.5 Bind immutable evidence packets to neutral decisions
- Preserve family identity, security object, analyzed scope, ordered semantics, observations, family fields, completeness, and provenance.
- Keep family decision codes separate from violation, satisfied, and indeterminate outcomes.
- Require bounded scope for satisfaction, named reasons for indeterminate outcomes, and a valid deterministic, model-corroborated, or execution-backed basis.
- Prevent incomplete or ambiguous required relations from producing violation or satisfaction.
- Done when positive, satisfied, and incomplete records round-trip canonically and every invalid packet/decision combination is refused.
- _Requirements: 1.1, 1.2, 1.3, 2.1, 2.2, 2.3, 2.4, 2.5, 10.1_
- [x] 6. Fact packs and consumer interfaces
- [x] 6.1 Fact-pack admission contract
- Ruleset loader admits family fact packs only with review evidence and provenance stamping; packs carry their family tag.
- Done when an unreviewed pack is refused and a reviewed pack stamps provenance in a test.
- _Requirements: 10.1_
- _Boundary: fact-pack contract_
- _Depends: 1.4_
- [x] 6.2 Consumer interfaces for push and proof
- Define the interfaces through which the pre-push transaction consumes traces/verdicts within its deadline and the proof layer attaches demonstrated evidence to a verdict; existing arms express behavior over the core contract.
- Done when both interfaces are exercised by conformance tests from a mock consumer.
- _Requirements: 9.2, 11.1, 11.2, 11.3_
- _Boundary: core consumer contracts_
- _Depends: 1.2, 3.2_
- [x] 6.3 Runners adopt core coverage, budget, provenance incrementally
- Migrate development_runner, arm_regression, prover_arm, run_manifests onto the core contracts one at a time, re-running the regression suite after each.
- Done when all runners consume the core contracts and the keyword regression passes at the pre-migration count plus new tests.
- _Requirements: 3.1, 4.1, 4.3, 6.1, 11.3, 12.1_
- _Boundary: benchmark runners_
- _Depends: 2.3, 3.2, 6.2_
- [ ] 3. Add vulnerability-family contracts and unlike reference families
- [ ] 3.1 Validate a closed vulnerability-family declaration
- Require family/schema versions, ecosystems, security-object fields, roles, decisions, outcome rules, completeness conditions, residual questions, and indeterminate reasons.
- Enforce typed family fields and exact ecosystem/schema scope.
- Declare whether each role accepts product evidence or requires reviewed excluded-dependency facts.
- Refuse undeclared fields, roles, decisions, reasons, or determinate outcomes without their minimum evidence.
- Done when the family registration matrix accepts only declared combinations and reports unsupported scope without cross-family fallback.
- _Requirements: 1.3, 2.1, 2.4, 3.1, 3.2, 3.3, 3.4, 4.4, 9.1_
- [ ] 3.2 Build exact family catalog resolution and capability declarations
- Re-derive every contract and catalog digest and require canonical ordering and unique family/version/schema identities.
- Resolve a contract only when family, digest, and ecosystem identify exactly one entry.
- Validate that each family capability names its contract, readiness fixtures, development population, and admission boundary.
- Done when duplicate/substituted catalog entries and each missing or corrupted capability field are refused.
- _Requirements: 3.1, 3.3, 3.4, 14.5_
- [ ] 3.3 Create obligation and taint-flow reference conformance fixtures
- Model an operation-plus-constraint family and an ordered source-to-sink family without target vocabulary.
- Provide positive, negative, and ambiguity or missing-evidence cases for both semantic shapes.
- Require each family to produce violation, bounded satisfaction, and indeterminate output through the same validators.
- Done when both fixture suites pass without adding a family-specific field to any shared core contract.
- _Requirements: 3.5, 8.2_
- [x] 7. Validation
- [x] 7.1 Fixed measurement schema enforced in reports
- Every report carries population/freeze digests, judgment vocabulary, rates with n and Wilson 95, coverage, instrument failures, runtime; vocabulary/population change starts a new baseline.
- Done when a schema-validation test rejects a report missing any field and accepts the 2026-10-30 report.
- [ ] 4. Expose operational coverage, budget, and readiness gates
- [ ] 4.1 (P) Account for instrument reads and every semantic-stage outcome
- Count reads, questions, sites considered and recognized, transitions, security objects, constraints, determinate outcomes, and findings.
- Put every considered relation in exactly one complete, incomplete, or unresolved bucket.
- Fail unopened or zero-byte tracked input and diagnose the last verified nonzero stage and first missing expected output.
- Done when stage-by-stage fixtures distinguish a valid zero from unread input and identify every injected first failure.
- _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 15.2, 15.4_
- _Boundary: Coverage Funnel_
- _Depends: 2.1, 2.5_
- [ ] 4.2 (P) Derive and enforce measured workload budgets and change scope
- Record changed regions, semantic-transition count, changed bytes, and total reply time.
- Derive workload-class budgets with sample count, percentile, reference workload, reference hardware, and canonical identity.
- Resolve the changed-byte measurement and refuse missing, zero, non-finite, or incompatible complexity data.
- Reject evidence and claim anchors outside the supplied affected scope and name the exceeded derivation on budget failure.
- Done when class derivation is deterministic and scope escape or elapsed-budget excess yields a named gate failure.
- _Requirements: 6.1, 6.2, 6.3, 6.4_
- _Boundary: Budget and Scope_
- _Depends: 1.2, 2.5_
- [ ] 4.3 Record ordered readiness evidence with explicit evidence-use roles
- Cover instrument, recognition, transitions, constraints, pairs, development, admission, and consumer integration in fixed order.
- Bind each result to arm/family/ecosystem, analysis identity, population use, fixtures, predecessors, measurement, and correction lever.
- Keep regression, development, repository-holdout, fresh-admission, and integration evidence uses non-interchangeable.
- Done when each stage accepts the correct positive/negative/ambiguity triplet and rejects the wrong evidence-use role.
- _Requirements: 8.1, 8.2, 8.6_
- [ ] 4.4 Stop promotion on failed, absent, or stale prerequisites
- Authorize a stage only when every ordered predecessor is current and passed.
- Derive transitive staleness from core schema, semantic vocabulary, family, facts, acquisition, configuration, model, and outcome mapping.
- Return the first failing layer and named correction lever without mutating historical results.
- Done when one changed identity makes all dependent later results stale and later population/integration gates refuse to run.
- _Requirements: 8.3, 8.4, 8.5, 15.5_
- [ ] 5. Replace semantic extension points with reviewed fact packs
- [ ] 5.1 Validate versioned packs and independent review evidence
- Bind schema, family, ecosystem, language, stable facts, declared roles, normalized content, and review evidence.
- Accept source, official-documentation, or independently reviewed behavioral evidence under a trusted review policy.
- Resolve conflict disclosures and require producer/reviewer independence where behavioral evidence is used.
- Reject label-derived semantic-fact ancestry, unknown scope/roles, missing review, and inconsistent digests.
- Done when the schema/scope/review matrix accepts only fully reviewed label-isolated packs.
- _Requirements: 11.1, 11.4, 13.3_
- [ ] 5.2 Validate complete structural observations and matcher lineage
- Match the declared requirement set, including qualified name, shape, receiver, target, argument, and configuration evidence as applicable.
- Resolve every observation artifact and require its identifier set to equal the fact’s complete structural requirements.
- Bind matcher revision, behavior digest, and current lineage to the exact evaluated product location.
- Prevent name-only, partial, fabricated, ambiguous, or protected-population-contaminated matches from becoming complete.
- Done when only a fully observed structural match with valid matcher lineage is eligible for effect application.
- _Requirements: 10.2, 11.2, 11.3, 13.4_
- [ ] 5.3 Apply declared fact effects with reproducible provenance
- Materialize only relation kinds, roles, source/target roles, conditions, and effects declared by the matched fact and family.
- Attach exact pack, fact, product location, structural observation, matcher, and match-reference identities to every derived relation.
- Preserve unmet conditions as incomplete evidence and conflicting eligible matches as unresolved ambiguity.
- Mark label-influenced matching behavior or promotion as spent before the relation can participate in evaluation.
- Done when valid matches yield canonical effects while partial and ambiguous cases remain non-determinate with named reasons.
- _Requirements: 4.1, 4.2, 4.3, 10.2, 11.2, 11.3, 11.5, 13.4_
- [ ] 5.4 Revalidate fact-derived relations and enforce the dependency boundary
- Re-open referenced packs, facts, match evidence, review policy, lineage, custody, and resolved bytes for every fact-derived relation.
- Reject undeclared roles, invalid scope, inconsistent content, and fact provenance that cannot be reproduced.
- Require reviewed semantic facts for determinate behavior attributed to excluded dependencies.
- Preserve missing dependency knowledge as named incomplete evidence rather than product-derived safety.
- Done when product-only dependency behavior is refused, a missing fact stays indeterminate, and the reviewed structurally matched equivalent is accepted.
- _Requirements: 2.4, 4.4, 10.4, 11.4, 11.5_
- [ ] 6. Add bounded model handoff and semantic shortcut probes
- [ ] 6.1 (P) Preserve reproducible model requests, results, and responses
- Present the canonical evidence, completeness snapshot, family declaration, and declared residual question without importing a model runtime.
- Freeze provider, model, program, prompt, examples, fitted component, configuration, adapter, decision mapping, and input-bundle identities before execution.
- Preserve normalized responses plus one bounded redacted inline value or trusted immutable artifact, including failures.
- Done when matching success/failure fixtures round-trip and substituted request, packet, response, program, or mapping identities are refused.
- _Requirements: 9.1, 9.2, 10.3_
- _Boundary: Model Handoff_
- _Depends: 2.4, 3.1_
- [ ] 6.2 Apply model results without rewriting deterministic evidence
- Validate current prompt/example/fitted-model/mapping lineage against the evaluated population and declared artifacts.
- Reject proposer/adjudicator role collision and results for another packet, family, residual, or mapping.
- Allow model-corroborated output only when deterministic residual prerequisites are complete.
- Prevent model output from creating semantic relations, satisfaction over missing evidence, or execution-backed basis.
- Done when accepted derivation leaves the evidence packet byte-identical and every incomplete or contaminated path remains indeterminate.
- _Requirements: 2.4, 2.5, 9.3, 9.4, 9.5, 13.1_
- [ ] 6.3 (P) Evaluate independently reviewed equal-semantics reversed-surface probes
- Derive semantic projections from records and family contracts rather than trusting caller summaries.
- Resolve distinct surface artifacts and independently reviewed semantic-equivalence evidence.
- Reject unequal semantics, digest-only review, non-finite tolerances, and fabricated passed summaries.
- Detect a surface-shape component while accepting a relation-based component within its pre-registered tolerance.
- Done when probe results are reproducible solely from canonical probe inputs and review policy.
- _Requirements: 13.2_
- _Boundary: Probes_
- _Depends: 3.3, 5.1_
- [ ] 7. Compose validated evidence into bounded decisions and consumer output
- [ ] 7.1 Revalidate deterministic and fact-derived records through one shared context
- Bind the exact catalog, packs, model declarations, lineage set, review policies, provenance trust, and custody policy.
- Resolve the family contract and all seven named provenance inputs for each raw record.
- Re-run family and fact validation and reject embedded pack, lineage, or named-input substitutes.
- Done when raw, forged, and context-mismatched records cannot reach admission or consumer decision code.
- _Requirements: 4.4, 10.1, 10.2, 10.4, 12.6, 14.1_
- [ ] 7.2 Replay model derivation at the shared validation seam
- Require deterministic records to omit model derivation and model-corroborated records to include both request and result.
- Re-run model application from the pre-model record and compare the derived record canonically.
- Refuse constructible “validated” markers or model programs outside the context.
- Done when admission and consumer tests exercise the same validator and neither can bypass it with a raw record.
- _Requirements: 9.2, 9.3, 9.4, 9.5, 10.3, 10.4, 12.6, 14.1_
- [ ] 7.3 (P) Derive defect claims, bounded satisfaction, and bounded silence
- Require exact change-tied locations and complete evidence for defect claims.
- Carry completed question identities and exact scope for satisfaction.
- Preserve named coverage reasons and record identities for every undecided question.
- Keep incomplete paths silent unless a separate complete path independently supports a claim.
- Done when mixed complete/incomplete fixtures produce supported claims plus bounded silence without any unbounded clean result.
- _Requirements: 2.2, 2.3, 7.1, 7.2, 7.3, 7.4, 14.4_
- _Boundary: Verdict and Proof_
- _Depends: 2.5, 3.1_
- [ ] 7.4 Validate differential proof attachment without enabling rollout
- Re-derive reviewed producer, sandbox, oracle, affected/safe side, control, witness, and verification identities.
- Resolve all proof artifacts under the supplied producer, reviewer, store, and byte-limit trust policy.
- Upgrade basis only for demonstrated affected-only behavior tied to the same family, packet, location, and change/control.
- Reject opaque, self-asserted, inconclusive, same-side, or mismatched proof records.
- Done when only the valid differential fixture yields execution-backed basis and no attachment changes rollout policy.
- _Requirements: 2.5, 9.6, 14.2_
- [ ] 7.5 Integrate validated records, proof, scope, and budget at the core consumer edge
- Accept raw validation envelopes and call the shared validator before creating any output.
- Preserve every bounded-silence reason and reject records or anchors outside the supplied change scope.
- Apply optional proof attachments through the proof validator and include all processing in total elapsed time.
- Return defects, satisfactions, silence, input identities, complexity, and measured budget identity.
- Done when a synthetic push returns a bounded verdict and scope, trust, or deadline violations fail explicitly.
- _Requirements: 6.4, 14.1, 14.4_
- _Depends: 4.2, 7.2, 7.4_
- [ ] 8. Implement fixed measurement and leakage-safe admission
- [ ] 8.1 (P) Define comparable baseline identities and fixed metric algebra
- Bind arm, population/freeze, claimed scopes, exact contract/fact/model sets, acquisition, analyzer, configuration, denominator, threshold set, and outcome mapping.
- Define closed numerator/denominator rules for pair correctness, recall, fixed and ordinary false alarms, precision, semantic coverage, and model success.
- Require Wilson 95 intervals with numerator and denominator; serialize zero denominators as null and prevent them from passing.
- Treat any identity mutation as a new baseline and refuse cross-arm or cross-scope aggregation.
- Done when a baseline mutation matrix and two-arm isolation fixture distinguish every non-comparable result.
- _Requirements: 11.6, 15.1, 15.3_
- _Boundary: Measurement_
- _Depends: 4.1, 4.4_
- [ ] 8.2 Build complete measurement records and typed repository-holdout reports
- Record family decision vocabulary, neutral outcomes, evidence bases, fixed metrics, semantic funnel, model outcomes/failures, instrument failures, runtime, and readiness ancestry.
- Require every later-stage record to carry current prerequisite readiness stage results and digests.
- Define a typed repository-holdout report with an explicit evidence-use role, training and holdout manifest references, repository identities, fitting lineage, and distinct measurement/report identities.
- Refuse zero or clean-looking results unless tracked input reads and prerequisite semantic stages passed.
- Done when missing metrics, counts, failure fields, readiness ancestry, or holdout-report identity fields invalidate the corresponding record.
- _Requirements: 13.2, 15.1, 15.2, 15.4, 15.5_
- [ ] 8.3 Validate independently reviewed census and manifest payloads
- Canonically bind every substantive census and manifest field into separate review-subject digests.
- Require producer/reviewer separation, admitted reviewer identity, allowed identity pairing, and resolver-validated conflict disclosure.
- Reject self-review, copied attestations, unknown reviewers, malformed review digests, and unreadable disclosures.
- Preserve aggregate census cardinalities without exposing case identities or per-case labels.
- Done when only independently attested label-blind payloads can proceed to manifest validation.
- _Requirements: 12.1, 12.2_
- _Boundary: measurement schema_
- _Depends: 4.2, 6.3_
- [x] 7.2 End-to-end: one push through core to verdict to admission report
- Run one dev-pool push end-to-end: scope, recognize, resolve, trace, verdict, digests, admission report on the dev pool.
- Done when the E2E test produces a verdict with witness or bounded silence plus a complete report.
- _Requirements: 5.1, 5.2, 7.3, 11.1_
- _Boundary: end-to-end integration_
- _Depends: 6.3, 7.1_
- [ ] 8.4 Validate exact admission context, readiness, scope, sets, and threshold lineage
- Bind the exact evidence-validation context, reviewed census, denominator, thresholds, readiness results, untouched population, and current custody head.
- Re-derive claimed scopes and exact contract, fact, model, threshold, and outcome-mapping sets from canonical objects.
- Require current instrument-through-development readiness with matching analysis identity and evidence-use roles.
- Validate prompt, example, fitted-model, fact, matcher, mapping, promotion, and threshold lineage before labels open.
- Reject label exposure, acquisition drift, context substitution, stale readiness, scope mismatch, or prior spend.
- Done when each refusal branch fails before feasibility and no digest-only set or lineage claim is accepted.
- _Requirements: 12.1, 12.2, 13.1_
- [ ] 8.5 Refuse arithmetically impossible threshold configurations
- Enumerate reachable counts from label-blind pair, case, semantic-site, and model-call capacities.
- Evaluate count, rate, per-100, and Wilson-bound thresholds with the fixed comparator direction for each metric.
- Require every grouped threshold to have at least one jointly attainable count assignment.
- Done when the historical impossible-denominator case is a configuration refusal rather than a measured no-go.
- _Requirements: 12.3, 12.4_
- [ ] 8.6 Validate the pre-label population-spend protocol inside core
- Construct a label-release request bound to the exact validated authorization and current custody-log head.
- Validate the returned receipt as the exact current-chain direct successor that changes the named population from available to spent.
- Reject missing, stale, mismatched, non-successor, or availability-restoring receipts before scoring authorization.
- Keep durable append, compare-and-swap execution, label release, persistence, crash handling, and concurrency outside core.
- Done when only the exact spent successor authorizes scoring and every other receipt shape is refused without an external side effect.
- _Requirements: 12.2, 13.4_
- [ ] 8.7 Derive scoped admission metrics from canonical case evidence
- Re-run shared evidence validation, instrument, budget, verdict, and proof validation for every case.
- Match case roles and pairs back to the reviewed census and derive decision vocabulary and counts from the exact catalog.
- Score quality only over pre-registered family/ecosystem scopes while retaining whole-population coverage.
- Compute paired behavior, ordinary false alarms, semantic coverage, model outcomes, and all standard metrics without trusting runner aggregates.
- Done when forged decisions, substituted contracts, invalid fact review, contaminated lineage, or case-count drift fail before metric production.
- _Requirements: 12.5, 12.6, 15.1, 15.2_
- [ ] 8.8 Assemble evidence-backed gates and the final admission report
- Emit instrument, budget, readiness, and quality gates with reasons and supporting identities.
- Bind the report to the validation context, authorization, pre-label spend receipt, census, and complete case-result set.
- Require every mandatory metric and gate, plus model success when applicable, before allowing GO.
- Keep model-corroborated, deterministic, and execution-backed counts separate.
- Done when any missing/failed mandatory gate forces NO-GO and a complete passing synthetic fixture yields a reproducible GO report.
- _Requirements: 12.6, 15.1, 15.2, 15.4, 15.5_
- [ ] 8.9 Authorize fitted components only from replayed probes and disjoint evaluation
- Resolve the component’s unique model declaration, family contract, review policy, and custody policy from the admission-bound context.
- Re-evaluate every canonical probe input and require the derived result set to match the component evidence.
- Consume the typed repository-holdout report from task 8.2 and validate its evidence-use role, training/holdout references, report identity, and fitting lineage.
- Prove the report’s training and repository-held-out repository sets are disjoint.
- Require an admission GO that was untouched before its bound label-release spend.
- Done when fabricated summaries, caller-selected policy substitutes, overlapping repositories, contaminated lineage, or mismatched admission are refused.
- _Requirements: 13.1, 13.2, 13.4_
- [ ] 8. Integration with existing specs (binding phase, roadmap 2026-10-30)
- [x] 8.1 Prover native conformance to the core trace contract
- access-control-prover's Trace implements core TraceRecord natively; hunt/core_adapter.py is removed once conformance tests pass; behavior byte-identical (admission suite regression gate).
- Done when test_core_adapter.py equivalents pass against native types and the adapter module is deleted.
- _Requirements: 1.1, 1.2, 11.3_
- _Boundary: prover arm_
- _Depends: 3.1_
- [ ] 8.2 Fact-pack workstream from the recognition-coverage table
- valid-detection-benchmark task 7.1's manifest-derived table becomes reviewed fact packs admitted via core/factpack.py, family-tagged and digest-stamped; measured on pool-train, admission-gated on the untouched pilot.
- Done when one new ecosystem pack passes admission gates and appears with its family in the report.
- _Requirements: 10.1, 10.2, 7.3_
- _Boundary: fact-pack contract + benchmark_
- _Depends: 6.1, 4.2_
- _Parked 2026-10-31: final gated round NO-GO #3 (rule 10 stop condition, maintainer strategy-review); done-when unreachable for this arm -- the unique-declaration member-call binding increment ships (dc28325, dev complete-share 11.9->19.2 with recall/FA unchanged) but the admission gap is entry recognition (4/12 repos unrecognised; coverage 8/12) and framework operation facts, not call resolution. See benchmarks/measurements/2026-10-31-access-control-admission-unique-binding/_
- [ ] 8.3 Pre-push hook consumes the core consumer contract
- pre-push-safety-net builds PushRequest and consumes PushVerdict via core/consumer.py; budget from the measured per-class derivation; one real dev-pool push E2E.
- Done when an E2E hook run on a dev-pool repo emits a PushVerdict with witnesses/reasons and a measured-class budget.
- _Requirements: 4.2, 11.1, 11.3_
- _Boundary: pre-push transaction_
- _Depends: 6.2, 2.2_
- [ ] 8.4 Proof layer attaches demonstrated evidence to verdicts
- search-with-proof attaches proof artifacts to PushVerdict claims; BLOCK only on proven claims; bounded-silence entries stay advisory.
- Done when a demonstrated-finding fixture blocks and an unproven claim stays advisory in a test.
- _Requirements: 9.2, 11.2_
- _Boundary: proof layer_
- _Depends: 6.2_
- [ ] 9. Integrate adjacent producers and benchmark-owned orchestration
- [ ] 9.1 (P) Translate the parked authorization trace conservatively into v2 evidence
- Move conversion out of the family trace domain and keep authorization vocabulary outside shared core modules.
- Map missing per-hop knowledge to incomplete relations and indeterminate outcomes without behavior lifting.
- Preserve existing parked-arm regression behavior and avoid any admission rerun or reactivation.
- Done when adapter conformance matches the frozen v1 behavior while the authorization domain no longer imports v2 core types directly.
- _Requirements: 3.5, 14.3_
- _Boundary: Authorization Adapter_
- _Depends: 3.3, 7.2_
- [ ] 9.2 (P) Translate hunt-owned model execution through the bounded handoff
- Convert structured executor inputs and outputs to the canonical request/result contracts.
- Keep prompt creation, provider calls, caching, parsing, and redaction in the hunt subsystem.
- Reject mismatched declared results through core validation without importing DSPy into core.
- Done when a fake executor success/failure pair crosses the adapter and core’s import census contains no model-runtime dependency.
- _Requirements: 9.1, 9.2, 9.3, 9.4, 9.5, 14.3_
- _Boundary: Model Adapter_
- _Depends: 6.2, 7.2_
- [ ] 9.3 (P) Translate demonstrated search results into trusted proof attachments
- Map existing typed affected/safe verification records without moving sandbox or oracle execution into core.
- Preserve producer, control, witness, storage, and verification identities required by proof validation.
- Refuse inconclusive and mismatched fixtures at the adapter/core seam.
- Done when a canonical demonstrated fixture attaches successfully and every negative proof shape remains at its prior evidence basis.
- _Requirements: 9.6, 14.2, 14.3_
- _Boundary: Proof Adapter_
- _Depends: 7.4_
- [ ] 9.4 (P) Migrate the existing entry-pack loader to the reviewed v2 schema
- Translate existing metadata and fact rows without adding framework vocabulary or changing target behavior.
- Supply complete structural match evidence and pack/fact/matcher provenance to the shared fact validator.
- Keep acquisition and review ownership outside core.
- Done when the migrated existing pack produces byte-stable behavior through v2 and partial/name-only matches remain incomplete.
- _Requirements: 10.2, 11.1, 11.2, 11.3, 11.4, 11.5, 14.3_
- _Boundary: Fact Pack Adapter_
- _Depends: 5.4_
- [ ] 9.5 (P) Produce readiness and development records through benchmark-owned adapters
- Construct canonical readiness results while leaving sequencing and persistence with benchmark workflows.
- Emit development observations with v2 identities, evidence-use roles, instrument receipts, and prerequisite ancestry.
- Stop a runner before population work when readiness is failed, absent, or stale.
- Done when development fixtures pass through v2 and an injected semantic identity change prevents the later run.
- _Requirements: 8.1, 8.3, 8.4, 8.5, 8.6, 14.3, 15.5_
- _Boundary: Benchmark Readiness Adapter_
- _Depends: 4.4, 8.2_
- [ ] 9.6 Integrate benchmark-owned admission scoring and guarded custody
- Delegate neutral scoring to core while retaining population persistence and run orchestration in the benchmark.
- Make the custodian the sole append-only writer and combine durable compare-and-swap spend plus label release as one guarded operation.
- Supply expanded manifest, lineage, resolver, and validation-context identities without exposing labels early.
- Exercise durable append, release, crash-before-append, crash-after-append, and concurrent-winner behavior at this external boundary.
- Done when synthetic admission completes end to end and injected crash/concurrency cases prove labels reach at most the successful compare-and-swap winner.
- _Requirements: 12.1, 12.2, 12.3, 12.4, 12.5, 12.6, 13.4_
- _Depends: 8.8, 9.5_
- [ ] 10. Complete the single-writer v2 migration
- [ ] 10.1 Prepare schema-routed legacy reads and cutover tests without publishing v2
- Add an internal read-only decoder that validates committed v1 traces and reports without constructing v2 determinate outcomes automatically.
- Route v1/v2 test reads by explicit schema version and prohibit all new writes through the legacy surface.
- Keep the top-level public API unchanged until the import and writer census reaches zero.
- Done when historical artifacts validate byte-unchanged, legacy write attempts fail, and cutover tests pass without publishing top-level v2 exports.
- _Requirements: 14.3, 15.3_
- [ ] 10.2 Stop v1 writers and publish the single v2 public surface
- Re-run repository import and artifact-writer censuses after all adjacent adapters use v2.
- Stop v1 writes and remove the compatibility shim, authorization-shaped public exports, and family-domain conversion only when both censuses reach zero.
- Publish dependency-free v2 contracts through the top-level core API after the zero-census gate.
- Retain the read-only legacy decoder and refuse comparison between v1 and v2 baseline identities.
- Done when no production writer/import targets the v1 contract, historical fixtures remain unchanged, and all public writes use v2.
- _Requirements: 14.3, 15.3_
- [ ] 11. Validate readiness, failure visibility, and implementation boundaries
- [ ] 11.1 Run both reference families through the complete synthetic readiness path
- Start from a verified nonzero input and run recognition, transitions, security object/constraints, pair, development, reviewed synthetic admission, guarded spend, and consumer verdict.
- Produce violation, bounded satisfaction, and bounded silence for both unlike families.
- Require current prerequisite readiness identities at every later stage.
- Done when both authored repositories complete the path without target-specific core fields or a real admission population.
- _Requirements: 3.5, 8.1, 8.2, 8.4, 8.6, 12.1, 14.1, 15.5_
- [ ] 11.2 Inject one failure at every semantic and governance stage
- Exercise unread/zero input, missing recognition, incomplete transition, missing constraint, pair mismatch, failed development, contaminated admission, and refused integration.
- Verify each run stops at the first failing stage with a named correction lever and no later metric or clean result.
- Mutate a semantic identity and verify transitive staleness through all dependent readiness and measurement records.
- Done when every injected failure is diagnosed at its source and no late admission-only surprise remains.
- _Requirements: 5.3, 5.4, 5.5, 8.3, 8.4, 8.5, 15.4, 15.5_
- [ ] 11.3 Verify bounded validation cost and total consumer deadline accounting
- Measure canonicalization and validation across increasing relation, observation, receipt, and case counts.
- Exercise large transition sets for linear behavior and bounded memory without weakening semantic checks.
- Include optional model and proof processing in the same measured consumer budget.
- Publish sample count, percentile, workload, hardware, and derived budget through the fixed measurement contract.
- Done when performance fixtures satisfy the derived workload envelope and any excess fails the named budget gate.
- _Requirements: 1.4, 6.1, 6.2, 6.3, 6.4_
- [ ] 11.4 Run the full migration, security, and regression gate
- Verify core imports only standard-library and lower core layers and contains no target, DSPy, benchmark, proof-execution, or analyzer dependencies.
- Run all unit, adapter, migration, reference-family, end-to-end, admission, and historical compatibility suites.
- Enforce function/module complexity caps and scan for remaining v1 imports or writers.
- Prove each test instrument read nonzero fixture bytes before interpreting zero findings or failures.
- Done when the complete suite passes, boundary/import censuses are clean, historical bytes are unchanged, and all declared adapter and consumer seams pass conformance.
- _Requirements: 1.4, 10.4, 14.3, 15.3, 15.4_