docs(detection-core): authenticate post-spend case release

This commit is contained in:
Marius Ciepluch
2026-11-01 10:03:58 +01:00
parent b28045810e
commit 007dd7aa77
4 changed files with 140 additions and 20 deletions
+79 -11
View File
@@ -229,7 +229,9 @@ src/openultrasast/core/
├── verdict.py # Claims, satisfaction, silence, proof attachment and aggregation
├── probes.py # Reviewed semantic-equivalence probes and fitted-component evidence gates
├── measurement.py # Complete baseline identity, rates, funnel and report schema v2
├── admission.py # Census, feasibility, evidence-derived scoring and fitted-component gate
├── admission_case.py # Case results and authenticated post-spend release bindings
├── admission_metrics.py # Evidence-derived scoped and whole-population metric algebra
├── admission.py # Census, feasibility, report assembly and fitted-component gate
├── consumer.py # Scoped push request and verdict consumption
└── legacy.py # Read-only v1 trace and report decoder and validator
@@ -374,16 +376,21 @@ sequenceDiagram
Custodian-->>Benchmark: Durable spend receipt then release labels
Benchmark->>Arm: Run frozen analyzer facts and model
Arm-->>Benchmark: Immutable case results
Benchmark->>Core: Preparation receipt case results and instrument receipts
Core->>Core: Revalidate canonical inputs and score scopes separately
Core-->>Benchmark: Report bound to authorization and spend receipt
Benchmark->>Custodian: Submit immutable case-result identities
Custodian-->>Benchmark: Attested case release bound to spend receipt
Benchmark->>Core: Preparation receipt case release results and instrument receipts
Core->>Core: Revalidate release bindings canonical inputs and scoped scores
Core-->>Benchmark: Report bound to authorization spend receipt and case release
```
Core never creates, edits, labels, or persists the admission population. The benchmark
custodian performs durable append and label release as one guarded operation: a crash before
the append releases no labels, while a crash after it leaves the population spent. Concurrent
attempts race on the same custody head, and only the compare-and-swap winner receives label
bytes.
bytes. The case release is constructed only after that spend. It discloses the per-case
role and pair mapping that the aggregate census deliberately withholds, and it binds those
labels to the exact immutable case-result and evidence/input identities submitted for
scoring.
## Requirements Traceability
@@ -400,7 +407,7 @@ bytes.
| 9.1, 9.2, 9.3, 9.4, 9.5, 9.6 | Bounded model and proof roles | Model Handoff, Family Contract, Verdict, Proof Attachment | Model residual and proof flow |
| 10.1, 10.2, 10.3, 10.4 | Complete reproducibility identities | Provenance and Lineage, Fact Pack, Model Handoff | All record flows |
| 11.1, 11.2, 11.3, 11.4, 11.5, 11.6 | Reviewed semantic packs and supported scope | Fact Pack, Semantic Evidence, Measurement | Pack ingestion and reports |
| 12.1, 12.2, 12.3, 12.4, 12.5, 12.6 | Admission prerequisites and scoped scoring | Admission, Readiness, Measurement | Admission sequence |
| 12.1, 12.2, 12.3, 12.4, 12.5, 12.6 | Admission prerequisites, authenticated case release and scoped scoring | Admission, Readiness, Measurement | Admission sequence |
| 13.1, 13.2, 13.3, 13.4 | Development and evaluation separation | Provenance and Lineage, Probes, Admission | Freeze and admission validation |
| 14.1, 14.2, 14.3, 14.4, 14.5 | Shared consumer and family-spec contract | Consumer, Verdict, Family Capability Declaration, Legacy Adapter | Push and migration flows |
| 15.1, 15.2, 15.3, 15.4, 15.5 | Fixed versioned measurement and baseline rules | Measurement, Coverage, Readiness | Development and admission reporting |
@@ -423,7 +430,7 @@ bytes.
| Verdict and Proof | Decision | Produce bounded outputs and attach proof | 7.1–7.4, 9.4, 9.6, 14.2, 14.4 | Evidence, Model | Service, State |
| Probes | Governance | Test semantic shortcut resistance | 13.2 | Evidence, Family, Fact Pack review policy | Service |
| Measurement | Evaluation | Create baseline identity and fixed report schema | 11.6, 15.1–15.5 | Coverage, Readiness | Service, State |
| Admission | Evaluation | Validate manifests, feasibility and scoped scores | 12.1–12.6, 13.4 | Readiness, Measurement, Evidence Validation, Verdict, Probes, Budget, Provenance | Service, Batch |
| Admission | Evaluation | Validate manifests, feasibility, post-spend case release and scoped scores | 12.1–12.6, 13.4 | Readiness, Measurement, Evidence Validation, Verdict, Probes, Budget, Provenance | Service, Batch |
| Consumer | Edge | Validate push requests and return bounded verdicts | 6.4, 14.1–14.5 | Budget, Evidence Validation, Verdict, Provenance, Canonical resolver | Service |
| Legacy Decoder | Migration | Read and validate historical v1 only | 14.3, 15.3 | Canonical Codec, Evidence | Service |
| External Adapters | Integration | Translate authorization, DSPy and proof records | 3.5, 9.1–9.6, 14.2–14.3 | Public core | Service |
@@ -1721,6 +1728,11 @@ authorization; the derived result digests must exactly equal the fitted-componen
separately.
- The census exposes only reviewed aggregate cardinalities, never case identities or
per-case labels. It is bound to the same population digest as the manifest.
- After the population is durably spent, the custodian produces an independently attested
`AdmissionCaseRelease`. The release is bound to the population, manifest, authorization,
preparation, exact spend receipt, and custodian identity. Its bindings associate each
case identity with one scope, role, scoped pair identity where applicable, result digest,
evidence-input digests, change-input digests, and input-manifest digest.
- Before labels open, feasibility enumerates the arithmetically possible counts using pair,
case, semantic-site, and model-call capacities and applies count, rate, per-100, or
Wilson-bound statistics. Every threshold group must have an attainable joint count. An
@@ -1731,6 +1743,10 @@ authorization; the derived result digests must exactly equal the fitted-componen
trusted. The scorer re-runs record/fact/verdict validation and refuses supplied
claims/satisfaction/silence that differ from the derived result. Core never persists or
relabels cases.
- Scoring treats the post-spend case release as the sole authority for case roles and pair
membership. It re-derives each case result and its operational projection, requires an
exact one-to-one binding, and refuses missing, extra, duplicate, substituted, swapped, or
cross-scope cases before producing any metric.
- Admission authorization produces a custody-head-bound label-release request. The
benchmark custodian must durably spend the population before releasing label bytes;
scoring revalidates that exact current-chain receipt and never emits a post-scoring spend
@@ -1898,6 +1914,32 @@ class AdmissionCaseResult:
total_reply_ms: int
result_digest: str
@dataclass(frozen=True)
class ReleasedCaseBinding:
case_id_digest: str
scope: ScopeKey
role: AdmissionCaseRole
pair_id_digest: str | None
case_result_digest: str
evidence_input_digests: tuple[str, ...]
change_input_digests: tuple[str, ...]
input_manifest_digest: str
binding_digest: str
@dataclass(frozen=True)
class AdmissionCaseRelease:
schema_version: str
population_digest: str
manifest_digest: str
authorization_digest: str
preparation_digest: str
label_release_receipt_digest: str
custodian_identity: str
bindings: tuple[ReleasedCaseBinding, ...]
release_payload_digest: str
review: IndependenceAttestation
release_digest: str
@dataclass(frozen=True)
class MetricResult:
metric_id: MetricId
@@ -1977,6 +2019,7 @@ class AdmissionReportV2:
validation_context_digest: str
authorization_digest: str
label_release_receipt_digest: str
case_release_digest: str
case_result_digests: tuple[str, ...]
in_scope_case_count: int
whole_population_case_count: int
@@ -2015,6 +2058,7 @@ def score_admission(
manifest: AdmissionManifestV2,
preparation: AdmissionPreparation,
label_release_receipt: PopulationUseReceipt,
case_release: AdmissionCaseRelease,
cases: Sequence[AdmissionCaseResult],
context: AdmissionValidationContext,
custody: PopulationCustodyPort,
@@ -2042,6 +2086,15 @@ and the conflict disclosure must resolve under the policy's store and byte limit
self-review, unknown reviewer, fabricated digest, or review copied from another payload is
refused before feasibility.
`AdmissionCaseRelease` is a post-spend contract, not part of the label-blind census or
manifest. Its payload digest includes every substantive field except the review attestation;
the release digest includes both. The review producer must equal both the declared custodian
and the validated spend receipt's custodian, while the reviewer must be distinct and admitted
by the context's admission-review policy. The attestation subject must equal the release
payload digest, and its conflict disclosure is resolved under the same store and byte limits.
The release population, manifest, authorization, preparation, and receipt digests must match
the exact values being scored.
The manifest identity, census population, claimed scopes,
contract-set, facts, acquisition, analyzer, configuration, model-program set, denominator,
threshold set, and outcome mapping must all match. `authorize_admission` re-derives the
@@ -2074,7 +2127,16 @@ append leaves the population spent, and concurrent losers receive no label bytes
`score_admission` requires that spend receipt to be the current-chain direct successor of
the authorization head and to match the preparation byte for byte. It refuses a different
manifest, context, readiness set, census, case population, or receipt. For every case it
manifest, context, readiness set, census, case population, receipt, or case release. It
validates the case release only after the spend receipt has been revalidated. Release
bindings are canonically ordered by case identity and unique; vulnerable and fixed cases
require one pair identity qualified by their `ScopeKey`, ordinary cases prohibit one, and
every scoped pair contains exactly one vulnerable and one fixed binding. For each supplied
case, scoring re-derives `result_digest`, the sorted evidence-input digests, the distinct
change-input digests from validated packet provenance, and the input-manifest digest and
requires exact equality with its release binding. The supplied scope, role, and pair must
also equal the binding. Missing, extra, duplicate, substituted, role-swapped, payload-swapped,
or cross-scope bindings fail before metrics. For every case it
calls `validate_evidence_input` for every `evidence_inputs` member, then performs instrument
and budget validation, and verdict/proof validation using the context-bound policies.
Proof attachments are applied only to those validated base records and the resulting claims
@@ -2086,7 +2148,8 @@ Claims, satisfaction, silence, decision vocabulary, metrics, and gates are then
from those revalidated objects. A forged family decision, substituted contract, invalid
fact review or match, contaminated lineage, or untrusted proof fails before any metric or
gate is produced. The final report binds the validation context, authorization, and
pre-label spend receipt; there is no post-scoring population-use request.
pre-label spend receipt, and authenticated post-spend case release; there is no
post-scoring population-use request.
The report validates case counts back to the census, reports paired vulnerable/fixed
behavior and ordinary false alarms through registered metrics, imports semantic-stage and
@@ -2340,7 +2403,8 @@ dependencies into core.
| `PopulationCensus` | Population, scope cardinalities, payload and independent review attestation | Aggregate and label-blind; no case IDs; same population as manifest; no self-review |
| `BaselineIdentity` | Arm, population/freeze, claimed scopes, contract set, fact schema/set, acquisition, analyzer, configuration, model set, denominator, threshold set and outcome mapping | Any changed field creates a new baseline |
| `AdmissionPreparation` | Manifest/context authorization and expected custody head | Feasible pre-registration; label-release request spends before bytes open |
| `AdmissionReportV2` | Baseline, validation context, authorization, spend receipt and case/result digests | Canonical input revalidation; in-scope scoring; all gates reported |
| `AdmissionCaseRelease` | Population, manifest, authorization, preparation, spend receipt, custodian and attested case bindings | Post-spend only; exact one-to-one case/result/evidence/change/input binding; scoped roles and pairs |
| `AdmissionReportV2` | Baseline, validation context, authorization, spend receipt, case release and case/result digests | Canonical input revalidation; in-scope scoring; all gates reported |
| `RepositoryHoldoutReport` | Component, disjoint repository manifests and measurement | Explicit holdout role; matches fitting lineage and component identity |
| `DifferentialProofRecord` | Reviewed producer, control-bound sides and resolved witness/verification receipt | Affected change identity, reviewed safe control, trusted producer/store policy |
| `PushVerdict` | Request and record digests | No unbounded clean result; proof basis does not imply rollout permission |
@@ -2381,6 +2445,7 @@ that can be mistaken for a clean scan.
| Provenance mismatch | Any digest cannot be re-derived | Refuse affected result and dependent later receipts |
| Review mismatch | Self-review, unknown reviewer, copied attestation, or unreadable disclosure | Refuse before feasibility or semantic use |
| Custody mismatch | Old/incomplete log, broken chain, or prior spend | Refuse use/admission; preserve monotonic spent status |
| Case release mismatch | Pre-spend release, wrong custodian/receipt, missing/extra case, role/pair swap, or operational payload substitution | Refuse before metric production |
### Monitoring
@@ -2443,7 +2508,10 @@ spec and consumes only status, counts, and digests.
covers spend-before-label crash/concurrency order and refuses a forged family decision,
substituted contract, invalid fact review, or contaminated model/threshold/matcher
lineage before metrics or gates. Manifest and census self-review, unknown reviewers,
copied attestations, and unresolved conflict disclosures fail before feasibility
copied attestations, and unresolved conflict disclosures fail before feasibility. A
post-spend release matrix rejects a wrong custodian or receipt, missing/extra/duplicate
bindings, cross-scope pairs, role swaps, result swaps, and rehashed evidence/change/input
substitutions before metric production
(`12.1–12.6`, `13.1`, `13.4`).
- `test_core_measurement.py`: required schema, basis stratification, Wilson counts, baseline
mutation matrix, zero receipts, and prerequisite stage result/digest references. A
+41
View File
@@ -523,3 +523,44 @@ requirements approval.
expectations. Fact content forbids all labelled ancestry; model, threshold, matcher, and
fitted-component validation protects the current evaluation population while allowing
separately spent development populations.
---
## Post-spend case-release trust closure (2026-11-01)
### Scope of this pass
- **Context:** task 8.7 demonstrated that aggregate census cardinalities and a valid spend
receipt do not authenticate which immutable operational result belongs to which released
case role and pair. A caller could swap vulnerable/fixed payloads, recompute result
digests, and change admission metrics while preserving every approved pre-label object.
- **Sources consulted:** requirements 12.1–12.6 and 15.1–15.5, the admission design and
custody contracts, the task-8.7 implementation diff and adversarial review reproducer,
and project steering on label isolation and fixed measurement.
- **Finding:** this is a design-contract gap, not a metric-algebra bug. The pre-label census
correctly withholds per-case identities and labels; therefore it cannot also authenticate
the post-opening mapping used for scoring.
### Decision: add an authenticated post-spend case release
- **Alternatives considered:** put per-case commitments in the pre-label census; trust
caller-supplied case roles after a valid spend; or add a separate post-spend release.
- **Selected approach:** the custodian produces an independently attested
`AdmissionCaseRelease` only after the exact label-opening spend. Each binding joins a case
identity and released scope/role/pair to the exact result, evidence-input, change-input,
and input-manifest identities that core re-derives during scoring.
- **Rationale:** the release preserves aggregate-only pre-registration while giving the
scorer an authority for the association that determines every numerator and denominator.
- **Trade-offs:** admission orchestration gains one immutable artifact and review step. In
exchange, case substitution, role swaps, cross-scope pair collisions, and rehashed payload
swaps fail before any measurement is emitted.
- **Boundary:** core defines and validates the release; the benchmark custodian remains the
sole producer, durable writer, and label releaser. No cryptographic signing system or
population persistence moves into core.
### Revalidation impact
- Admission scoring and report assembly must consume and bind the release digest.
- Benchmark-owned guarded custody must produce the release after the winning spend and
prove crash/concurrency behavior at that external boundary.
- The pre-label census and feasibility contracts remain unchanged and label-blind.
+1 -1
View File
@@ -1,7 +1,7 @@
{
"feature_name": "detection-core",
"created_at": "2026-10-30T13:55:30Z",
"updated_at": "2026-10-31T10:59:35Z",
"updated_at": "2026-11-01T09:03:42Z",
"language": "en",
"phase": "tasks-generated",
"approvals": {
+19 -8
View File
@@ -263,21 +263,31 @@
- Keep durable append, compare-and-swap execution, label release, persistence, crash handling, and concurrency outside core.
- Done when only the exact spent successor authorizes scoring and every other receipt shape is refused without an external side effect.
- _Requirements: 12.2, 13.4_
- [ ] 8.7 Derive scoped admission metrics from canonical case evidence
- [ ] 8.7 Validate the authenticated post-spend case release
- Define canonical released-case bindings over case identity, scope, role, scoped pair identity, exact result digest, evidence-input digests, change-input digests, and input-manifest digest.
- Bind the release to the population, manifest, authorization, preparation, exact current-chain spend receipt, custodian identity, and an independent review attestation.
- Validate the release only after the exact spend successor and require one canonical binding for every supplied case and no others.
- Reject missing, extra, duplicate, cross-scope, role-swapped, pair-swapped, result-swapped, or rehashed operational-payload substitutions before metric production.
- Keep case identities and labels out of the pre-label census and make the external custodian, not the runner, authoritative for their post-spend association.
- Done when the reproduced vulnerable/fixed payload swap and every release-field mutation fail before any admission metric is returned.
- _Requirements: 12.2, 12.5, 12.6, 13.4, 15.1_
- _Boundary: Admission Case Release_
- [ ] 8.8 Derive scoped admission metrics from canonical case evidence
- Re-run shared evidence validation, instrument, budget, verdict, and proof validation for every case.
- Match case roles and pairs back to the reviewed census and derive decision vocabulary and counts from the exact catalog.
- Consume only cases that exactly match the authenticated post-spend release; use its roles and scoped pairs while reconciling aggregate counts to the reviewed census.
- Score quality only over pre-registered family/ecosystem scopes while retaining whole-population coverage.
- Compute paired behavior, ordinary false alarms, semantic coverage, model outcomes, and all standard metrics without trusting runner aggregates.
- Done when forged decisions, substituted contracts, invalid fact review, contaminated lineage, or case-count drift fail before metric production.
- _Requirements: 12.5, 12.6, 15.1, 15.2_
- [ ] 8.8 Assemble evidence-backed gates and the final admission report
- _Depends: 8.7_
- [ ] 8.9 Assemble evidence-backed gates and the final admission report
- Emit instrument, budget, readiness, and quality gates with reasons and supporting identities.
- Bind the report to the validation context, authorization, pre-label spend receipt, census, and complete case-result set.
- Bind the report to the validation context, authorization, pre-label spend receipt, authenticated case release, census, and complete case-result set.
- Require every mandatory metric and gate, plus model success when applicable, before allowing GO.
- Keep model-corroborated, deterministic, and execution-backed counts separate.
- Done when any missing/failed mandatory gate forces NO-GO and a complete passing synthetic fixture yields a reproducible GO report.
- _Requirements: 12.6, 15.1, 15.2, 15.4, 15.5_
- [ ] 8.9 Authorize fitted components only from replayed probes and disjoint evaluation
- [ ] 8.10 Authorize fitted components only from replayed probes and disjoint evaluation
- Resolve the component’s unique model declaration, family contract, review policy, and custody policy from the admission-bound context.
- Re-evaluate every canonical probe input and require the derived result set to match the component evidence.
- Consume the typed repository-holdout report from task 8.2 and validate its evidence-use role, training/holdout references, report identity, and fitting lineage.
@@ -330,11 +340,12 @@
- [ ] 9.6 Integrate benchmark-owned admission scoring and guarded custody
- Delegate neutral scoring to core while retaining population persistence and run orchestration in the benchmark.
- Make the custodian the sole append-only writer and combine durable compare-and-swap spend plus label release as one guarded operation.
- After the winning spend, produce and independently attest the canonical case release over the exact immutable result, evidence, change, and input identities supplied to scoring.
- Supply expanded manifest, lineage, resolver, and validation-context identities without exposing labels early.
- Exercise durable append, release, crash-before-append, crash-after-append, and concurrent-winner behavior at this external boundary.
- Done when synthetic admission completes end to end and injected crash/concurrency cases prove labels reach at most the successful compare-and-swap winner.
- Exercise durable append, label release, case-release production, crash-before-append, crash-after-append, and concurrent-winner behavior at this external boundary.
- Done when synthetic admission completes end to end, injected crash/concurrency cases prove labels reach at most the successful compare-and-swap winner, and swapped operational payloads are refused by core.
- _Requirements: 12.1, 12.2, 12.3, 12.4, 12.5, 12.6, 13.4_
- _Depends: 8.8, 9.5_
- _Depends: 8.9, 9.5_
- [ ] 10. Complete the single-writer v2 migration
- [ ] 10.1 Prepare schema-routed legacy reads and cutover tests without publishing v2