docs(detection-core): add authoritative admission replay

This commit is contained in:
Marius Ciepluch
2026-11-01 11:16:38 +01:00
parent c8ce1c9e63
commit 6a1bc2023a
4 changed files with 100 additions and 12 deletions
+58 -9
View File
@@ -231,7 +231,8 @@ src/openultrasast/core/
├── measurement.py # Complete baseline identity, rates, funnel and report schema v2 ├── measurement.py # Complete baseline identity, rates, funnel and report schema v2
├── admission_case.py # Case results and authenticated post-spend release bindings ├── admission_case.py # Case results and authenticated post-spend release bindings
├── admission_metrics.py # Evidence-derived scoped and whole-population metric algebra ├── admission_metrics.py # Evidence-derived scoped and whole-population metric algebra
├── admission.py # Census, feasibility, report assembly and fitted-component gate ├── admission.py # Census, feasibility and fitted-component gate
├── admission_report.py # Evidence-backed gates, report assembly and replay validation
├── consumer.py # Scoped push request and verdict consumption ├── consumer.py # Scoped push request and verdict consumption
└── legacy.py # Read-only v1 trace and report decoder and validator └── legacy.py # Read-only v1 trace and report decoder and validator
@@ -1747,6 +1748,9 @@ authorization; the derived result digests must exactly equal the fitted-componen
membership. It re-derives each case result and its operational projection, requires an membership. It re-derives each case result and its operational projection, requires an
exact one-to-one binding, and refuses missing, extra, duplicate, substituted, swapped, or exact one-to-one binding, and refuses missing, extra, duplicate, substituted, swapped, or
cross-scope cases before producing any metric. cross-scope cases before producing any metric.
- Scoring returns an immutable `AdmissionReplay`, not a bare report. The replay carries the
exact authoritative objects needed to reproduce the report. A raw `AdmissionReportV2` is
serializable result data, never an authorization token.
- Admission authorization produces a custody-head-bound label-release request. The - Admission authorization produces a custody-head-bound label-release request. The
benchmark custodian must durably spend the population before releasing label bytes; benchmark custodian must durably spend the population before releasing label bytes;
scoring revalidates that exact current-chain receipt and never emits a post-scoring spend scoring revalidates that exact current-chain receipt and never emits a post-scoring spend
@@ -1757,8 +1761,9 @@ authorization; the derived result digests must exactly equal the fitted-componen
`measurement.py` owns `ScopeKey`, baseline identity, the closed metric algebra, interval and `measurement.py` owns `ScopeKey`, baseline identity, the closed metric algebra, interval and
gate results, readiness references, `MeasurementRecordV2`, and gate results, readiness references, `MeasurementRecordV2`, and
`RepositoryHoldoutReport`. `admission.py` imports those types and owns census, metric rules, `RepositoryHoldoutReport`. `admission.py` imports those types and owns census, metric rules,
thresholds, manifests, case inputs, authorization, scoring, admission reports, and fitted thresholds, manifests, case inputs, authorization, and fitted-component authorization.
component authorization. Measurement never imports admission. `admission_report.py` owns gate derivation, report assembly, the authoritative replay bundle,
and replay validation. Measurement never imports admission modules.
```python ```python
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -2031,6 +2036,18 @@ class AdmissionReportV2:
decision: AdmissionDecision decision: AdmissionDecision
report_digest: str report_digest: str
@dataclass(frozen=True)
class AdmissionReplay:
manifest: AdmissionManifestV2
denominator: DenominatorPolicy
readiness: tuple[ReadinessResult, ...]
preparation: AdmissionPreparation
label_release_receipt: PopulationUseReceipt
case_release: AdmissionCaseRelease
cases: tuple[AdmissionCaseResult, ...]
report: AdmissionReportV2
replay_digest: str
@dataclass(frozen=True) @dataclass(frozen=True)
class AdmissionAuthorization: class AdmissionAuthorization:
manifest_digest: str manifest_digest: str
@@ -2056,6 +2073,8 @@ def authorize_admission(
def score_admission( def score_admission(
manifest: AdmissionManifestV2, manifest: AdmissionManifestV2,
denominator: DenominatorPolicy,
readiness: Sequence[ReadinessResult],
preparation: AdmissionPreparation, preparation: AdmissionPreparation,
label_release_receipt: PopulationUseReceipt, label_release_receipt: PopulationUseReceipt,
case_release: AdmissionCaseRelease, case_release: AdmissionCaseRelease,
@@ -2063,6 +2082,13 @@ def score_admission(
context: AdmissionValidationContext, context: AdmissionValidationContext,
custody: PopulationCustodyPort, custody: PopulationCustodyPort,
resolver: ContentResolver, resolver: ContentResolver,
) -> AdmissionReplay: ...
def validate_admission_replay(
replay: AdmissionReplay,
context: AdmissionValidationContext,
custody: PopulationCustodyPort,
resolver: ContentResolver,
) -> AdmissionReportV2: ... ) -> AdmissionReportV2: ...
def authorize_fitted_component( def authorize_fitted_component(
@@ -2071,7 +2097,7 @@ def authorize_fitted_component(
context: AdmissionValidationContext, context: AdmissionValidationContext,
fitting_lineage: LineageManifest, fitting_lineage: LineageManifest,
repository_holdout: RepositoryHoldoutReport, repository_holdout: RepositoryHoldoutReport,
admission: AdmissionReportV2, admission: AdmissionReplay,
custody: PopulationCustodyPort, custody: PopulationCustodyPort,
resolver: ContentResolver, resolver: ContentResolver,
) -> None: ... ) -> None: ...
@@ -2151,6 +2177,15 @@ gate is produced. The final report binds the validation context, authorization,
pre-label spend receipt, and authenticated post-spend case release; there is no pre-label spend receipt, and authenticated post-spend case release; there is no
post-scoring population-use request. post-scoring population-use request.
`score_admission` freezes the denominator, readiness, and cases exactly once, derives the
report, and returns them with every other authoritative scoring input in an
`AdmissionReplay`. Its `replay_digest` protects transport integrity of the bundle but is not
a validation token. `validate_admission_replay` revalidates every carried object against an
independently supplied current context, custody port, and content resolver, re-runs report
derivation, and requires exact report and replay equality. A caller cannot authorize from a
raw report or from a coordinated set of rehashed report, threshold, readiness, or case
objects.
The report validates case counts back to the census, reports paired vulnerable/fixed The report validates case counts back to the census, reports paired vulnerable/fixed
behavior and ordinary false alarms through registered metrics, imports semantic-stage and behavior and ordinary false alarms through registered metrics, imports semantic-stage and
model-call outcomes, and stratifies findings by evidence basis so model-corroborated and model-call outcomes, and stratifies findings by evidence basis so model-corroborated and
@@ -2171,7 +2206,9 @@ disjoint, matches component/training/lineage/report digests, validates current c
the fitting lineage with subject `component_digest` and a required `FITTED_MODEL` the fitting lineage with subject `component_digest` and a required `FITTED_MODEL`
expectation for that digest, and requires expectation for that digest, and requires
the admission report to be a `GO` produced from a population that was untouched before its the admission report to be a `GO` produced from a population that was untouched before its
bound label-release spend. It re-derives `context.context_digest`, requires it to equal the bound label-release spend. It first calls `validate_admission_replay`; a bare
`AdmissionReportV2` or a replay whose carried inputs no longer validate cannot authorize a
component. It re-derives `context.context_digest`, requires it to equal the validated
report's validation-context digest, and requires exactly one baseline-listed declaration in report's validation-context digest, and requires exactly one baseline-listed declaration in
`context.evidence.model_programs` whose `fitted_component_digest` equals `context.evidence.model_programs` whose `fitted_component_digest` equals
`evidence.component_digest`. The selected declaration supplies the family-contract digest; `evidence.component_digest`. The selected declaration supplies the family-contract digest;
@@ -2188,9 +2225,15 @@ authorize the component.
Every admission report must contain `PAIRED_CORRECTNESS`, `VULNERABLE_RECALL`, Every admission report must contain `PAIRED_CORRECTNESS`, `VULNERABLE_RECALL`,
`FIXED_FALSE_ALARM_RATE`, `ORDINARY_FALSE_ALARMS_PER_100`, `PRECISION`, and `FIXED_FALSE_ALARM_RATE`, `ORDINARY_FALSE_ALARMS_PER_100`, `PRECISION`, and
`SEMANTIC_STAGE_COVERAGE` with nonzero pre-registered denominators, plus model success when `SEMANTIC_STAGE_COVERAGE` with nonzero pre-registered denominators, plus model success when
applicable. It must also contain evidence-backed instrument, budget, readiness, and quality applicable. Model success is applicable when the frozen baseline configures at least one
gate results. `GO` is invalid if any mandatory metric or gate is absent, any instrument model program **or** validated claimed-scope cases contain declared model calls. An
receipt failed, or any required gate did not pass. applicable model with zero attempted calls, no model-success threshold, or any failed model
outcome forces `NO_GO`; calls wholly outside the claimed scopes do not make the model
applicable. Claimed-scope model calls without a baseline-listed declaration are invalid
evidence and are refused before report production. The report must also contain
evidence-backed instrument, budget, readiness, and quality gate results. `GO` is invalid if
any mandatory metric or gate is absent, any instrument receipt failed, or any required gate
did not pass.
### Decision and Consumer Edge ### Decision and Consumer Edge
@@ -2404,7 +2447,8 @@ dependencies into core.
| `BaselineIdentity` | Arm, population/freeze, claimed scopes, contract set, fact schema/set, acquisition, analyzer, configuration, model set, denominator, threshold set and outcome mapping | Any changed field creates a new baseline | | `BaselineIdentity` | Arm, population/freeze, claimed scopes, contract set, fact schema/set, acquisition, analyzer, configuration, model set, denominator, threshold set and outcome mapping | Any changed field creates a new baseline |
| `AdmissionPreparation` | Manifest/context authorization and expected custody head | Feasible pre-registration; label-release request spends before bytes open | | `AdmissionPreparation` | Manifest/context authorization and expected custody head | Feasible pre-registration; label-release request spends before bytes open |
| `AdmissionCaseRelease` | Population, manifest, authorization, preparation, spend receipt, custodian and attested case bindings | Post-spend only; exact one-to-one case/result/evidence/change/input binding; scoped roles and pairs | | `AdmissionCaseRelease` | Population, manifest, authorization, preparation, spend receipt, custodian and attested case bindings | Post-spend only; exact one-to-one case/result/evidence/change/input binding; scoped roles and pairs |
| `AdmissionReportV2` | Baseline, validation context, authorization, spend receipt, case release and case/result digests | Canonical input revalidation; in-scope scoring; all gates reported | | `AdmissionReportV2` | Baseline, validation context, authorization, spend receipt, case release and case/result digests | Canonical result data; in-scope scoring; all gates reported; never an authorization token by itself |
| `AdmissionReplay` | Canonical digest of the exact authoritative scoring inputs and derived report | Immutable transport bundle; independently replayed against current context, custody and content before trust |
| `RepositoryHoldoutReport` | Component, disjoint repository manifests and measurement | Explicit holdout role; matches fitting lineage and component identity | | `RepositoryHoldoutReport` | Component, disjoint repository manifests and measurement | Explicit holdout role; matches fitting lineage and component identity |
| `DifferentialProofRecord` | Reviewed producer, control-bound sides and resolved witness/verification receipt | Affected change identity, reviewed safe control, trusted producer/store policy | | `DifferentialProofRecord` | Reviewed producer, control-bound sides and resolved witness/verification receipt | Affected change identity, reviewed safe control, trusted producer/store policy |
| `PushVerdict` | Request and record digests | No unbounded clean result; proof basis does not imply rollout permission | | `PushVerdict` | Request and record digests | No unbounded clean result; proof basis does not imply rollout permission |
@@ -2513,6 +2557,11 @@ spec and consumes only status, counts, and digests.
bindings, cross-scope pairs, role swaps, result swaps, and rehashed evidence/change/input bindings, cross-scope pairs, role swaps, result swaps, and rehashed evidence/change/input
substitutions before metric production substitutions before metric production
(`12.1–12.6`, `13.1`, `13.4`). (`12.1–12.6`, `13.1`, `13.4`).
- `test_core_admission_report.py`: evidence-backed gate derivation, exact report identity,
`AdmissionReplay` construction and authoritative revalidation, configured-model zero-call
refusal, claimed-scope undeclared-model refusal, out-of-scope-only model-call isolation,
and coordinated report/readiness/threshold/case substitution refusal. Raw reports cannot
authorize later consumers (`12.6`, `13.1–13.2`, `15.1–15.5`).
- `test_core_measurement.py`: required schema, basis stratification, Wilson counts, baseline - `test_core_measurement.py`: required schema, basis stratification, Wilson counts, baseline
mutation matrix, zero receipts, and prerequisite stage result/digest references. A mutation matrix, zero receipts, and prerequisite stage result/digest references. A
two-arm case over one population keeps family/ecosystem scope, census rows, denominator two-arm case over one population keeps family/ecosystem scope, census rows, denominator
+34
View File
@@ -564,3 +564,37 @@ requirements approval.
- Benchmark-owned guarded custody must produce the release after the winning spend and - Benchmark-owned guarded custody must produce the release after the winning spend and
prove crash/concurrency behavior at that external boundary. prove crash/concurrency behavior at that external boundary.
- The pre-label census and feasibility contracts remain unchanged and label-blind. - The pre-label census and feasibility contracts remain unchanged and label-blind.
---
## Authoritative admission replay closure (2026-11-01)
### Scope of this pass
- **Context:** task 8.9 showed that a structurally valid, self-consistent report cannot be
authenticated later without the authoritative manifest, denominator, readiness,
preparation, spend receipt, case release, and case results from which it was derived.
- **Finding:** contextual report validation inside the scoring call fixed local gate forgery
but did not give task 8.10 a safe consumer contract. Accepting a bare report would either
trust its digest as a capability or require every consumer to recover unbound inputs.
- **Classification:** specification conflict at the 8.9/8.10 boundary, not an isolated code
defect. The report is result data; later authorization needs replayable evidence.
### Decision: carry and independently replay authoritative inputs
- **Alternatives considered:** treat `report_digest` as an authorization token; add more
nested evidence to the report; make every fitted-component caller pass the original inputs
separately; or return one immutable replay bundle from scoring.
- **Selected approach:** `score_admission` returns `AdmissionReplay`, containing the exact
frozen authoritative inputs and the derived report. `validate_admission_replay` accepts
only that bundle plus independently supplied current context, custody, and content
resolution, re-runs the full derivation, and compares exact output.
- **Rationale:** the bundle prevents accidental input loss without promoting a digest or
caller-provided aggregate into a capability. Current external trust state remains an
explicit validator input.
- **Model applicability closure:** model success is mandatory when a model is frozen in the
baseline or validated claimed-scope calls occurred. Configured zero-call, missing model
threshold, or failed declared calls force `NO_GO`; undeclared claimed-scope calls are
refused before report production, and out-of-scope-only calls do not alter the arm's gate.
- **Boundary:** replay production remains part of task 8.9. Task 8.10 consumes and validates
the replay before fitted-component checks and refuses raw reports.
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"feature_name": "detection-core", "feature_name": "detection-core",
"created_at": "2026-10-30T13:55:30Z", "created_at": "2026-10-30T13:55:30Z",
"updated_at": "2026-11-01T09:03:42Z", "updated_at": "2026-11-01T10:15:05Z",
"language": "en", "language": "en",
"phase": "tasks-generated", "phase": "tasks-generated",
"approvals": { "approvals": {
+7 -2
View File
@@ -283,11 +283,15 @@
- [ ] 8.9 Assemble evidence-backed gates and the final admission report - [ ] 8.9 Assemble evidence-backed gates and the final admission report
- Emit instrument, budget, readiness, and quality gates with reasons and supporting identities. - Emit instrument, budget, readiness, and quality gates with reasons and supporting identities.
- Bind the report to the validation context, authorization, pre-label spend receipt, authenticated case release, census, and complete case-result set. - Bind the report to the validation context, authorization, pre-label spend receipt, authenticated case release, census, and complete case-result set.
- Require every mandatory metric and gate, plus model success when applicable, before allowing GO. - Require every mandatory metric and gate before allowing GO; make model success mandatory when a model is baseline-configured or validated claimed-scope calls occur. Refuse undeclared claimed-scope calls before report production; make configured zero-call, missing-threshold, and failed declared-call cases NO-GO while ignoring out-of-scope-only calls.
- Return an immutable admission replay bundle carrying the exact manifest, denominator, readiness, preparation, spend receipt, case release, cases, and derived report; give the bundle a canonical transport digest without treating it as a validation token.
- Revalidate the replay against independently supplied current context, custody, and content resolution by re-running the complete derivation and requiring exact report and replay equality.
- Keep model-corroborated, deterministic, and execution-backed counts separate. - Keep model-corroborated, deterministic, and execution-backed counts separate.
- Done when any missing/failed mandatory gate forces NO-GO and a complete passing synthetic fixture yields a reproducible GO report. - Done when any missing/failed mandatory gate forces NO-GO, coordinated report/readiness/threshold/case substitutions are refused, and a complete passing synthetic fixture yields a reproducible replayed GO report.
- _Requirements: 12.6, 15.1, 15.2, 15.4, 15.5_ - _Requirements: 12.6, 15.1, 15.2, 15.4, 15.5_
- _Depends: 8.8_
- [ ] 8.10 Authorize fitted components only from replayed probes and disjoint evaluation - [ ] 8.10 Authorize fitted components only from replayed probes and disjoint evaluation
- Accept only the typed admission replay bundle from task 8.9, validate it first against current authoritative context/custody/content, and refuse raw admission reports or coordinated replay substitutions.
- Resolve the component’s unique model declaration, family contract, review policy, and custody policy from the admission-bound context. - Resolve the component’s unique model declaration, family contract, review policy, and custody policy from the admission-bound context.
- Re-evaluate every canonical probe input and require the derived result set to match the component evidence. - Re-evaluate every canonical probe input and require the derived result set to match the component evidence.
- Consume the typed repository-holdout report from task 8.2 and validate its evidence-use role, training/holdout references, report identity, and fitting lineage. - Consume the typed repository-holdout report from task 8.2 and validate its evidence-use role, training/holdout references, report identity, and fitting lineage.
@@ -295,6 +299,7 @@
- Require an admission GO that was untouched before its bound label-release spend. - Require an admission GO that was untouched before its bound label-release spend.
- Done when fabricated summaries, caller-selected policy substitutes, overlapping repositories, contaminated lineage, or mismatched admission are refused. - Done when fabricated summaries, caller-selected policy substitutes, overlapping repositories, contaminated lineage, or mismatched admission are refused.
- _Requirements: 13.1, 13.2, 13.4_ - _Requirements: 13.1, 13.2, 13.4_
- _Depends: 8.2, 8.9_
- [ ] 9. Integrate adjacent producers and benchmark-owned orchestration - [ ] 9. Integrate adjacent producers and benchmark-owned orchestration
- [ ] 9.1 (P) Translate the parked authorization trace conservatively into v2 evidence - [ ] 9.1 (P) Translate the parked authorization trace conservatively into v2 evidence