docs(access-control-prover): own reviewed fact authority

This commit is contained in:
Marius Ciepluch
2026-11-01 14:58:22 +01:00
parent 24724e6da4
commit 8991f44ee3
9 changed files with 360 additions and 16 deletions
+163
View File
@@ -31,6 +31,8 @@ initialization job produces that knowledge, and evaluation freezes its digest be
- Resource- and authority-sensitive operation/witness matching.
- Prover-mode control-flow dominance without textual-order fallback.
- Acquired entry, operation, discharger and authority knowledge needed by this family.
- The production access-control detection-core family contract, independently reviewed
target-knowledge authority, structural matcher, and canonical observation/match writer.
- Deterministic traces, failure-visible coverage, development arms and the admission-run contract.
### Out of Boundary
@@ -39,17 +41,23 @@ initialization job produces that knowledge, and evaluation freezes its digest be
- Runtime demonstrations, owned by `search-with-proof`.
- Hook admission and rollout, owned by `pre-push-safety-net`.
- Continuous fact evolution after the one-time initialization pass.
- Family-neutral fact, match, evidence, readiness and admission validation, owned by
`detection-core`.
### Allowed Dependencies
- `openultrasast.source`, `openultrasast.mapping`, the existing semantic IR and CFG, and the existing obligation
facts/checker contracts.
- The hunter `Context` caches, valid-benchmark metrics, replay record envelopes and reserved-population guard.
- Public detection-core v2 contracts and validators through an adapter dependency from this
spec to core; detection-core never imports access-control or target-aware modules.
- No new third-party dependency.
### Revalidation Triggers
- A change to `EntryPointRecord`, `Operation`, `Discharge`, `Trace`, hunt-defect or replay-record contracts.
- A change to call-site receiver/signature capture, target-knowledge schemas or authority implication semantics.
- A change to path aggregation, judgment ordering, admission population isolation or the 120-second budget.
- A change to detection-core family, fact-pack, fact-match, provenance, custody or canonical
serialization contracts; the adapter must re-run downstream conformance before use.
## Architecture
@@ -65,6 +73,9 @@ values before that boolean boundary and adding an opt-in three-way judgment path
flowchart LR
TK[Target knowledge] --> MR[Mapping registry]
TK --> FI[Fact interpreter]
TK --> FA[Reviewed fact authority]
FA --> FM[Structural matcher and artifact writer]
FM --> DC[Detection core validation]
MR --> CTX[Per change context]
FI --> OP[Operations and witnesses]
CTX --> SI[Symbol index]
@@ -97,6 +108,7 @@ adapters may import engine contracts.
|---|---|---|
| Analysis | Python 3.11, existing semantic IR and structural CFG | symbols, calls, summaries, dominance |
| Knowledge | Existing TOML fact files and mapping strategies | target-specific vocabulary |
| Core contract | detection-core v2 typed records and validators | family-neutral conformance |
| Scoring | Existing valid-benchmark metrics and replay records | development and admission results |
## File Structure Plan
@@ -105,6 +117,10 @@ src/openultrasast/
├── mapping/
│ ├── registry.py # strategy registration and dispatch
│ ├── registration_sites.py # generic call-site registration adapter
│ ├── fact_authority.py # authenticate installed reviewed authority
│ ├── entry_packs.py # compatibility projection and v2 structural adapter
│ ├── fact_match_store.py # persist canonical observations and match artifacts
│ ├── legacy.py # production caller migrates to authority-bearing adapter
│ └── <target>.py # one target-aware strategy per migrated concern
├── semantic/
│ ├── ir.py # CallSite receiver and signature evidence
@@ -126,6 +142,12 @@ src/openultrasast/
└── tools/
└── fact_init.py # label-blind extract/propose/verify/promote workflow
ruleset/obligations/*.toml # promoted operation, witness and authority knowledge
src/openultrasast/ruleset/entries/
├── bootstrap.json # host trust policy, external to authenticated bundle
├── authority.json # reviewed bundle payload and immutable references
├── express.javascript.v2.json # reviewed JavaScript FactPack
├── express.typescript.v2.json # reviewed TypeScript FactPack
└── express.json # authenticated non-authoritative compatibility projection
benchmarks/valid/
├── prover_arm.py # records, stage funnel, gates and four-arm comparison
└── prover_admission.py # frozen-manifest admission orchestration
@@ -145,6 +167,8 @@ tests/
- Existing facts, IR, checker, dominance, trace, prover and arm modules receive additive or opt-in contracts.
- Existing callers retain legacy defaults; only the prover selects three-way judgment and strict CFG semantics.
- Existing completed benchmark records remain immutable development baselines.
- `mapping/entry_packs.py` keeps legacy pattern output byte-stable while adding an explicit
authority-bearing v2 path; no-context legacy loading cannot emit detection-core evidence.
## System Flows
@@ -180,6 +204,24 @@ flowchart LR
GATE -->|fail| NOGO[No go]
~~~
### Reviewed fact adaptation
~~~mermaid
flowchart LR
BP[Host bootstrap policy] --> AUTH[Authenticate authority bundle]
DATA[Reviewed contract packs and projection] --> AUTH
AUTH --> MATCH[Match typed repository structure]
MATCH --> OBS[Persist canonical observations]
OBS --> EVID[Build and persist FactMatchEvidence]
EVID --> CORE[Detection core validate and apply]
LEGACY[Legacy pattern projection] --> RECOG[Compatibility recognition only]
LEGACY -. forbidden .-> CORE
~~~
The access-control adapter writes each observation before constructing the match, writes the
canonical match bytes next, and only then calls core with the immutable references. The
resolver exposed to core is read-only. A regex or name hit can preserve recognition during
migration but cannot substitute for typed structural observations.
## Requirements Traceability
| Requirement | Components |
|---|---|
@@ -195,6 +237,7 @@ flowchart LR
| 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7 | strict CFG dominance and legacy order compatibility |
| 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 11.8 | abstract domain, path and operation aggregation, transition report |
| 12.1, 12.2, 12.3, 12.4, 12.5, 12.6, 12.7, 12.8, 12.9 | acquisition record, freeze manifest, admission runner |
| 13.1, 13.2, 13.3, 13.4, 13.5, 13.6, 13.7, 13.8 | access-control family contract, reviewed fact authority, structural matcher, artifact writer, detection-core conformance, caller cutover |
## Components and Interfaces
@@ -340,6 +383,115 @@ The mapping registry and fact interpreter are target-aware adapters. The initial
Admission mode prevents the acquisition process from receiving benchmark labels or fix locations. A changed output
digest after labels are opened spends the population.
### Reviewed detection-core fact adapter
The production contract is family-specific but target-neutral. Its initial reviewed
identity is `family_id=access_control`, `family_version=2.0.0`,
`schema_version=openultrasast.detection-core.evidence-packet.v2`, and
`ecosystems=(express,)`; later ecosystems require a reviewed contract-version change. It
uses the security object `entry_operation_authorization` with required text identity fields
`operation` and `boundary`. Its closed vocabulary is:
| Role | Contract |
|---|---|
| `operation` | no fields; `OPERATION_EFFECT`; `product_or_reviewed_fact` |
| `constraint` | no fields; `CONSTRAINT_EFFECT`; `product_or_reviewed_fact` |
| `security_object` | required text fields `operation` and `boundary`; no relation kinds; `product_or_reviewed_fact` |
| `framework_factory` | no fields; `REGISTRATION`; `product_or_reviewed_fact` |
| `framework_instance` | no fields; `REGISTRATION`; `product_or_reviewed_fact` |
| `entry_handler` | no fields; `INVOCATION`; `product_or_reviewed_fact` |
| `entry_registration` | no fields; `REGISTRATION`; `product_or_reviewed_fact` |
The decision rules are exactly `unguarded -> violation`, `discharged -> satisfied`, and
`indeterminate -> indeterminate`. The two determinate decisions require `operation`,
`constraint`, and `security_object` with allowed statuses `(COMPLETE,)`; the indeterminate
decision requires `constraint` with allowed statuses `(INCOMPLETE, UNRESOLVED)`. Residual question
`is_path_authorized` requires `operation`, `constraint`, and `security_object` and permits
all three decisions. The initial indeterminate-reason set includes
`missing_constraint_relation` and `ambiguous_constraint_relation`; additions require a
contract-version change and downstream revalidation.
```python
@dataclass(frozen=True)
class EntryFactAuthorityBundle:
contract: FamilyContract
packs: tuple[FactPack, ...]
recognition_projection: ContentRef
lineages: tuple[LineageManifest, ...]
custody_policy: PopulationCustodyPolicy
review_policy: ReviewIndependencePolicy
matcher_revision: str
matcher_behavior_digest: str
matcher_lineage_manifest_digest: str
authority_review: IndependenceAttestation
bundle_digest: str
@dataclass(frozen=True)
class EntryAuthorityBootstrapPolicy:
reviewed_bundle_reviewer_digests: tuple[str, ...]
forbidden_identity_pairs: tuple[tuple[str, str], ...]
trusted_store_identities: tuple[str, ...]
maximum_bundle_bytes: int
policy_digest: str
@dataclass(frozen=True)
class EntryFactAuthorityContext:
bundle: EntryFactAuthorityBundle
custody: PopulationCustodyPort
resolver: ContentResolver
class FactMatchArtifactWriter(Protocol):
def persist_observation(self, value: StructuralObservation) -> ContentRef: ...
def persist_match(self, value: FactMatchEvidence) -> ContentRef: ...
```
The authority review subject is the canonical bundle payload digest with
`authority_review` omitted. The host supplies the bootstrap policy separately; neither the
bundle nor mapping code may choose its own trusted reviewers or stores. Validation
re-derives policy, attestation, disclosure, pack, projection, lineage, and bundle digests;
requires producer-reviewer separation; applies forbidden identity pairs; resolves every
referenced byte from an admitted store; and refuses missing, duplicate, or substituted
language packs. Because `FactPack.language` is singular, a multi-language target has one
reviewed pack per language with shared semantic intent but distinct canonical identities.
The shipped host trust policy is loaded from `bootstrap.json`, never referenced as authority
by the bundle it authenticates, and is replaceable by host configuration. The artifact
writer receives an explicit store root, store identity, and retention-policy digest; it
writes content-addressed canonical bytes and returns a read-only resolver plus re-derivable
storage receipts admitted by that bootstrap policy.
The structural adapter consumes promoted recognition output and repository structure. For
each fact it emits the complete typed requirement set (qualified target, AST shape,
receiver, call target, argument shape, or configuration as declared), persists every
canonical `StructuralObservation`, builds `FactMatchEvidence`, persists
`fact_match_bytes(match)`, and then calls detection-core with the match plus its content
reference. Match construction never accepts a future self-reference: the reference is an
input only to core application after persistence.
#### Initial compatibility migration evidence
The first reviewed migration preserves the existing Express recognition rows as data, not
engine vocabulary. `express.javascript.v2.json` and `express.typescript.v2.json` carry the
same semantic expansion under distinct language-scoped pack identities. The authenticated
projection uses schema `openultrasast.entry-recognition-projection.v1` and closed row fields
`legacy_fact_id`, `languages`, `defines_handler`, `handler_kind`, and `pattern`; its content
reference is covered by the authority bundle digest. The projection keeps legacy regular
expressions and IDs byte-stable while the v2 packs use these exact facts:
| Legacy data row | Reviewed v2 fact | Required observations | Effect |
|---|---|---|---|
| `entry:express:app-route` | for each `method` in `get`, `post`, `put`, `delete`, `patch`, `all`, `use`: ID `entry:express:app-route:<method>`, fact role `entry_registration` | `qualified_target`: `QUALIFIED_NAME`, field `qualified_target`, expected `app.<method>`; `call_shape`: `AST_SHAPE`, field `call_shape`, expected `member_call`; `receiver`: `RECEIVER_TYPE`, field `receiver`, expected `express_application`; `arguments`: `ARGUMENT_SHAPE`, field `arguments`, expected `route_path_and_handler` | kind `REGISTRATION`, role `entry_registration`, source `framework_instance`, target `entry_handler`, conditions `()` |
| `entry:express:router-route` | for the same methods: ID `entry:express:router-route:<method>`, fact role `entry_registration` | `qualified_target`: `QUALIFIED_NAME`, field `qualified_target`, expected `router.<method>`; `call_shape`: `AST_SHAPE`, field `call_shape`, expected `member_call`; `receiver`: `RECEIVER_TYPE`, field `receiver`, expected `express_router`; `arguments`: `ARGUMENT_SHAPE`, field `arguments`, expected `route_path_and_handler` | kind `REGISTRATION`, role `entry_registration`, source `framework_instance`, target `entry_handler`, conditions `()` |
| `entry:express:express-app` | ID `entry:express:express-app`, fact role `framework_instance` | `qualified_target`: `QUALIFIED_NAME`, field `qualified_target`, expected `express`; `call_shape`: `AST_SHAPE`, field `call_shape`, expected `factory_call`; `binding`: `ARGUMENT_SHAPE`, field `binding`, expected `application_instance_binding` | kind `REGISTRATION`, role `framework_instance`, source `framework_factory`, target `framework_instance`, conditions `()` |
The promoted route-method values are read from reviewed projection data rather than copied
into engine branches. A pattern hit alone is `INCOMPLETE`; it cannot produce
`FactMatchEvidence`, enter readiness, or support admission. The no-context legacy loader is
removed only after `mapping/legacy.py` uses the authority-bearing path, a repository census
finds no caller that treats the projection as v2 evidence or bypasses authority, and the
core conformance handoff passes. `analyze_entry_points` remains the stable public entry;
bootstrap and adapter construction occur at its target-aware mapping boundary rather than
being pushed into neutral engine callers.
### Prover integration
`prove_change` orchestrates cached reads, recognition, resolution, summaries, paths, strict dominance and
judgment. It emits findings only for `unguarded` pairs and preserves all three judgments as traces. Coverage
@@ -408,11 +560,22 @@ digest; a manifest cannot redefine or synthesize the baseline.
- Runner: baseline/candidate bundle materialisation, four-arm attribution, paired vulnerable/fixed scoring,
reviewed-manifest validation, freeze-digest guard, zero-byte and stale-anchor refusal, indeterminate rate, and
hard 120-second gate.
- Fact authority: bootstrap-policy independence, canonical bundle identity, reviewer/store
admission, forbidden identity pairs, disclosure resolution, one pack per language, and
refusal of substituted or self-authored artifacts.
- Fact matching: canonical observation and match persistence, complete structural matches,
name-only/partial ambiguity, lineage/custody contamination, and read-only core resolution.
- Compatibility: byte-stable legacy recognition projection and proof that it cannot construct
v2 evidence, readiness, or admission; core imports remain target-neutral.
## Migration Strategy
1. Add abstract types and adapters without changing legacy checker defaults.
2. Add strict resolution, summaries and judgment behind the prover's opt-in path.
3. Migrate traces and coverage, then enable the amended development arms.
4. Freeze facts and analyzer before running a separate admission population.
5. Authenticate the existing entry projection, publish reviewed language-scoped v2 packs,
and run the authority-bearing structural adapter beside the legacy loader.
6. Pass detection-core downstream conformance, migrate production callers, then remove the
no-context compatibility path at the single-writer cutover.
Rollback is the previous prover arm and fact-set digest; baseline records remain immutable and comparable.
@@ -277,3 +277,34 @@ facts are frozen, so that a better development score is not mistaken for general
and shall not enable the arm in pre-push admission.
9. The workflow shall retain the current 23-repository population as development regression evidence and shall
require a separate untouched population for admission claims.
### Requirement 13: Reviewed detection-core fact authority and migration
**Objective:** As a maintainer, I want access-control target knowledge translated into independently reviewed,
persisted detection-core evidence, so that entry recognition can support later vulnerability-class specs without
making the neutral core target-aware or allowing a pattern match to authorize itself.
#### Acceptance Criteria
1. The access-control adapter shall define a versioned production family contract whose security object, roles,
decisions, residual questions and incompleteness reasons exactly represent access-control operation,
constraint and entry-registration semantics.
2. For each promoted target-knowledge set, the adapter shall provide reviewed language-scoped fact packs and an
independently authenticated authority bundle naming the contract, pack references, review and custody
policies, semantic-fact and matcher lineages, matcher identity and every immutable evidence reference.
3. The host bootstrap policy shall be external to the bundle it authenticates, shall constrain admitted reviewer
and store identities, shall enforce producer-reviewer separation and forbidden identity pairs, and shall
refuse unresolved conflict disclosures or a non-canonical bundle digest.
4. When legacy recognition knowledge is migrated, the adapter shall preserve its recognition behavior as a
canonical compatibility projection while mapping each promoted row to explicit contract roles, typed
structural requirements and declared relation effects. The projection alone shall remain non-authoritative.
5. For every structural match, the adapter shall persist canonical bytes for each `StructuralObservation` and
the resulting `FactMatchEvidence` before invoking detection-core, then supply the immutable content references
through a read-only resolver. Detection-core shall not write or synthesize those artifacts.
6. The adapter shall bind fact and matcher lineage, custody and evaluation-population identity so that label
access for the same admission population makes the match incomplete or refused; result labels shall not
influence recognition or matching for that admission run.
7. The adapter shall pass the detection-core downstream conformance gate for complete evidence and shall remain
incomplete or refused for name-only, partial, substituted, self-authored, legacy-only or contaminated inputs,
without detection-core importing access-control or target modules.
8. Before the compatibility path is removed, the production entry-mapping caller shall load the externally
bootstrapped authority-bearing adapter, and a repository census shall report zero callers that use the legacy
projection as v2 evidence or bypass the reviewed authority context.
@@ -284,3 +284,51 @@ next lever is entry-recognition and call-resolution breadth for the valid pool's
ecosystems, gated by the same admission workflow. The pilot pool remains unspent: no
acquisition observed its labels; only admission scoring read them. Standing rules:
`steering/detection-quality.md`.
---
## Detection-core fact-authority ownership amendment (2026-11-01)
### Context
Detection-core task 9.4 reached a real boundary gap while migrating the installed entry
pack. The generic core already validates fact packs, structural matches, lineages, custody,
and relation application, but no owner supplied the production access-control contract,
reviewed target authority, structural matcher, or persisted match artifacts. Putting those
objects into core would make a vulnerability-neutral package own access-control and target
vocabulary.
The instrument was verified before diagnosis: the installed entry file was readable at 963
bytes, and the focused legacy/v2 fact suites passed. The blocker was therefore ownership and
missing authority, not an unread input or broken validator.
### Synthesis and decision
1. **Dependency direction:** `access-control-prover` imports detection-core contracts and
supplies family-owned objects. Detection-core never imports access-control or target code.
2. **Authority:** a checked-in immutable bundle references the production family contract,
one reviewed fact pack per language, recognition projection, policies, lineages, matcher
identity, and canonical evidence. A separately configured bootstrap policy authenticates
the bundle's reviewer and store; the bundle cannot trust itself.
3. **Evidence custody:** the downstream adapter persists canonical structural observations
and match bytes before calling core. Core receives only typed objects, immutable content
references, and read-only custody/resolver ports.
4. **Compatibility:** existing regex rows remain a byte-stable recognition projection during
migration. They are explicitly non-authoritative; exact typed observations are required
for complete fact evidence.
5. **Task routing:** access-control-prover tasks 14.1–14.6 define the family contract,
authenticate authority, migrate reviewed data, persist evidence, match structure, and
publish the conformance handoff. Detection-core task 9.4 then runs the target-neutral
handoff-acceptance gate. Access-control-prover task 14.7 subsequently migrates the
production caller and proves zero bypasses before detection-core task 10.2 cuts over.
### Risks and mitigations
- **Self-authorized trust:** keep bootstrap policy outside the bundle, re-derive every digest,
require producer-reviewer separation, and enforce forbidden identity pairs.
- **Pattern authority inflation:** preserve recognition behavior separately and require the
complete typed predicate set before core application.
- **Circular match references:** persist the completed canonical match first, then pass its
reference to core; the reference is never part of match-digest construction.
- **Admission contamination:** bind fact and matcher lineages to current custody and the
evaluation population so label access refuses the same run.
+3 -3
View File
@@ -1,7 +1,7 @@
{
"feature_name": "access-control-prover",
"created_at": "2026-10-14T13:48:18Z",
"updated_at": "2026-10-29T13:05:00Z",
"updated_at": "2026-11-01T13:57:54Z",
"language": "en",
"phase": "tasks-generated",
"approvals": {
@@ -10,6 +10,6 @@
"tasks": {"generated": true, "approved": true}
},
"ready_for_implementation": true,
"depends_on": ["authorization-obligations", "valid-detection-benchmark", "hunter-learning-loop"],
"approval_note": "2026-10-14 maintainer directive in session: spec the prover and queue implementation with Codex (fast-track; the three gates were passed on that instruction, review the documents before the second delivery). 2026-10-29: requirements amended after the 2026-10-28 gates 1-5 measurements (trace resolution, programmatic entry facts, failure visibility). 2026-10-29 (later): maintainer instruction in session ('can you address the issues?') treated as requirements approval and as the directive to regenerate the design. 2026-10-29 (capability lift): per-target vocabulary became acquired data; engine/target split, pattern discipline and complexity caps were recorded in steering; all phases reopened. 2026-10-29 (abstract-interpretation amendment): the maintainer accepted the design-review recommendations by asking to make the amendments. Requirements and design were regenerated with a three-way judgment, explicit evidence algebra, stable symbols, resource/authority domains, label-blind acquisition and separate admission measurement; that directive is recorded as approval to regenerate tasks. The merged task plan passed an independent task-graph sanity review after two design repairs. Tasks await human approval before implementation. 2026-10-29 (implementation start): the maintainer invoked /kiro-impl access-control-prover in session; that directive is recorded as tasks approval."
"depends_on": ["authorization-obligations", "valid-detection-benchmark", "hunter-learning-loop", "detection-core"],
"approval_note": "2026-10-14 maintainer directive in session: spec the prover and queue implementation with Codex (fast-track; the three gates were passed on that instruction, review the documents before the second delivery). 2026-10-29: requirements amended after the 2026-10-28 gates 1-5 measurements (trace resolution, programmatic entry facts, failure visibility). 2026-10-29 (later): maintainer instruction in session ('can you address the issues?') treated as requirements approval and as the directive to regenerate the design. 2026-10-29 (capability lift): per-target vocabulary became acquired data; engine/target split, pattern discipline and complexity caps were recorded in steering; all phases reopened. 2026-10-29 (abstract-interpretation amendment): the maintainer accepted the design-review recommendations by asking to make the amendments. Requirements and design were regenerated with a three-way judgment, explicit evidence algebra, stable symbols, resource/authority domains, label-blind acquisition and separate admission measurement; that directive is recorded as approval to regenerate tasks. The merged task plan passed an independent task-graph sanity review after two design repairs. Tasks await human approval before implementation. 2026-10-29 (implementation start): the maintainer invoked /kiro-impl access-control-prover in session; that directive is recorded as tasks approval. 2026-11-01: maintainer approved moving the production fact authority, matcher and artifact writer from detection-core into access-control-prover; Requirement 13, design ownership and the repaired tasks 14.1-14.7 are approved for implementation."
}
@@ -330,6 +330,61 @@ its final resolution, dominance, fact-acquisition and measurement semantics with
- _Boundary: Admission Measurement_
- _Depends: 12.6, 13.3_
- [ ] 14. Reviewed detection-core fact adapter
- [ ] 14.1 Define the production access-control family contract
- Encode the versioned access-control security object, roles, three decision rules, residual question and closed incompleteness reasons through public detection-core types.
- Keep target names outside the contract while pinning every digest-affecting field, status, origin policy and ecosystem declared by the design.
- Done when the canonical production contract validates, round-trips deterministically, and rejects every altered or undeclared role, decision, field and reason fixture.
- _Requirements: 13.1_
- _Boundary: Access-Control Family Contract_
- _Depends: 11.5_
- [ ] 14.2 Implement externally bootstrapped fact authority
- Define the immutable authority bundle and separate host bootstrap policy over reviewed pack/projection references, policies, lineages, matcher identity, reviewer/store admission, forbidden identity pairs and conflict disclosure.
- Re-derive canonical policy, attestation, disclosure and bundle identities before any target data is consumed; never let the bundle choose its own trust roots.
- Done when a complete independently reviewed authority fixture authenticates and every missing, duplicate, substituted, self-authored or non-canonical variant fails before pack loading.
- _Requirements: 13.2, 13.3_
- _Boundary: Fact Authority_
- _Depends: 14.1_
- [ ] 14.3 Migrate the installed entry knowledge into reviewed v2 packs
- Produce one language-scoped v2 pack per promoted language and a separately authenticated canonical compatibility projection from the existing reviewed entry data.
- Map every migrated row to declared roles, typed structural requirements and relation effects without adding target vocabulary to engine modules.
- Preserve legacy pattern recognition byte-for-byte while preventing the projection from constructing detection-core evidence or readiness.
- Done when legacy recognition fixtures are unchanged, both reviewed language packs validate under task 14.2 authority, and projection-only input remains non-authoritative.
- _Requirements: 1.1, 1.5, 1.7, 2.9, 2.10, 13.2, 13.4_
- _Boundary: Entry Knowledge Migration_
- _Depends: 14.1, 14.2_
- [ ] 14.4 Add canonical structural-observation and fact-match persistence
- Provide the downstream write boundary that persists canonical `StructuralObservation` bytes and returns immutable references through a read-only resolver.
- Persist completed `fact_match_bytes` only after its observation references exist, then return the match reference without inserting a circular self-reference into the match digest.
- Done when stored observation and match bytes resolve canonically and byte or reference substitution is refused deterministically.
- _Requirements: 13.5_
- _Boundary: Fact-Match Artifact Writer_
- [ ] 14.5 Implement the authority-bearing structural matcher
- Build typed observations from repository structure for the requirements declared by the selected reviewed fact; do not infer missing structure from a regex or name hit.
- Use task 14.4 to persist every observation and completed `FactMatchEvidence`, then pass the immutable match reference with family contract, policies, lineages, custody and read-only resolver to detection-core.
- Bind matcher behavior and fact lineage to current custody and the evaluation population so same-population label influence is incomplete or refused.
- Done when complete fixtures apply the declared relations, while name-only, partial, ambiguous, substituted and contaminated fixtures cannot produce complete evidence.
- _Requirements: 1.1, 1.2, 1.3, 1.5, 7.2, 9.4, 13.5, 13.6_
- _Boundary: Structural Matcher_
- _Depends: 14.2, 14.3, 14.4_
- [ ] 14.6 Produce the downstream conformance handoff
- Run detection-core fact-pack, fact-match, application and full-evidence validators over target-owned fixtures after proving nonzero fixture reads.
- Publish an immutable manifest naming every fixture, artifact digest, validator result and negative-case reason for detection-core task 9.4 to accept; do not add target fixtures to core.
- Exercise authority substitution, producer/reviewer conflict, resolver byte substitution, lineage omission, custody contamination and deterministic replay.
- Done when the complete target-owned fixture passes, every incomplete/self-authored/substituted/contaminated/legacy-only variant is refused with a stable reason, and the immutable handoff manifest is reproducible.
- _Requirements: 7.2, 7.4, 12.2, 12.3, 13.7_
- _Boundary: Downstream Adapter Conformance Handoff_
- _Depends: 14.5_
- [ ] 14.7 Migrate the production entry caller and prove zero bypasses
- Wire the externally bootstrapped authority-bearing adapter into the target-aware `mapping/legacy.py` entry path while keeping `analyze_entry_points` stable for its callers.
- Retain the legacy projection only for compatibility recognition; prohibit it from supplying v2 evidence, readiness or admission.
- Run import/caller and artifact-writer censuses proving no production path bypasses authority or treats legacy projection output as v2 evidence.
- Done when existing entry recognition remains byte-stable, explicit v2 matching uses the reviewed persisted path, and both production bypass censuses are zero.
- _Requirements: 1.1, 1.5, 1.7, 13.4, 13.7, 13.8_
- _Boundary: Production Mapping Integration_
- _Depends: 14.6_
- _Blocked: detection-core task 9.4 must accept the immutable handoff from task 14.6 before production caller migration_
## Task constraints
- Complexity: McCabe at most 10 per touched function, at most 50 logical lines per function and 400 per new module.
+20 -4
View File
@@ -70,6 +70,8 @@ remain readable but are never rewritten or used as a v2 trend baseline.
orchestration.
- Proof search, sandbox execution, oracle semantics, and demonstration qualification.
- Push scoping policy, alert rendering, capability rollout, and BLOCK enablement.
- Production vulnerability-family contracts, target-specific authority bundles, structural
matchers, and match-artifact persistence. Those belong to the downstream family spec.
### Allowed Dependencies
@@ -85,6 +87,9 @@ remain readable but are never rewritten or used as a v2 trend baseline.
- Authorization, DSPy, search-proof, benchmark, and push adapters may import the public core
contract.
- Development and admission populations are read-only identities at the core boundary.
- Downstream family adapters may inject typed contracts, packs, matches, policies, lineages,
custody ports, content resolvers, and persisted content references. Core validates and
applies those objects but never imports the target adapter or creates its authority.
- No core module imports `openultrasast.semantic`, `openultrasast.hunt`,
`openultrasast.search`, `openultrasast.tools`, `benchmarks`, or `dspy`.
@@ -293,10 +298,6 @@ not execute proof or family analysis.
for core types; remains the authorization-family trace domain.
- `src/openultrasast/hunt/prover.py` — sends authorization results through
`semantic/obligations/core_adapter.py`; it does not define neutral core types.
- `src/openultrasast/mapping/entry_packs.py` — consumes versioned typed packs and supplies
structural match evidence.
- `src/openultrasast/ruleset/entries/express.json` — migrates metadata and fact rows to the
v2 pack schema without changing target behavior.
- Existing `tests/test_core_*.py` files — authorization assumptions move to adapter or
legacy tests; neutral tests use the two reference contracts.
- `tests/test_core_contract.py` — replaces the assertion that no model path exists with the
@@ -1069,6 +1070,17 @@ record's evaluation population. Matching behavior influenced by the admission
population's labels therefore spends that population and cannot complete a relation in the
same admission run.
#### Downstream Fact-Adapter Seam
A downstream vulnerability-family adapter owns its production `FamilyContract`, reviewed
packs, structural matcher, and persistence of canonical `StructuralObservation` and
`FactMatchEvidence` bytes. It supplies those typed objects, their content references,
lineages, policies, custody port, and resolver to the core. Core only revalidates the pack,
match, provenance, and relation application; it neither imports target modules nor mints
missing trust or evidence. A legacy recognition projection may coexist during downstream
migration, but it is non-authoritative and cannot construct v2 evidence, readiness, or
admission by itself.
#### Evidence Validation Composition
| Field | Detail |
@@ -2545,6 +2557,10 @@ spec and consumes only status, counts, and digests.
pack plus valid structural match. Direct `apply_fact` use with an unreviewed pack, wrong
review policy, or protected-population matcher contamination is refused (`4.4`, `10.4`,
`11.1–11.5`, `13.3–13.4`).
- Downstream conformance fixture: inject a complete family-owned contract, reviewed pack,
persisted structural match, lineages, policies, custody and resolver; prove generic core
validation succeeds while name-only, partial, substituted, self-authored, contaminated,
or legacy-projection-only inputs are refused (`10.2`, `11.1–11.5`, `14.3`).
- `test_core_admission.py`: consume-only refusal order, readiness prerequisites, threshold
feasibility over label-blind census capacities and Wilson bounds, the historical
impossible-denominator regression, scoring derived from full case evidence, fixed
+28
View File
@@ -598,3 +598,31 @@ requirements approval.
refused before report production, and out-of-scope-only calls do not alter the arm's gate.
- **Boundary:** replay production remains part of task 8.9. Task 8.10 consumes and validates
the replay before fitted-component checks and refuses raw reports.
---
## Downstream fact-authority boundary correction (2026-11-01)
### Scope and finding
- **Trigger:** task 9.4 could not migrate the legacy Express pattern pack without either
inventing reviewer/custodian authority inside mapping or removing current recognition.
- **Instrument evidence:** the installed `express.json` was read at 963 bytes; the legacy
and v2 fact suites passed 80 tests, confirming this was not an unread-input or broken-core
diagnosis.
- **Finding:** the attempted core task decomposition put a production access-control
contract, target authority, matcher, and artifact writer into the neutral core. That
violated the engine/target split even though the generic validators were already sound.
### Decision: downstream ownership plus core conformance
- `access-control-prover` owns its production family contract, reviewed target packs,
externally authenticated authority bundle, matcher, and canonical observation/match
artifact writer. Its bootstrap trust remains external to the authenticated bundle.
- Detection-core owns only family-neutral validation and relation application. Its task 9.4
becomes a conformance gate over downstream-produced objects and cannot import target code.
- The legacy recognition projection remains downstream-owned, byte-stable, and explicitly
non-authoritative until cutover. Name-only recognition cannot produce complete v2 evidence.
- The downstream spec must first deliver authority, reviewed packs, persistence, and the
structural adapter. Core conformance then proves those objects satisfy the generic
contract before readiness, admission, or single-writer cutover can depend on them.
+1 -1
View File
@@ -1,7 +1,7 @@
{
"feature_name": "detection-core",
"created_at": "2026-10-30T13:55:30Z",
"updated_at": "2026-11-01T10:15:05Z",
"updated_at": "2026-11-01T13:57:54Z",
"language": "en",
"phase": "tasks-generated",
"approvals": {
+11 -8
View File
@@ -326,15 +326,15 @@
- _Requirements: 9.6, 14.2, 14.3_
- _Boundary: Proof Adapter_
- _Depends: 7.4_
- [ ] 9.4 (P) Migrate the existing entry-pack loader to the reviewed v2 schema
- Translate existing metadata and fact rows without adding framework vocabulary or changing target behavior.
- Supply complete structural match evidence and pack/fact/matcher provenance to the shared fact validator.
- Keep acquisition and review ownership outside core.
- Done when the migrated existing pack produces byte-stable behavior through v2 and partial/name-only matches remain incomplete.
- _Requirements: 10.2, 11.1, 11.2, 11.3, 11.4, 11.5, 14.3_
- _Boundary: Fact Pack Adapter_
- [ ] 9.4 Accept downstream reviewed fact-adapter conformance evidence
- Consume the immutable conformance fixture and manifest produced by the downstream family adapter without constructing its contract, packs, matches, authority, or target data in core.
- Re-run only the family-neutral fact-pack, fact-match, relation-application, full-evidence, canonical-reference, and import-boundary checks named by that handoff.
- Verify the manifest accounts for complete, name-only, partial, substituted, self-authored, protected-population-contaminated, and legacy-projection-only cases.
- Done when the downstream handoff is accepted by generic validators and core remains free of target imports; adapter implementation and target-specific fixtures remain downstream-owned.
- _Requirements: 10.2, 10.4, 11.1, 11.2, 11.3, 11.4, 11.5, 13.4, 14.3_
- _Boundary: Fact Pack Adapter Conformance_
- _Depends: 5.4_
- _Blocked: task decomposition is missing an externally owned production family contract, reviewed authority bundle, lineage/custody wiring, matcher identity, and approved legacy-to-v2 semantic mapping; self-authorizing defaults are prohibited_
- _Blocked: consume the handoff only after access-control-prover tasks 14.1–14.6 produce the downstream artifacts and immutable conformance manifest_
- [ ] 9.5 (P) Produce readiness and development records through benchmark-owned adapters
- Construct canonical readiness results while leaving sequencing and persistence with benchmark workflows.
- Emit development observations with v2 identities, evidence-use roles, instrument receipts, and prerequisite ancestry.
@@ -367,6 +367,8 @@
- Retain the read-only legacy decoder and refuse comparison between v1 and v2 baseline identities.
- Done when no production writer/import targets the v1 contract, historical fixtures remain unchanged, and all public writes use v2.
- _Requirements: 14.3, 15.3_
- _Depends: 9.4_
- _Blocked: access-control-prover task 14.7 must migrate the production caller and prove a zero-bypass census after task 9.4 accepts the handoff_
- [ ] 11. Validate readiness, failure visibility, and implementation boundaries
- [ ] 11.1 Run both reference families through the complete synthetic readiness path
@@ -395,6 +397,7 @@
- Prove each test instrument read nonzero fixture bytes before interpreting zero findings or failures.
- Done when the complete suite passes, boundary/import censuses are clean, historical bytes are unchanged, and all declared adapter and consumer seams pass conformance.
- _Requirements: 1.4, 10.4, 14.3, 15.3, 15.4_
- _Depends: 9.4_
## Implementation Notes