From 8991f44ee3da409be560d302ebc7bb478c059521 Mon Sep 17 00:00:00 2001 From: Marius Ciepluch <11855163+norandom@users.noreply.github.com> Date: Sun, 1 Nov 2026 14:58:22 +0100 Subject: [PATCH] docs(access-control-prover): own reviewed fact authority --- .kiro/specs/access-control-prover/design.md | 163 ++++++++++++++++++ .../access-control-prover/requirements.md | 31 ++++ .kiro/specs/access-control-prover/research.md | 48 ++++++ .kiro/specs/access-control-prover/spec.json | 6 +- .kiro/specs/access-control-prover/tasks.md | 55 ++++++ .kiro/specs/detection-core/design.md | 24 ++- .kiro/specs/detection-core/research.md | 28 +++ .kiro/specs/detection-core/spec.json | 2 +- .kiro/specs/detection-core/tasks.md | 19 +- 9 files changed, 360 insertions(+), 16 deletions(-) diff --git a/.kiro/specs/access-control-prover/design.md b/.kiro/specs/access-control-prover/design.md index 1e557bf..4919988 100644 --- a/.kiro/specs/access-control-prover/design.md +++ b/.kiro/specs/access-control-prover/design.md @@ -31,6 +31,8 @@ initialization job produces that knowledge, and evaluation freezes its digest be - Resource- and authority-sensitive operation/witness matching. - Prover-mode control-flow dominance without textual-order fallback. - Acquired entry, operation, discharger and authority knowledge needed by this family. +- The production access-control detection-core family contract, independently reviewed + target-knowledge authority, structural matcher, and canonical observation/match writer. - Deterministic traces, failure-visible coverage, development arms and the admission-run contract. ### Out of Boundary @@ -39,17 +41,23 @@ initialization job produces that knowledge, and evaluation freezes its digest be - Runtime demonstrations, owned by `search-with-proof`. - Hook admission and rollout, owned by `pre-push-safety-net`. - Continuous fact evolution after the one-time initialization pass. +- Family-neutral fact, match, evidence, readiness and admission validation, owned by + `detection-core`. ### Allowed Dependencies - `openultrasast.source`, `openultrasast.mapping`, the existing semantic IR and CFG, and the existing obligation facts/checker contracts. - The hunter `Context` caches, valid-benchmark metrics, replay record envelopes and reserved-population guard. +- Public detection-core v2 contracts and validators through an adapter dependency from this + spec to core; detection-core never imports access-control or target-aware modules. - No new third-party dependency. ### Revalidation Triggers - A change to `EntryPointRecord`, `Operation`, `Discharge`, `Trace`, hunt-defect or replay-record contracts. - A change to call-site receiver/signature capture, target-knowledge schemas or authority implication semantics. - A change to path aggregation, judgment ordering, admission population isolation or the 120-second budget. +- A change to detection-core family, fact-pack, fact-match, provenance, custody or canonical + serialization contracts; the adapter must re-run downstream conformance before use. ## Architecture @@ -65,6 +73,9 @@ values before that boolean boundary and adding an opt-in three-way judgment path flowchart LR TK[Target knowledge] --> MR[Mapping registry] TK --> FI[Fact interpreter] + TK --> FA[Reviewed fact authority] + FA --> FM[Structural matcher and artifact writer] + FM --> DC[Detection core validation] MR --> CTX[Per change context] FI --> OP[Operations and witnesses] CTX --> SI[Symbol index] @@ -97,6 +108,7 @@ adapters may import engine contracts. |---|---|---| | Analysis | Python 3.11, existing semantic IR and structural CFG | symbols, calls, summaries, dominance | | Knowledge | Existing TOML fact files and mapping strategies | target-specific vocabulary | +| Core contract | detection-core v2 typed records and validators | family-neutral conformance | | Scoring | Existing valid-benchmark metrics and replay records | development and admission results | ## File Structure Plan @@ -105,6 +117,10 @@ src/openultrasast/ ├── mapping/ │ ├── registry.py # strategy registration and dispatch │ ├── registration_sites.py # generic call-site registration adapter +│ ├── fact_authority.py # authenticate installed reviewed authority +│ ├── entry_packs.py # compatibility projection and v2 structural adapter +│ ├── fact_match_store.py # persist canonical observations and match artifacts +│ ├── legacy.py # production caller migrates to authority-bearing adapter │ └── .py # one target-aware strategy per migrated concern ├── semantic/ │ ├── ir.py # CallSite receiver and signature evidence @@ -126,6 +142,12 @@ src/openultrasast/ └── tools/ └── fact_init.py # label-blind extract/propose/verify/promote workflow ruleset/obligations/*.toml # promoted operation, witness and authority knowledge +src/openultrasast/ruleset/entries/ +├── bootstrap.json # host trust policy, external to authenticated bundle +├── authority.json # reviewed bundle payload and immutable references +├── express.javascript.v2.json # reviewed JavaScript FactPack +├── express.typescript.v2.json # reviewed TypeScript FactPack +└── express.json # authenticated non-authoritative compatibility projection benchmarks/valid/ ├── prover_arm.py # records, stage funnel, gates and four-arm comparison └── prover_admission.py # frozen-manifest admission orchestration @@ -145,6 +167,8 @@ tests/ - Existing facts, IR, checker, dominance, trace, prover and arm modules receive additive or opt-in contracts. - Existing callers retain legacy defaults; only the prover selects three-way judgment and strict CFG semantics. - Existing completed benchmark records remain immutable development baselines. +- `mapping/entry_packs.py` keeps legacy pattern output byte-stable while adding an explicit + authority-bearing v2 path; no-context legacy loading cannot emit detection-core evidence. ## System Flows @@ -180,6 +204,24 @@ flowchart LR GATE -->|fail| NOGO[No go] ~~~ +### Reviewed fact adaptation +~~~mermaid +flowchart LR + BP[Host bootstrap policy] --> AUTH[Authenticate authority bundle] + DATA[Reviewed contract packs and projection] --> AUTH + AUTH --> MATCH[Match typed repository structure] + MATCH --> OBS[Persist canonical observations] + OBS --> EVID[Build and persist FactMatchEvidence] + EVID --> CORE[Detection core validate and apply] + LEGACY[Legacy pattern projection] --> RECOG[Compatibility recognition only] + LEGACY -. forbidden .-> CORE +~~~ + +The access-control adapter writes each observation before constructing the match, writes the +canonical match bytes next, and only then calls core with the immutable references. The +resolver exposed to core is read-only. A regex or name hit can preserve recognition during +migration but cannot substitute for typed structural observations. + ## Requirements Traceability | Requirement | Components | |---|---| @@ -195,6 +237,7 @@ flowchart LR | 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7 | strict CFG dominance and legacy order compatibility | | 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 11.8 | abstract domain, path and operation aggregation, transition report | | 12.1, 12.2, 12.3, 12.4, 12.5, 12.6, 12.7, 12.8, 12.9 | acquisition record, freeze manifest, admission runner | +| 13.1, 13.2, 13.3, 13.4, 13.5, 13.6, 13.7, 13.8 | access-control family contract, reviewed fact authority, structural matcher, artifact writer, detection-core conformance, caller cutover | ## Components and Interfaces @@ -340,6 +383,115 @@ The mapping registry and fact interpreter are target-aware adapters. The initial Admission mode prevents the acquisition process from receiving benchmark labels or fix locations. A changed output digest after labels are opened spends the population. +### Reviewed detection-core fact adapter + +The production contract is family-specific but target-neutral. Its initial reviewed +identity is `family_id=access_control`, `family_version=2.0.0`, +`schema_version=openultrasast.detection-core.evidence-packet.v2`, and +`ecosystems=(express,)`; later ecosystems require a reviewed contract-version change. It +uses the security object `entry_operation_authorization` with required text identity fields +`operation` and `boundary`. Its closed vocabulary is: + +| Role | Contract | +|---|---| +| `operation` | no fields; `OPERATION_EFFECT`; `product_or_reviewed_fact` | +| `constraint` | no fields; `CONSTRAINT_EFFECT`; `product_or_reviewed_fact` | +| `security_object` | required text fields `operation` and `boundary`; no relation kinds; `product_or_reviewed_fact` | +| `framework_factory` | no fields; `REGISTRATION`; `product_or_reviewed_fact` | +| `framework_instance` | no fields; `REGISTRATION`; `product_or_reviewed_fact` | +| `entry_handler` | no fields; `INVOCATION`; `product_or_reviewed_fact` | +| `entry_registration` | no fields; `REGISTRATION`; `product_or_reviewed_fact` | + +The decision rules are exactly `unguarded -> violation`, `discharged -> satisfied`, and +`indeterminate -> indeterminate`. The two determinate decisions require `operation`, +`constraint`, and `security_object` with allowed statuses `(COMPLETE,)`; the indeterminate +decision requires `constraint` with allowed statuses `(INCOMPLETE, UNRESOLVED)`. Residual question +`is_path_authorized` requires `operation`, `constraint`, and `security_object` and permits +all three decisions. The initial indeterminate-reason set includes +`missing_constraint_relation` and `ambiguous_constraint_relation`; additions require a +contract-version change and downstream revalidation. + +```python +@dataclass(frozen=True) +class EntryFactAuthorityBundle: + contract: FamilyContract + packs: tuple[FactPack, ...] + recognition_projection: ContentRef + lineages: tuple[LineageManifest, ...] + custody_policy: PopulationCustodyPolicy + review_policy: ReviewIndependencePolicy + matcher_revision: str + matcher_behavior_digest: str + matcher_lineage_manifest_digest: str + authority_review: IndependenceAttestation + bundle_digest: str + +@dataclass(frozen=True) +class EntryAuthorityBootstrapPolicy: + reviewed_bundle_reviewer_digests: tuple[str, ...] + forbidden_identity_pairs: tuple[tuple[str, str], ...] + trusted_store_identities: tuple[str, ...] + maximum_bundle_bytes: int + policy_digest: str + +@dataclass(frozen=True) +class EntryFactAuthorityContext: + bundle: EntryFactAuthorityBundle + custody: PopulationCustodyPort + resolver: ContentResolver + +class FactMatchArtifactWriter(Protocol): + def persist_observation(self, value: StructuralObservation) -> ContentRef: ... + def persist_match(self, value: FactMatchEvidence) -> ContentRef: ... +``` + +The authority review subject is the canonical bundle payload digest with +`authority_review` omitted. The host supplies the bootstrap policy separately; neither the +bundle nor mapping code may choose its own trusted reviewers or stores. Validation +re-derives policy, attestation, disclosure, pack, projection, lineage, and bundle digests; +requires producer-reviewer separation; applies forbidden identity pairs; resolves every +referenced byte from an admitted store; and refuses missing, duplicate, or substituted +language packs. Because `FactPack.language` is singular, a multi-language target has one +reviewed pack per language with shared semantic intent but distinct canonical identities. +The shipped host trust policy is loaded from `bootstrap.json`, never referenced as authority +by the bundle it authenticates, and is replaceable by host configuration. The artifact +writer receives an explicit store root, store identity, and retention-policy digest; it +writes content-addressed canonical bytes and returns a read-only resolver plus re-derivable +storage receipts admitted by that bootstrap policy. + +The structural adapter consumes promoted recognition output and repository structure. For +each fact it emits the complete typed requirement set (qualified target, AST shape, +receiver, call target, argument shape, or configuration as declared), persists every +canonical `StructuralObservation`, builds `FactMatchEvidence`, persists +`fact_match_bytes(match)`, and then calls detection-core with the match plus its content +reference. Match construction never accepts a future self-reference: the reference is an +input only to core application after persistence. + +#### Initial compatibility migration evidence + +The first reviewed migration preserves the existing Express recognition rows as data, not +engine vocabulary. `express.javascript.v2.json` and `express.typescript.v2.json` carry the +same semantic expansion under distinct language-scoped pack identities. The authenticated +projection uses schema `openultrasast.entry-recognition-projection.v1` and closed row fields +`legacy_fact_id`, `languages`, `defines_handler`, `handler_kind`, and `pattern`; its content +reference is covered by the authority bundle digest. The projection keeps legacy regular +expressions and IDs byte-stable while the v2 packs use these exact facts: + +| Legacy data row | Reviewed v2 fact | Required observations | Effect | +|---|---|---|---| +| `entry:express:app-route` | for each `method` in `get`, `post`, `put`, `delete`, `patch`, `all`, `use`: ID `entry:express:app-route:`, fact role `entry_registration` | `qualified_target`: `QUALIFIED_NAME`, field `qualified_target`, expected `app.`; `call_shape`: `AST_SHAPE`, field `call_shape`, expected `member_call`; `receiver`: `RECEIVER_TYPE`, field `receiver`, expected `express_application`; `arguments`: `ARGUMENT_SHAPE`, field `arguments`, expected `route_path_and_handler` | kind `REGISTRATION`, role `entry_registration`, source `framework_instance`, target `entry_handler`, conditions `()` | +| `entry:express:router-route` | for the same methods: ID `entry:express:router-route:`, fact role `entry_registration` | `qualified_target`: `QUALIFIED_NAME`, field `qualified_target`, expected `router.`; `call_shape`: `AST_SHAPE`, field `call_shape`, expected `member_call`; `receiver`: `RECEIVER_TYPE`, field `receiver`, expected `express_router`; `arguments`: `ARGUMENT_SHAPE`, field `arguments`, expected `route_path_and_handler` | kind `REGISTRATION`, role `entry_registration`, source `framework_instance`, target `entry_handler`, conditions `()` | +| `entry:express:express-app` | ID `entry:express:express-app`, fact role `framework_instance` | `qualified_target`: `QUALIFIED_NAME`, field `qualified_target`, expected `express`; `call_shape`: `AST_SHAPE`, field `call_shape`, expected `factory_call`; `binding`: `ARGUMENT_SHAPE`, field `binding`, expected `application_instance_binding` | kind `REGISTRATION`, role `framework_instance`, source `framework_factory`, target `framework_instance`, conditions `()` | + +The promoted route-method values are read from reviewed projection data rather than copied +into engine branches. A pattern hit alone is `INCOMPLETE`; it cannot produce +`FactMatchEvidence`, enter readiness, or support admission. The no-context legacy loader is +removed only after `mapping/legacy.py` uses the authority-bearing path, a repository census +finds no caller that treats the projection as v2 evidence or bypasses authority, and the +core conformance handoff passes. `analyze_entry_points` remains the stable public entry; +bootstrap and adapter construction occur at its target-aware mapping boundary rather than +being pushed into neutral engine callers. + ### Prover integration `prove_change` orchestrates cached reads, recognition, resolution, summaries, paths, strict dominance and judgment. It emits findings only for `unguarded` pairs and preserves all three judgments as traces. Coverage @@ -408,11 +560,22 @@ digest; a manifest cannot redefine or synthesize the baseline. - Runner: baseline/candidate bundle materialisation, four-arm attribution, paired vulnerable/fixed scoring, reviewed-manifest validation, freeze-digest guard, zero-byte and stale-anchor refusal, indeterminate rate, and hard 120-second gate. +- Fact authority: bootstrap-policy independence, canonical bundle identity, reviewer/store + admission, forbidden identity pairs, disclosure resolution, one pack per language, and + refusal of substituted or self-authored artifacts. +- Fact matching: canonical observation and match persistence, complete structural matches, + name-only/partial ambiguity, lineage/custody contamination, and read-only core resolution. +- Compatibility: byte-stable legacy recognition projection and proof that it cannot construct + v2 evidence, readiness, or admission; core imports remain target-neutral. ## Migration Strategy 1. Add abstract types and adapters without changing legacy checker defaults. 2. Add strict resolution, summaries and judgment behind the prover's opt-in path. 3. Migrate traces and coverage, then enable the amended development arms. 4. Freeze facts and analyzer before running a separate admission population. +5. Authenticate the existing entry projection, publish reviewed language-scoped v2 packs, + and run the authority-bearing structural adapter beside the legacy loader. +6. Pass detection-core downstream conformance, migrate production callers, then remove the + no-context compatibility path at the single-writer cutover. Rollback is the previous prover arm and fact-set digest; baseline records remain immutable and comparable. diff --git a/.kiro/specs/access-control-prover/requirements.md b/.kiro/specs/access-control-prover/requirements.md index ea71423..cea64fc 100644 --- a/.kiro/specs/access-control-prover/requirements.md +++ b/.kiro/specs/access-control-prover/requirements.md @@ -277,3 +277,34 @@ facts are frozen, so that a better development score is not mistaken for general and shall not enable the arm in pre-push admission. 9. The workflow shall retain the current 23-repository population as development regression evidence and shall require a separate untouched population for admission claims. + +### Requirement 13: Reviewed detection-core fact authority and migration +**Objective:** As a maintainer, I want access-control target knowledge translated into independently reviewed, +persisted detection-core evidence, so that entry recognition can support later vulnerability-class specs without +making the neutral core target-aware or allowing a pattern match to authorize itself. + +#### Acceptance Criteria +1. The access-control adapter shall define a versioned production family contract whose security object, roles, + decisions, residual questions and incompleteness reasons exactly represent access-control operation, + constraint and entry-registration semantics. +2. For each promoted target-knowledge set, the adapter shall provide reviewed language-scoped fact packs and an + independently authenticated authority bundle naming the contract, pack references, review and custody + policies, semantic-fact and matcher lineages, matcher identity and every immutable evidence reference. +3. The host bootstrap policy shall be external to the bundle it authenticates, shall constrain admitted reviewer + and store identities, shall enforce producer-reviewer separation and forbidden identity pairs, and shall + refuse unresolved conflict disclosures or a non-canonical bundle digest. +4. When legacy recognition knowledge is migrated, the adapter shall preserve its recognition behavior as a + canonical compatibility projection while mapping each promoted row to explicit contract roles, typed + structural requirements and declared relation effects. The projection alone shall remain non-authoritative. +5. For every structural match, the adapter shall persist canonical bytes for each `StructuralObservation` and + the resulting `FactMatchEvidence` before invoking detection-core, then supply the immutable content references + through a read-only resolver. Detection-core shall not write or synthesize those artifacts. +6. The adapter shall bind fact and matcher lineage, custody and evaluation-population identity so that label + access for the same admission population makes the match incomplete or refused; result labels shall not + influence recognition or matching for that admission run. +7. The adapter shall pass the detection-core downstream conformance gate for complete evidence and shall remain + incomplete or refused for name-only, partial, substituted, self-authored, legacy-only or contaminated inputs, + without detection-core importing access-control or target modules. +8. Before the compatibility path is removed, the production entry-mapping caller shall load the externally + bootstrapped authority-bearing adapter, and a repository census shall report zero callers that use the legacy + projection as v2 evidence or bypass the reviewed authority context. diff --git a/.kiro/specs/access-control-prover/research.md b/.kiro/specs/access-control-prover/research.md index 903ad47..72aed8e 100644 --- a/.kiro/specs/access-control-prover/research.md +++ b/.kiro/specs/access-control-prover/research.md @@ -284,3 +284,51 @@ next lever is entry-recognition and call-resolution breadth for the valid pool's ecosystems, gated by the same admission workflow. The pilot pool remains unspent: no acquisition observed its labels; only admission scoring read them. Standing rules: `steering/detection-quality.md`. + +--- + +## Detection-core fact-authority ownership amendment (2026-11-01) + +### Context + +Detection-core task 9.4 reached a real boundary gap while migrating the installed entry +pack. The generic core already validates fact packs, structural matches, lineages, custody, +and relation application, but no owner supplied the production access-control contract, +reviewed target authority, structural matcher, or persisted match artifacts. Putting those +objects into core would make a vulnerability-neutral package own access-control and target +vocabulary. + +The instrument was verified before diagnosis: the installed entry file was readable at 963 +bytes, and the focused legacy/v2 fact suites passed. The blocker was therefore ownership and +missing authority, not an unread input or broken validator. + +### Synthesis and decision + +1. **Dependency direction:** `access-control-prover` imports detection-core contracts and + supplies family-owned objects. Detection-core never imports access-control or target code. +2. **Authority:** a checked-in immutable bundle references the production family contract, + one reviewed fact pack per language, recognition projection, policies, lineages, matcher + identity, and canonical evidence. A separately configured bootstrap policy authenticates + the bundle's reviewer and store; the bundle cannot trust itself. +3. **Evidence custody:** the downstream adapter persists canonical structural observations + and match bytes before calling core. Core receives only typed objects, immutable content + references, and read-only custody/resolver ports. +4. **Compatibility:** existing regex rows remain a byte-stable recognition projection during + migration. They are explicitly non-authoritative; exact typed observations are required + for complete fact evidence. +5. **Task routing:** access-control-prover tasks 14.1–14.6 define the family contract, + authenticate authority, migrate reviewed data, persist evidence, match structure, and + publish the conformance handoff. Detection-core task 9.4 then runs the target-neutral + handoff-acceptance gate. Access-control-prover task 14.7 subsequently migrates the + production caller and proves zero bypasses before detection-core task 10.2 cuts over. + +### Risks and mitigations + +- **Self-authorized trust:** keep bootstrap policy outside the bundle, re-derive every digest, + require producer-reviewer separation, and enforce forbidden identity pairs. +- **Pattern authority inflation:** preserve recognition behavior separately and require the + complete typed predicate set before core application. +- **Circular match references:** persist the completed canonical match first, then pass its + reference to core; the reference is never part of match-digest construction. +- **Admission contamination:** bind fact and matcher lineages to current custody and the + evaluation population so label access refuses the same run. diff --git a/.kiro/specs/access-control-prover/spec.json b/.kiro/specs/access-control-prover/spec.json index 08a02f4..91fe950 100644 --- a/.kiro/specs/access-control-prover/spec.json +++ b/.kiro/specs/access-control-prover/spec.json @@ -1,7 +1,7 @@ { "feature_name": "access-control-prover", "created_at": "2026-10-14T13:48:18Z", - "updated_at": "2026-10-29T13:05:00Z", + "updated_at": "2026-11-01T13:57:54Z", "language": "en", "phase": "tasks-generated", "approvals": { @@ -10,6 +10,6 @@ "tasks": {"generated": true, "approved": true} }, "ready_for_implementation": true, - "depends_on": ["authorization-obligations", "valid-detection-benchmark", "hunter-learning-loop"], - "approval_note": "2026-10-14 maintainer directive in session: spec the prover and queue implementation with Codex (fast-track; the three gates were passed on that instruction, review the documents before the second delivery). 2026-10-29: requirements amended after the 2026-10-28 gates 1-5 measurements (trace resolution, programmatic entry facts, failure visibility). 2026-10-29 (later): maintainer instruction in session ('can you address the issues?') treated as requirements approval and as the directive to regenerate the design. 2026-10-29 (capability lift): per-target vocabulary became acquired data; engine/target split, pattern discipline and complexity caps were recorded in steering; all phases reopened. 2026-10-29 (abstract-interpretation amendment): the maintainer accepted the design-review recommendations by asking to make the amendments. Requirements and design were regenerated with a three-way judgment, explicit evidence algebra, stable symbols, resource/authority domains, label-blind acquisition and separate admission measurement; that directive is recorded as approval to regenerate tasks. The merged task plan passed an independent task-graph sanity review after two design repairs. Tasks await human approval before implementation. 2026-10-29 (implementation start): the maintainer invoked /kiro-impl access-control-prover in session; that directive is recorded as tasks approval." + "depends_on": ["authorization-obligations", "valid-detection-benchmark", "hunter-learning-loop", "detection-core"], + "approval_note": "2026-10-14 maintainer directive in session: spec the prover and queue implementation with Codex (fast-track; the three gates were passed on that instruction, review the documents before the second delivery). 2026-10-29: requirements amended after the 2026-10-28 gates 1-5 measurements (trace resolution, programmatic entry facts, failure visibility). 2026-10-29 (later): maintainer instruction in session ('can you address the issues?') treated as requirements approval and as the directive to regenerate the design. 2026-10-29 (capability lift): per-target vocabulary became acquired data; engine/target split, pattern discipline and complexity caps were recorded in steering; all phases reopened. 2026-10-29 (abstract-interpretation amendment): the maintainer accepted the design-review recommendations by asking to make the amendments. Requirements and design were regenerated with a three-way judgment, explicit evidence algebra, stable symbols, resource/authority domains, label-blind acquisition and separate admission measurement; that directive is recorded as approval to regenerate tasks. The merged task plan passed an independent task-graph sanity review after two design repairs. Tasks await human approval before implementation. 2026-10-29 (implementation start): the maintainer invoked /kiro-impl access-control-prover in session; that directive is recorded as tasks approval. 2026-11-01: maintainer approved moving the production fact authority, matcher and artifact writer from detection-core into access-control-prover; Requirement 13, design ownership and the repaired tasks 14.1-14.7 are approved for implementation." } diff --git a/.kiro/specs/access-control-prover/tasks.md b/.kiro/specs/access-control-prover/tasks.md index effc19b..04a8558 100644 --- a/.kiro/specs/access-control-prover/tasks.md +++ b/.kiro/specs/access-control-prover/tasks.md @@ -330,6 +330,61 @@ its final resolution, dominance, fact-acquisition and measurement semantics with - _Boundary: Admission Measurement_ - _Depends: 12.6, 13.3_ +- [ ] 14. Reviewed detection-core fact adapter +- [ ] 14.1 Define the production access-control family contract + - Encode the versioned access-control security object, roles, three decision rules, residual question and closed incompleteness reasons through public detection-core types. + - Keep target names outside the contract while pinning every digest-affecting field, status, origin policy and ecosystem declared by the design. + - Done when the canonical production contract validates, round-trips deterministically, and rejects every altered or undeclared role, decision, field and reason fixture. + - _Requirements: 13.1_ + - _Boundary: Access-Control Family Contract_ + - _Depends: 11.5_ +- [ ] 14.2 Implement externally bootstrapped fact authority + - Define the immutable authority bundle and separate host bootstrap policy over reviewed pack/projection references, policies, lineages, matcher identity, reviewer/store admission, forbidden identity pairs and conflict disclosure. + - Re-derive canonical policy, attestation, disclosure and bundle identities before any target data is consumed; never let the bundle choose its own trust roots. + - Done when a complete independently reviewed authority fixture authenticates and every missing, duplicate, substituted, self-authored or non-canonical variant fails before pack loading. + - _Requirements: 13.2, 13.3_ + - _Boundary: Fact Authority_ + - _Depends: 14.1_ +- [ ] 14.3 Migrate the installed entry knowledge into reviewed v2 packs + - Produce one language-scoped v2 pack per promoted language and a separately authenticated canonical compatibility projection from the existing reviewed entry data. + - Map every migrated row to declared roles, typed structural requirements and relation effects without adding target vocabulary to engine modules. + - Preserve legacy pattern recognition byte-for-byte while preventing the projection from constructing detection-core evidence or readiness. + - Done when legacy recognition fixtures are unchanged, both reviewed language packs validate under task 14.2 authority, and projection-only input remains non-authoritative. + - _Requirements: 1.1, 1.5, 1.7, 2.9, 2.10, 13.2, 13.4_ + - _Boundary: Entry Knowledge Migration_ + - _Depends: 14.1, 14.2_ +- [ ] 14.4 Add canonical structural-observation and fact-match persistence + - Provide the downstream write boundary that persists canonical `StructuralObservation` bytes and returns immutable references through a read-only resolver. + - Persist completed `fact_match_bytes` only after its observation references exist, then return the match reference without inserting a circular self-reference into the match digest. + - Done when stored observation and match bytes resolve canonically and byte or reference substitution is refused deterministically. + - _Requirements: 13.5_ + - _Boundary: Fact-Match Artifact Writer_ +- [ ] 14.5 Implement the authority-bearing structural matcher + - Build typed observations from repository structure for the requirements declared by the selected reviewed fact; do not infer missing structure from a regex or name hit. + - Use task 14.4 to persist every observation and completed `FactMatchEvidence`, then pass the immutable match reference with family contract, policies, lineages, custody and read-only resolver to detection-core. + - Bind matcher behavior and fact lineage to current custody and the evaluation population so same-population label influence is incomplete or refused. + - Done when complete fixtures apply the declared relations, while name-only, partial, ambiguous, substituted and contaminated fixtures cannot produce complete evidence. + - _Requirements: 1.1, 1.2, 1.3, 1.5, 7.2, 9.4, 13.5, 13.6_ + - _Boundary: Structural Matcher_ + - _Depends: 14.2, 14.3, 14.4_ +- [ ] 14.6 Produce the downstream conformance handoff + - Run detection-core fact-pack, fact-match, application and full-evidence validators over target-owned fixtures after proving nonzero fixture reads. + - Publish an immutable manifest naming every fixture, artifact digest, validator result and negative-case reason for detection-core task 9.4 to accept; do not add target fixtures to core. + - Exercise authority substitution, producer/reviewer conflict, resolver byte substitution, lineage omission, custody contamination and deterministic replay. + - Done when the complete target-owned fixture passes, every incomplete/self-authored/substituted/contaminated/legacy-only variant is refused with a stable reason, and the immutable handoff manifest is reproducible. + - _Requirements: 7.2, 7.4, 12.2, 12.3, 13.7_ + - _Boundary: Downstream Adapter Conformance Handoff_ + - _Depends: 14.5_ +- [ ] 14.7 Migrate the production entry caller and prove zero bypasses + - Wire the externally bootstrapped authority-bearing adapter into the target-aware `mapping/legacy.py` entry path while keeping `analyze_entry_points` stable for its callers. + - Retain the legacy projection only for compatibility recognition; prohibit it from supplying v2 evidence, readiness or admission. + - Run import/caller and artifact-writer censuses proving no production path bypasses authority or treats legacy projection output as v2 evidence. + - Done when existing entry recognition remains byte-stable, explicit v2 matching uses the reviewed persisted path, and both production bypass censuses are zero. + - _Requirements: 1.1, 1.5, 1.7, 13.4, 13.7, 13.8_ + - _Boundary: Production Mapping Integration_ + - _Depends: 14.6_ + - _Blocked: detection-core task 9.4 must accept the immutable handoff from task 14.6 before production caller migration_ + ## Task constraints - Complexity: McCabe at most 10 per touched function, at most 50 logical lines per function and 400 per new module. diff --git a/.kiro/specs/detection-core/design.md b/.kiro/specs/detection-core/design.md index d9a5ee7..9b3a524 100644 --- a/.kiro/specs/detection-core/design.md +++ b/.kiro/specs/detection-core/design.md @@ -70,6 +70,8 @@ remain readable but are never rewritten or used as a v2 trend baseline. orchestration. - Proof search, sandbox execution, oracle semantics, and demonstration qualification. - Push scoping policy, alert rendering, capability rollout, and BLOCK enablement. +- Production vulnerability-family contracts, target-specific authority bundles, structural + matchers, and match-artifact persistence. Those belong to the downstream family spec. ### Allowed Dependencies @@ -85,6 +87,9 @@ remain readable but are never rewritten or used as a v2 trend baseline. - Authorization, DSPy, search-proof, benchmark, and push adapters may import the public core contract. - Development and admission populations are read-only identities at the core boundary. +- Downstream family adapters may inject typed contracts, packs, matches, policies, lineages, + custody ports, content resolvers, and persisted content references. Core validates and + applies those objects but never imports the target adapter or creates its authority. - No core module imports `openultrasast.semantic`, `openultrasast.hunt`, `openultrasast.search`, `openultrasast.tools`, `benchmarks`, or `dspy`. @@ -293,10 +298,6 @@ not execute proof or family analysis. for core types; remains the authorization-family trace domain. - `src/openultrasast/hunt/prover.py` — sends authorization results through `semantic/obligations/core_adapter.py`; it does not define neutral core types. -- `src/openultrasast/mapping/entry_packs.py` — consumes versioned typed packs and supplies - structural match evidence. -- `src/openultrasast/ruleset/entries/express.json` — migrates metadata and fact rows to the - v2 pack schema without changing target behavior. - Existing `tests/test_core_*.py` files — authorization assumptions move to adapter or legacy tests; neutral tests use the two reference contracts. - `tests/test_core_contract.py` — replaces the assertion that no model path exists with the @@ -1069,6 +1070,17 @@ record's evaluation population. Matching behavior influenced by the admission population's labels therefore spends that population and cannot complete a relation in the same admission run. +#### Downstream Fact-Adapter Seam + +A downstream vulnerability-family adapter owns its production `FamilyContract`, reviewed +packs, structural matcher, and persistence of canonical `StructuralObservation` and +`FactMatchEvidence` bytes. It supplies those typed objects, their content references, +lineages, policies, custody port, and resolver to the core. Core only revalidates the pack, +match, provenance, and relation application; it neither imports target modules nor mints +missing trust or evidence. A legacy recognition projection may coexist during downstream +migration, but it is non-authoritative and cannot construct v2 evidence, readiness, or +admission by itself. + #### Evidence Validation Composition | Field | Detail | @@ -2545,6 +2557,10 @@ spec and consumes only status, counts, and digests. pack plus valid structural match. Direct `apply_fact` use with an unreviewed pack, wrong review policy, or protected-population matcher contamination is refused (`4.4`, `10.4`, `11.1–11.5`, `13.3–13.4`). +- Downstream conformance fixture: inject a complete family-owned contract, reviewed pack, + persisted structural match, lineages, policies, custody and resolver; prove generic core + validation succeeds while name-only, partial, substituted, self-authored, contaminated, + or legacy-projection-only inputs are refused (`10.2`, `11.1–11.5`, `14.3`). - `test_core_admission.py`: consume-only refusal order, readiness prerequisites, threshold feasibility over label-blind census capacities and Wilson bounds, the historical impossible-denominator regression, scoring derived from full case evidence, fixed diff --git a/.kiro/specs/detection-core/research.md b/.kiro/specs/detection-core/research.md index add3d80..e291412 100644 --- a/.kiro/specs/detection-core/research.md +++ b/.kiro/specs/detection-core/research.md @@ -598,3 +598,31 @@ requirements approval. refused before report production, and out-of-scope-only calls do not alter the arm's gate. - **Boundary:** replay production remains part of task 8.9. Task 8.10 consumes and validates the replay before fitted-component checks and refuses raw reports. + +--- + +## Downstream fact-authority boundary correction (2026-11-01) + +### Scope and finding + +- **Trigger:** task 9.4 could not migrate the legacy Express pattern pack without either + inventing reviewer/custodian authority inside mapping or removing current recognition. +- **Instrument evidence:** the installed `express.json` was read at 963 bytes; the legacy + and v2 fact suites passed 80 tests, confirming this was not an unread-input or broken-core + diagnosis. +- **Finding:** the attempted core task decomposition put a production access-control + contract, target authority, matcher, and artifact writer into the neutral core. That + violated the engine/target split even though the generic validators were already sound. + +### Decision: downstream ownership plus core conformance + +- `access-control-prover` owns its production family contract, reviewed target packs, + externally authenticated authority bundle, matcher, and canonical observation/match + artifact writer. Its bootstrap trust remains external to the authenticated bundle. +- Detection-core owns only family-neutral validation and relation application. Its task 9.4 + becomes a conformance gate over downstream-produced objects and cannot import target code. +- The legacy recognition projection remains downstream-owned, byte-stable, and explicitly + non-authoritative until cutover. Name-only recognition cannot produce complete v2 evidence. +- The downstream spec must first deliver authority, reviewed packs, persistence, and the + structural adapter. Core conformance then proves those objects satisfy the generic + contract before readiness, admission, or single-writer cutover can depend on them. diff --git a/.kiro/specs/detection-core/spec.json b/.kiro/specs/detection-core/spec.json index 77313e6..1a370d6 100644 --- a/.kiro/specs/detection-core/spec.json +++ b/.kiro/specs/detection-core/spec.json @@ -1,7 +1,7 @@ { "feature_name": "detection-core", "created_at": "2026-10-30T13:55:30Z", - "updated_at": "2026-11-01T10:15:05Z", + "updated_at": "2026-11-01T13:57:54Z", "language": "en", "phase": "tasks-generated", "approvals": { diff --git a/.kiro/specs/detection-core/tasks.md b/.kiro/specs/detection-core/tasks.md index b966963..44ab711 100644 --- a/.kiro/specs/detection-core/tasks.md +++ b/.kiro/specs/detection-core/tasks.md @@ -326,15 +326,15 @@ - _Requirements: 9.6, 14.2, 14.3_ - _Boundary: Proof Adapter_ - _Depends: 7.4_ -- [ ] 9.4 (P) Migrate the existing entry-pack loader to the reviewed v2 schema - - Translate existing metadata and fact rows without adding framework vocabulary or changing target behavior. - - Supply complete structural match evidence and pack/fact/matcher provenance to the shared fact validator. - - Keep acquisition and review ownership outside core. - - Done when the migrated existing pack produces byte-stable behavior through v2 and partial/name-only matches remain incomplete. - - _Requirements: 10.2, 11.1, 11.2, 11.3, 11.4, 11.5, 14.3_ - - _Boundary: Fact Pack Adapter_ +- [ ] 9.4 Accept downstream reviewed fact-adapter conformance evidence + - Consume the immutable conformance fixture and manifest produced by the downstream family adapter without constructing its contract, packs, matches, authority, or target data in core. + - Re-run only the family-neutral fact-pack, fact-match, relation-application, full-evidence, canonical-reference, and import-boundary checks named by that handoff. + - Verify the manifest accounts for complete, name-only, partial, substituted, self-authored, protected-population-contaminated, and legacy-projection-only cases. + - Done when the downstream handoff is accepted by generic validators and core remains free of target imports; adapter implementation and target-specific fixtures remain downstream-owned. + - _Requirements: 10.2, 10.4, 11.1, 11.2, 11.3, 11.4, 11.5, 13.4, 14.3_ + - _Boundary: Fact Pack Adapter Conformance_ - _Depends: 5.4_ - - _Blocked: task decomposition is missing an externally owned production family contract, reviewed authority bundle, lineage/custody wiring, matcher identity, and approved legacy-to-v2 semantic mapping; self-authorizing defaults are prohibited_ + - _Blocked: consume the handoff only after access-control-prover tasks 14.1–14.6 produce the downstream artifacts and immutable conformance manifest_ - [ ] 9.5 (P) Produce readiness and development records through benchmark-owned adapters - Construct canonical readiness results while leaving sequencing and persistence with benchmark workflows. - Emit development observations with v2 identities, evidence-use roles, instrument receipts, and prerequisite ancestry. @@ -367,6 +367,8 @@ - Retain the read-only legacy decoder and refuse comparison between v1 and v2 baseline identities. - Done when no production writer/import targets the v1 contract, historical fixtures remain unchanged, and all public writes use v2. - _Requirements: 14.3, 15.3_ + - _Depends: 9.4_ + - _Blocked: access-control-prover task 14.7 must migrate the production caller and prove a zero-bypass census after task 9.4 accepts the handoff_ - [ ] 11. Validate readiness, failure visibility, and implementation boundaries - [ ] 11.1 Run both reference families through the complete synthetic readiness path @@ -395,6 +397,7 @@ - Prove each test instrument read nonzero fixture bytes before interpreting zero findings or failures. - Done when the complete suite passes, boundary/import censuses are clean, historical bytes are unchanged, and all declared adapter and consumer seams pass conformance. - _Requirements: 1.4, 10.4, 14.3, 15.3, 15.4_ + - _Depends: 9.4_ ## Implementation Notes