spec(harnessx-removal): remove completely; the memory layer carries evolution and the loop

Maintainer amendment: no HarnessX path stays, optional or not; the plane memory
layer replaces the HarnessX evolution proposer and carries loop engineering
(Requirement 6: persistent run memory, proposals from it through the existing
validator and gate, the loop as a plane Run, provenance per proposal).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-09-30 10:12:55 +02:00
co-authored by Claude Opus 5.5
parent 095eeee12e
commit 6eee8e6b1d
2 changed files with 36 additions and 9 deletions
+35 -8
View File
@@ -10,6 +10,12 @@ deterministic stages under a HarnessX processor. The ai-service-plane increment
HarnessX from the product and moves every LLM capability it carried either onto the plane or into a recorded
retirement. Inventory: `brief.md` (verified 2026-09-30).
Maintainer, 2026-09-30: "Please remove harnessx completely. the memory layer that saves tokens takes over the
evolutionary improvements as well as loop engineering." So nothing of HarnessX stays, not even as an optional
path, and the self-improving cycle HarnessX was meant to drive (the LLM `MetaAgent.evolve` proposer plugged into
`improve/evolve.py`) is carried by the plane's memory layer instead: what runs record (repository facts,
verdicts, disputes, costs) becomes the input to proposals and to the run-measure-propose-gate loop.
## Boundary Context
- **In scope**: the HarnessX-only modules (`harness_ext.py`, `hunter_harness.py`); the guarded branches in
@@ -17,8 +23,10 @@ retirement. Inventory: `brief.md` (verified 2026-09-30).
`stage_processors.py` (`host_under_harnessx`), `config.py` (`HarnessxConfig`); the `harnessx` extra, its mypy
override and lock entry; the tests that exercise HarnessX; README and docs sections; dated retirement notes in
the specs and steering that name HarnessX.
- **Out of scope**: changing the deterministic engine, rulesets or `evolve`'s validated/gated machinery; new
detection logic; qualification on an independent population; the plane's own precision work (judge design) --
- **Also in scope**: the plane memory layer as the source of evolutionary proposals and the loop (Requirement 6),
replacing the HarnessX proposer.
- **Out of scope**: changing the deterministic engine, rulesets or `evolve`'s validator and gate (proposals change
source, not how they are validated and gated); new detection logic; qualification on an independent population; the plane's own precision work (judge design) --
where a capability moves to the plane, this spec wires it, it does not improve it.
- **Adjacent expectations**: every deterministic path (quick mode, `regress`, `benchmark`, `gate`, `evolve`)
produces byte-identical output before and after; `vulnerabilities-over-plumbing` applies, so capabilities are
@@ -26,16 +34,17 @@ retirement. Inventory: `brief.md` (verified 2026-09-30).
## Requirements
### Requirement 1: Each HarnessX capability is moved or retired by an explicit decision
### Requirement 1: HarnessX is removed completely; the plane is the only agentic path
**User Story:** As the maintainer, I want every LLM capability HarnessX provided to have a recorded fate, so that
nothing is lost silently.
**User Story:** As the maintainer, I want no HarnessX path left, optional or not, and every LLM capability either
on the plane or explicitly retired, so that nothing is lost silently.
#### Acceptance Criteria
1. The design lists each capability -- LLM hunter pool (`ousast scan --llm`), LLM judge (`verify_judge`), LLM
fusion panels (`fusion`), deterministic stages hosted under HarnessX (`host_under_harnessx`) -- with one of:
moved to a plane task (named), or retired (with the evidence or reason).
1. No HarnessX code path remains, optional or guarded. The design lists each capability -- LLM hunter pool
(`ousast scan --llm`), LLM judge (`verify_judge`), LLM fusion panels (`fusion`), deterministic stages hosted
under HarnessX (`host_under_harnessx`), the LLM evolution proposer -- with its fate: carried by a plane task
(named), carried by the memory layer (Requirement 6), or retired with the deterministic path kept.
2. A capability moved to the plane is reachable from the CLI through the plane (`ousast plane run` with a
documented Run, or a CLI flag that submits one), binds its own Model per task (ai-service-plane Req 7), and
has a fixture test like the other plane tasks.
@@ -96,3 +105,21 @@ deterministic run produces.
1. Order: baseline (3.1) -> capabilities moved or retired (1) -> code and packaging removed (2) -> equality proof
(3.2) -> docs and specs (4). Each step is a commit gated on the full suite's own exit code, ruff and mypy.
2. No step deletes code whose replacement (Requirement 1) has not landed and passed its fixture test.
### Requirement 6: The memory layer carries evolutionary improvement and loop engineering
**User Story:** As the maintainer, I want the token-saving memory layer to be what the system learns from, so
that improvement proposals and the improvement loop come from recorded evidence instead of a HarnessX agent.
#### Acceptance Criteria
1. Plane runs persist their memory beyond the run directory in one store keyed by repository and pin: repository
facts (`repo-facts`), per-candidate verdicts per pass with the final decision, disputes and tie-breaks, and cost
and tool turns per unit; later runs reuse facts for an unchanged pin instead of recomputing them.
2. `evolve`'s proposals are generated from that store (for example: families or rules whose candidates are often
disputed, missed declared sites, repeated false alerts), deterministically and without a HarnessX agent; they
pass through the existing validator and gate unchanged.
3. The improvement loop -- run, measure, propose, validate, gate -- is expressible as a plane Run whose tasks bind
their own Models and budgets (ai-service-plane Req 7), so each iteration's token spend is attributed per task.
4. A proposal records which memory entries it came from, so a later reader can trace every accepted change to the
evidence behind it; proposals never train on the population used to qualify them (the train-on-test rule).
+1 -1
View File
@@ -1,7 +1,7 @@
{
"feature_name": "harnessx-removal",
"created_at": "2026-09-30T08:00:00Z",
"updated_at": "2026-09-30T08:00:00Z",
"updated_at": "2026-09-30T09:00:00Z",
"language": "en",
"phase": "requirements-generated",
"approvals": {