refactor(memory): S3-generic store on boto3; RustFS is the tested server (no MinIO naming)

The plane's memory store no longer names one vendor. `MinioStore`/`SdkClient`/
`minio_settings`/`MINIO_KEYS` become `S3Store`/`S3Client`/`s3_settings`/`S3_KEYS`;
the URL scheme is `s3://<bucket>[/<prefix>]` (`s3://` alone uses `S3_BUCKET`); the
settings are `S3_ENDPOINT` (a full URL), `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`,
optional `AWS_SESSION_TOKEN` and `S3_REGION`. The client is boto3 (extra `s3`,
imported lazily): SigV4, path-style addressing, SDK checksum trailers off, paginated
listing, versions and delete markers listed and deleted by version (`purge`, used by
the contract tests' teardown so the bucket keeps nothing of a run).

Behaviour is unchanged: the store never configures its bucket and verifies
versioning, the `runs/` expiry rule, S3 Select and tag reads at startup; no local
fallback; every row carries every queryable field with "" placeholders. The admin
hints read `--endpoint-url "$S3_ENDPOINT"`. The fake client still reproduces RustFS's
two Select failures (unbound leading-row field, null-typed column).

Docs: rustfs.md stays the tested server's setup page (policy JSON unchanged, env
names updated, any S3-compatible store with versioning, lifecycle, tags and Select
works); memory.md, ops/ax/README.md, README.md, steering and the two current spec
designs use the S3 naming. Retired specs and committed measurements are untouched.

Real server (RustFS, OUSAST_MEMORY_TEST_S3=1): test_plane_memory + test_learn_compile
75 passed, 1 skipped. Store copy of the live local store to a temporary prefix: every
count matched (776 row objects, 2982 rows, 22 facts, 788 excerpts, 2601 responses,
2 programs, 1793 index entries), 4190 versions purged afterwards.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 10:15:19 +02:00
co-authored by Claude Fable 5.1
parent 285b6a367f
commit 51d064e381
22 changed files with 410 additions and 437 deletions
+6 -6
View File
@@ -11,7 +11,7 @@ from pathlib import Path
import pytest
from learn_fixtures import ScriptedChat, by_id, corpus, default_answer, example
from test_plane_memory import FakeObjects, _real_minio
from test_plane_memory import FakeObjects, _real_s3
from openultrasast.learn import compile as compile_module
from openultrasast.learn.compile import (
@@ -34,7 +34,7 @@ from openultrasast.learn.examples import Example
from openultrasast.learn.folds import Fold, FoldError
from openultrasast.learn.program import Caller, Demo
from openultrasast.plane.budget import MeteredClient
from openultrasast.plane.memory import FileStore, MemoryStore, MinioStore
from openultrasast.plane.memory import FileStore, MemoryStore, S3Store
PRICES = {"cache_hit_per_m": 0.014, "input_per_m": 0.44, "output_per_m": 1.32}
MEMORY = corpus(groups=24, per_group=4)
@@ -58,14 +58,14 @@ def answer(messages: Sequence[Mapping[str, object]], temperature: float | None)
return json.dumps(data)
@pytest.fixture(params=["file", "fake-minio", "minio"])
@pytest.fixture(params=["file", "fake-s3-select", "s3"])
def store(request: pytest.FixtureRequest, tmp_path: Path) -> Iterator[MemoryStore]:
if request.param == "file":
yield FileStore(tmp_path / "memory")
elif request.param == "minio":
yield from _real_minio()
elif request.param == "s3":
yield from _real_s3()
else:
yield MinioStore(FakeObjects("works"), "bucket", "p")
yield S3Store(FakeObjects("works"), "bucket", "p")
def _caller(chat: ScriptedChat, store: MemoryStore | None = None, **budget: object) -> Caller: