refactor(memory): S3-generic store on boto3; RustFS is the tested server (no MinIO naming)

The plane's memory store no longer names one vendor. `MinioStore`/`SdkClient`/
`minio_settings`/`MINIO_KEYS` become `S3Store`/`S3Client`/`s3_settings`/`S3_KEYS`;
the URL scheme is `s3://<bucket>[/<prefix>]` (`s3://` alone uses `S3_BUCKET`); the
settings are `S3_ENDPOINT` (a full URL), `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`,
optional `AWS_SESSION_TOKEN` and `S3_REGION`. The client is boto3 (extra `s3`,
imported lazily): SigV4, path-style addressing, SDK checksum trailers off, paginated
listing, versions and delete markers listed and deleted by version (`purge`, used by
the contract tests' teardown so the bucket keeps nothing of a run).

Behaviour is unchanged: the store never configures its bucket and verifies
versioning, the `runs/` expiry rule, S3 Select and tag reads at startup; no local
fallback; every row carries every queryable field with "" placeholders. The admin
hints read `--endpoint-url "$S3_ENDPOINT"`. The fake client still reproduces RustFS's
two Select failures (unbound leading-row field, null-typed column).

Docs: rustfs.md stays the tested server's setup page (policy JSON unchanged, env
names updated, any S3-compatible store with versioning, lifecycle, tags and Select
works); memory.md, ops/ax/README.md, README.md, steering and the two current spec
designs use the S3 naming. Retired specs and committed measurements are untouched.

Real server (RustFS, OUSAST_MEMORY_TEST_S3=1): test_plane_memory + test_learn_compile
75 passed, 1 skipped. Store copy of the live local store to a temporary prefix: every
count matched (776 row objects, 2982 rows, 22 facts, 788 excerpts, 2601 responses,
2 programs, 1793 index entries), 4190 versions purged afterwards.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
This commit is contained in:
Marius Ciepluch
2026-10-02 10:15:19 +02:00
co-authored by Claude Fable 5.1
parent 285b6a367f
commit 51d064e381
22 changed files with 410 additions and 437 deletions
+7 -6
View File
@@ -16,10 +16,10 @@ docs = [
"mkdocs>=1.6,<2",
"mkdocs-material>=9.5,<10",
]
# The plane's memory store on MinIO (OUSAST_MEMORY=minio://<bucket>, plane/memory.py). Imported lazily; the
# core install and the file store need nothing beyond PyYAML.
minio = [
"minio>=7.2,<8",
# The plane's memory store on an S3-compatible bucket (OUSAST_MEMORY=s3://<bucket>, plane/memory.py; RustFS is the
# tested server). Imported lazily; the core install and the file store need nothing beyond PyYAML.
s3 = [
"boto3>=1.35,<2",
]
# Structured parse for overlay (JS/C/C++/Java/Python). Core install stays empty.
# Wheels are MIT; import is lazy via openultrasast.semantic.extra.
@@ -69,9 +69,10 @@ warn_unused_configs = true
module = "tree_sitter"
ignore_missing_imports = true
# The minio SDK is an optional extra, absent in the type-check environment (plane/memory.py imports it lazily).
# boto3 is an optional extra without inline types, absent in the type-check environment (plane/memory.py imports
# it lazily).
[[tool.mypy.overrides]]
module = ["minio", "minio.*"]
module = ["boto3", "botocore.*"]
ignore_missing_imports = true
# PyYAML ships no inline types and types-PyYAML is not a dev dependency.