unseen: bucket-cache get_repo (kill the repeated used-set canonicalization tax)

The real bottleneck was not enumeration or extraction: canonicalize_used
resolves the entire exclusion set (measured: 4,499 names) via serial
GitHub get_repo on every launch, before any harvesting -- ~30 min of
network-latency-bound work that also burns ~4,499 of GitHub's 5,000/hr
budget before real work starts (hence the low CPU and the "never
finishes"). The earlier resolution-cache only banked via
BankedGitHub.resolve, but canonicalize_used calls eligibility.resolve ->
get_repo directly, so it was never cached and re-ran every time.

Cache get_repo at the real choke point. CorpusBank gains cache_repo /
get_cached_repo storing one object per normalized name at
<prefix>/repos/<sha256(name)>.json -- NOT seed-scoped, since repo
metadata is seed-independent, so every seed/run shares it. 404s are
cached as None too (a found-but-None hit) so known-missing repos are not
re-fetched. BankedGitHub.get_repo now checks the local journal, then the
bucket, and only then GitHub (banking the result). canonicalize_used,
advisory_index and candidate resolution all funnel through get_repo, so
all of them are covered.

Effect: the first run pays the lookups once and banks them; every later
launch reads them from the bucket (no GitHub, no rate-limit spend), so
the rate budget goes to actual candidate harvesting and restarts are
cheap. Proven live against ousast-corpus (repos/ fills 1:1 with
resolutions). 206 unseen tests + new repo-cache test pass, ruff clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

tests/test_unseen_bank.py

@@ -38,6 +38,25 @@ class FakeObjects:
         return [key for key in reversed(self.objects) if key.startswith(prefix)]
 
 
+def test_cache_repo_roundtrip_404_and_shared_prefix():
+    objects = FakeObjects()
+    bank = CorpusBank(objects, seed=20260601, prefix="unseen")
+    # the get_repo cache is shared across seeds, not under the seed prefix
+    assert bank.repos_prefix == "unseen/repos/"
+    bank.cache_repo("org/repo", {"full_name": "org/repo", "size": 10}, contract="v1")
+    bank.cache_repo("org/gone", None, contract="v1")
+    assert bank.get_cached_repo("org/repo", contract="v1") == (True, {"full_name": "org/repo", "size": 10})
+    # a cached 404 is a hit (found True, metadata None): it must not re-hit GitHub
+    assert bank.get_cached_repo("org/gone", contract="v1") == (True, None)
+    assert bank.get_cached_repo("org/never", contract="v1") == (False, None)
+    # a stale contract is a miss so the lookup is refreshed
+    assert bank.get_cached_repo("org/repo", contract="v2") == (False, None)
+    # idempotent: re-caching the same metadata does not rewrite
+    puts = objects.calls.count("put")
+    bank.cache_repo("org/repo", {"full_name": "org/repo", "size": 10}, contract="v1")
+    assert objects.calls.count("put") == puts
+
+
 def settings_env():
     return {
         "CORPUS_S3_ENDPOINT": "https://objects.invalid",