unseen: bucket-cache get_repo (kill the repeated used-set canonicalization tax)

The real bottleneck was not enumeration or extraction: canonicalize_used
resolves the entire exclusion set (measured: 4,499 names) via serial
GitHub get_repo on every launch, before any harvesting -- ~30 min of
network-latency-bound work that also burns ~4,499 of GitHub's 5,000/hr
budget before real work starts (hence the low CPU and the "never
finishes"). The earlier resolution-cache only banked via
BankedGitHub.resolve, but canonicalize_used calls eligibility.resolve ->
get_repo directly, so it was never cached and re-ran every time.

Cache get_repo at the real choke point. CorpusBank gains cache_repo /
get_cached_repo storing one object per normalized name at
<prefix>/repos/<sha256(name)>.json -- NOT seed-scoped, since repo
metadata is seed-independent, so every seed/run shares it. 404s are
cached as None too (a found-but-None hit) so known-missing repos are not
re-fetched. BankedGitHub.get_repo now checks the local journal, then the
bucket, and only then GitHub (banking the result). canonicalize_used,
advisory_index and candidate resolution all funnel through get_repo, so
all of them are covered.

Effect: the first run pays the lookups once and banks them; every later
launch reads them from the bucket (no GitHub, no rate-limit spend), so
the rate budget goes to actual candidate harvesting and restarts are
cheap. Proven live against ousast-corpus (repos/ fills 1:1 with
resolutions). 206 unseen tests + new repo-cache test pass, ruff clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

benchmarks/unseen/bank.py

@@ -108,6 +108,9 @@ class CorpusBank:
         self.resolved_prefix = f"{prefix}/{seed}/resolved/"
         self.advisories_key = f"{prefix}/{seed}/advisories/list.json"
         self.repositories_prefix = f"{prefix}/{seed}/repositories/"
+        # get_repo metadata is seed-independent GitHub data: share it across seeds
+        # so the ~thousands of used-set canonicalization lookups are paid once ever.
+        self.repos_prefix = f"{prefix}/repos/"
 
     @staticmethod
     def advisory_key(advisory_id: str) -> str:
@@ -192,6 +195,24 @@ class CorpusBank:
             if body["resolution_contract"] == contract
         }
 
+    def cache_repo(self, name: str, metadata: dict | None, *, contract: str) -> None:
+        """Persist one get_repo result (metadata, or None for a 404) by normalized name."""
+        self._put(
+            self.repos_prefix + hashlib.sha256(name.encode()).hexdigest() + ".json",
+            {"name": name, "metadata": metadata, "resolution_contract": contract},
+            labels={"contract": contract},
+        )
+
+    def get_cached_repo(self, name: str, *, contract: str) -> tuple[bool, dict | None]:
+        """Return (found, metadata). found is True even when metadata is None (a cached 404)."""
+        stored = _retry(self.client.get, self.repos_prefix + hashlib.sha256(name.encode()).hexdigest() + ".json")
+        if stored is None:
+            return (False, None)
+        body = json.loads(stored[0])
+        if body.get("name") != name or body.get("resolution_contract") != contract:
+            return (False, None)
+        return (True, body["metadata"])
+
     @staticmethod
     def key_for(repository: str, fix: str) -> str:
         return hashlib.sha256(f"{repository}\n{fix}".encode()).hexdigest()