fix(cpg): reclaim the scratch tree, and sweep what earlier runs abandoned

Every build allocated a temp directory for the graph, its request files and
joern's own working copy of the CPG. Nothing ever reclaimed it. A day of
scanning left 1,055 directories and 355MB in /tmp on a disk already at 95%,
and the runs that leaked hardest were the ones that FAILED -- which is exactly
backwards, since a failed scan has the least to show for the space it took.

Two mechanisms, because they cover different deaths:

  * `CpgResult.cleanup` disposes of the tree, and `scan_repository` calls it
    when the graph has answered everything it is going to. Never raises: a scan
    that found things must not fail while tidying up.
  * A sweep at build time removes trees older than six hours. A process killed
    mid-scan never runs its cleanup -- which is how most of the 1,055 died --
    so the disposal alone is not enough. Only directories under the system temp
    root carrying this module's own prefix are touched, and a scan running in
    another process right now is left alone.

Note for the record what was NOT a product leak: 6.3GB of /tmp/joern-cli-*
was three copies of the Joern CLI I made by hand during today's container
experiments, and 63MB of workspace/ in the repo came from running joern from
the repo root instead of a scratch directory. The product already runs joern
with cwd set to the scratch tree, so its workspace is contained and now
disposed with it. Both are cleaned up; neither was the code's fault, and
conflating my litter with its leak would have hidden the real one.

Disk went from 6.1GB free to 19GB.

857 tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
2026-09-10 15:54:50 +02:00
co-authored by Claude Opus 5
parent 37eca758d4
commit f496f41be1
3 changed files with 110 additions and 1 deletions
+13
View File
@@ -277,6 +277,9 @@ def scan_repository(
scanned = len(judged)
findings = _ordered(_deduplicated(collected))
# The graph has answered everything it is going to. Its scratch tree holds the CPG, the request files and
# joern's own working copy -- tens of megabytes per scan -- and nothing else reclaims it.
_dispose(cpg)
return ModelScanResult(
findings=findings,
by_rung=_tally(findings),
@@ -293,6 +296,16 @@ def scan_repository(
)
def _dispose(cpg: object) -> None:
"""Release a built CPG's scratch tree. Never raises: cleanup must not turn a good scan into a failure."""
cleanup = getattr(cpg, "cleanup", None)
if callable(cleanup):
try:
cleanup()
except Exception as exc: # noqa: BLE001 -- a scan that found things must not fail while tidying up
logger.warning("could not remove the cpg scratch directory: %s", exc)
def _build(backend: Any, root: Path, language: str) -> Any:
"""Build through the backend, passing the language when the backend can use it."""
try: