mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 19:59:26 +00:00
Marius CiepluchandClaude Opus 5.5
6b85a60ff1
The loop the maintainer called out: four scale defects in the PHP transfer replay, found one per two-hour Joern run, all Python bookkeeping between engine calls. benchmarks/push/python_overhead.py ends that: the real replay() on the real PMPro pins, with a JoernBackend whose graphs are stubs and whose queries answer instantly with rows recorded from the real engine. Engine time is zero, so everything measured is ours, in about a minute, under the sampling profiler. Its first pass named two more walls: - Region discovery re-parsed the ruleset for every region: 89,127 TOML parses, 275 s of a 339 s discovery, twice per push. The fact, obligation and taxonomy loaders now keep the parsed (immutable) result keyed by each file's name, size and mtime -- an edit in place or an added file reloads. Discovery 339 s -> 7 s; output digest identical on PMPro. - affected_context decoded every span's and every line anchor's path for every context row -- a changed file has an anchor per unchanged line -- 78% of a stubbed push. Decoded once per call, anchors indexed by line and bisected, original order kept. A randomised control compares it against the row-by-row matching over 200 contexts (>100 attachments). Result: the whole stubbed PMPro push, 27,542 answered requests and 2,688 outcomes, runs in 58.5 s of Python. What is left is the engine. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
34 lines
1.7 KiB
Python
34 lines
1.7 KiB
Python
"""The parsed ruleset is shared across calls, and never outlives a change to the files it came from.
|
|
|
|
Region discovery asked for the facts once per region and re-parsed the TOML every time: 89,127 parses and
|
|
275 s of a 339 s discovery on a WordPress plugin, twice per push.
|
|
"""
|
|
|
|
import shutil
|
|
|
|
from openultrasast.semantic.facts import DEFAULT_FACTS_DIR, load_facts
|
|
from openultrasast.semantic.obligations.facts import DEFAULT_OBLIGATION_FACTS_DIR, load_obligation_facts
|
|
|
|
|
|
def test_facts_are_parsed_once_and_reread_when_a_file_changes(tmp_path):
|
|
root = tmp_path / "semantic"
|
|
shutil.copytree(DEFAULT_FACTS_DIR, root)
|
|
first = load_facts(root)
|
|
assert load_facts(root) is first, "an unchanged ruleset must not be parsed again"
|
|
python = root / "python.toml"
|
|
python.write_text(python.read_text() + '\n[[sink]]\nid = "memo-probe"\ncwe = "CWE-78"\ncalls = ["memo_probe_call"]\n')
|
|
edited = load_facts(root)
|
|
assert edited is not first and any("memo_probe_call" in sink.calls for sink in edited.sinks), "an edit in place was not seen"
|
|
(root / "zz_added.toml").write_text('language = "python"\n[[sink]]\nid = "added"\ncwe = "CWE-78"\ncalls = ["added_call"]\n')
|
|
added = load_facts(root)
|
|
assert any("added_call" in sink.calls for sink in added.sinks), "a new fact file was not seen"
|
|
|
|
|
|
def test_obligation_facts_are_parsed_once_and_reread_when_a_file_changes(tmp_path):
|
|
root = tmp_path / "obligations"
|
|
shutil.copytree(DEFAULT_OBLIGATION_FACTS_DIR, root)
|
|
first = load_obligation_facts(root)
|
|
assert load_obligation_facts(root) is first
|
|
(root / "python.toml").write_text((root / "python.toml").read_text().replace('version = "', 'version = "memo-', 1))
|
|
assert load_obligation_facts(root) is not first
|