chore(p0): retire regress, the skill router, the module audit tool and vocabulary.sc

Off every active line (PATHFINDER-2026-10-13/03-unified-proposal.md, cuts).
Deleted: src/openultrasast/regress/ and the deep-mode REGRESS branch (deep
now records a regress_retired degradation and equals standard; proof is the
search-with-proof arm's job), the sandbox docker runner and probe that only
regress used, --fail-on worth-fixing, RegressConfig, the two ledger helpers
only regress called; skills.py and index.py with the hunter pool's skill
routing; model/audit.py; cpg/queries/vocabulary.sc with
benchmarks/push/ontology_gap.py. Docs and the scan skill describe the
retirement. Net: 3,895 lines deleted, 86 added. Targeted tests: 180 passed,
28 skipped; 3,331 tests collect.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
This commit is contained in:
Marius Ciepluch
2026-10-13 10:50:01 +02:00
co-authored by Claude Fable 5.1
parent 94e96c6885
commit f5a904b7f2
50 changed files with 86 additions and 3895 deletions
-101
View File
@@ -1,101 +0,0 @@
// What this repository calls that the ontology does not model (contributor-scan 7.1).
//
// Five of seven families established nothing in the 2026-09 family census, and every one of them failed for
// an ontology reason with the flow engine ready and nothing to aim at: no HTTP client for a server-side
// request forgery, no collection read for an object-reference defect, no WordPress spelling for a
// deserialization. Each was found by investigating one family at a time. This answers the question directly.
//
// It needs no dataflow. The structural pass already walks every call, which is why a census that took three
// investigations can be one query.
//
// What it reports is a CANDIDATE LIST, never an ontology. A name here becomes a fact only when someone
// resolves it to a library API, states the weakness class and records provenance. Adding a name because a
// subject called it is how a general-purpose tool becomes a fixture of its own benchmark.
//
// Parameters: cpgFile, modelled (comma-separated tokens any fact table names), sources (comma-separated
// source patterns, used only to say whether a call sits in a method that reads untrusted input).
// Output: fenced JSON array of {name, spelling, count, modelled, nearSource, file, line}.
@main def exec(cpgFile: String, modelled: String = "", sources: String = "") = {
importCpg(cpgFile)
def split(raw: String): List[String] = raw.split(",").map(_.trim).filter(_.nonEmpty).toList
val modelledTokens = split(modelled)
val sourcePatterns = split(sources)
def boundedPattern(t: String) =
java.util.regex.Pattern.compile("(?<![A-Za-z0-9_$])" + java.util.regex.Pattern.quote(t) + "(?![A-Za-z0-9_$])")
val patterns = modelledTokens.map(t => (t, boundedPattern(t)))
// The callee, which is the only part of a call that names what is being called. Matching a whole node's
// text is the defect this project has now fixed six times.
def calleeText(code: String): String = {
val open = code.indexOf('(')
val head = if (open < 0) code else code.substring(0, open)
head.trim.takeRight(120)
}
// Names the FRONTEND synthesised are not names the code calls. jssrc2cpg lowers a destructuring to
// `_tmp_3.push` and an array literal to `__ecma.Array.factory`, and on the first measured subject those
// were the two most frequent candidates in the list -- noise at the top of a worklist is how a worklist
// stops being read.
def synthesised(name: String): Boolean =
name.startsWith("<") || name.startsWith("__ecma") || name.matches("^_tmp_?[0-9]*\\..*") || name.startsWith("_tmp")
val calls = cpg.call.filterNot(c => c.name.startsWith("<operator") || synthesised(c.name)).l
// Memoised per method: does this method read something the ontology calls untrusted input? A gap that sits
// beside a source is worth more attention than one that does not, and that is a ranking hint rather than a
// claim about any flow.
val sourceMemo = scala.collection.mutable.Map.empty[String, Boolean]
def nearSource(c: io.shiftleft.codepropertygraph.generated.nodes.Call): Boolean =
if (sourcePatterns.isEmpty) false
// CPS-IT/mailqueue Extension::KEY has a null .method; it cannot be near a method source.
else Option(c.method).exists { m =>
sourceMemo.getOrElseUpdate(
m.fullName,
m.ast.isCall.exists(x => sourcePatterns.exists(p => boundedPattern(p).matcher(x.code).find()))
)
}
def isModelled(c: io.shiftleft.codepropertygraph.generated.nodes.Call): Boolean = {
val callee = calleeText(c.code)
patterns.exists { case (token, pattern) => c.name == token || pattern.matcher(callee).find() }
}
case class Row(name: String, spelling: String, modelledHere: Boolean, near: Boolean, file: String, line: Int)
val rows = calls.map { c =>
Row(
c.name,
calleeText(c.code),
isModelled(c),
nearSource(c),
c.file.name.l.headOption.getOrElse(""),
c.lineNumber.getOrElse(-1)
)
}
val grouped = rows.groupBy(_.name).toList.map { case (name, items) =>
val first = items.head
ujson.Obj(
"name" -> name,
// The commonest qualified spelling, because `get` and `needle.get` are different leads.
"spelling" -> items.groupBy(_.spelling).toList.maxBy(_._2.size)._1,
"count" -> items.size,
"modelled" -> items.exists(_.modelledHere),
"nearSource" -> items.count(_.near),
"file" -> first.file,
"spellings" -> items.map(_.spelling).distinct.size,
"line" -> first.line
)
}
// Unmodelled first and by frequency, because that ordering IS the worklist.
val ordered = grouped.sortBy(o => (o("modelled").bool, -o("count").num.toInt))
println("---OUSAST-CPG-BEGIN---")
println(ujson.write(ujson.Arr(ordered: _*)))
println("---OUSAST-CPG-END---")
}