Commit Graph
11 Commits
Author SHA1 Message Date
Marius CiepluchandClaude Opus 5.5 3358631448 fix(scan,taint): TypeScript's limit is owned by TypeScript; a source pattern is a token
Two defects found by the cause-B rescan of the independent population (exploratory,
a retune after protocol-v1):

- `typescript_property_support_unvalidated` still reached the whole repository.
  YesWiki is a PHP application that ships a few `.ts` files, and all 1,128 of its PHP
  questions were unresolved. The limit is now owned by the TypeScript partition. A
  TypeScript project (Mockoon, Signal K) is unchanged.
- `isSourceCall`, and the method-level "reads a source" heuristics, matched source
  patterns as substrings. Flask's `request.get` therefore claimed Django's
  `request.get_host()`. wger's fix passes that host to
  `url_has_allowed_host_and_scheme`, and both fixed functions were reported as open
  redirects from it. Patterns are now matched as bounded tokens, like sinks and
  sanitizers. wger's vulnerable-pin detections rest on `request.GET`, which still
  matches.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 15:13:30 +02:00
Marius CiepluchandClaude Opus 5.5 65dcb46b5e fix(scan): symlink, unsupported-language and unparsed-file gaps are owned by what they name
This extends dbcb230 to the remaining graph-integrity gaps. In the independent
evaluation these gaps were repository-wide:
- alerta: one symlink, `alerta/app.wsgi`, which is not even source, made 261 of 261
  questions unresolved.
- wger and Signal K: a language no frontend reads did the same.
- YesWiki: unparsed files did the same.

Now:
- the partition build records the paths a symlink boundary skipped (a directory
  ends in `/`) and the languages no frontend reads;
- `files_unparsed` carries its full paths;
- the scan demotes only questions about an owned path or under an owned directory,
  and only questions in an owned language.
Every gap stays in the report as a boundary. A gap that names nothing keeps its
partition-wide or repository-wide reach. Tier-0 pruning stays off, as before.

The tests that pinned repository-wide reach now assert ownership. The one about a
vendor exclusion masking an integrity failure keeps its intent, using a census gap
that names no file.

Retune after protocol-v1; exploratory only.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 14:30:36 +02:00
Marius CiepluchandClaude Opus 5.5 dbcb230990 fix(scan): a census gap that names its missing files is owned by those files
The first independent evaluation (results-v1.json) completed no question on eight of
eleven projects. On the cases traced, the cause was a census gap: `.eslintrc.js` or
`webpack.config.js` missing from a JavaScript graph, or `contrib/apache/api.wsgi` from a
Python graph, and every question in that language was then marked unresolved. Now only
questions about a named missing file are demoted. The gap is still reported as a
boundary. A gap that names no files still reaches its whole partition.

This is a retune after the evaluation. Per protocol-v1 it cannot be qualified on the
same population.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-25 11:16:45 +02:00
Marius CiepluchandClaude Opus 5.5 da0ee576f8 fix(scan): a catch-all function's parameters are not attacker input; field reads need the same field
Two classes the traced PMPro survivors named after 1f91c0b:

- The mapper records every named PHP function as an entry of kind
  `function` (3,507 of PMPro's 4,412), and every entry's parameters were
  sources, so internal helpers became injections: sendCancelAdminEmail's
  $user, a gateway hook callback's $user_id/$morder fired by do_action.
  Regions now carry the entry kind; parameters are input only for request
  entries (route, parser, privileged surface). Call depth is unchanged, so
  such a region is still followed into its callees. JavaScript and Python
  emit no `function` entries.
- The object rule is general: a field read continues a flow through the
  bare object only if the taint came through the SAME field, or was
  assigned to the variable or passed in as a parameter. It now also cuts
  $this->Gateway -> this -> $this->user_id (memberorder.php:705).

plausible() memoises the sink test per node and skips the object walk for
paths with no field access (the filter had cost one outlier 101 s -> 160 s).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-24 02:26:07 +02:00
Marius CiepluchandClaude Opus 5.5 1b28465a61 feat(scan): size portions from measured per-answer cost and isolate the kill culprit
The 7.8 trace showed the completion loss was a sizer death spiral, not
expensive families: after two kills the budget fell to 3 sink visits and
never recovered, because portions finishing in 25-30 s sat under an assumed
70 s fixed start and read as "no information", so the budget could only
shrink.

- taint.sc streams each answer's own cost (`ms`); the backend carries it as a
  reserved `__timing__` list through shards, partitions and reuse.
- The sizer budgets predicted seconds from per-family seconds-per-visit rates
  learned from those costs, and measures the fixed start as the wall time the
  answers do not explain (median), instead of assuming it.
- A kill names its culprit: the first unanswered id in request order. The ids
  behind it never started and are re-queued at once; the culprit is asked
  alone after the rest. One that still fails alone after more than a
  portion's target is reported as `query_too_expensive`, apart from
  `query_failed`.
- Portions grow id by id from the queue head, and a single-portion scan now
  goes through the queue too, so a kill there re-asks its remainder.
- Fix: the reuse wrapper wrapped only `run_batch`, so since 7.8 every
  portioned query bypassed the graph-reuse cache and its source-root
  normalization. `run_batch_once` is wrapped too.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-23 07:55:33 +02:00
Marius CiepluchandClaude Fable 5.1 1aa7f175e9 fix(scan): a portion sizer that never extrapolates from a sample with no work in it
Measured on the plugin, first run of the adaptive sizer. A 600-visit portion answered in 184 s and
the budget was correctly cut to 262. The next portion, 199 visits, answered in 73 s -- three seconds
of work above the 70 s fixed start -- and 3 s over 199 visits read as 0.015 s each, so the budget
grew thirteen-fold to 3,404 visits. The ceiling killed that portion. The harvest kept the 13
answers it had streamed, which is the mechanism working; the sizing was not.

Two rules. A sample with less than ten seconds of work above the fixed start is not extrapolated
from, because it carries no information about cost per visit. And a budget may double per portion,
never more: the sizing may still be wrong, but by at most one factor of two, which a kill undoes.

1,423 tests pass, nineteen skipped; the sizer control asserts both rules.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 22:17:30 +02:00
Marius CiepluchandClaude Fable 5.1 a8b226a5c8 feat(cpg): portion the work instead of capping it -- a kill keeps what finished
A hard per-invocation ceiling is all-or-nothing. A batch killed at 301 s lost every answer it had
already computed, and the retry paid a fresh JVM start per part to ask them all again: measured, 116
requests killed at 300 s, then eight restarts carrying the same everywhere-sinks in every part.

Three coordinated changes make portioning, not the cap, the thing that governs cost.

  The taint query STREAMS. The census first, then one line per request as it is answered, flushed as
  it goes, inside the unchanged fence. The other queries do not stream and need not; the driver
  reads both forms, and a legacy single document parses exactly as before.

  The driver HARVESTS a kill. The process group is terminated first -- a first cut read the pipe
  before killing and a pipe whose writer is alive never reaches end-of-file; the suite own
  hanging-engine control sat on that read for twenty-eight minutes -- then the pipe is drained and
  every complete line becomes an answer, marked partial. The retry asks only the remainder.

  Portion sizing ADAPTS. After each portion the observed seconds per sink visit, less the fixed
  start, resizes the rest toward a 120 s target; a portion that did not answer halves the budget
  rather than being fitted, because its true cost is unknown and above the ceiling. A portion cut
  down to fit puts its remainder back at the front of the queue.

A census LINE carries nothing but the census. A first cut misread a legacy document that carried the
census beside its answers as a census line and dropped every answer it sat beside; a control caught
it. Task 7.8 in contributor-scan.

Verified: 1,423 host tests pass, nineteen skipped, including new controls for the assembler in both
forms and under a kill, for a retry that asks only the remainder, and for the sizer. Not yet
verified: the joern-gated streaming check and the plugin measurement, which wait for the transfer
replay to free the machine and are stated as open in the task.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01EmM1j46qgKMh3zqD1QMwEo
2026-09-22 21:03:04 +02:00
Marius Ciepluch 7c1c2d4475 fix(contributor-scan): retain declared Gruntfile and report census gaps honestly 2026-09-14 08:07:06 +02:00
Marius Ciepluch 8fe3a7e51e fix(pre-push-safety-net): stop semantic planning at the shared deadline 2026-09-13 15:11:42 +02:00
Marius Ciepluch 6ba2b06353 feat(pre-push-safety-net): exclude vendor inputs and partition frontend coverage 2026-09-13 11:53:29 +02:00
Marius Ciepluch 546db24b8e feat(pre-push-safety-net): preserve exact ranker scope and outcomes 2026-09-13 10:16:22 +02:00