mirror of
https://github.com/norandom/OpenUltraSAST.git
synced 2026-10-16 03:49:26 +00:00
docs(learning-harness): brief for a classified, self-improving detector set; fold closed-loop-integrity
Measured 2026-09-06 on the vibe-py holdout: the existing tool hunter, with no learning, scores 12/14 pair-correct (Youden +0.857, no leaks) once detection and silence are keyed on the labeled class and function and the byte-identical twins are excluded; the overlay scores 5/14 on the same rows. The published hunter number (0/17, Youden -0.588) was a scoring artifact: the scorer wanted the literal CWE id in the finding text and the prompt never asked for it. The brief proposes an auto-classifier as the routing key, one evolved hunter configuration per class, a class-aware scorer with unscorable denominators, the split enforced by construction, and the HarnessX evolve loop with per-class attribution as the round primitive. Raw measurements are committed under reports/measurements. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01T7XD7x5KonMdmGRkp63EQG
This commit is contained in:
Marius Ciepluch
co-authored by
Claude Fable 5.1
Claude Fable 5.1
parent
e277c2423c
commit
3259d0469f
@@ -1,73 +0,0 @@
|
||||
# Brief: closed-loop-integrity
|
||||
|
||||
## Problem
|
||||
|
||||
The closed loop cannot tell a better detector from a corpus artifact. Measured 2026-09-05 while wiring the obligation checker to the real pair corpus:
|
||||
|
||||
- **The lever trains on the test set.** `ousast mechanisms export` seeds candidate shapes from every vendored pair of a slice, and `improve.propose_mechanism_edits` admits a candidate because it recovers a missed holdout pair. A shape taught by a holdout pair can recover itself. On vibe-py (35 vendored, 18 train, 17 holdout) 5 of 11 candidates were taught by a holdout pair; holdout Youden is +0.059 when every pair teaches and **−0.059 when only the train split teaches**. The reported +0.353 holdout gain from `corpus-seeded-mechanisms` is inflated by an unknown amount. `evaluate_loo` is honest; only the lever leaks.
|
||||
- **Five vibe-py twins are byte-identical** once the provenance header is stripped (`dsvw-dsvw-do-get`, `python-insecure-app-main-try-hack-me`, `pythonssti-main-read-root`, `vampi-books-get-by-title`, `vulnpy-deserialization-do-pickle-load`). No detector can be loud on one side and silent on the other; the slice ceiling is 30/35 and every published vibe-py number carries those five as permanent losses.
|
||||
- **TypeScript has no grammar.** The semantic extra declares python, javascript, c, cpp and java. Every `.ts`/`.tsx` row parses as `language_unsupported`: 18 agent-vfc rows plus most of the absence rows. The row count is scored as failure, not as unscorable.
|
||||
- **Excerpts carry no registration context.** Vendored absence excerpts are the handler function alone, so no entry point is named and anything gated on reachability reports nothing. The `handler_context` harvest mode exists (2026-09-05) but no row has been re-harvested with it.
|
||||
- **Unit fixtures are easier than the corpus.** Fixtures bind identity in one hop; the real vampi fix binds it in two. Green suites therefore do not evidence corpus behaviour, and nothing in CI compares a fixture's difficulty against the rows it stands in for.
|
||||
|
||||
The consequence is that three specs (`corpus-seeded-mechanisms`, `authorization-obligations`, and the unwritten `guard-dominance-regime`) each added a detector and a measurement, and none of the measurements is trustworthy. Detection logic is ~16% of the source; the other 84% is harness that currently cannot say whether the 16% works.
|
||||
|
||||
## Desired Outcome
|
||||
|
||||
One number per slice that a maintainer can believe, before any detector is tuned against it. Concretely: the improve lever never sees a holdout excerpt while it learns; unscorable rows are excluded from the denominator and listed by reason; every language the corpus carries either parses or is counted as unsupported rather than as a miss; every absence excerpt carries the context its label needs; and a fixture-difficulty check fails CI when a unit fixture is strictly easier than the corpus rows its test claims to cover.
|
||||
|
||||
Then, and only then, the evolutionary loop (HarnessX `MetaAgent.evolve` or the existing bounded levers) is given more search freedom, because a fitness function that rewards memorizing the holdout will be optimized for exactly that.
|
||||
|
||||
Principle for the roadmap: **the ruler is repaired before the thing it measures is tuned; no published number without its unscorable denominator.**
|
||||
|
||||
## Approach
|
||||
|
||||
1. **Split enforcement.** The exporter takes a split and defaults to `train`; the lever refuses a candidate whose `record.pairs` intersect the holdout set and records the refusal. Re-measure `corpus-seeded-mechanisms` and rewrite its roadmap row with the honest number.
|
||||
2. **Unscorable rows.** The pair loader flags `identical_twin` (excerpt bodies equal after header strip) and `language_unsupported` rows; scorer and LOO report them under a `unscorable` counter with the reason and drop them from recall, silence and Youden denominators. Nothing is deleted; the catalog row gains a `known_limit` reason instead, which the existing achievable-metric machinery already honors.
|
||||
3. **TypeScript grammar.** `tree-sitter-typescript` joins the semantic extra; the walker maps `typescript`/`tsx` to it. Existing grammars unchanged.
|
||||
4. **Context re-harvest.** The absence rows of vibe-py and agent-vfc are re-harvested in `handler_context` mode (maintainer, network, license already on the recipe); pointer rows through the cache. Task 4.3 of `authorization-obligations` moves here.
|
||||
5. **Fixture-difficulty gate.** A test-only check derives a small difficulty vector from a fixture (bind depth to the identity source, registration present, cross-file registration) and compares it against the corpus rows named in the test's docstring; strictly easier fails.
|
||||
6. **Evolution under the repaired ruler.** Only after 1–5: expose the HarnessX evolve loop over the existing levers (rules, policy, mechanisms) with the per-profile holdout clause as the fitness function and the split enforced by construction. This step gets its own requirements once 1–5 are measured.
|
||||
|
||||
## Scope
|
||||
|
||||
- **In**: exporter split filter and lever refusal; unscorable classification in the loader, scorer and LOO; TypeScript grammar in the semantic extra; re-harvest of absence rows; fixture-difficulty check; re-measurement and roadmap rewrite; a closure decision on `authorization-obligations` tasks 4.3, 5 and 6 (they measure against this repaired ruler or not at all).
|
||||
- **Out**: new detectors; new shape families; changing dispositions or evidence rungs; any merge gate on non-local slices; vendoring new pairs beyond re-harvesting existing recipes.
|
||||
|
||||
## Boundary Candidates
|
||||
|
||||
- Corpus honesty (loader, scorer, LOO, catalogs) vs detectors (unchanged)
|
||||
- Lever admission rule (split-aware) vs lever validator and store schema (unchanged)
|
||||
- Semantic extra dependency set (one grammar added) vs walker semantics (unchanged)
|
||||
|
||||
## Out of Boundary
|
||||
|
||||
- Taint semantics, facts, dominance (`guard-dominance-regime`)
|
||||
- Call graph, path records (`reachability-flow-model`)
|
||||
- Obligation checker internals (`authorization-obligations`), except that its corpus tasks are re-homed here
|
||||
|
||||
## Upstream / Downstream
|
||||
|
||||
- **Upstream**: `pair-corpus-honesty` (labels, tiers, splits, `known_limit`), `corpus-seeded-mechanisms` (exporter, lever, LOO), `tree-sitter-overlay-extra` (grammar set), `authorization-obligations` (`handler_context`, obligation label).
|
||||
- **Downstream**: every roadmap number; `harnessx-self-improving-rulesets` (evolve loop gets a trustworthy fitness); `authorization-obligations` 4.3/5/6.
|
||||
|
||||
## Existing Spec Touchpoints
|
||||
|
||||
- **Extends**: `pair-corpus-honesty` Req 9/10 (one more row classification), `corpus-seeded-mechanisms` Req 4/5 (split-aware seeding and admission), `tree-sitter-overlay-extra` (grammar list).
|
||||
- **Re-homes**: `authorization-obligations` task 4.3 and the corpus half of 6.1.
|
||||
- **Blocks**: `harnessx-self-improving-rulesets` evolve step until 1–5 are measured.
|
||||
|
||||
## Constraints
|
||||
|
||||
Core `dependencies = []`; the grammar lands in the optional extra only. No row is deleted; unscorable rows stay in the catalog with a reason. No LLM-authored labels, shapes or policy. Gates (`gate`, `map_gate`, `pair_gate`) stay byte-identical: the local slice has no identical twins, no TypeScript and no lever. Every re-harvested excerpt is re-redacted and keeps its license line. All numbers in the roadmap are rewritten in one commit with the measurement command that produced them.
|
||||
|
||||
## Viability of `authorization-obligations` under this brief
|
||||
|
||||
The detector is sound where it can see: with a registration line present it fires on the real vampi vulnerable excerpt and stays silent on a constrained twin. It cannot be evaluated today because its inputs (named entry points, registration context, TypeScript parse) are exactly the defects above. Recommendation: keep tasks 1–3 and 4.1/4.2 (implemented, reviewed, pushed), re-home 4.3 and 6.1's corpus half here, and hold 5 (the lever) until the split is enforced, because a lever admitted under the leaking rule would be admitted for the wrong reason.
|
||||
|
||||
## Open Questions for the maintainer
|
||||
|
||||
1. Treat the five identical twins as `known_limit` (excluded from achievable, kept for silence) or drop them from the vibe-py catalog?
|
||||
2. Is `tree-sitter-typescript` acceptable in the semantic extra, or does TypeScript wait for a broader overlay-ir-completeness spec?
|
||||
3. Should the fixture-difficulty gate be CI-failing from day one or reported first?
|
||||
4. Does the HarnessX evolve step (approach 6) get its own spec, or stays as the final task group here once 1–5 are measured?
|
||||
@@ -0,0 +1,90 @@
|
||||
# Brief: learning-harness
|
||||
|
||||
## Problem
|
||||
|
||||
The tool is a hand-authored static analyzer with a learning loop bolted on the side, and the loop cannot learn. Three deliberate choices cap it: closed vocabularies (sinks, sources, obligation kinds) that grow only by writing a new spec; a corpus that is the only teacher and a shape that is the only lesson; and a ban on retaining anything the model discovers. Results have ceilings by construction, the loop trains on its own test set, and there is no failure analysis feeding the next round. Detection logic is ~16% of the source and the other 84% cannot say whether the 16% works.
|
||||
|
||||
Measured 2026-09-06 on the vibe-py holdout (17 vendored pairs), the existing tool hunter (Sonnet 5 through OpenRouter, 4 steps, tools `read_file`, `grep_repo`, `find_refs`, prompted from file-level hotspots, no learning of any kind) against the overlay:
|
||||
|
||||
| scorer | published number | class-aware, identical twins excluded (n = 14) |
|
||||
|---|---|---|
|
||||
| tool hunter | 0/17 correct, Youden −0.588 | **12/14 correct, Youden +0.857, 0 leaks** |
|
||||
| overlay (facts + shapes) | 5/17 correct, Youden +0.294 | 5/14 correct, Youden +0.357 |
|
||||
|
||||
The published hunter number is a scoring artifact: the pair scorer counts a hunter finding only when the finding text contains the literal CWE id and the prompt never asks for one, so 24 of 25 vulnerable-side findings that sit inside the labeled function and name the labeled bug (SQL injection in `resolve_pastes`, BOLA in `get_by_title`, SSTI in `read_root`, missing authorization on the profile lookup) were thrown away. The 24 "leaks" split into 3 findings on byte-identical twins (unscorable by construction) and 21 findings outside the labeled function on the trap twin, most of them real defects in a deliberately vulnerable repository. The two remaining misses are `before_request` (`loads(b64decode(cookie))` with no import in the excerpt) and `bad_mvc.do_GET` (labeled CWE-79, the hunter reported the SQL concatenation on the same lines; the label may be wrong).
|
||||
|
||||
Three consequences follow.
|
||||
|
||||
1. **The model with tools is already the best detector we own, by a wide margin, without any learning.** The roadmap thesis ("recall from the model with tools plus proof; precision and silence from static checks") is confirmed, and the harness has been optimizing the wrong component.
|
||||
2. **Detection, silence and leak are meaningless without a class.** The same finding is a detection in one class, a leak in another, and noise in a third. Every scorer, lever and gate in the tool currently uses text tokens (CWE strings, sink names) as a proxy for class, and the corpus's own class labels are 32 free-text strings across 7 slices, with 130 of 176 vfc rows labeled `other`.
|
||||
3. **Improvements without a class dimension oscillate.** A change that helps SQL injection and hurts access control reads as "no change" on an aggregate Youden. The loop then reverts it, or keeps it, for the wrong reason, and the system degrades in ways nobody predicted. This is the maintainer's observation and it is exactly what the corpus-seeded lever does today.
|
||||
|
||||
Repairs already measured and required regardless (from the folded `closed-loop-integrity` brief): the mechanism lever seeds candidates from holdout pairs (holdout Youden +0.059 with every pair teaching, −0.059 with the train split only); five vibe-py twins are byte-identical; TypeScript has no grammar; absence excerpts carry no registration context; unit fixtures are easier than the corpus rows they stand in for.
|
||||
|
||||
## Desired Outcome
|
||||
|
||||
A harness that improves a **set of classified detectors** round by round, where every change is attributed to the class it was meant to move, every unpredicted regression in another class counts against it, and nothing is published without its unscorable denominator.
|
||||
|
||||
- **Auto-classifier.** Every unit of work (a labeled pair, a finding, a scan target region) is assigned to a class from a closed, versioned taxonomy by a classifier that is itself measured: agreement with maintainer labels on the reviewed pairs, confusion matrix per round. The classifier is the routing key for everything downstream: which detector runs, which skills load, which holdout the change is validated on, which journal row records it.
|
||||
- **Per-class detectors.** One hunter configuration per class: system prompt, skills, tool set, step budget, evidence checklist, known counterexamples. A round evolves one class's configuration and is validated on that class's holdout plus a regression sweep over every other class. The static machinery (facts, overlay, obligations, dominance, sandbox) becomes tools the detector calls and verifiers that check its claims; it stops being the detector.
|
||||
- **Class-aware scorer.** Detection = a finding in the labeled function whose class matches the label. Silence = no finding of the labeled class in the labeled function on the fixed side. Unscorable rows (identical twin, unsupported language, unresolved label) are listed by reason and removed from the denominator, never deleted. The text-token proxy goes away.
|
||||
- **Learning unit = skill, not regex.** A round's output is a natural-language skill, a tool recipe, a checklist or a counterexample, drafted from a failure analysis, tested on the class's train split, validated on its holdout, journaled with a hypothesis, the predicted affected class, and the measured attribution. HarnessX's `MetaAgent.evolve` with its journal (`hypothesis_id`, `levers`, `predicted_affected`, precision discounted by `regressed_unpredicted`) is the loop primitive; the tool supplies tasks, trajectories, the evaluator and the class routing.
|
||||
- **Honest fitness.** Split enforced by construction (a candidate touching a holdout pair is refused, not scored); per-class Youden on scorable rows; sandbox proof as the tiebreaker where a class has recipes; the roadmap rewritten from one command with the unscorable denominator next to every number.
|
||||
|
||||
Principle for the roadmap: **classify first, then detect, then improve one class at a time under a fitness that punishes what it did not predict.**
|
||||
|
||||
## Approach
|
||||
|
||||
1. **Class-aware scorer and unscorable rows** (repairs the ruler; smallest change, biggest number). Detection and silence keyed on class and labeled function; identical-twin and unsupported-language rows flagged with a reason and excluded from denominators; the hunter's findings carry a class field the scorer reads. Re-publish every slice number from one command.
|
||||
2. **Taxonomy and auto-classifier.** A closed, versioned class taxonomy (seeded from the 32 free-text labels, the mechanism vocabulary and the obligation kinds; CWE families as the spine). A classifier with two tiers: a deterministic tier from static signals (rule id, sink family, obligation kind, mechanism) and a model tier for what static signals cannot place, each answer carrying which tier decided. Measured against maintainer labels on the reviewed pairs; disagreements are the review queue. The vfc slice's 130 `other` rows are its first job.
|
||||
3. **Per-class hunter configurations.** Split the single hunter into one configuration per class: prompt, skills directory, tools, budget, evidence checklist. Start by cloning the current hunter into every class so round zero is a controlled baseline, then let rounds diverge them. The classifier routes a target region to its detector; a region can route to several.
|
||||
4. **Split enforcement and lever refusal.** The exporter and every future lever take the train split by construction and refuse candidates that touch a holdout pair. Re-measure `corpus-seeded-mechanisms` and rewrite its roadmap row.
|
||||
5. **Evolve loop on HarnessX.** Tasks = classified pairs (`RLTask.task_type` = class); trajectories = the hunter's journaled runs; evaluator = the class-aware scorer; `MetaAgent.evolve` proposes one configuration change per round for one class with `predicted_affected` set to that class; the round is accepted only if the class's holdout improves and no other class regresses beyond tolerance; the journal's attribution table is the round report. Budget-capped per round.
|
||||
6. **Corpus repairs that the loop needs.** TypeScript grammar in the semantic extra; absence rows re-harvested with `handler_context`; identical twins marked `known_limit` with reason `identical_twin`; a fixture-difficulty check that fails when a unit fixture is strictly easier than the corpus rows its test claims to cover.
|
||||
7. **Static machinery as tools and verifiers.** Facts, overlay dispositions, obligations, entry points and the sandbox are exposed to the hunter as tools (`entry_points`, `flows`, `obligations`, `prove`) and to the scorer as verifiers that can raise a hunter suspicion to `static_corroboration` or a proof rung. No detector logic is deleted; it changes role.
|
||||
|
||||
## Scope
|
||||
|
||||
- **In**: class-aware scorer; taxonomy and classifier with its own measurement; per-class hunter configurations; split enforcement; HarnessX evolve loop wiring with tasks, trajectories, evaluator and journal; the corpus repairs listed in 6; static machinery exposed as hunter tools; roadmap rewrite from one command.
|
||||
- **Out**: new static shape families; new hand-written facts beyond what a round proposes and validates; changing the evidence ladder or dispositions; any merge gate on non-local slices; vendoring new pairs beyond re-harvesting existing recipes; fine-tuning model weights (skills and configurations only).
|
||||
|
||||
## Boundary Candidates
|
||||
|
||||
- Taxonomy and classifier (new, owns class assignment) vs pair labels (unchanged, the reference the classifier is measured against)
|
||||
- Per-class detector configurations (new, evolved) vs static analyzers (unchanged internally, exposed as tools)
|
||||
- Evolve loop (new, HarnessX) vs existing bounded levers (rules, policy, mechanisms; kept, subordinated to the split and the class attribution)
|
||||
- Class-aware scorer (replaces the text-token rule) vs catalog schema (one additive `unscorable` reason)
|
||||
|
||||
## Out of Boundary
|
||||
|
||||
- Taint semantics, walker, dominance internals (`guard-dominance-regime`, `overlay-ir-completeness`)
|
||||
- Call graph and path records (`reachability-flow-model`), except as a future hunter tool
|
||||
- Obligation checker internals (`authorization-obligations`); its corpus tasks 4.3 and 6.1 move here, its lever task 5 waits for the split
|
||||
|
||||
## Upstream / Downstream
|
||||
|
||||
- **Upstream**: `pair-corpus-honesty` (labels, tiers, splits, `known_limit`), `corpus-seeded-mechanisms` (exporter, LOO, lever), `tool_hunter` and `provider/openrouter`, `harness_ext` seam, HarnessX 0.1.0 (`meta_harness`, `rl`, `tracing.journal`), `tree-sitter-overlay-extra`.
|
||||
- **Downstream**: every roadmap number; `harnessx-self-improving-rulesets` (its evolve step is this spec); `authorization-obligations` 4.3/5/6; scan reports (findings carry a class and the detector that produced them).
|
||||
|
||||
## Existing Spec Touchpoints
|
||||
|
||||
- **Extends**: `pair-corpus-honesty` Req 1 (scorer rule) and Req 9/10 (row classification); `corpus-seeded-mechanisms` Req 4/5 (split by construction); `three-stage-scan` MAP (per-class hunters as proposers); `harnessx-self-improving-rulesets` (the evolve loop lands here).
|
||||
- **Folds in**: the `closed-loop-integrity` brief (its five repairs are approach items 1, 4 and 6).
|
||||
- **Re-homes**: `authorization-obligations` 4.3 and the corpus half of 6.1; holds 5.
|
||||
|
||||
## Constraints
|
||||
|
||||
Core `dependencies = []`; HarnessX and grammars stay optional extras; every loop step degrades to a recorded reason without them. No row deleted; unscorable rows keep a reason. Labels stay maintainer-owned: the classifier proposes, disagreements queue for review, and a class change on a reviewed pair is a reviewed edit. Model-authored skills are retained only after held-out validation and are journaled with their attribution; nothing a round proposes reaches a scan without passing its class holdout and the cross-class regression sweep. Split enforced by construction, never by convention. Gates (`gate`, `map_gate`, `pair_gate`) stay byte-identical: the local slice is not touched by any of this. Redaction applies to every prompt and every journaled trajectory. Per-round cost cap; a round that exceeds it is recorded and reverted.
|
||||
|
||||
## Open Questions for the maintainer
|
||||
|
||||
1. Taxonomy spine: CWE families, the mechanism vocabulary, or a new closed list with both as attributes?
|
||||
2. Round zero: clone the current hunter into every class (controlled baseline) or start each class from a hand-written seed prompt?
|
||||
3. Cross-class regression tolerance: zero, or a per-class floor proportional to class size?
|
||||
4. Does `authorization-obligations` task 5 (the lever) stay a spec task or become one class's evolve round?
|
||||
5. Model for the detectors and for the meta-agent: same model, or a cheaper detector with a stronger meta-agent?
|
||||
|
||||
## Evidence
|
||||
|
||||
- `benchmarks/measurements/2026-09-06-vibe-py-holdout-hunter-sonnet5.json`: `ousast pairs --slice vibe-py --split holdout --hunter --hunter-model anthropic/claude-sonnet-5 --json` (2026-09-06).
|
||||
- `benchmarks/measurements/2026-09-06-vibe-py-holdout-hunter-findings.json`: every hunter finding per side for the 17 holdout pairs with `in_labeled_function` and the scorer's verdict; the class-aware re-score uses a keyword classifier written for this measurement only and is the reason approach item 2 exists.
|
||||
@@ -0,0 +1,841 @@
|
||||
[
|
||||
{
|
||||
"pair": "damn-vulnerable-graphql-application-views-resolve-pastes",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-89",
|
||||
"vulnerability_class": "sql_injection",
|
||||
"path": "core/views.py",
|
||||
"evidence": "SQL injection via unsanitized 'filter' GraphQL argument directly interpolated into raw SQL using Python string formatting: result.filter(text(\"title = '%s' or content = '%s'\" % (filter, filter))). An attacker can inject arbitrary SQL via th",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "resolve_pastes",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "core/views.py",
|
||||
"line": 20,
|
||||
"title": "SQL Injection via unsanitized 'filter' parameter in resolve_pastes",
|
||||
"rationale": "The GraphQL resolver builds a raw SQL WHERE clause using Python string interpolation with the user-supplied 'filter' argument: `text(\"title = '%s' or content = '%s'\" % (filter, filter))`. Since 'filter' is passed directly from the GraphQL query without escaping or parameterization, an attacker can i",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "core/views.py",
|
||||
"line": 14,
|
||||
"title": "Plaintext password comparison enables credential stuffing/timing attacks",
|
||||
"rationale": "User.query.filter_by(username=username, password=password) compares raw password directly against a stored value with no hashing (e.g., bcrypt) and no constant-time comparison, suggesting passwords are stored in plaintext in the database. This exposes credentials to trivial theft on any DB read acce",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "core/views.py",
|
||||
"line": 14,
|
||||
"title": "No account lockout / rate limiting on login mutation",
|
||||
"rationale": "The mutate() resolver performs an unrestricted DB lookup with username+password on every call, with no rate limiting or lockout logic visible, making brute-force credential guessing against this GraphQL mutation straightforward given GraphQL's higher-density request surface.",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "core/views.py",
|
||||
"line": 16,
|
||||
"title": "Generic exception message but no differentiation prevents user enumeration only weakly",
|
||||
"rationale": "While raising 'Authentication Failure' avoids leaking whether the username or password was wrong, the surrounding logic still allows unlimited attempts, and any variance in Audit.create_audit_entry timing/behavior before the check could still leak information about valid usernames.",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "core/views.py",
|
||||
"line": 19,
|
||||
"title": "Tokens minted from username only, without verifying additional session/secret binding",
|
||||
"rationale": "create_access_token(username) and create_refresh_token(username) are generated using just the username string; if the underlying JWT implementation lacks proper signing/expiry configuration (not shown here), tokens could be forged or reused improperly, and there's no per-session revocation reference",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "djangoat-analytics-hits-by-ip",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-89",
|
||||
"vulnerability_class": "sql_injection",
|
||||
"path": "app/models/Analytics/analytics.py",
|
||||
"evidence": "SQL injection via string formatting in raw query. Both ip and col parameters are interpolated directly into SQL: 'SELECT %s FROM %s WHERE ip_address=\\'%s\\' ORDER BY id DESC' % (col, table_name, ip). The ip parameter comes from request.GET['",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "hits_by_ip",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "app/models/Analytics/analytics.py",
|
||||
"line": 13,
|
||||
"title": "SQL Injection via unparameterized string interpolation in hits_by_ip",
|
||||
"rationale": "The 'ip' and 'col' parameters are directly interpolated into a raw SQL string using Python %-formatting instead of using parameterized queries (e.g., cursor.execute(cmd, [ip])). If 'ip' or 'col' originates from user-controllable input (such as an HTTP request parameter), an attacker could inject arb",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "app/models/Analytics/analytics.py",
|
||||
"line": 13,
|
||||
"title": "Orphaned/incomplete analytics function with no visible callers or surrounding query logic",
|
||||
"rationale": "The file only contains a nested parse_field() helper that whitelists field names ('ip_address','referrer','user_agent') defaulting to '1' for anything else, evidently used to sanitize a column name for dynamic SQL (as suggested by the header comment referencing hits_by_ip and 'source_reaches_sink').",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "djangoat-views-user-view",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-89",
|
||||
"vulnerability_class": "sql_injection",
|
||||
"path": "app/views/users/views.py",
|
||||
"evidence": "SQL injection via string formatting in raw query. user_id_form from POST data is interpolated directly into SQL: User.objects.raw('SELECT * FROM %s WHERE user_id=\\'%s\\' ORDER BY is_admin=\\'0\\'' % (table_name, user_id_form)). An attacker can",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "user_view",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "app/views/users/views.py",
|
||||
"line": 18,
|
||||
"title": "Broad exception swallowing hides injected/malformed query errors",
|
||||
"rationale": "The `try/except` around `users[0]` catches all exceptions silently and just returns a generic 'NOT FOUND' message. This makes it harder to detect exploitation attempts (e.g., malformed SQL injection payloads causing query errors) and could mask underlying security-relevant failures, aiding attackers",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "app/views/users/views.py",
|
||||
"line": 21,
|
||||
"title": "SQL Injection via raw() query using unsanitized user_id from POST data",
|
||||
"rationale": "The function takes `user_id` directly from `request.POST['user_id']` (form['user_id']) and interpolates it unsanitized into a raw SQL string via Python string formatting (`% (table_name, user_id_form)`), then executes it with `User.objects.raw(...)`. An attacker controlling the POST body can inject ",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "app/views/users/views.py",
|
||||
"line": 36,
|
||||
"title": "Mass-assignment / unchecked mutable update dict onto arbitrary model instance",
|
||||
"rationale": "The `update` dict is built from `User.validate_update_form(form, user, update)` and then applied directly via `User.objects.filter(pk=user.pk).update(**update)` or `user.__dict__.update(update)` + `user.save()`. If the validation function does not strictly whitelist fields (e.g., it could allow `is_",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "app/views/users/views.py",
|
||||
"line": 13,
|
||||
"title": "Whitelist-based field validator with no visible call site or sink usage",
|
||||
"rationale": "The file only contains a nested `parse_field` helper that whitelists `ip_address`, `referrer`, `user_agent` and falls back to the literal `'1'` for anything else. This pattern is typical of sanitizing a column/field name before interpolating it into a raw SQL query (e.g., ORDER BY or WHERE clause bu",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "dsvpwa-dsvpwa-main",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-326",
|
||||
"vulnerability_class": "security_misconfiguration",
|
||||
"path": "dsvpwa.py",
|
||||
"evidence": "Insecure SSL/TLS configuration in main(). Three dangerous SSL options are explicitly enabled: (1) SSLv3 re-enabled via ctx.options &= ~ssl.OP_NO_SSLv3 (vulnerable to POODLE), (2) TLS compression re-enabled via &= ~ssl.OP_NO_COMPRESSION (vul",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "main",
|
||||
"sink": null,
|
||||
"mechanism": "permissive_default",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "dsvpwa.py",
|
||||
"line": 22,
|
||||
"title": "SSL enablement defaults from an environment variable with weak type handling",
|
||||
"rationale": "The --ssl flag defaults to os.getenv('DSVPWA_SSL', 0) but is defined as store_true with a non-boolean default; combined with the insecure SSL context modifications above, any deployment relying on environment-based configuration is likely to end up with a downgraded/misconfigured TLS setup by defaul",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvpwa.py",
|
||||
"line": 33,
|
||||
"title": "SSL context re-enables SSLv3 support",
|
||||
"rationale": "The code explicitly clears ssl.OP_NO_SSLv3 from ctx.options, re-enabling the obsolete and insecure SSLv3 protocol which is vulnerable to POODLE and other downgrade attacks. This is a permissive default that weakens transport security for any client that can negotiate SSLv3.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvpwa.py",
|
||||
"line": 34,
|
||||
"title": "SSL context disables compression protection",
|
||||
"rationale": "Clearing ssl.OP_NO_COMPRESSION re-enables TLS compression, which exposes the connection to CRIME-style compression side-channel attacks that can leak sensitive session data (e.g., cookies).",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvpwa.py",
|
||||
"line": 35,
|
||||
"title": "Server cipher preference explicitly disabled",
|
||||
"rationale": "Clearing ssl.OP_CIPHER_SERVER_PREFERENCE allows the client to dictate the negotiated cipher suite instead of the server, letting a malicious or misconfigured client force use of a weak cipher, undermining the server's ability to enforce strong cryptography.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "dsvpwa.py",
|
||||
"line": 17,
|
||||
"title": "Unvalidated/unsanitized comment input accepted without length or content checks",
|
||||
"rationale": "The 'comment' parameter is taken directly from request params with only a default empty-string fallback and no validation, sanitization, or length limiting before being persisted to the comments table, allowing arbitrary and potentially malicious content (e.g., script payloads, oversized data) to be",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvpwa.py",
|
||||
"line": 20,
|
||||
"title": "Stored XSS via unescaped comment output in guestbook",
|
||||
"rationale": "User-supplied 'comment' parameter is inserted into the database via a parameterized query (safe from SQLi) but later rendered directly into HTML via string formatting ('<td>{}</td>'.format(column)) without any HTML encoding. Any comment containing HTML/JS (e.g., <script>) is stored and then rendered",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "dsvw-dsvw-do-get",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-89",
|
||||
"vulnerability_class": "sql_injection",
|
||||
"path": "dsvw.py",
|
||||
"evidence": "SQL injection via id parameter directly concatenated into SELECT query without sanitization",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "do_GET",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 19,
|
||||
"title": "SQL Injection via 'id' parameter",
|
||||
"rationale": "params[\"id\"] is concatenated directly into a SQL SELECT statement without parameterization or sanitization, allowing arbitrary SQL injection through the GET query string.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 21,
|
||||
"title": "Insecure deserialization via pickle.loads on untrusted input",
|
||||
"rationale": "The 'object' GET parameter is base64/urlencoded attacker-controlled data fed directly into pickle.loads(), enabling arbitrary code execution during unpickling.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 21,
|
||||
"title": "Reflected XSS via 'v' parameter",
|
||||
"rationale": "params[\"v\"] is substituted directly into HTML content via re.sub without HTML-escaping, allowing reflected cross-site scripting through crafted query values.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 22,
|
||||
"title": "Path traversal / SSRF via 'path' parameter",
|
||||
"rationale": "params[\"path\"] is used directly in os.path.abspath()/open() or urllib.request.urlopen() with no validation, allowing local file disclosure or server-side request forgery to arbitrary URLs/schemes.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 23,
|
||||
"title": "OS command injection via 'domain' parameter",
|
||||
"rationale": "params[\"domain\"] is concatenated into a shell command string passed to subprocess.run with shell=True, allowing arbitrary command execution via shell metacharacters.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 24,
|
||||
"title": "XXE (XML External Entity) injection via 'xml' parameter",
|
||||
"rationale": "User-supplied XML is parsed with lxml.etree.XMLParser(load_dtd=True, resolve_entities=True, no_network=False), enabling external entity expansion, SSRF, and local file disclosure through crafted DTDs/entities.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 26,
|
||||
"title": "XPath injection via 'name' parameter",
|
||||
"rationale": "params[\"name\"] is interpolated directly into an XPath expression string (.//user[name/text()='%s']) without escaping, allowing XPath injection to bypass filters or extract arbitrary XML data.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 29,
|
||||
"title": "Uncontrolled resource consumption (DoS) via 'size' parameter",
|
||||
"rationale": "params[\"size\"] is used unchecked as both string repeat count and loop range for int(), allowing extremely large values to cause massive memory/CPU consumption (denial of service).",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 33,
|
||||
"title": "SQL Injection via 'comment' parameter (stored)",
|
||||
"rationale": "params[\"comment\"] is directly formatted into an INSERT SQL statement without parameter binding, enabling stored SQL injection.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 39,
|
||||
"title": "Reflected/stored XSS in rendered comments and results output",
|
||||
"rationale": "Database query results (comments, user rows) are directly interpolated into HTML output without escaping, enabling stored cross-site scripting if malicious HTML/JS is inserted via the comment or id-injected data.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 42,
|
||||
"title": "Remote code execution via 'include' parameter and exec()",
|
||||
"rationale": "Content fetched from a user-controlled local path or arbitrary remote URL (params[\"include\"]) is passed directly to exec(), allowing full arbitrary Python code execution on the server (RCE), plus SSRF/LFI via the URL/path fetch itself.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 47,
|
||||
"title": "Open redirect via 'redir' parameter",
|
||||
"rationale": "params[\"redir\"] is inserted unescaped into a meta-refresh HTML tag controlling redirection target, allowing open redirect and potential HTML/attribute injection since it is not sanitized.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "dsvw.py",
|
||||
"line": 55,
|
||||
"title": "SQL Injection in login (username/password) leading to authentication bypass",
|
||||
"rationale": "params.get(\"password\", \"\") is concatenated unsanitized into the SQL WHERE clause of the login query (only username is filtered via regex), enabling SQL injection-based authentication bypass.",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "dvpwa-middlewares-session-middleware",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-1004",
|
||||
"vulnerability_class": "security_misconfiguration",
|
||||
"path": "sqli/middlewares.py",
|
||||
"evidence": "Session cookie created with httponly=False. The RedisStorage is configured with httponly=False, meaning the AIOHTTP_SESSION cookie is accessible to JavaScript via document.cookie. Combined with the stored XSS vulnerabilities, this enables s",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "session_middleware",
|
||||
"sink": null,
|
||||
"mechanism": "permissive_default",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "sqli/middlewares.py",
|
||||
"line": 21,
|
||||
"title": "Session cookie created with httponly=False",
|
||||
"rationale": "The RedisStorage session cookie is explicitly configured with httponly=False, which disables the HttpOnly flag and allows client-side JavaScript to read the session cookie via document.cookie. This is a permissive-default misconfiguration that significantly increases the impact of any XSS vulnerabil",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "extremely-vulnerable-flask-app-account-before-request",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-502",
|
||||
"vulnerability_class": "insecure_deserialization",
|
||||
"path": "routes/account.py",
|
||||
"evidence": "Remote code execution via insecure deserialization. The 'preferences' cookie is base64-decoded and passed directly to pickle.loads() (imported as 'loads' from pickle). An attacker can craft a malicious pickle payload, base64-encode it, and ",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "before_request",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [],
|
||||
"fixed": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "extremely-vulnerable-flask-app-signup-validate-token",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-89",
|
||||
"vulnerability_class": "sql_injection",
|
||||
"path": "routes/signup.py",
|
||||
"evidence": "SQL injection via f-string interpolation of user-supplied registration code directly into a raw SQL query: text(f\"SELECT id, code FROM ... WHERE code = '{code}'\"). An attacker can inject arbitrary SQL through the registration_code form fiel",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "validate_token",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "routes/signup.py",
|
||||
"line": 16,
|
||||
"title": "SQL Injection via unsanitized `code` parameter in validate_token",
|
||||
"rationale": "The `code` parameter is directly interpolated into a raw SQL query string using an f-string (`WHERE code = '{code}'`) and executed via `session.execute(text(...))`. Since `code` is user-controlled (e.g., from a registration/signup token field) and not parameterized or escaped, an attacker can inject",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "pythonssti-main-read-root",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-79",
|
||||
"vulnerability_class": "reflected_xss",
|
||||
"path": "main.py",
|
||||
"evidence": "The Jinja2 Environment is created with default settings (autoescape=False). User input from the 'username' query parameter is concatenated into the template string and the rendered output is returned as HTMLResponse. An attacker can inject ",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "read_root",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "main.py",
|
||||
"line": 19,
|
||||
"title": "Server-Side Template Injection (SSTI) via unsanitized username parameter",
|
||||
"rationale": "The `username` parameter, which originates from user-controlled input (e.g., a query/path parameter in a FastAPI/Starlette route), is directly concatenated into a Jinja2 template string via `Jinja2.from_string(\"Welcome \" + username + \"!\")` and then rendered. Because the user input becomes part of th",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "main.py",
|
||||
"line": 18,
|
||||
"title": "Server-Side Template Injection (SSTI) via unsanitized 'username' concatenated into Jinja2 template string",
|
||||
"rationale": "The `username` parameter (attacker-controlled input) is directly concatenated into a string that is passed to `Jinja2.from_string()` and then rendered. Since the value becomes part of the template source rather than being passed as render context, an attacker can inject Jinja2 template syntax (e.g. ",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "threatbyte-api-v1-get",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-639",
|
||||
"vulnerability_class": "broken_access_control",
|
||||
"path": "server/api/v1/api_v1.py",
|
||||
"evidence": "BOLA (Broken Object Level Authorization) on the API profile endpoint. The @token_required decorator is commented out (line 152-153), so /api/v1/profile?user_id=X returns any user's profile (username, email, country, role, permissions, team)",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "get",
|
||||
"sink": null,
|
||||
"mechanism": "missing_auth_guard",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "server/api/v1/api_v1.py",
|
||||
"line": 14,
|
||||
"title": "Missing authentication/authorization on profile lookup endpoint",
|
||||
"rationale": "The get() handler retrieves a user's username, email, country, role, permissions, and team directly from the `user_id` query argument with no visible authentication decorator, session check, or verification that the requester is authorized to view that profile (e.g., no comparison to the current log",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "server/api/v1/api_v1.py",
|
||||
"line": 17,
|
||||
"title": "Unvalidated user_id parameter enabling enumeration",
|
||||
"rationale": "`user_id = args.get('user_id')` is taken directly from request arguments without validating that it belongs to the requesting session/user. Combined with the missing auth guard, this enables attackers to iterate over IDs to harvest all user profiles including role/permissions/team fields, which is s",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "vampi-books-get-by-title",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-639",
|
||||
"vulnerability_class": "broken_access_control",
|
||||
"path": "api_views/books.py",
|
||||
"evidence": "Broken Object Level Authorization (BOLA). When vuln=1, get_by_title queries the book by title only (Book.query.filter_by(book_title=str(book_title)).first()) without verifying the authenticated user is the owner. Any authenticated user can ",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "get_by_title",
|
||||
"sink": null,
|
||||
"mechanism": "missing_auth_guard",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "api_views/books.py",
|
||||
"line": 17,
|
||||
"title": "Broken Object Level Authorization (BOLA) in get_by_title",
|
||||
"rationale": "When the `vuln` flag is enabled, the function queries the Book table by book_title alone, without filtering by the authenticated user. This returns the 'secret_content' and owner information for any book title regardless of ownership, allowing any authenticated user to read another user's private bo",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "api_views/books.py",
|
||||
"line": 17,
|
||||
"title": "Broken Object Level Authorization (BOLA) in get_by_title when 'vuln' flag is enabled",
|
||||
"rationale": "When the global 'vuln' flag is truthy, the function looks up a Book solely by book_title without verifying that the authenticated user (resp['sub']) is the owner. This allows any authenticated user to retrieve another user's secret_content by supplying a known/guessable book_title, exposing sensitiv",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "vampi-users-update-password",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-639",
|
||||
"vulnerability_class": "broken_access_control",
|
||||
"path": "api_views/users.py",
|
||||
"evidence": "Unauthorized password change (IDOR). When vuln=1, the update_password function uses the username from the URL path parameter to look up the user whose password to change, instead of using the authenticated user from the JWT token (resp['sub",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "update_password",
|
||||
"sink": null,
|
||||
"mechanism": "missing_auth_guard",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "api_views/users.py",
|
||||
"line": 20,
|
||||
"title": "Broken Object Level Authorization (BOLA) in update_password",
|
||||
"rationale": "When the `vuln` flag is enabled, the function updates the password for the `username` supplied in the URL path without verifying that it matches the authenticated user (`resp['sub']`) from the JWT token. Any authenticated user can therefore change any other user's password by simply calling the endp",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "vfapi-main-put-user",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-89",
|
||||
"vulnerability_class": "sql_injection",
|
||||
"path": "main.py",
|
||||
"evidence": "SQL injection via f-string interpolation of all user-supplied fields (name, username, password, address, email, contact) into an INSERT SQL query. All fields from the User model body are directly embedded in the query string without paramet",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "put_user",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "main.py",
|
||||
"line": 16,
|
||||
"title": "Use of weak/broken MD5 hash for password storage",
|
||||
"rationale": "Passwords are hashed with unsalted MD5 (`md5(user.password.encode()).hexdigest()`) before being stored in both the SQL database and MongoDB. MD5 is cryptographically broken and fast to brute-force/rainbow-table, and the lack of a per-user salt makes stored password hashes trivially crackable if the ",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "main.py",
|
||||
"line": 17,
|
||||
"title": "SQL Injection in put_user via unsanitized user fields in f-string SQL query",
|
||||
"rationale": "The function builds an INSERT statement by directly interpolating user-controlled fields (user.name, user.username, user.password, user.address, user.email, user.contact) into an f-string without parameterization or escaping, then executes it via run_sql_query. An attacker supplying a payload like a",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "main.py",
|
||||
"line": 27,
|
||||
"title": "Duplicate user data write to MongoDB without validation, enabling data inconsistency/injection surface",
|
||||
"rationale": "After the raw SQL insert, the same unsanitized user-controlled fields are inserted into a MongoDB collection with an `_id` derived from a raw SQL SELECT result. This mirrors the injectable input into a second datastore, increasing the impact of any injected payload and introducing potential NoSQL da",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "main.py",
|
||||
"line": 15,
|
||||
"title": "Potential NoSQL injection via unsanitized username in query construction",
|
||||
"rationale": "The function builds a MongoDB-style query dict directly from the `username` parameter (`{'username': username}`) and passes it to `get_nosql_users`. If `username` originates from untrusted user input (e.g., a JSON request body) rather than being strictly validated as a string, an attacker could supp",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "vulnerable-python-apps-bad-mvc-do-get",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-79",
|
||||
"vulnerability_class": "reflected_xss",
|
||||
"path": "Insecure_Frameworks/bad_mvc.py",
|
||||
"evidence": "XSS via unescaped output. The username variable (derived from decrypting a user-supplied cookie at line 26) is rendered into HTML without any escaping: bytes('Hello %s' % username, 'utf-8'). An attacker who controls the encrypted cookie val",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "do_GET",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "Insecure_Frameworks/bad_mvc.py",
|
||||
"line": 14,
|
||||
"title": "Cookie parsing without validation of session_id format prior to cryptographic operation",
|
||||
"rationale": "SimpleCookie(self.headers.get('Cookie')) parses attacker-controlled HTTP headers directly, and the raw 'session_id' value is passed straight into decrypt() with no length, charset, or format validation. This lack of input validation on attacker-supplied data flowing into a decryption/identity-resolu",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "Insecure_Frameworks/bad_mvc.py",
|
||||
"line": 17,
|
||||
"title": "Sensitive session cookie value passed to an unverified 'decrypt' routine with broad exception swallowing",
|
||||
"rationale": "The session_id cookie is passed to self.decrypt(), but the decrypt method is not defined anywhere in the visible file or elsewhere in the scanned repository (per find_refs), making it unclear what cryptographic scheme (if any) is used, whether it validates integrity (e.g., HMAC/AEAD) or is a weak/re",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "Insecure_Frameworks/bad_mvc.py",
|
||||
"line": 30,
|
||||
"title": "Reflected/Stored XSS via unescaped 'username' written into HTTP response",
|
||||
"rationale": "The value derived from the 'session_id' cookie (via self.decrypt) is embedded directly into the HTML response body using string formatting ('Hello %s' % username) without any HTML-encoding or escaping. If an attacker can control or influence the decrypted cookie value (e.g., by supplying a crafted s",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "Insecure_Frameworks/bad_mvc.py",
|
||||
"line": 14,
|
||||
"title": "Undefined function get_db_connection() with no import or definition",
|
||||
"rationale": "get_db_connection is called but never defined or imported in this file, meaning the snippet cannot execute as-is; this indicates the file is an isolated fragment extracted from a larger MVC app, and any real analysis of injection risk must be done in the context of the full application (e.g., where ",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "Insecure_Frameworks/bad_mvc.py",
|
||||
"line": 14,
|
||||
"title": "No exception handling around database connection/close",
|
||||
"rationale": "If conn.execute() raises an exception (e.g., malformed post_id causing a type error, or database lock), conn.close() on line 17 will never execute, potentially leaking the database connection under error conditions - a resource-management issue that could be leveraged for a denial-of-service resourc",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "Insecure_Frameworks/bad_mvc.py",
|
||||
"line": 15,
|
||||
"title": "SQL query uses parameterized placeholder but truncated/partial statement",
|
||||
"rationale": "The query 'SELECT * FROM posts WHERE id = ?' with (post_id,) does use a parameter binding, which is the safe pattern; however since this is presented as the 'bad_mvc.py' (vulnerable) sample paired with a 'fixed' commit per the provenance header, the file may not represent the actual vulnerable varia",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "Insecure_Frameworks/bad_mvc.py",
|
||||
"line": 19,
|
||||
"title": "abort() called without importing flask.abort",
|
||||
"rationale": "abort(404) is used but there's no import statement (e.g., from flask import abort) in this fragment, again reinforcing that this is a partial extraction; missing error-handling context could hide whether user input (post_id) is validated/sanitized before reaching the abort/404 path or whether except",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "vulnpy-cmdi-do-subprocess-popen",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-78",
|
||||
"vulnerability_class": "command_injection",
|
||||
"path": "src/vulnpy/trigger/cmdi.py",
|
||||
"evidence": "User-controlled command passed to subprocess.Popen with shell=True, enabling arbitrary OS command execution",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "do_subprocess_popen",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "src/vulnpy/trigger/cmdi.py",
|
||||
"line": 13,
|
||||
"title": "OS Command Injection via subprocess.Popen with shell=True",
|
||||
"rationale": "do_subprocess_popen passes an unvalidated 'command' argument directly into subprocess.Popen with shell=True, allowing arbitrary shell command execution if the argument is influenced by user input. No sanitization, allow-listing, or use of shell=False with argument lists is performed, making this a c",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "src/vulnpy/trigger/cmdi.py",
|
||||
"line": 13,
|
||||
"title": "Mislabeled/incomplete command-injection trigger function",
|
||||
"rationale": "The file's provenance header states this is the (fixed) `do_subprocess_popen` command-injection PoC from vulnpy, but the actual code defines an unrelated `_execute(db_func)` helper that calls an undefined `_db_reset()` and references an undefined `SELECT_ALL` constant. No subprocess/os.system call o",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "vulpy-libuser-create",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-89",
|
||||
"vulnerability_class": "sql_injection",
|
||||
"path": "bad/libuser.py",
|
||||
"evidence": "SQL injection via %-formatting in INSERT statement: c.execute(\"INSERT INTO users ... VALUES ('%s', '%s', ...\" %(username, password, ...)). User-supplied username and password are interpolated via % string formatting into the SQL query.",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "create",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "bad/libuser.py",
|
||||
"line": 19,
|
||||
"title": "SQL Injection via string formatting in user creation",
|
||||
"rationale": "The create() function builds a SQL INSERT statement using Python's % string formatting to directly interpolate the 'username' and 'password' parameters into the query without any parameterization, escaping, or input validation. An attacker-controlled username or password value (e.g., containing a si",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": []
|
||||
}
|
||||
},
|
||||
{
|
||||
"pair": "vulpy-libuser-password-change",
|
||||
"expected": [
|
||||
{
|
||||
"cwe": "CWE-89",
|
||||
"vulnerability_class": "sql_injection",
|
||||
"path": "bad/libuser.py",
|
||||
"evidence": "SQL injection via string format in UPDATE statement: c.execute(\"UPDATE users SET password = '{}' WHERE username = '{}'\".format(password, username)). The password parameter from user input is directly interpolated into the SQL query.",
|
||||
"rule_id": null,
|
||||
"line": null,
|
||||
"function": "password_change",
|
||||
"sink": null,
|
||||
"mechanism": "source_reaches_sink",
|
||||
"obligation": null
|
||||
}
|
||||
],
|
||||
"sides": {
|
||||
"vuln": [
|
||||
{
|
||||
"path": "bad/libuser.py",
|
||||
"line": 19,
|
||||
"title": "SQL Injection via unparameterized query in password_change",
|
||||
"rationale": "The function builds a raw SQL UPDATE statement using Python str.format() to directly embed the user-controlled 'username' and 'password' parameters into the query string, then executes it via c.execute(). Neither value is sanitized or passed as a bound parameter, allowing an attacker to inject arbit",
|
||||
"in_labeled_function": true,
|
||||
"matches": false
|
||||
}
|
||||
],
|
||||
"fixed": [
|
||||
{
|
||||
"path": "bad/libuser.py",
|
||||
"line": 13,
|
||||
"title": "Metadata/function mismatch suggests incomplete or misattributed sample",
|
||||
"rationale": "The file's provenance header claims this is the 'password_change' function from vulpy, but the actual code defines get_posts with no password-changing logic. This inconsistency suggests the corpus entry may not reflect the intended vulnerable code path (e.g., a missing password_change function elsew",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "bad/libuser.py",
|
||||
"line": 15,
|
||||
"title": "SQL trace callback leaks query data via print/stdout",
|
||||
"rationale": "conn.set_trace_callback(print) causes every SQL statement (including bound parameter values such as usernames) to be printed to stdout/logs. In production this can leak sensitive data (usernames, potentially other PII) into logs, and is a debugging leftover that should not ship in a security-sensiti",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
},
|
||||
{
|
||||
"path": "bad/libuser.py",
|
||||
"line": 15,
|
||||
"title": "New DB connection per call without proper resource closing",
|
||||
"rationale": "get_posts opens a new sqlite3 connection on every call and never closes the connection or cursor (no try/finally or context manager). Under concurrent or repeated invocation this can exhaust file descriptors/connections, a resource-exhaustion/DoS-adjacent issue, and is a common source of subtle bugs",
|
||||
"in_labeled_function": false,
|
||||
"matches": false
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user