Orchestrating 7 security scanners into one report

The Analyst's Stack · Showcase

One command runs a codebase through 6 security scanners, or a live URL through a seventh, and hands back a single list of problems. The interesting part is not the 7.

Why I built it

I write small tools that touch real material, and before one goes near that material I want to know 3 boring things: is a credential committed in it, is it standing on a dependency with a known hole, is the container config wrong.

There is a good free scanner for each of those questions and none that answers all of them. So you run several. That part is easy — each is one command.

Then you have several reports. One is a JSON object with a Results array. One has a results array that means something different. One grades findings ERROR, WARNING, INFO. One grades them CRITICAL through LOW. One has no severity field at all, because everything it finds is a leaked credential and it considers that self-evidently bad.

What you do next, if you are honest, is open a spreadsheet. I did that 5 times before admitting it was a permanent job.

What it solves

One command, one report, one row per problem. The scanners run in parallel, up to 6 at a time, and the slowest one sets the wall clock instead of the sum.

The ratio is the point: the code that runs the scanners is 271 lines. The code that makes their output comparable is 740.

Two bars drawn to scale. The code that runs the 7 scanners is 271 lines. The code that makes their output comparable is 740 lines, nearly 3 times as much. — **Where the code went.** Orchestration is the cheap part; agreeing on what a finding is costs nearly 3 times as much.

$ NO_COLOR=1 COLUMNS=100 scan vulnapp --no-fail --out /tmp/p16-demo/scan-reports
                    Findings by scanner / severity
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━┳━━━━━━━━┳━━━━━┳━━━━━━┳━━━━━━━━━┓
┃ Scanner           ┃ Critical ┃ High ┃ Medium ┃ Low ┃ Info ┃ Unknown ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━╇━━━━━━━━╇━━━━━╇━━━━━━╇━━━━━━━━━┩
│ checkov           │ 0        │ 0    │ 15     │ 0   │ 0    │ 0       │
│ osv-scanner       │ 0        │ 7    │ 7      │ 0   │ 0    │ 0       │
│ osv-scanner+trivy │ 2        │ 6    │ 9      │ 0   │ 0    │ 0       │
│ semgrep           │ 0        │ 6    │ 3      │ 0   │ 0    │ 0       │
│ trivy             │ 0        │ 8    │ 1      │ 5   │ 0    │ 0       │
└───────────────────┴──────────┴──────┴────────┴─────┴──────┴─────────┘
Total: 2 critical · 27 high · 35 medium · 5 low · 0 info · 0 unknown
Wrote json: /tmp/p16-demo/scan-reports/scan-report.json
Wrote html: /tmp/p16-demo/scan-reports/scan-report.html

Six tools, one table. The scan is the part everyone pictures. One run over an intentionally vulnerable demo project: the 6 scanners that read a folder ran, and the 2 that found nothing have no row. The seventh probes a live URL, so it runs on its own with --url. The osv-scanner+trivy row is the merge described below. Real output, with colour codes stripped and the tool’s own name replaced by scan.

How it works

Five stages, left to right: Collect runs 7 scanners, Normalize turns every result into one finding shape, Analyze deduplicates and merges, Alert applies the severity gate, Present writes JSON, HTML and SARIF. Normalize is highlighted. — **Stage 2, doing all the work.** There are 5 boxes, not 6 — the run holds its own state, and the report files are the only thing that outlives it.

Collect. Seven scanners, each a plugin that registers itself. The dispatcher picks by target, so a folder gets the filesystem scanners and a URL gets the one that probes live services. A scanner whose binary is missing is recorded as skipped, with a reason, rather than silently dropped — an empty report that looks clean is the failure this stage exists to avoid.

Normalize. Every output becomes the same record: scan type, tool, title, severity, file, line range, description, a CVE if there is one, a suggested fix, a rule id, a fingerprint. Six severity levels, with each tool’s vocabulary mapped onto them.

One hardcoded credential reported by 3 scanners. Each arrives under its own rule name and its own severity vocabulary: the first grades it WARNING, the second grades it CRITICAL, the third has no severity field at all. A normalize step in the middle turns all 3 into one record shape on 6 levels, so they come out as severity medium, severity critical and severity high. — **The third row is the expensive one.** Downstream, a severity somebody picked on a tool’s behalf is indistinguishable from one the tool actually assigned.

Analyze. Findings collapse by fingerprint. Then the 2 dependency scanners get a second pass: the same CVE, package and file from both becomes one finding whose tool name is both names joined. Optionally, a model triages what is left.

Alert. A severity gate. If any finding matches the levels you asked to fail on — critical or high by default — the command exits 1. That is the whole notification layer, and in a pipeline it is enough.

Present. The same findings written 3 ways: JSON for another program, HTML with summary cards for a person, SARIF for a code host that can annotate a diff.

The last box, sized honestly. Cards, a table, and the fix text the scanner supplied. Nothing here decides anything. The report from the same demo run, so every finding on screen is a demo finding. The heading that carries the tool’s name is hidden, as is the table’s Line column, empty for these rows.

How I built it

Python, a command-line framework, and a thread pool. Each scanner is a small class whose job is to turn one tool’s output into a list of the common record. Adding an eighth is an afternoon; that was the point of the plugin shape.

Normalization was not an afternoon. It went wrong in ways that stayed invisible until I wrote tests for them.

The merge that downgraded things. When 2 dependency scanners report the same CVE I keep the more severe grading. The internal ranking puts critical at 0 and unknown at 5, so the most severe member of a group is the minimum. I wrote max. A low-severity duplicate then quietly overrode a critical finding and sailed through the default gate, and the build stayed green — the worst category of bug in a tool whose job is to stop a build.

The internal severity ranking runs from critical at 0, through high, medium, low and info, to unknown at 5, so the most severe finding has the smallest number. Below it, 2 dependency scanners report the same CVE, one grading it critical and the other low. Keeping the maximum picks low, which passes the default gate and leaves the build green. Keeping the minimum picks critical, which fails the gate as it should. — **One wrong function name.** On a scale where critical is 0, *keep the more severe* means the minimum.

The merge that grouped strangers. My first version grouped duplicate CVEs on the CVE identifier alone. The same CVE in 2 lock files is 2 dependency trees and 2 different upgrades; merging them hides one. The key is now the CVE, the scan type, the file and the package.

The fingerprint includes the tool’s name. Deliberate, and the thing I would most like to improve. It removes exact repeats but does not collapse across tools, because 2 tools disagreeing about one line is information rather than noise.

The AI step is off by default. It reads normalized findings rather than raw scanner output and scores each for whether it looks real. Below a confidence of 0.3 the finding is relabelled as informational, with the model’s reasoning attached. Nothing a model says here removes a row from the report.

And it is not allowed to see everything. A secrets finding points at the exact line a credential sits on, so the usual 5-lines-either-side excerpt is the credential. Send that to a hosted model and you have exfiltrated the thing you were scanning for. The payload for a secrets finding carries no source excerpt — not a redacted one, none — and the model judges it from the rule name, the path and the line, which is enough.

A look at the code

The fingerprint, rewritten to stand alone:

import hashlib

def fingerprint(scanner, rule_id, file_path, line):
    """A finding's identity: what found it, which rule fired, and where.

    `line` gets a sentinel when it is missing, so a finding with no line
    number cannot collide with one that genuinely points at line 0.
    """
    line_part = "none" if line is None else str(line)
    parts = f"{scanner}|{rule_id or ''}|{file_path or ''}|{line_part}"
    return hashlib.sha256(parts.encode()).hexdigest()[:16]

# The same finding on two runs an hour apart collapses to one row:
fingerprint("semgrep", "python.lang.security.audit.exec-detected",
            "app/tasks.py", 41)            # 'df85100c6ff5444a'

# A different tool, same file, same line, is a different row — on purpose:
fingerprint("bearer", "python_lang_exec", "app/tasks.py", 41)
                                          # '940fc3513fa5bca3'

And the rule about what a model is allowed to look at:

def triage_item(finding, read_source_excerpt):
    """Build the one payload item a hosted model sees for this finding.

    A secrets finding points at the exact line a credential sits on, so the
    usual line-plus-or-minus-five excerpt is the credential itself. No
    excerpt is safe here, so none is built. This must hold whether or not a
    redaction pattern happens to recognise the credential's format, which is
    why the check is on the finding's type and not on its contents.
    """
    excerpt = "" if finding.scan_type == "secrets" else read_source_excerpt(finding)
    return {
        "fingerprint": finding.fingerprint,
        "scanner": finding.scanner,
        "title": finding.title,
        "severity": finding.severity,
        "file_path": finding.file_path,
        "line_start": finding.line_start,
        "code_excerpt": excerpt,
    }

What it can’t do

Deduplication is mostly within a tool, not across them

Identical findings from one scanner collapse cleanly. Across scanners only one case is handled: the same CVE, package and file from both dependency scanners. Everything else stays duplicated — 2 static-analysis tools flagging one line with no CVE between them produce 2 rows, and no amount of comparing their descriptions makes a machine certain they are the same problem.

Normalizing a severity throws information away

Six levels have to absorb every scanner’s scale. One tool’s WARNING becomes medium. One has no severity field, so every credential it finds is high — including the expired one in a test fixture. Anything arriving with nothing usable defaults to medium, a guess wearing a label. And the gate is exact membership rather than a threshold: asking it to fail on medium and low will not fail it on a high finding. That is in the tests now because it surprised me.

It reports; it does not judge

Scanners are loud. A clean run means nobody found anything, not that there is nothing. The optional triage step only re-labels: below the threshold a finding is demoted, above it the model may nudge a severity either way. That is an opinion printed next to a finding, not a decision. And a default run is local but not hermetic — some scanners fetch their vulnerability data, and one sends your dependency inventory to a public service to ask about it.

Two tools, one row. The joined tool name is the visible trace of the merge, and the alias list carries the identifiers both tools used for the flaw. The row keeps one description, not both. From the same demo run.

Where this sits in the series

This is stage 2 of the spine with nothing else competing for attention. It has 7 sources, 7 shapes, and a pipeline that can do nothing at all until they are one shape. Change “scanner” to “platform” and it is every collection system in this series.

The credential rule is the argument from the do’s and don’ts showing up as an if statement. You can hold the principle — do not hand a third party the sensitive thing — and still ship the code that hands it over, because it arrived inside a routine that was only ever meant to add context. Principles become real when they are a line of code with a test behind them.

Both halves of this post continue elsewhere. Does your AI step actually work? is how you would find out whether the triage confidence score deserves the weight it carries. Storing and searching what you collect is the general form of the fingerprint: deciding when 2 records are the same thing is the same problem whether they are findings or events.