Skip to content

About

Example code for using the OpenVAS scanner container with recommended settings.

Resources

Stars

1 star

Watchers

1 watching

Forks

Repository files navigation

scan-examples

Example code for using the OpenVAS scanner container REST API with Greenbone community scanner containers and feed data synchronized by greenbone-feed-sync.

Warning

This repository contains example code only. It is not officially supported by Greenbone and is not intended for production use.

License

MIT. See LICENSE.

What is in this repo?

  • Python CLI example for the OpenVAS scanner REST API
  • scannerctl based conversion of feed scan configs into scan JSON, plus direct custom scan JSON payloads
  • Docker image for the example CLI
  • Docker Compose environment with:
    • greenbone-feed-sync for retrieving NASL, Notus, and data-object feeds into persistent named volumes
    • openvasd REST API plus the required ospd-openvas scanner service/socket wiring
    • a kirscht/metasploitable3-ub1404 target container for end-to-end scans
  • Unit tests plus GitHub Actions workflows for Ruff linting and self-hosted end-to-end validation

Configuration

This example targets the Greenbone community scanner container deployment shown in docker-compose.yml, with feed data retrieved by the greenbone-feed-sync container instead of the community feed data-copy containers.

  • Scanner API endpoint inside Compose: http://openvasd:80
  • Scanner API endpoint from the host: http://localhost:3000
  • Feed mounts expected by the example:
    • data objects synchronized by greenbone-feed-sync --type gvmd-data: /feed/data-objects
    • NASL vulnerability tests synchronized by greenbone-feed-sync --type nasl: /feed/vulnerability-tests
    • Notus advisories synchronized by greenbone-feed-sync --type notus: /var/lib/notus/advisories
  • Certificates:
    • the bundled Compose environment uses plain HTTP internally, so no client certificate files are required for the example workflow
    • if you point the CLI at an HTTPS scanner endpoint, trust the server certificate in your runtime or use --insecure only for throwaway lab testing

Supported environment variables:

  • SCANNER_API_URL — scanner REST API base URL
  • SCANNER_API_TIMEOUT — HTTP timeout in seconds
  • DATA_OBJECTS_PATH — mounted Greenbone data-objects feed path
  • VT_PATH — mounted Greenbone vulnerability-tests feed path (used for NASL content and vt-metadata.json enrichment lookups)
  • NOTUS_PATH — optional path to the mounted Notus advisory feed (.notus files) used for Notus-specific result enrichment
  • SCAP_PATH — optional path to SCAP/NVD CVE JSON data for second-stage CVE enrichment after VT OID matching
  • SCANNERCTL_BIN — path to scannerctl
  • SCAN_EXAMPLES_ENRICHMENT_ENGINE — default enrichment engine selection (auto, python, or rust)
  • SCAN_EXAMPLES_RUST_ENRICHMENT_BIN — optional explicit path to the scan-enrich-results Rust binary
  • SCAN_CONFIG — scan config to convert with scannerctl (defaults to full-and-fast)
  • SCAN_CONFIG_JSON — path to a custom scanner API scan config JSON payload, or a .zip containing one JSON file; when set, the e2e flow skips scannerctl conversion and uses this payload as a template
  • TARGET_TCP_PORTS — optional comma-separated TCP ports for the target definition; omit it to use the scan config defaults
  • TARGET_SSH_USERNAME / TARGET_SSH_PASSWORD / TARGET_SSH_PORT — SSH credentials to include in the scan target definition (defaults: msfadmin / msfadmin / 22 for the bundled target)
  • WAIT_BEFORE_RESULTS — initial delay before polling for scan results in the e2e flow
  • RESULTS_TIMEOUT / RESULTS_POLL_INTERVAL — controls for waiting until findings appear
  • E2E_MIN_RESULTS — minimum result count for first-results mode; CI uses 1000
  • CREATE_SCAN_RETRIES / CREATE_SCAN_RETRY_DELAY — API warm-up retry controls for scan creation

CLI commands

The Docker image and local package expose openvas-example for scanner lifecycle examples and openvas-enrich-results for standalone result enrichment.

Quick commit smoke test with openvas-mock-scanner

For fast commit-time coverage, this repo can exercise its real HTTP client and CLI against the baseline mock scanner from clawosiris/openvas-mock-sanner. CI pins the published ghcr.io/clawosiris/openvas-mock-scanner:0.2.1 release instead of a floating :latest tag so mock-backed validation only changes when this repository intentionally updates the selected mock scanner release. This is much lighter than the full Compose/OpenVAS stack and is intended as the default scanner-facing gate for short-lived pull requests.

Local example against the published container image:

OPENVAS_MOCK_SCANNER_IMAGE=ghcr.io/clawosiris/openvas-mock-scanner:0.2.1 \
  uv run pytest tests/test_mock_server_smoke.py

The mock test suite includes a feed-backed compatibility scenario using fixture data under tests/data/mock-feed. Those fixtures mirror the Greenbone feed shapes consumed by the real scan enrichment flow: VT metadata, Notus-style package advisory data, SCAP/CVE metadata, target services, and a larger paged result set.

If you do not have a container runtime available, the smoke test still supports a local source checkout fallback:

git clone https://github.com/clawosiris/openvas-mock-sanner tmp/openvas-mock-sanner
OPENVAS_MOCK_SANNER_REPO=$PWD/tmp/openvas-mock-sanner \
  uv run pytest tests/test_mock_server_smoke.py

Convert a scan configuration

openvas-example convert-config \
  --host target \
  --scan-config full-and-fast \
  --data-objects-path /feed/data-objects \
  --vt-path /feed/vulnerability-tests \
  --output scan.json

Scan lifecycle commands

openvas-example create-scan scan.json
openvas-example start-scan <scan-id>
openvas-example stop-scan <scan-id>
openvas-example get-results <scan-id>
openvas-example delete-scan <scan-id>

Enrich scanner results offline

There are now two enrichment entry points:

  • openvas-enrich-results — the Python-facing wrapper CLI shipped by this package
  • scan-enrich-results — the Rust binary built from rust/scan-enrichment

openvas-enrich-results is the stable user-facing command for post-processing saved scanner output. It uses the same enrichment logic as the e2e flow and defaults to the Rust engine when the binary is available, while still letting you explicitly select the Python reference implementation.

Required inputs:

  • --results — scanner results JSON, either a raw result array or an object containing a results array
  • at least one feed metadata source:
    • --vt-metadata — path to vt-metadata.json or a directory containing it
    • --notus-path — path to a Notus advisory file or directory containing .notus files

Optional inputs:

  • --scap-path — Greenbone/NVD SCAP CVE JSON data for second-stage CVE enrichment
  • --engine {auto,python,rust} — choose the enrichment engine; auto prefers Rust when the binary is available, while python keeps the readable reference implementation available
  • --rust-bin — explicit path to the scan-enrich-results binary when you do not want auto-discovery
  • --output — output file; omit it to print enriched JSON to stdout
openvas-enrich-results \
  --results scan-results.json \
  --vt-metadata /feed/vulnerability-tests/vt-metadata.json \
  --notus-path /var/lib/notus/advisories \
  --scap-path /feed/scap-data \
  --output enriched-results.json

By default the CLI and e2e flow prefer the Rust enrichment binary when it is present. The current Python implementation stays available as an explicit --engine python reference/debug path. For the standalone offline enrichment CLI, the Python wrapper now passes large file-based jobs straight through to the Rust binary instead of loading the enriched payload back into Python memory.

If you want to invoke the Rust binary directly, build it first and call scan-enrich-results:

cargo build --release -p scan-enrichment
./target/release/scan-enrich-results \
  --results scan-results.json \
  --vt-metadata /feed/vulnerability-tests/vt-metadata.json \
  --notus-path /var/lib/notus/advisories \
  --scap-path /feed/scap-data \
  --output enriched-results.json

The Python API is available from scan_examples.enrichment for callers that want to embed the same logic directly:

from scan_examples.enrichment import enrich_results_from_files

enriched = enrich_results_from_files(
    results_path="scan-results.json",
    vt_metadata_path="/feed/vulnerability-tests/vt-metadata.json",
    notus_path="/var/lib/notus/advisories",
    scap_path="/feed/scap-data",
)

Run the end-to-end flow

openvas-example e2e --host target

This command:

  1. Resolves the mounted feed layout and either converts the requested scan config with scannerctl or loads a custom scan config JSON payload
  2. Retries scan creation while openvasd is still warming up
  3. Starts the scan
  4. Polls according to the configured completion mode: quick checks stop after first findings; full checks wait for the scan status to reach succeeded
  5. Fetches results in JSON format
  6. Enriches each result with matching NASL VT metadata from vt-metadata.json and/or matching Notus advisory metadata from .notus files when available (Rust by default when bundled, Python optionally for reference/debugging)
    • the Rust CLI now performs a two-pass, streaming file-based enrichment path so large offline result sets do not need to be materialized in memory all at once
  7. Stops the scan after first findings in quick mode, or lets it finish naturally in full mode
  8. Deletes the scan

While it runs, the CLI now emits step-by-step progress logs to stderr (handy in CI), pretty-prints the enriched findings in the log, and writes final JSON that includes the final scan status, raw results, enriched_results, and a findings_summary block with grouped counts by severity and type.

Each enriched_results entry keeps the original scanner result fields at the top level and adds enrichment fields such as feed-metadata-source, vt-metadata, vt-metadata-status, notus-metadata, notus-metadata-status, cve-ids, cve-metadata, and cve-metadata-status.

Example raw result entry:

[
  {
    "id": 3,
    "type": "alarm",
    "ip_address": "127.0.0.1",
    "hostname": "localhost",
    "oid": "1.3.6.1.4.1.25623.1.0.147696",
    "protocol": "tcp",
    "message": "Installed version: 9.53.3\nFixed version: 9.55\nInstallation\npath / port: /usr/bin/gs"
  }
]

Example entry enriched with VT metadata only:

[
  {
    "id": 3,
    "type": "alarm",
    "ip_address": "127.0.0.1",
    "hostname": "localhost",
    "oid": "1.3.6.1.4.1.25623.1.0.147696",
    "protocol": "tcp",
    "message": "Installed version: 9.53.3\nFixed version:     9.55\nInstallation\npath / port:       /usr/bin/gs",
    "vt-metadata": {
      "oid": "1.3.6.1.4.1.25623.1.0.147696",
      "name": "Ghostscript 9.50 < 9.55.0 Sandbox Escape Vulnerability - Linux",
      "filename": "2022/artifex/gb_ghostscript_sandbox_escape_vuln_sep21_lin.nasl",
      "tag": {
        "affected": "Ghostscript version 9.50 through 9.54.x.",
        "creation_date": 1645676695,
        "cvss_base_vector": "AV:N/AC:M/Au:N/C:C/I:C/A:C",
        "insight": "The file access protection built into Ghostscript proved\n  insufficient for the '%pipe%' PostScript device, when combined with Ghostscript's requirement to\n  be able to create and control temporary files in the conventional temporary file directories (for\n  example, '/tmp' or '/temp').",
        "last_modification": 1646190255,
        "qod_type": "executable_version_unreliable",
        "severity_date": 1646077920,
        "severity_origin": "NVD",
        "severity_vector": "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H",
        "solution": "Update to version 9.55 or later.",
        "solution_type": "VendorFix",
        "summary": "Ghostscript is prone to a sandbox escape vulnerability.",
        "vuldetect": "Checks if a vulnerable version is present on the target host."
      },
      "dependencies": [
        "secpod_ghostscript_detect_lin.nasl"
      ],
      "required_keys": [],
      "mandatory_keys": [
        "artifex/ghostscript/lin/detected"
      ],
      "excluded_keys": [],
      "required_ports": [],
      "required_udp_ports": [],
      "references": [
        {
          "class": "cve",
          "id": "CVE-2021-3781"
        },
        {
          "class": "URL",
          "id": "https://ghostscript.com/blog/CVE-2021-3781.html"
        }
      ],
      "preferences": [],
      "category": "gather_info",
      "family": "General"
    }
  }
]

The bundled target is kirscht/metasploitable3-ub1404 with FTP, SSH, HTTP, SMB, and MySQL enabled. Compose explicitly sets the target password to msfadmin, and the e2e flow includes the matching SSH credential (msfadmin / msfadmin on port 22) in the scan target so authenticated SSH checks can run. By default, scannerctl-based conversion tries to use the scan config's default ports. If the feed does not include the referenced default port-list XML, the example falls back to the bundled metasploitable service ports 21,22,80,139,445,3306 so local runs and CI stay stable. If you want a custom target port set, pass --tcp-ports (or set TARGET_TCP_PORTS); scannerctl conversion will generate an override port list, and custom JSON payloads will have their target ports replaced.

For custom scan configs, pass --scan-config-json or set SCAN_CONFIG_JSON. The payload is treated as a template: the e2e flow replaces target.hosts, injects SSH credentials when configured, and preserves the payload's own ports unless --tcp-ports is provided. The repository includes scanconfigs/scanconfig-modified.json.zip from issue #17, and CI uses it as the default e2e scan config.

The e2e completion behavior is controlled by --completion-mode / E2E_COMPLETION_MODE:

  • first-results (default): quick validation for commits and PRs; stop once the configured minimum number of findings is available. Use --min-results / E2E_MIN_RESULTS to raise that threshold; CI waits for the first 1000 results.
  • scan-complete: full validation for pushes to main; keep polling status and results until the scan finishes successfully.

For long-running CI scans, --no-findings-increment-timeout / E2E_NO_FINDINGS_INCREMENT_TIMEOUT can stop a still-running scan after the finding count has not increased for the configured number of seconds. CI sets this to 1500 seconds (25 minutes) for main push scans, keeping the findings collected so far and avoiding a long tail where OpenVAS keeps running without producing new results. Set it to 0 to disable the idle heuristic.

The GitHub Actions workflow can also be triggered manually. Its inputs let you choose the completion mode, results timeout, and no-findings-increment timeout.

To run a full scan manually from GitHub Actions:

  1. Open the repository's Actions tab.
  2. Select the tests workflow.
  3. Click Run workflow.
  4. Choose the branch to run, usually main.
  5. Set full_scan to true.
  6. Optionally raise results_timeout if you expect the scan to take longer than the default 1800 seconds.
  7. Click Run workflow.

When full_scan is true, the workflow forces scan-complete mode and sets the no-findings idle timeout to 0, so the scan waits until OpenVAS reports natural completion. The workflow uploads scan-results.json and docker-compose.log as the scan-examples-e2e-artifacts artifact.

The enrichment step first uses vt-metadata.json from the mounted NASL vulnerability-test feed. The code checks both <VT_PATH>/vt-metadata.json and <VT_PATH>/nasl/vt-metadata.json. If the file is unavailable, malformed, or shaped unexpectedly, the workflow still returns raw results and marks VT enrichment as unavailable on each enriched result entry instead of faceplanting.

For Notus-backed results, the example can also index the mounted .notus files from <NOTUS_PATH> (for example /var/lib/notus/advisories). In practice, the Notus feed can expose both sparse product/package mappings and richer advisory records for the same OID. The enrichment logic prefers the richer advisory-style data when present and merges in complementary package/fixed- version details from the product-style records.

If --scap-path / SCAP_PATH points at SCAP/NVD CVE JSON data, enrichment then uses CVE references found in the matched VT metadata to attach CVE details such as descriptions, publication timestamps, references, CWE weaknesses, CVSS metrics, and affected CPEs. SCAP enrichment is optional; missing or unreadable SCAP data is logged and each enriched result entry marks CVE metadata as unavailable rather than failing the scan workflow.

One gotcha we hit: openvasd does not actually run scans by itself in this community-container setup. The Compose stack also needs ospd-openvas plus the shared scanner socket volume, otherwise scans get created and then stall with the very helpful classic of “OSPD socket ... does not exist.”

Compose-based test environment

Synchronize the Greenbone feed into persistent named volumes, then start the scanner stack and target:

docker compose up greenbone-feed-sync
docker compose up -d gpg-data redis-server configure-openvas ospd-openvas openvasd target

The greenbone-feed-sync service downloads NASL VTs, Notus data, and GVMD data objects (scan-configs, port-lists, etc.) using rsync. The named volumes (vt_data_vol, notus_data_vol, and data_objects_vol) are reused by later runs, so subsequent synchronizations only fetch deltas instead of starting from empty feed containers.

Run the example container against that stack:

docker compose run --rm example e2e --host target

Use a different scannerctl scan config, custom scan JSON, or explicit ports if you want to override the defaults:

docker compose run --rm \
  -e SCAN_CONFIG=full-and-fast \
  -e TARGET_TCP_PORTS=21,22,80,139,445,3306 \
  -e TARGET_SSH_USERNAME=msfadmin \
  -e TARGET_SSH_PASSWORD=msfadmin \
  example e2e --host target

docker compose run --rm \
  -e SCAN_CONFIG_JSON=/app/scanconfigs/scanconfig-modified.json.zip \
  example e2e --host target

Local development with uv

Create the project environment and install dev dependencies:

uv sync --locked --extra dev

Run the tests or CLI inside the managed environment:

uv run pytest
uv run openvas-example --help
uv run openvas-enrich-results --help

Build the Rust binary locally when you want to test the default Rust-backed path outside the Docker image:

cargo build --release -p scan-enrichment
./target/release/scan-enrich-results --help

Generate synthetic large-result fixtures and compare Python vs Rust enrichment performance:

python3 scripts/generate_synthetic_results.py \
  --output generated/synthetic-scan-results-500k.json \
  --vt-metadata-output generated/synthetic-vt-metadata.json

python3 scripts/benchmark_enrichment.py \
  --results generated/synthetic-scan-results-500k.json \
  --vt-metadata generated/synthetic-vt-metadata.json \
  --runs 3

The benchmark script builds the release Rust binary if needed, runs both engines against the same fixture, verifies semantic output parity by normalized JSON SHA-256 (and also reports byte-for-byte output parity separately), and writes a JSON report to generated/enrichment-benchmark-report.json.

This keeps the project virtualenv in .venv/ and avoids ad-hoc pip install drift.

VS Code devcontainer

A .devcontainer/devcontainer.json is included for VS Code / Dev Containers users. It provisions:

  • Python 3.11
  • Rust stable
  • Docker-outside-of-Docker support for running the local Compose stack
  • uv sync --locked --extra dev on first create

It also preconfigures common Python/Rust extensions and forwards port 3000 for the local openvasd API published by the Compose stack.

GitHub Actions

The repo includes:

  • .github/workflows/lint.yml for uv + Ruff lint/format validation on ubuntu-latest
  • .github/workflows/tests.yml with:
    • unit on ubuntu-latest
    • e2e on the self-hosted runner label scan-examples-e2e
  • .github/workflows/nightly-rust-release.yml for a nightly/manual Rust release build that:
    • installs Python + Rust toolchains
    • runs the Python test suite first
    • builds scan-enrich-results in release mode
    • uploads the Linux x86_64 binary plus a SHA-256 sidecar as a GitHub Actions artifact

That runner is intended to map to the dedicated Hetzner runner named hetzner-vps-scan-examples.

The self-hosted e2e workflow runs greenbone-feed-sync before starting the scanner stack and intentionally keeps the named feed volumes (vt_data_vol, notus_data_vol, data_objects_vol, gpg_data_vol) between runs so Greenbone feed data is updated incrementally instead of re-fetched from scratch every time. Transient scanner state volumes are removed during teardown.

CI is split by validation cost. Unit tests and the pinned mock-backed compatibility test run for normal pull requests. The heavyweight real scan runs in addition to the mock-backed checks for pull requests targeting main, manual workflow_dispatch validation, pushes to main, and release tag pushes such as v1.2.3.

Reference docs

About

Example code for using the OpenVAS scanner container with recommended settings.

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages