Skip to content

feat(telemetry): report unusable scan targets and count them as failure loops - #109

Merged
lab700xdev merged 1 commit into
mainfrom
slice-126-target-error-telemetry
Sep 13, 2026
Merged

feat(telemetry): report unusable scan targets and count them as failure loops#109
lab700xdev merged 1 commit into
mainfrom
slice-126-target-error-telemetry

Conversation

@lab700xdev

Copy link
Copy Markdown
Contributor

Why

Since the fix that made an unusable scan target exit 1, a missing path, broken symlink, or unsupported single-file target fails correctly — but silently:

  • No cli_error. A target error has no exception object, so it never reached an emit site. It only showed up in parse_error_count, indistinguishable from a corrupt model.
  • It reset the stale-loop counter. The loop detector only counted fetch failures, so every run of a scheduled job pointed at a typo'd path called record_success — the exact unattended-CI case the nudge exists for could never trip it.

What changed

  • scanner.py: _record_target_error now requires a reason from the closed set TARGET_ERROR_TYPES = MissingTarget / UnsupportedFileType / NotAFileOrDirectory, chosen at the call site. Unknown values raise. The path, extension, and message never feed it.
  • cli.py:
    • One cli_error per target error: command, error_type (the reason), http_status: "none", token_present, target_type, consecutive_failures.
    • Target errors feed loop_state (a fetch failure, when present, still owns the fingerprint). The nudge prints after the "Cannot scan" line with a check-the-path hint instead of the auth hint.
    • cli_scan gains target_error_count. parse_error_count is unchanged so existing reports stay comparable.
  • README.md Telemetry & Privacy: documents target_error_count, the target-error cli_error and its fixed error_type values, and the widened loop_state.json fingerprint. "What's never collected" remains true verbatim.

Privacy

No new class of data: target_type was already a bucket, and the only addition is a three-value reason string plus a count. A test plants a distinctive directory name and extension and asserts neither — nor the user-facing message — appears anywhere in the emitted events. AISBOM_NO_TELEMETRY=1 suppresses the event (asserted at the _do_post layer); the local-only loop_state.json is still written.

Verification

  • poetry run pytest --cov=aisbom --cov-fail-under=85 → 1170 passed, 92.91%
  • poetry run aisbom bypass-scorecard --check → passed
  • New tests/test_target_error_telemetry.py (22 tests): reason per cause (missing, dangling symlink, unsupported, FIFO), closed set under odd extensions, exact payload, no-leak, opt-out, nudge on 3rd run + ordering, clean scan clears the loop, different causes don't share a count, target_error_count vs parse_error_count. Existing fetch-failure telemetry tests pass unchanged.
  • Manual run of the real CLI against a missing path ×3 with an isolated state dir: consecutive_failures 1→2→3, nudge on run 3, loop_state.json cleared by a subsequent clean local scan.

…re loops

A missing path, a broken symlink, or a named file no scanner claims fails
the scan with exit 1, but was invisible: it reached no cli_error emit site,
was folded into parse_error_count, and called loop_state.record_success, so
a scheduled job pointed at a typo'd path reset the stale-loop counter on
every run and the nudge could never fire.

- The scanner tags each target error with a closed-set reason
  (MissingTarget / UnsupportedFileType / NotAFileOrDirectory), chosen at
  the call site and never derived from the path, extension, or message.
- Each target error emits one cli_error with that error_type and
  http_status "none"; fetch-failure events are unchanged.
- Target errors feed the loop detector; the nudge names the scan path as
  the likely cause.
- cli_scan gains target_error_count; parse_error_count keeps its meaning
  so existing history stays comparable.
- README Telemetry & Privacy updated in the same change.
@lab700xdev
lab700xdev merged commit cfa8437 into main Sep 13, 2026
2 checks passed
@lab700xdev
lab700xdev deleted the slice-126-target-error-telemetry branch September 13, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant