feat(telemetry): report unusable scan targets and count them as failure loops - #109
Merged
Merged
Conversation
…re loops A missing path, a broken symlink, or a named file no scanner claims fails the scan with exit 1, but was invisible: it reached no cli_error emit site, was folded into parse_error_count, and called loop_state.record_success, so a scheduled job pointed at a typo'd path reset the stale-loop counter on every run and the nudge could never fire. - The scanner tags each target error with a closed-set reason (MissingTarget / UnsupportedFileType / NotAFileOrDirectory), chosen at the call site and never derived from the path, extension, or message. - Each target error emits one cli_error with that error_type and http_status "none"; fetch-failure events are unchanged. - Target errors feed the loop detector; the nudge names the scan path as the likely cause. - cli_scan gains target_error_count; parse_error_count keeps its meaning so existing history stays comparable. - README Telemetry & Privacy updated in the same change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Since the fix that made an unusable scan target exit 1, a missing path, broken symlink, or unsupported single-file target fails correctly — but silently:
cli_error. A target error has no exception object, so it never reached an emit site. It only showed up inparse_error_count, indistinguishable from a corrupt model.record_success— the exact unattended-CI case the nudge exists for could never trip it.What changed
scanner.py:_record_target_errornow requires a reason from the closed setTARGET_ERROR_TYPES=MissingTarget/UnsupportedFileType/NotAFileOrDirectory, chosen at the call site. Unknown values raise. The path, extension, and message never feed it.cli.py:cli_errorper target error:command,error_type(the reason),http_status: "none",token_present,target_type,consecutive_failures.loop_state(a fetch failure, when present, still owns the fingerprint). The nudge prints after the "Cannot scan" line with a check-the-path hint instead of the auth hint.cli_scangainstarget_error_count.parse_error_countis unchanged so existing reports stay comparable.README.mdTelemetry & Privacy: documentstarget_error_count, the target-errorcli_errorand its fixederror_typevalues, and the widenedloop_state.jsonfingerprint. "What's never collected" remains true verbatim.Privacy
No new class of data:
target_typewas already a bucket, and the only addition is a three-value reason string plus a count. A test plants a distinctive directory name and extension and asserts neither — nor the user-facing message — appears anywhere in the emitted events.AISBOM_NO_TELEMETRY=1suppresses the event (asserted at the_do_postlayer); the local-onlyloop_state.jsonis still written.Verification
poetry run pytest --cov=aisbom --cov-fail-under=85→ 1170 passed, 92.91%poetry run aisbom bypass-scorecard --check→ passedtests/test_target_error_telemetry.py(22 tests): reason per cause (missing, dangling symlink, unsupported, FIFO), closed set under odd extensions, exact payload, no-leak, opt-out, nudge on 3rd run + ordering, clean scan clears the loop, different causes don't share a count,target_error_countvsparse_error_count. Existing fetch-failure telemetry tests pass unchanged.consecutive_failures1→2→3, nudge on run 3,loop_state.jsoncleared by a subsequent clean local scan.