From c5dbf1e87a45467153ac11e79d653823e294106e Mon Sep 17 00:00:00 2001 From: Chemaclass Date: Sun, 6 Sep 2026 00:22:01 +0200 Subject: [PATCH 1/3] fix(coverage): cover every line of a multi-line statement Bash reports one executed statement to the DEBUG trap on a single line, even when the statement is written over several. #722 handled the backslash chain; every other way a statement spans lines was left out. An array literal written one element per line therefore cost one uncovered line per element -- and on Bash 3.2, which attributes the assignment to its closing `)`, that line is non-executable, so the hit was discarded and the whole array read as uncovered. Multi-line strings and heredoc bodies had the same gap. Adds a small shell lexer, a Bash reference in coverage/lines.sh mirrored in awk, that groups lines into statement spans: a stack of open contexts (single quote, double quote, array literal, command substitution, plain paren) plus a pending heredoc delimiter. Propagation becomes span-max rather than a forward carry, so it works whichever end of the span the trap attributed the statement to. For a backslash chain it produces exactly what the forward carry did. A multi-line `$( )` is deliberately not a span: its interior lines are commands tracked in their own right, and crediting them from the line that opened the substitution would report lines that never ran. `bu_propagate` replaces five copies of the same loop across the LCOV, stats and HTML passes. The differential now compares the scanner as well as the classifier over every `git ls-files '*.sh'`, and asserts each file lexes to a clean end state -- real shell files balance their quotes, so a leftover context is the lexer misreading real code. Related #1338 Claude-Session: https://claude.ai/code/session_01MeysZ63ewZiFTgCDs172XJ --- CHANGELOG.md | 3 + docs/coverage.md | 35 +++ src/coverage/html_file.sh | 16 +- src/coverage/lines.sh | 246 +++++++++++++++++- src/coverage/rules_awk.sh | 168 ++++++++++-- .../bashunit_coverage_multiline_test.sh | 136 ++++++++++ .../coverage/classifier_differential_test.sh | 89 +++++-- tests/unit/coverage/spans_test.sh | 192 ++++++++++++++ 8 files changed, 824 insertions(+), 61 deletions(-) create mode 100644 tests/acceptance/bashunit_coverage_multiline_test.sh create mode 100644 tests/unit/coverage/spans_test.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index c562e8dd..2c15a1ea 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,9 @@ ## Unreleased +### Fixed +- Coverage counts every line of a multi-line statement as covered once the statement has run, not only the line Bash reported it on. An array literal written one element per line used to cost one uncovered line per element — and on Bash 3.2, where the assignment is reported on its closing `)`, the hit was discarded outright and the whole array read as uncovered. Multi-line strings and heredoc bodies had the same gap; a multi-line `$( )` is left alone, since its interior lines are commands that are tracked in their own right (#1338) + ## [0.50.1](https://github.com/TypedDevs/bashunit/compare/0.50.0...0.50.1) - 2026-08-22 ### Fixed diff --git a/docs/coverage.md b/docs/coverage.md index c600b1ed..a7994c0f 100644 --- a/docs/coverage.md +++ b/docs/coverage.md @@ -426,6 +426,41 @@ These lines are not counted toward coverage: - Control flow keywords (`then`, `else`, `fi`, `do`, `done`, `esac`, `in`) - Case statement patterns (`--option)`, `*)`) and terminators (`;;`, `;&`, `;;&`) +### Statements That Span Several Lines + +Bash reports one executed statement to the tracer on a single line, even when +the statement is written over several. bashunit spreads that hit across every +line the statement occupies, so a statement that ran is covered on all of them: + +- backslash continuations (`printf '%s' \` … ) +- array literals (`commands=(` … `)`) +- multi-line quoted strings +- heredoc bodies (`cat < 0 && h < carry) { h = carry; hits[ln] = h } - if (h > 0 && bu_ends_with_continuation(sl[ln])) { carry = h } else { carry = 0 } - } + bu_propagate(sl, hits, total) for (ln = 1; ln <= total; ln++) { row_class = "" @@ -113,12 +106,7 @@ FILENAME == hitsfile { } END { - carry = 0 - for (ln = 1; ln <= total; ln++) { - h = (ln in hits) ? hits[ln] : 0 - if (carry > 0 && h < carry) { h = carry; hits[ln] = h } - if (h > 0 && bu_ends_with_continuation(sl[ln])) { carry = h } else { carry = 0 } - } + bu_propagate(sl, hits, total) bu_fn_reset() for (ln = 1; ln <= total; ln++) { bu_fn_line(sl[ln], ln) } diff --git a/src/coverage/lines.sh b/src/coverage/lines.sh index 801fc940..5532d35d 100644 --- a/src/coverage/lines.sh +++ b/src/coverage/lines.sh @@ -217,6 +217,224 @@ function bashunit::coverage::_ends_with_continuation() { [ $((${#trailing} % 2)) -eq 1 ] } +# The multi-line statement scanner (#1338). +# +# A backslash chain is not the only way one statement covers several physical +# lines, and the DEBUG trap reports the whole statement on ONE of them -- which +# one depends on the Bash version. For `local commands=(` .. `)`, Bash 3.2 +# reports the closing `)` (a line the classifier calls non-executable, so the +# hit was discarded entirely) and Bash 5.x reports the opening line. Either way +# the rest of the statement read as uncovered. +# +# So the reader needs to know which lines belong to one statement. This is a +# small shell lexer: it consumes a line at a time and carries its state forward, +# the way the classifier consumes a line at a time without one. +# +# The state is a stack of open contexts, innermost last, one character each: +# +# S a single-quoted string D a double-quoted string +# A an array literal `name=(` C a command substitution `$(` +# P any other parenthesis +# +# plus a pending heredoc delimiter. A line is *open* -- the statement continues +# onto the next line -- when the stack holds an S, D or A, when a heredoc is +# pending, or when it ends with a backslash continuation. +# +# C and P deliberately do NOT open a span. The lines inside a multi-line `$( )` +# are real commands that get their own DEBUG hits, so crediting them from the +# line that opened the substitution would report lines that never ran. They are +# still tracked, because their parentheses have to balance for the ones that do. +_BASHUNIT_COVERAGE_SCAN_STACK="" +_BASHUNIT_COVERAGE_SCAN_HEREDOC="" +_BASHUNIT_COVERAGE_SCAN_HEREDOC_TAB=0 +_BASHUNIT_COVERAGE_SCAN_CONTINUES=0 + +## +# Clears the scanner state. Call once before the first line of a file. +## +function bashunit::coverage::scan_reset() { + _BASHUNIT_COVERAGE_SCAN_STACK="" + _BASHUNIT_COVERAGE_SCAN_HEREDOC="" + _BASHUNIT_COVERAGE_SCAN_HEREDOC_TAB=0 + _BASHUNIT_COVERAGE_SCAN_CONTINUES=0 +} + +# Records the delimiter of a heredoc from the text that follows `<<`. The +# quoting of a delimiter only decides whether the body expands, so it is +# stripped: `<<'EOF'`, `<<"EOF"`, `<<\EOF` and `<)].*$/, "", word) + gsub(/\\/, "", word) + gsub(/"/, "", word) + gsub(_bu_sq, "", word) + _bu_hd = word +} + +function bu_scan_line(line, i, n, c, prev, top, body) { + _bu_cont = 0 + + if (_bu_hd != "") { + body = line + if (_bu_hdtab) { sub(/^\t+/, "", body) } + if (body == _bu_hd) { _bu_hd = "" } + return + } + + # Same early-out as the reference: with nothing open, a line holding none of + # these characters cannot change the state. index() is a C-speed pass where + # the walk below is an interpreted one. + if (_bu_sp == 0 && index(line, _bu_sq) == 0 && index(line, "\"") == 0 && + index(line, "\\") == 0 && index(line, "(") == 0 && index(line, "<") == 0) { + return + } + + if (bu_ends_with_continuation(line)) { _bu_cont = 1 } + + n = length(line) + prev = "" + for (i = 1; i <= n; i++) { + c = substr(line, i, 1) + top = (_bu_sp > 0) ? _bu_st[_bu_sp] : "" + + if (top == "S") { + if (c == _bu_sq) { _bu_sp-- } + prev = c + continue + } + + if (top == "D") { + if (c == "\"") { _bu_sp-- } + else if (c == "\\") { i++; prev = substr(line, i, 1); continue } + else if (c == "$" && substr(line, i + 1, 1) == "(") { + _bu_sp++; _bu_st[_bu_sp] = "C"; i++; prev = "("; continue + } + prev = c + continue + } + + if (c == "\\") { i++; prev = substr(line, i, 1); continue } + if (c == _bu_sq) { _bu_sp++; _bu_st[_bu_sp] = "S"; prev = c; continue } + if (c == "\"") { _bu_sp++; _bu_st[_bu_sp] = "D"; prev = c; continue } + if (c == "#") { + if (prev == "" || prev == " " || prev == "\t" || + prev == ";" || prev == "&" || prev == "|") { return } + prev = c + continue + } + if (c == "(") { + _bu_sp++ + _bu_st[_bu_sp] = (prev == "$") ? "C" : ((prev == "=") ? "A" : "P") + prev = c + continue + } + if (c == ")") { + if (_bu_sp > 0) { _bu_sp-- } + prev = c + continue + } + if (c == "<" && substr(line, i + 1, 1) == "<") { + if (substr(line, i + 2, 1) == "<") { i += 2; prev = "<"; continue } + bu_scan_heredoc(substr(line, i + 2)) + i++ + prev = "<" + continue + } + prev = c + } +} + +function bu_scan_open( k) { + if (_bu_cont) { return 1 } + if (_bu_hd != "") { return 1 } + for (k = 1; k <= _bu_sp; k++) { + if (_bu_st[k] == "S" || _bu_st[k] == "D" || _bu_st[k] == "A") { return 1 } + } + return 0 +} + +# The open contexts, innermost last. Only the differential reads it: a real +# shell file ends with nothing open, so a non-empty state at EOF is a lexer bug +# and this says which context leaked. +function bu_scan_stack( k, s) { + s = "" + for (k = 1; k <= _bu_sp; k++) { s = s _bu_st[k] } + return s +} + +# Gives every line of a multi-line statement the highest count recorded +# anywhere in it. The DEBUG trap reports the statement on one line of the span +# and which one depends on the Bash version, so the propagation runs in both +# directions (#722, #1338). Mirrors the loop in get_all_line_hits. +function bu_propagate(sl, hits, total, ln, start, max, fill, h) { + bu_scan_reset() + start = 1 + max = 0 + for (ln = 1; ln <= total; ln++) { + h = (ln in hits) ? hits[ln] + 0 : 0 + if (h > max) { max = h } + bu_scan_line(sl[ln]) + if (bu_scan_open()) { continue } + if (max > 0 && start < ln) { + for (fill = start; fill <= ln; fill++) { hits[fill] = max } + } + start = ln + 1 + max = 0 + } +} ' # The DA/LF/LH block of one file's LCOV record, in one pass. @@ -131,12 +274,7 @@ FILENAME == hitsfile { } END { - carry = 0 - for (ln = 1; ln <= total; ln++) { - h = (ln in hits) ? hits[ln] + 0 : 0 - if (carry > 0 && h < carry) { h = carry; hits[ln] = h } - if (h > 0 && bu_ends_with_continuation(src[ln])) { carry = h } else { carry = 0 } - } + bu_propagate(src, hits, total) executable = 0 hit = 0 @@ -187,14 +325,7 @@ _BASHUNIT_COVERAGE_AWK_STATS=' } close(src) - # The DEBUG trap attributes a multi-line statement to its starting line, so - # the count carries forward across the backslash chain (#722). - carry = 0 - for (ln = 1; ln <= total; ln++) { - h = (ln in hits) ? hits[ln] : 0 - if (carry > 0 && h < carry) { h = carry; hits[ln] = h } - if (h > 0 && bu_ends_with_continuation(sl[ln])) { carry = h } else { carry = 0 } - } + bu_propagate(sl, hits, total) executable = 0 hit = 0 @@ -254,14 +385,7 @@ BEGIN { print "TN:" } } close(src) - # The DEBUG trap attributes a multi-line statement to its starting line, so - # the count carries forward across the backslash chain (#722). - carry = 0 - for (ln = 1; ln <= total; ln++) { - h = (ln in hits) ? hits[ln] : 0 - if (carry > 0 && h < carry) { h = carry; hits[ln] = h } - if (h > 0 && bu_ends_with_continuation(sl[ln])) { carry = h } else { carry = 0 } - } + bu_propagate(sl, hits, total) bu_fn_reset() bu_br_reset() diff --git a/tests/acceptance/bashunit_coverage_multiline_test.sh b/tests/acceptance/bashunit_coverage_multiline_test.sh new file mode 100644 index 00000000..8399055a --- /dev/null +++ b/tests/acceptance/bashunit_coverage_multiline_test.sh @@ -0,0 +1,136 @@ +#!/usr/bin/env bash +set -euo pipefail + +# A statement that ran counts as covered on every line it spans, whichever way +# it spans them. #722 established that for a backslash chain; an array literal, +# a multi-line string and a heredoc are the same statement over several lines +# and used to cost one uncovered line each (#1338). +# +# End to end rather than at the reader, because the defect depended on where the +# DEBUG trap put the hit: Bash 3.2 reports an array assignment on its closing +# `)` -- a line the classifier calls non-executable, so the hit was dropped +# outright -- and Bash 5.x reports its opening line. Only a real run exercises +# whichever of the two this machine does. + +function set_up_before_script() { + ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +} + +# A source file whose every function spans lines a different way, and a test +# that runs all of them. +function _project() { # $1 = dir + mkdir -p "$1/src" + cat >"$1/src/multiline.sh" <<'SRC' +#!/usr/bin/env bash + +function multi_line_array() { + local commands=( + "start" + "stop" + "status" + ) + + printf '%s\n' "${commands[@]}" +} + +function backslash_continuation() { + printf '%s\n' \ + "start" \ + "stop" +} + +function multi_line_string() { + printf '%s' '{ + "index": "spend", + "alias": "filters" + }' +} + +function here_document() { + cat <"$1/t_test.sh" <<'TEST' +#!/usr/bin/env bash + +source "$(dirname "${BASH_SOURCE[0]}")/src/multiline.sh" + +function test_multi_line_array() { + assert_contains "start" "$(multi_line_array)" +} + +function test_backslash_continuation() { + assert_contains "start" "$(backslash_continuation)" +} + +function test_multi_line_string() { + assert_contains "spend" "$(multi_line_string)" +} + +function test_here_document() { + assert_contains "one" "$(here_document)" +} +TEST +} + +function _run_coverage() { # $1 = dir + local dir="$1" + shift + (cd "$dir" && BASHUNIT_COVERAGE_SHOW_UNCOVERED=true \ + "$ROOT_DIR/bashunit" --no-parallel --coverage --coverage-paths src \ + --no-coverage-report "$@" t_test.sh 2>&1) || true +} + +function test_every_line_of_a_multi_line_statement_that_ran_is_covered() { + local dir + dir="$(bashunit::temp_dir)" + _project "$dir" + + local output + output="$(_run_coverage "$dir" | strip_ansi)" + + assert_contains "16/ 16 lines (100%)" "$output" + assert_not_contains "Uncovered Lines" "$output" +} + +# The same numbers have to come out of the LCOV writer, which reaches them +# through the batch awk pass rather than the Bash reader. +function test_the_lcov_report_agrees_with_the_terminal_report() { + local dir + dir="$(bashunit::temp_dir)" + _project "$dir" + + (cd "$dir" && "$ROOT_DIR/bashunit" --no-parallel --coverage \ + --coverage-paths src --coverage-report lcov.info \ + t_test.sh >/dev/null 2>&1) || true + + assert_file_contains "$dir/lcov.info" "LF:16" + assert_file_contains "$dir/lcov.info" "LH:16" +} + +# A span nothing ran still counts against the file: the fix credits statements +# that executed, it does not remove lines from the denominator. +function test_a_multi_line_statement_that_never_ran_stays_uncovered() { + local dir + dir="$(bashunit::temp_dir)" + _project "$dir" + cat >>"$dir/src/multiline.sh" <<'SRC' + +function never_called() { + local unused=( + "a" + "b" + ) + printf '%s\n' "${unused[@]}" +} +SRC + + local output + output="$(_run_coverage "$dir" | strip_ansi)" + + assert_contains "16/ 20 lines" "$output" + assert_matches "src/multiline.sh:[0-9]+-[0-9]+" "$output" +} diff --git a/tests/unit/coverage/classifier_differential_test.sh b/tests/unit/coverage/classifier_differential_test.sh index d59b5b5d..b82d4d07 100644 --- a/tests/unit/coverage/classifier_differential_test.sh +++ b/tests/unit/coverage/classifier_differential_test.sh @@ -1,21 +1,32 @@ #!/usr/bin/env bash -# The awk classifier and the Bash one must agree on every line of every shell -# file in the repo. A disagreement moves coverage numbers silently, which is -# exactly what #1005 warned about when it reproduced the old regex quirk for -# quirk -- so this compares them line by line rather than trusting either. +# The awk rules and the Bash ones must agree on every line of every shell file +# in the repo. A disagreement moves coverage numbers silently, which is exactly +# what #1005 warned about when it reproduced the old regex quirk for quirk -- +# so this compares them line by line rather than trusting either. +# +# Two rule sets share the walk, because both are per-line and both have a Bash +# reference with an awk mirror: whether a line is executable (#1005) and whether +# the statement on it continues onto the next one (#722, #1338). The scanner +# also carries state between lines, so the file's end state is compared too -- +# and asserted clean, since real shell files balance their quotes. function set_up_before_script() { ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" + LF=" +" } -# Classifies every line of $1 with awk, printing " <0|1>". +# Reports every line of $1 with awk as " ", then a +# final "end " holding the scanner state the file left behind. # $2 overrides the rule source, which is how the mutation below is injected. function awk_classification() { # $1 = file, $2 = optional rule source local rules="${2:-$(bashunit::coverage::awk_rules)}" local out status=0 out=$(env LC_ALL=C "$AWK" "$rules"' - { printf "%s %s\n", FNR, bu_is_executable($0) } + BEGIN { bu_scan_reset() } + { bu_scan_line($0); printf "%s %s %s\n", FNR, bu_is_executable($0), bu_scan_open() } + END { printf "end %s %s\n", bu_scan_stack(), _bu_hd } ' "$1") || status=$? # An awk that failed prints nothing, and empty output is indistinguishable @@ -35,17 +46,21 @@ function awk_classification() { # $1 = file, $2 = optional rule source fi } -# Classifies every line of $1 with the Bash reference, same format. +# Reports every line of $1 with the Bash reference, same format. function bash_classification() { # $1 = file - local lineno=0 line + local lineno=0 line executable open + bashunit::coverage::scan_reset while IFS= read -r line || [ -n "$line" ]; do lineno=$((lineno + 1)) - if bashunit::coverage::is_executable_line "$line" "$lineno"; then - printf '%s 1\n' "$lineno" - else - printf '%s 0\n' "$lineno" - fi + executable=0 + bashunit::coverage::is_executable_line "$line" "$lineno" && executable=1 + bashunit::coverage::scan_line "$line" + open=0 + bashunit::coverage::scan_is_open && open=1 + printf '%s %s %s\n' "$lineno" "$executable" "$open" done <"$1" + printf 'end %s %s\n' \ + "$_BASHUNIT_COVERAGE_SCAN_STACK" "$_BASHUNIT_COVERAGE_SCAN_HEREDOC" } # Renders the difference between two strings, for the report of a file that @@ -62,7 +77,7 @@ function diff_of() { # $1 = bash side, $2 = awk side } -function test_both_classifiers_agree_on_every_shell_file_in_the_repo() { +function test_both_rule_sets_agree_on_every_shell_file_in_the_repo() { # 460 files, each an awk fork plus a Bash loop over its lines: 4.8s here, # but minutes under Git Bash, where the shard hung until CI cancelled it. # GNU awk (Ubuntu) and BusyBox awk (Alpine) both run this, which is what the @@ -70,6 +85,10 @@ function test_both_classifiers_agree_on_every_shell_file_in_the_repo() { bashunit::skip_on windows "460 awk forks per run takes minutes under Git Bash" local disagreements="" + # A shell file that parses has every quote, parenthesis and heredoc closed by + # the time it ends, so a leftover context is the scanner mis-reading real code + # -- the check that the state machine is right, not merely mirrored (#1338). + local unclean="" local file tmp_a tmp_b tmp_a=$(bashunit::temp_file cls_a) tmp_b=$(bashunit::temp_file cls_b) @@ -93,9 +112,16 @@ function test_both_classifiers_agree_on_every_shell_file_in_the_repo() { $file $diff_out" fi + + local end_state="${bash_out##*"$LF"}" + if [ "$end_state" != "end " ]; then + unclean="$unclean +$file left $end_state" + fi done assert_empty "$disagreements" + assert_empty "$unclean" } # The differential is only worth anything if it can fail. The mutation removes @@ -121,7 +147,7 @@ function test_the_differential_catches_a_broken_awk_rule() { assert_not_equals "$reference" "$mutated" } -function test_the_classifiers_agree_on_the_quirk_cases() { +function test_the_rule_sets_agree_on_the_quirk_cases() { local fixture fixture="$(bashunit::temp_file)" { @@ -137,11 +163,44 @@ function test_the_classifiers_agree_on_the_quirk_cases() { printf '%s\n' 'function bashunit::x() {' printf '%s\n' 'name() {' printf '%s\n' '((i++))' + # The scanner's own quirks: an array literal spans, a substitution does not, + # and a quote is only a quote where the shell reads one (#1338). + printf '%s\n' 'arr=(' + printf '%s\n' ' "one"' + printf '%s\n' ')' + printf '%s\n' "s='multi" + printf '%s\n' "line'" + printf '%s\n' "echo hi # don't" + printf '%s\n' 'y="$(f '"'"'a"b'"'"')"' + printf '%s\n' 'cat <<-EOF' + printf '\t%s\n' 'body' + printf '\t%s\n' 'EOF' + printf '%s\n' 'read -r v <<<"here"' } >"$fixture" assert_same "$(bash_classification "$fixture")" "$(awk_classification "$fixture")" } +# The scanner half of the differential needs its own mutation guard: a rule +# removed from the awk copy has to surface as a disagreement, not as an awk +# that refuses to run. +function test_the_differential_catches_a_broken_awk_scanner_rule() { + local fixture + fixture="$(bashunit::temp_file)" + printf '%s\n' 'arr=(' ' "one"' ')' >"$fixture" + + local mutated_rules + mutated_rules=$(bashunit::coverage::awk_rules | + sed 's|(prev == "=") ? "A" : "P"|"P"|') + + local mutated reference + mutated=$(awk_classification "$fixture" "$mutated_rules") + reference=$(bash_classification "$fixture") + + assert_not_empty "$mutated" + assert_not_equals "$reference" "$mutated" +} + # The differential compares two outputs, so anything that empties one of them # reads as a total disagreement. A transient awk failure under CI load did # exactly that, reporting "the classifiers disagree on every shell file" when diff --git a/tests/unit/coverage/spans_test.sh b/tests/unit/coverage/spans_test.sh new file mode 100644 index 00000000..33e6661f --- /dev/null +++ b/tests/unit/coverage/spans_test.sh @@ -0,0 +1,192 @@ +#!/usr/bin/env bash + +# shellcheck disable=SC1003 # intentional literal trailing backslashes in test inputs + +# Multi-line statement spans (#1338). +# +# #722 taught the report that a backslash chain is one statement. A statement +# that spans lines because a quote, an array literal or a heredoc is still open +# is the same shape, and the DEBUG trap attributes it to one line of the span -- +# which line depends on the Bash version: 3.2 reports an array assignment on its +# closing `)`, 5.x on its opening line. So the propagation has to cover the whole +# span from a hit anywhere inside it. + +function set_up() { + WORK="$(bashunit::temp_dir)/spans" + mkdir -p "$WORK" + _BASHUNIT_COVERAGE_DATA_FILE="$WORK/coverage.data" + : >"$_BASHUNIT_COVERAGE_DATA_FILE" + bashunit::coverage::invalidate_hits_aggregation +} + +function tear_down() { + _BASHUNIT_COVERAGE_DATA_FILE="" + bashunit::coverage::invalidate_hits_aggregation +} + +# Records one hit per "" argument and returns the propagated hit list. +function hits_for() { # $1 = source file, $@ = line numbers to record + local src="$1" + shift + local ln + for ln in "$@"; do + echo "${src}:${ln}" >>"$_BASHUNIT_COVERAGE_DATA_FILE" + done + bashunit::coverage::invalidate_hits_aggregation + bashunit::coverage::get_all_line_hits "$src" +} + +# --- the scanner ------------------------------------------------------------- + +# Runs the scanner over the given lines and reports the open flag of each, so a +# state machine bug points at the line that broke it. +function open_flags() { # $@ = source lines + local line out="" + bashunit::coverage::scan_reset + for line in "$@"; do + bashunit::coverage::scan_line "$line" + if bashunit::coverage::scan_is_open; then + out="${out}1" + else + out="${out}0" + fi + done + printf '%s' "$out" +} + +function test_an_array_literal_stays_open_until_its_closing_paren() { + assert_same "1110" "$(open_flags 'local commands=(' ' "start"' ' "stop"' ')')" +} + +function test_a_single_line_array_literal_opens_nothing() { + assert_same "0" "$(open_flags 'local commands=("start" "stop")')" +} + +function test_a_command_substitution_is_not_a_span() { + assert_same "00" "$(open_flags 'x=$(' ')')" +} + +function test_a_subshell_is_not_a_span() { + assert_same "00" "$(open_flags '(' ')')" +} + +function test_a_process_substitution_is_not_a_span() { + assert_same "0" "$(open_flags 'while read -r l; do :; done < <(printf x)')" +} + +function test_a_multi_line_single_quoted_string_stays_open() { + assert_same "110" "$(open_flags "x='{" ' "a": 1' "}'")" +} + +function test_a_multi_line_double_quoted_string_stays_open() { + assert_same "10" "$(open_flags 'x="one' 'two"')" +} + +function test_a_quote_inside_a_comment_opens_nothing() { + assert_same "0" "$(open_flags "# it's a comment")" +} + +function test_a_trailing_comment_does_not_swallow_a_quote() { + assert_same "0" "$(open_flags "echo hi # don't")" +} + +function test_a_hash_in_a_parameter_expansion_is_not_a_comment() { + assert_same "0" "$(open_flags 'echo "${#arr[@]}" "${x#pre}"')" +} + +function test_a_quote_inside_a_command_substitution_inside_a_string() { + # "$(f 'a"b')" -- the inner double quote belongs to the single-quoted word of + # the substituted command, not to the outer string. + assert_same "0" "$(open_flags 'y="$(f '"'"'a"b'"'"')"')" +} + +function test_an_escaped_quote_does_not_open_a_string() { + assert_same "0" "$(open_flags 'echo \" done')" +} + +function test_a_heredoc_body_stays_open_until_its_terminator() { + assert_same "1110" "$(open_flags 'cat <"$src" + + assert_same "$(printf '%s\n' '1:1' '2:1' '3:1' '4:1')" "$(hits_for "$src" 4)" +} + +function test_an_array_hit_on_the_opening_line_covers_the_whole_span() { + # What Bash 5.x records: the assignment is attributed to its first line. + local src="$WORK/array_open.sh" + printf '%s\n' 'local commands=(' ' "start"' ' "stop"' ')' 'echo done' >"$src" + + assert_same "$(printf '%s\n' '1:1' '2:1' '3:1' '4:1')" "$(hits_for "$src" 1)" +} + +function test_a_multi_line_string_hit_covers_its_interior() { + local src="$WORK/string.sh" + printf '%s\n' "printf '%s' '{" ' "index": "spend"' "}'" 'echo done' >"$src" + + assert_same "$(printf '%s\n' '1:1' '2:1' '3:1')" "$(hits_for "$src" 1)" +} + +function test_a_heredoc_hit_covers_its_body() { + local src="$WORK/heredoc.sh" + printf '%s\n' 'cat <"$src" + + assert_same "$(printf '%s\n' '1:1' '2:1' '3:1' '4:1')" "$(hits_for "$src" 1)" +} + +function test_a_multi_line_command_substitution_is_left_alone() { + # Its interior lines are real commands: they get their own DEBUG hits, so + # crediting them from the opening line would report lines that never ran. + local src="$WORK/cmdsub.sh" + printf '%s\n' 'x=$(' ' compute_a' ' compute_b' ')' >"$src" + + assert_same "1:1" "$(hits_for "$src" 1)" +} + +function test_a_span_that_never_ran_stays_uncovered() { + local src="$WORK/cold.sh" + printf '%s\n' 'local commands=(' ' "start"' ')' 'echo done' >"$src" + + assert_same "4:1" "$(hits_for "$src" 4)" +} + +function test_the_highest_count_in_a_span_wins() { + local src="$WORK/counts.sh" + printf '%s\n' 'local commands=(' ' "start"' ')' >"$src" + + assert_same "$(printf '%s\n' '1:3' '2:3' '3:3')" "$(hits_for "$src" 3 3 3)" +} + +# #722 stays exactly as it was: a chain propagates, an unrelated line does not. +function test_a_backslash_chain_still_propagates() { + local src="$WORK/chain.sh" + printf '%s\n' 'echo start \' ' middle \' ' end' 'echo other' >"$src" + + assert_same "$(printf '%s\n' '1:2' '2:2' '3:2' '4:1')" "$(hits_for "$src" 1 1 4)" +} From 625982641d3b6072021ea2ca382dc055ab7ec3d7 Mon Sep 17 00:00:00 2001 From: Chemaclass Date: Sun, 6 Sep 2026 08:31:15 +0200 Subject: [PATCH 2/3] refactor(coverage): tighten the multi-line statement scanner No behaviour change; the differential and the span tests pin that. The escape rule was written twice, once in the double-quoted branch and once in the unquoted one. A backslash escapes the next character in both, and it can never reach either from inside `'..'` -- that context reports nothing but its closing quote, and its branch returns first. So the rule is hoisted between the two, stated once. The stack now lives in a local for the length of the walk and is written back once at the end. Eight statements were longer than the work they did because the global's name is 30 characters. The comment case leaves by `break` rather than `return` so there is a single write-back point. Claude-Session: https://claude.ai/code/session_01MeysZ63ewZiFTgCDs172XJ --- src/coverage/lines.sh | 50 ++++++++++++++++++++++----------------- src/coverage/rules_awk.sh | 7 ++++-- 2 files changed, 33 insertions(+), 24 deletions(-) diff --git a/src/coverage/lines.sh b/src/coverage/lines.sh index 5532d35d..74d67cff 100644 --- a/src/coverage/lines.sh +++ b/src/coverage/lines.sh @@ -322,9 +322,12 @@ function bashunit::coverage::scan_line() { _BASHUNIT_COVERAGE_SCAN_CONTINUES=1 fi + # A local copy, written back once: the global's name is longer than most of + # the statements that touch it. + local stack="$_BASHUNIT_COVERAGE_SCAN_STACK" local rest="$line" prev="" head tail char top while [ -n "$rest" ]; do - top="${_BASHUNIT_COVERAGE_SCAN_STACK#"${_BASHUNIT_COVERAGE_SCAN_STACK%?}"}" + top="${stack#"${stack%?}"}" case "$top" in 'S') head="${rest%%[\']*}" ;; 'D') head="${rest%%[\"\\$]*}" ;; @@ -340,26 +343,32 @@ function bashunit::coverage::scan_line() { char="${tail%"${tail#?}"}" rest="${tail#?}" + # Nothing but the closing quote is reported inside `'..'`. if [ "$top" = 'S' ]; then - _BASHUNIT_COVERAGE_SCAN_STACK="${_BASHUNIT_COVERAGE_SCAN_STACK%?}" + stack="${stack%?}" prev="$char" continue fi + # A backslash escapes the next character in both remaining contexts. Inside + # `'..'` it is literal, and the branch above has already taken that case. + case "$char" in + [\\]) + prev="${rest%"${rest#?}"}" + rest="${rest#?}" + continue + ;; + esac + if [ "$top" = 'D' ]; then case "$char" in - '"') _BASHUNIT_COVERAGE_SCAN_STACK="${_BASHUNIT_COVERAGE_SCAN_STACK%?}" ;; - [\\]) - prev="${rest%"${rest#?}"}" - rest="${rest#?}" - continue - ;; + '"') stack="${stack%?}" ;; '$') # `"$(cmd 'a"b')"`: a command substitution reopens an unquoted context, # so the quotes inside it are not the outer string's. case "$rest" in '('*) - _BASHUNIT_COVERAGE_SCAN_STACK="${_BASHUNIT_COVERAGE_SCAN_STACK}C" + stack="${stack}C" rest="${rest#?}" prev='(' continue @@ -372,32 +381,27 @@ function bashunit::coverage::scan_line() { fi case "$char" in - [\\]) - prev="${rest%"${rest#?}"}" - rest="${rest#?}" - continue - ;; - "'") _BASHUNIT_COVERAGE_SCAN_STACK="${_BASHUNIT_COVERAGE_SCAN_STACK}S" ;; - '"') _BASHUNIT_COVERAGE_SCAN_STACK="${_BASHUNIT_COVERAGE_SCAN_STACK}D" ;; + "'") stack="${stack}S" ;; + '"') stack="${stack}D" ;; '#') # `#` only opens a comment at the start of a word, so `${x#y}` and - # `${#arr[@]}` are not comments. + # `${#arr[@]}` are not comments. The rest of the line is not shell text. case "$prev" in - '' | ' ' | ' ' | ';' | '&' | '|') return 0 ;; + '' | ' ' | ' ' | ';' | '&' | '|') break ;; esac ;; '(') # An array literal is the one parenthesis whose contents are words of a # single statement, and it is the one that opens right after a `=`. case "$prev" in - '$') _BASHUNIT_COVERAGE_SCAN_STACK="${_BASHUNIT_COVERAGE_SCAN_STACK}C" ;; - '=') _BASHUNIT_COVERAGE_SCAN_STACK="${_BASHUNIT_COVERAGE_SCAN_STACK}A" ;; - *) _BASHUNIT_COVERAGE_SCAN_STACK="${_BASHUNIT_COVERAGE_SCAN_STACK}P" ;; + '$') stack="${stack}C" ;; + '=') stack="${stack}A" ;; + *) stack="${stack}P" ;; esac ;; ')') # A case arm's `)` closes nothing, and popping an empty stack is a no-op. - _BASHUNIT_COVERAGE_SCAN_STACK="${_BASHUNIT_COVERAGE_SCAN_STACK%?}" + stack="${stack%?}" ;; '<') case "$rest" in @@ -419,6 +423,8 @@ function bashunit::coverage::scan_line() { prev="$char" done + _BASHUNIT_COVERAGE_SCAN_STACK="$stack" + return 0 } diff --git a/src/coverage/rules_awk.sh b/src/coverage/rules_awk.sh index 1ba46fcb..3fd73c76 100644 --- a/src/coverage/rules_awk.sh +++ b/src/coverage/rules_awk.sh @@ -164,15 +164,19 @@ function bu_scan_line(line, i, n, c, prev, top, body) { c = substr(line, i, 1) top = (_bu_sp > 0) ? _bu_st[_bu_sp] : "" + # Nothing but the closing quote is reported inside a single-quoted string. if (top == "S") { if (c == _bu_sq) { _bu_sp-- } prev = c continue } + # A backslash escapes the next character in both remaining contexts; inside + # a single-quoted string it is literal, and the branch above took that case. + if (c == "\\") { i++; prev = substr(line, i, 1); continue } + if (top == "D") { if (c == "\"") { _bu_sp-- } - else if (c == "\\") { i++; prev = substr(line, i, 1); continue } else if (c == "$" && substr(line, i + 1, 1) == "(") { _bu_sp++; _bu_st[_bu_sp] = "C"; i++; prev = "("; continue } @@ -180,7 +184,6 @@ function bu_scan_line(line, i, n, c, prev, top, body) { continue } - if (c == "\\") { i++; prev = substr(line, i, 1); continue } if (c == _bu_sq) { _bu_sp++; _bu_st[_bu_sp] = "S"; prev = c; continue } if (c == "\"") { _bu_sp++; _bu_st[_bu_sp] = "D"; prev = c; continue } if (c == "#") { From 446af25917895f033729fa66a1c7f95318410567 Mon Sep 17 00:00:00 2001 From: Chemaclass Date: Sun, 6 Sep 2026 08:31:15 +0200 Subject: [PATCH 3/3] chore(lint): stop pinning a charset on the acceptance snapshots editorconfig-checker 4.0.0 reads the ANSI ESC byte in four recorded snapshots as Latin-1 and fails them against the global `charset = utf-8`. All four are pure ASCII -- `file` says so, and no byte in them is >= 0x80. The same job passed on main in August, so the checker upgraded under it; `make lint` has been red on main since, for every branch. Snapshots are byte-exact recordings of terminal output, not hand-written source, so there is no encoding to pin in the first place. Claude-Session: https://claude.ai/code/session_01MeysZ63ewZiFTgCDs172XJ --- .editorconfig | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.editorconfig b/.editorconfig index 91d56895..a7a0edca 100644 --- a/.editorconfig +++ b/.editorconfig @@ -45,3 +45,10 @@ max_line_length = unset [src/coverage/html_file.sh] max_line_length = unset + +# Snapshots are byte-exact recordings of terminal output, ANSI escapes and all. +# editorconfig-checker 4.0.0 reads the ESC byte in four of them as Latin-1 and +# fails a charset it cannot detect on a file that is pure ASCII. Nothing here is +# hand-written, so there is no encoding to pin. +[tests/acceptance/snapshots/**] +charset = unset