Skip to content

tr: stop an absurd [c*N] repeat count from crashing or hanging - #14317

Open
arbelonson-source wants to merge 1 commit into
uutils:mainfrom
arbelonson-source:fix/tr-huge-repeat-count-crash
Open

tr: stop an absurd [c*N] repeat count from crashing or hanging#14317
arbelonson-source wants to merge 1 commit into
uutils:mainfrom
arbelonson-source:fix/tr-huge-repeat-count-crash

Conversation

@arbelonson-source

Copy link
Copy Markdown
Contributor

What

An explicit repeat count in a [c*N] construct (as opposed to the
open-ended [c*], which is already bounded by the other set's real
length) was taken completely literally: Sequence::flatten's lazy
std::iter::repeat_n(c, n) got collected into an N-byte Vec<u8>
with no upper bound, both when solving set1 and when solving set2.

$ echo abc | tr '[a*999999999999]' x
memory allocation of 999999999999 bytes failed
Aborted (SIGABRT), exit code 134

A merely huge rather than astronomical N -- still far more than any
real command-line argument could ever need -- hangs for seconds to
minutes instead of aborting outright, since materializing and then
iterating N bytes has a real cost even when the allocation itself
succeeds.

Fix

Every position past the point where the other set's real length
runs out collapses to the same result regardless of exactly how much
further N reaches, since that's tr's own "pad the shorter set with
its last element" rule. Confirmed empirically before assuming it: GNU
tr's output for [a*N] x is identical for every N from 1 up to the
largest it can complete in reasonable time (50000), and for a matched
pair of one-million-character sets, this fix's output is byte-for-byte
identical to GNU's own.

GNU itself is not immune to this class of input, though, so this
isn't a "fix the wording to match GNU" PR like most of the rest of
this series -- [a*999999999999] x hangs GNU tr too (confirmed
directly with a real 15s timeout; an earlier, unverified pass at this
had reported GNU handled the value gracefully, which turned out to be
wrong). a [x*999999999999] does not hang GNU, though, and is the
more common practical case in the first place (padding one set to
match a long literal replacement) -- and completes instantly on both
GNU and, after this fix, on this implementation.

Capping the parsed repeat count at 2 MiB keeps every realistic use
working exactly as before (confirmed with a real matched-pair test at
100,000 characters, byte-identical to GNU) while turning both the
crash and the hang into an instant, correct result. 2 MiB is far more
headroom than any single shell argument can realistically contain --
confirmed directly, since even attempting to construct the equivalent
real command-line arguments to test right at that boundary hit the
OS's own ARG_MAX before this cap ever could.

Testing

  • cargo test -p uu_tr / full tests/by-util/test_tr.rs suite: 169 passed, 0 failed.
  • Added a regression test covering the crash (translate mode, set1), the hang (translate mode, set2), and delete mode (-d), all completing in well under a second.
  • Manually diffed [a*N] x for N = 1, 5, 50, 500, 5000, 50000 against GNU tr 9.11 -- byte-identical output throughout.
  • Manually diffed a matched 100,000/50,000+50,000-character set pair (md5sum-compared) against GNU -- identical.
  • Manually confirmed the fix covers -d, -s, and -c modes too, not just plain translation.
  • cargo clippy -p uu_tr --all-targets -- -D warnings and cargo fmt --check: clean.

This PR was written with AI assistance (Claude Opus 5, via Claude Code). I've tested the changes but please review the code carefully.

An explicit repeat count in a [c*N] construct (as opposed to the
open-ended [c*], which is already bounded by the other set's real
length) was taken completely literally: Sequence::flatten's lazy
std::iter::repeat_n(c, n) got collected into an N-byte Vec<u8> with no
upper bound, both when solving set1 and when solving set2.

tr '[a*999999999999]' x aborts the process outright (a ~1 TB
allocation request); a merely huge rather than astronomical N -- still
far more than any real command-line argument could need -- hangs for
seconds to minutes instead, since materializing and then iterating N
bytes has a real cost even when the allocation itself succeeds.

Every position past the point where the other set's real (bounded, by
construction, to whatever a single shell argument can hold) length
runs out collapses to the same result regardless of exactly how much
further N reaches, since that's tr's own "pad the shorter set with
its last element" rule -- confirmed empirically: GNU tr's output for
[a*N] x is identical for N from 1 up to the largest N it can complete
in reasonable time (50000), and for a matched pair of one-million
character sets, this fix's output is byte-identical to GNU's own. GNU
itself is not immune to this class of input, though: [a*999999999999]
x hangs GNU tr too (confirmed directly, contradicting an initial,
unverified report that it handled the value gracefully) -- a
[x*999999999999] does not, which is the more common practical case
(padding one set to match a long literal replacement) and completes
instantly on both GNU and, after this fix, on this implementation.

Capping the parsed repeat count at 2 MiB -- far more headroom than any
single shell argument can realistically contain, confirmed directly:
attempting to even construct the equivalent real arguments to test at
exactly that boundary hit the OS's own ARG_MAX first -- keeps every
realistic use working exactly as before while turning both the crash
and the hang into an instant, correct result.

AI-assisted-by: Claude Opus 5, via Claude Code
// it N actually reaches; not capping it means materializing N literal
// bytes downstream, which is either an out-of-memory abort or, for a
// merely huge rather than astronomical N, a multi-second hang.
const MAX_CHAR_REPEAT: usize = 2 * 1024 * 1024;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should catch actual allocation's error instead of adding such bound. Or it panics when RAM is actually lacking.

.map(|(l, a)| (l, Ok(Self::CharStar(a))))
}

// `[c*N]` only ever needs to be as long as the other set it is padding

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need 9 lines of comments for this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants