Skip to content

Test framework: fix the old test bugs in llcommon's tests, and LLApp::instance() after the app is gone - #460

Merged
RyeMutt merged 5 commits into
rye/tests-tm20from
rye/tm20-test-fixes
Oct 13, 2026
Merged

RyeMutt merged 5 commits into
rye/tests-tm20from
rye/tm20-test-fixes

Conversation

@RyeMutt

@RyeMutt RyeMutt commented Oct 13, 2026

Copy link
Copy Markdown
Member

Description

Fixes the bugs in llcommon's own tests that the Catch2 conversion in #459 found and kept, since a conversion must not change what a test does. Each was a test that:

  • could not fail for the reason it existed;
  • checked something other than what it said;
  • read memory it had not checked, leaked, or could flake.

It also fixes one bug in production code that the conversion found: LLApp::instance() dangling after the app is deleted.

It stacks on #459 (rye/tests-tm20). Apart from llapp.cpp, it changes only test files.

Production: 98982d2, LLApp::instance() finds no app once it is gone.

  • The bug: ~LLApp ended LLCommon but left sApplication pointing at the app it was destroying. Anything asking afterwards got a deleted object.
  • Why it hadn't shown: every caller runs while the app is alive.
  • Why it matters now: test binaries make and end apps in turn.
  • The fix: ~LLApp clears sApplication as it ends, if it names the app being destroyed.
  • Test: a new llapp_test case checks this, and fails without the fix.

Strings (dac4b3b)

  • llstreamtools:
    • return values were never checked;
    • fullread compared one byte short;
    • a word-splitting comparison never held the hyphen it described;
    • an empty case tested a function that doesn't exist.
  • llstring:
    • a format branch checked the result instead of its key;
    • F32/F64 limit checks were commented out, and now run;
    • a case that tested only std::string is removed.
  • stringize: the wstringize case checked nothing off Windows.
  • llstring_utf: comments that no longer described the code.

LLSD and events (bd86330)

  • llsdserialize:
    • the "map" case began from an array;
    • comments were copied, cut off or wrong;
    • a buffer leaked when a check failed.
  • llsdserialize_python: when another waitpid() reaped the child, the case passed without checking the exit code. It now skips and says why.
  • llsd: a REQUIRE(true).
  • lleventdispatcher: three matchers had actual and expected swapped.
  • lazyeventapi: a case passed trivially and relied on an earlier case.
  • llpounceable: an empty compile-fail case, now a comment.

Registries, singletons, units, UUIDs, timers (b111002)

  • alregex: a check passed whenever firstByteRange() said no.
  • alrecoverystore:
    • front() called on lists whose size was never checked;
    • write results never checked.
  • llinstancetracker:
    • unchecked iterator dereferences;
    • a constructor that should throw: its object leaked if it didn't, and nothing checked that it threw.
  • llsingleton:
    • an empty case;
    • an unreachable throw;;
    • REQUIREs that could leave singletons alive for later cases.
  • llunits: values built from the wrong types.
  • lluuid: ^= and commutativity claimed, never tested.
  • lltrace: comments that didn't match the code.
  • lltimer: bounds a loaded machine could miss. They now hold against scheduling delays and still catch a clock 0.5% off.

Timers, dates, files, processes, LEAP (cdd2de1)

  • lldeadmantimer: two checks that couldn't fail.
  • lldate:
    • the compiled-out boundary block wouldn't have compiled;
    • its leap-day input was a valid date;
    • a day-of-week check checked only a length.
  • llfile:
    • sizeof of a pointer;
    • an isfile() never asked.
  • commonmisc:
    • an unchecked &bin[0];
    • a wrong max_bytes;
    • dead code.
  • llprocess:
    • one find() checked twice;
    • the "History:" dump never printed for a failed CHECK;
    • five polling loops raced the script's truncating write;
    • Windows-only, not compiled here: leaked process handles.
  • llleap:
    • the reader script's end-of-input test could never be true;
    • each failing size in the size search now says which size it was.

Related Issues

  • Please link to a relevant GitHub issue for additional context.

Issue Link: none. The bugs were found by #459's idiom pass and listed in c8b7e58's message.


Checklist

  • I have provided a clear title and detailed description for this pull request.
  • If useful, I have included media such as screenshots and video to show off my changes.
  • I have tested the changes locally and verified they work as intended.
  • All new and existing tests pass.
  • Code follows the project's style guidelines.
  • Documentation has been updated if needed.
  • Any dependent changes have been merged and published in downstream modules.
  • I have reviewed the contributing guidelines.

Additional Notes

  • Tested on macOS arm64, RelWithDebInfo:
    • every llcommon entry passes, and each of the four binaries passes 10 random orders and 20 repeats, run whole, with no state guard report;
    • the full build is clean, all 454 tests of ctest -LE manual pass, and so do all 10 pending combined runs.
  • Each new or corrected check fails when it should. 55 small changes to the tests' own inputs or expected values were run, and each was reverted after.
    • 52 were meant to fail a new or corrected check, and did.
    • Three were meant to pass, and did. llstring's format key and llprocess's race would each have failed on the old code. stringize's third is a check made on Windows only.
    • Reaping the Python child early made each llsdserialize_python case skip, saying why.
    • llpounceable's documented misuse still fails to compile.
  • Counts changed only where described, for example: llstreamtools 103 to 123 assertions; stringize 10 to 17; alrecoverystore 295 to 301.
    • Fewer cases: llstring, llsingleton and llpounceable each lost a case that tested nothing of ours.
    • lldate lost a case that tested nothing: it is compiled out with the boundary block it belonged to.
  • Merge order: this should follow Test framework, stage 3: llcommon's tests in Catch2 #459. Once Test framework, stage 3: llcommon's tests in Catch2 #459 merges, its base becomes develop.

🤖 Generated with Claude Code

RyeMutt and others added 5 commits October 12, 2026 22:21
~LLApp ended LLCommon and left sApplication pointing at the app it was
destroying, so LLApp::instance() handed out a deleted object to anything
that asked afterwards. Every caller found runs while the app is alive,
in the viewer and in the tests, so nothing reached it yet, but a test
binary makes and ends apps in turn: lleventcoro's and lllogin's process
apps end in the harness's teardown, and llapp_test ends one in every
case.

~LLApp now clears it as it ends, when the app ending is the one it
names. A new llapp_test case deletes its app and checks instance() is
null; it fails without the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- llstreamtools:
  - skip_to_end_of_next_keyword's, get_word's, remove_last_char's and
    fullread's return values were taken and never checked; they are;
  - fullread compared one byte short of what it read, so the final
    newline was never checked;
  - the get_word/skip_to_next_word comparison's comments described a
    hyphen the input never held: both streams used an underscore and
    landed on the same word. The hyphen stream now shows get_word
    splitting only at whitespace where skip_to_next_word splits at it;
  - get_brace_count's case was empty, and there is no such function;
  - wrong messages and comments: a lone quote is removed, not kept; a
    stale "remainder of Second"; six unused locals.
- llstring:
  - format's no-substitution branch checked s1 after format rewrote it,
    not the key, and passed only because "?" and "[]" map to
    themselves; it checks the key;
  - the F32 and F64 conversions' limit checks were commented out, the
    F64 one calling convertToF32. Each now accepts its type's largest
    magnitude and rejects one past it;
  - the first case tested std::string's own constructors, nothing of
    ours, and goes; so do an unused local and stale comments.
- stringize: the wstringize case asserted nothing off Windows, its #else
  commented out. It checks every type it can on every platform, and the
  Windows checks stay as they were.
- llstring_utf: comments that no longer described the code: the
  conversions it has (three, not six), isPictographBase's range (U+2000
  is rightly rejected), and walkers and helpers since removed.

Each new or corrected check was shown to fail under a small change to the
test's own input or expected value. llstreamtools_test passes 123
assertions in 25 cases (103 in 26), llstring_test 371 in 46 (383 in 47:
the std::string case's 18 went), stringize_test 17 (10).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- llsdserialize:
  - the cross-compatible "map" case began from an empty array, which
    became a map only at its first key; it begins from an empty map;
  - comments copied between cases, cut off, or claiming exceptions and
    failures that never happen are corrected, and the binary size
    labels say which way the size field is wrong;
  - srand(ii) on every pass is kept, deliberately: it makes each pass's
    buffer reproducible whatever drew from rand() before, as a comment
    now says;
  - found here: the navmesh case leaked its unzipped buffer whenever a
    check failed. A unique_ptr with a free() deleter owns it.
- llsdserialize_python: when waitpid() failed with ECHILD the case
  passed without the exit code it exists to check. It skips, saying the
  code was taken by another waitpid(), as LLProcess allows a SIGCHLD
  handler may; any other failure fails.
- llsd: a REQUIRE(true) inside the branch it meant to show was taken is
  a flag set there and REQUIREd after.
- lleventdispatcher: three matchers had actual and expected swapped, so
  a failure would have been reported backwards.
- lazyeventapi: "without a LazyEventAPI" passed whatever happened, and
  its comment leaned on case 1 having run first. It makes and destroys
  its own registrar, and shows the pump is not made after.
- llpounceable: the compile-fail case was empty and always passed. What
  it documents is a compile error, so it is a comment, and the case
  goes; uncommenting the line still fails to compile.

Each new or corrected check was shown to fail under a small change to the
test's own input or expected value, and the reaping of the Python child
by another waitpid() to skip each case. llpounceable_test has six cases
(seven).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… they say they check

- alregex: a check passed whenever firstByteRange() answered false. It
  now expects false where any byte may start a match, and a new check
  expects the range where a word must.
- alrecoverystore: three front() calls on lists whose size nothing
  checked, and two writes whose result nothing checked; each is
  REQUIREd or CHECKed. A taken entry was never checked to be there.
- llinstancetracker: a key snapshot's iterator was dereferenced three
  times with its size unchecked; the keys are compared whole. A
  throwing constructor's object leaked when it did not throw, and
  nothing checked that it threw: CHECK_THROWS_AS.
- llsingleton: the first case was empty and goes. A `throw;` with no
  exception active could not run: initSingleton()'s getInstance()
  returns the instance being made, which the case now checks.
  require_order's REQUIREs skipped deleteAll() when one failed, leaving
  the singletons for later cases; they are CHECKs.
- llunits: the U32 unit was built from the F32 value where the case
  meant the signed integer one, and an explicit cast stood where the
  case meant an implicit conversion. accept_explicit_quatloos always
  returned true; it returns what it was given.
- lluuid: the XOR case claimed ^= and commutativity and tested neither;
  it tests both. A set was iterated with its size unchecked.
- lltrace: comments named cups and powers of two the code doesn't use.
- lltimer: bounds of 0.04 s and 5 ms that a loaded machine could miss.
  The expiry case sleeps 250 ms against bounds of 0.2 s; each
  totalTime() read is held between two steady_clock reads with 1 ms
  of slack, so a thread kept off the processor no longer fails it,
  while a clock 0.5% fast or slow still does.

Each new or corrected check was shown to fail under a small change to the
test's own input or expected value. alrecoverystore_test passes 301
assertions (295), alregex_test 137 (133), lluuid_test 36,997 (36,993),
lltimer_test 13 (12), llinstancetracker_test 30 (32: three dereferences
are one comparison), llsingleton_test 37 in 14 cases (36 in 15).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…say they check

- lldeadmantimer: in the CPU-metrics block a second isExpired() call
  left out the CPU arguments, so the two checks after it could not
  fail.
- lldate: the parser boundary block, compiled out, used inputs that were
  commented out, and its leap-day input was a valid date. The parser
  does not reject out-of-range fields (timegm normalises them), so the
  block stays out, now whole case and all, and says why. Found here: a
  case claimed to check the RFC 1123 day of week and checked only a
  length.
- llfile: sizeof a const char*, the pointer's size; and a case named for
  isfile() after a write never asked it.
- commonmisc: &bin[0] on a vector not checked non-empty is a whole
  comparison; a max_bytes shorter than the text it parses is that
  text's length (the parser ignores it for quoted strings, so nothing
  changes); dead stubs, an #if 0 block, an empty namespace and stale
  headers go.
- llprocess:
  - one find("ghi") was checked twice where the second meant from
    offset 6;
  - the "History:" dump printed only for a failed REQUIRE, inside a try
    that a CHECK never left; it is a scoped INFO, so any failure shows it;
  - five polling loops raced readfile(), which fails on an empty file,
    against the script's truncating write: they poll with
    readfile_if_present(), which reads that as not yet;
  - Windows only, not compiled here: the autokill helper's process
    handles leaked when a REQUIRE failed; they are owned.
- llleap: the reader script's end-of-input test could never be true, and
  the mismatch report gave the last differing chunk, not the first.
  test_or_split's search records each failing size, as Catch2 records a
  failed REQUIRE before it throws; each now names its size.

Each new or corrected check was shown to fail under a small change to the
test's own input or expected value, and llprocess's script, made to
pause between truncating and writing, to pass. lldate_test passes 40
assertions in 9 cases (39 in 10: the empty boundary case is compiled out
with its block), llfile_test 149 (148).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: cf12b1c8-96c0-4faa-b5b1-e67fbd04c7ef
📥 Commits

Reviewing files that changed from the base of the PR and between 9f14a44 and cdd2de1.

📒 Files selected for processing (26)
  • indra/llcommon/llapp.cpp
  • indra/llcommon/tests/alrecoverystore_test.cpp
  • indra/llcommon/tests/alregex_test.cpp
  • indra/llcommon/tests/commonmisc_test.cpp
  • indra/llcommon/tests/lazyeventapi_test.cpp
  • indra/llcommon/tests/llapp_test.cpp
  • indra/llcommon/tests/lldate_test.cpp
  • indra/llcommon/tests/lldeadmantimer_test.cpp
  • indra/llcommon/tests/lleventdispatcher_test.cpp
  • indra/llcommon/tests/llfile_test.cpp
  • indra/llcommon/tests/llinstancetracker_test.cpp
  • indra/llcommon/tests/llleap_test.cpp
  • indra/llcommon/tests/llpounceable_test.cpp
  • indra/llcommon/tests/llprocess_test.cpp
  • indra/llcommon/tests/llsd_test.cpp
  • indra/llcommon/tests/llsdserialize_python_test.cpp
  • indra/llcommon/tests/llsdserialize_test.cpp
  • indra/llcommon/tests/llsingleton_test.cpp
  • indra/llcommon/tests/llstreamtools_test.cpp
  • indra/llcommon/tests/llstring_test.cpp
  • indra/llcommon/tests/llstring_utf_test.cpp
  • indra/llcommon/tests/lltimer_test.cpp
  • indra/llcommon/tests/lltrace_test.cpp
  • indra/llcommon/tests/llunits_test.cpp
  • indra/llcommon/tests/lluuid_test.cpp
  • indra/llcommon/tests/stringize_test.cpp
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@RyeMutt
RyeMutt added this pull request to stack #461 October 13, 2026 04:04
@RyeMutt
RyeMutt merged commit c538293 into develop Oct 13, 2026
21 checks passed
@RyeMutt
RyeMutt deleted the rye/tm20-test-fixes branch October 13, 2026 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant