Skip to content

Test framework, stage 5: TUT removed - #479

Merged
RyeMutt merged 5 commits into
developfrom
rye/tests-tut-removed
Oct 14, 2026
Merged

RyeMutt merged 5 commits into
developfrom
rye/tests-tut-removed

Conversation

@RyeMutt

@RyeMutt RyeMutt commented Oct 14, 2026

Copy link
Copy Markdown
Member

Description

Stage 5's first step of the test framework's move from TUT to Catch2 (TM26): TUT is removed. Every test in the tree is Catch2 now, and how to write one is documented.

It is on develop, after #478 and #477.

llkdu's test (c63e717)

  • What changed: llimagej2ckdu_test, the last TUT test, is a Catch2 binary of its own. It compiles the codec in, over the test's stubs of llimage and of the KDU calls the codec makes, and links what the TUT test linked.
  • Its cases: the three tests are cases with checks, named for what they check. They hold their images as locals, where the TUT tests leaked them.
  • Not compiled: it is built only with AL_USE_KDU, and KDU has no vcpkg port yet, so the conversion is by hand and unchecked until a build with KDU exists.

No test keeps namespace tut (d1a287a)

  • Befriended fixtures: the three that production code names as friends (alincludesearch_data, llfontregistry_data and llfontregistry_gl_data) move to ll_test with their friend declarations, beside the probes already there.
  • Everything else: the other blocks the conversion left in namespace tut, in eight llcommon and llmessage test files, are anonymous, as everything local to a test file is.

TUT goes (d0986e5)

  • Deleted:
    • externals/tut and ll::tut;
    • the TUT runner (lltut.h, lltut.cpp, test.cpp, chained_callback.h, lltut_runner_lib);
    • setenv.h, which nothing included;
    • al_add_test and LLAddBuildTest.cmake. The library path a test runs with, AL_TEST_ENVIRONMENT, moves to ALAddTestBinary.cmake.
  • Kept helpers: sync.h, namedtempfile.h and the other helpers Catch2 tests still use are listed with the harness.
  • AL_TEST_CATCH2 goes, and the TUT branches it chose between:
    • sync.h fails a timeout with FAIL;
    • the headless UI and the shared GL context have one body each, so no object library built without the define can give either a second static. That was the cause of Test framework, stage 3: llui's tests in Catch2 #471's CI failures.
    • newview_test_fakes now links the harness for its headers, which the define used to hide from it.

Comments (fdc261a)

  • Updated: comments that told of TUT as the tests' framework now say what Catch2 does. These include the include orders around lltut.h, test.cpp as the main, fixtures made per test method, and the TUT banners.
  • Kept: comments that say why a helper does as TUT did.

Docs (3186eb2)

  • doc/TESTING.md is new. It covers:
    • running the tests: a file, a case, another order, one case to a process;
    • writing one, to the rules a test binary asks of its files: names, CHECK and REQUIRE, linkage, static init, fakes, the state guard, process-wide fixtures;
    • al_add_test_binary's options;
    • benchmarks;
    • upstream TUT tests, which are converted by hand.
  • CLAUDE.md's and BUILD.md's Tests sections now say the tests are Catch2, and point at it.

Related Issues

  • Please link to a relevant GitHub issue for additional context.

Issue Link: none. This follows #471, #472, #474, #475 and #478.


Checklist

  • I have provided a clear title and detailed description for this pull request.
  • If useful, I have included media such as screenshots and video to show off my changes.
  • I have tested the changes locally and verified they work as intended.
  • All new and existing tests pass.
  • Code follows the project's style guidelines.
  • Documentation has been updated if needed.
  • Any dependent changes have been merged and published in downstream modules.
  • I have reviewed the contributing guidelines.

Additional Notes

  • Tested on macOS arm64, RelWithDebInfo:
    • the full build is clean;
    • all 477 tests of ctest -LE manual pass, and so do all 19 pending combined runs;
    • nothing in the build graph is a TUT target;
    • no anonymous type is used with a name-keyed registry in two files of any of the 46 test binaries.
  • Not compiled here: llkdu's test, as above.
  • Left as they are:
    • scripts/testing/tut_parity.py cannot run without TUT.
    • tut2catch.py needs a checkout from before this, since it parses a file with TUT's headers; TESTING.md says so.
    • test_cost_report.py's default pattern still names the old executables. TM28, which measures with it, updates it.

🤖 Generated with Claude Code

RyeMutt and others added 5 commits October 14, 2026 07:45
The last TUT test: llimagej2ckdu_test, the codec compiled into its
test over stubs of llimage and of the KDU calls it makes, becomes a
binary of its own (al_add_test_binary with UNIT), linking what the
TUT test linked. Its three tests are cases with checks, named for what
they check, and hold the images they pass as locals where the TUT
tests leaked them.

It is built only with AL_USE_KDU, and KDU has no vcpkg port yet, so
this has not been compiled: the conversion is by hand and unchecked
until a build with KDU exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fixtures production code befriends by name -- alincludesearch_data
and the two llfontregistry data structs -- move to ll_test with their
friend declarations, beside the probes already there. The other test
files the conversion left in namespace tut (llcommon's date, error,
event dispatcher, LEAP, process, LLSD and Python serialisation tests,
and llmessage's io test) put it in an anonymous namespace, as
everything local to a test file is: tut::call_exc and tut::call_logerr
are ::call_exc and ::call_logerr, which members of those names still
reach past, and tut::TestRecorder is TestRecorder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every test is Catch2 now (TM26):

- externals/tut, ll::tut, the TUT runner (lltut.h, lltut.cpp,
  test.cpp, chained_callback.h, lltut_runner_lib) and setenv.h, which
  nothing included, are deleted.
- al_add_test and LLAddBuildTest.cmake are deleted. The library path a
  test runs with, AL_TEST_ENVIRONMENT, is defined in
  ALAddTestBinary.cmake now.
- The helpers TUT tests shared and Catch2 tests still use -- sync.h,
  namedtempfile.h and the rest -- are listed with the harness.
- AL_TEST_CATCH2 goes, and the TUT branches it chose between: sync.h
  fails a timeout with FAIL, and the headless UI and the shared GL
  context have the one body each, so no object library built without
  the define can give one of them a second static.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The comments that told of TUT as the tests' framework -- headers to
include before lltut.h, test.cpp as the main, fixtures made per test
method, test functions run in sequence, a TUT banner over each file's
tests -- say what Catch2 does. Those that say why a helper does as TUT
did are left. CI's LOGFAIL is the tests' main's, and llprocess's
comment names llleap's test rather than its old target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
doc/TESTING.md says how to run the tests, a file, a case, in another
order or a case to a process; how to write one, to the rules a test
binary asks of its files (names, CHECK and REQUIRE, linkage, static
init, fakes, the state guard, process-wide fixtures); what
al_add_test_binary's options are for; how a benchmark is written; and
that an upstream TUT test is converted by hand. CLAUDE.md's and
BUILD.md's tests sections say the tests are Catch2 and point at it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fffc8670-0888-4e7e-a401-fbf5f7e0d6d4
📥 Commits

Reviewing files that changed from the base of the PR and between 0f896df and 3186eb2.

📒 Files selected for processing: 81
  • .github/workflows/build.yaml
  • CLAUDE.md
  • doc/BUILD.md
  • doc/TESTING.md
  • indra/CMakeLists.txt
  • indra/alscript/preprocessor/alincludesearch.h
  • indra/alscript/tests/alincludesearch_test.cpp
  • indra/cmake/ALAddTestBinary.cmake
  • indra/cmake/Dependencies.cmake
  • indra/cmake/LLAddBuildTest.cmake
  • indra/externals/README.md
  • indra/externals/tut/LICENSE.txt
  • indra/externals/tut/tut/tut.hpp
  • indra/externals/tut/tut/tut_assert.hpp
  • indra/externals/tut/tut/tut_exception.hpp
  • indra/externals/tut/tut/tut_posix.hpp
  • indra/externals/tut/tut/tut_reporter.hpp
  • indra/externals/tut/tut/tut_restartable.hpp
  • indra/externals/tut/tut/tut_result.hpp
  • indra/externals/tut/tut/tut_runner.hpp
  • indra/llcommon/is_approx_equal_fraction.h
  • indra/llcommon/llprocess.cpp
  • indra/llcommon/tests/StringVec.h
  • indra/llcommon/tests/apply_test.cpp
  • indra/llcommon/tests/classic_callback_test.cpp
  • indra/llcommon/tests/lazyeventapi_test.cpp
  • indra/llcommon/tests/llcond_test.cpp
  • indra/llcommon/tests/lldate_test.cpp
  • indra/llcommon/tests/lldependencies_test.cpp
  • indra/llcommon/tests/llerror_test.cpp
  • indra/llcommon/tests/lleventcoro_test.cpp
  • indra/llcommon/tests/lleventdispatcher_test.cpp
  • indra/llcommon/tests/lleventfilter_test.cpp
  • indra/llcommon/tests/llevents_test.cpp
  • indra/llcommon/tests/llheteromap_test.cpp
  • indra/llcommon/tests/llleap_test.cpp
  • indra/llcommon/tests/llmainthreadtask_test.cpp
  • indra/llcommon/tests/llpounceable_test.cpp
  • indra/llcommon/tests/llprocess_test.cpp
  • indra/llcommon/tests/llsd_test.cpp
  • indra/llcommon/tests/llsdserialize_python_test.cpp
  • indra/llcommon/tests/lltreeiterators_test.cpp
  • indra/llcommon/tests/threadsafeschedule_test.cpp
  • indra/llcommon/tests/tuple_test.cpp
  • indra/llcommon/tests/workqueue_test.cpp
  • indra/llcorehttp/tests/test_llcorehttp_peer.py
  • indra/llimage/tests/llimageworker_test.cpp
  • indra/llkdu/CMakeLists.txt
  • indra/llkdu/tests/llimagej2ckdu_test.cpp
  • indra/llmath/llmath.h
  • indra/llmessage/tests/io_test.cpp
  • indra/llrender/llfontregistry.h
  • indra/llrender/tests/llfontgl_test.cpp
  • indra/llrender/tests/llfontregistry_gl_test.cpp
  • indra/llrender/tests/llfontregistry_test.cpp
  • indra/llrender/tests/llfonttextcache_test.cpp
  • indra/llrender/tests/llheadlessgl_fixture.h
  • indra/llui/tests/alheadlessui_fixture.h
  • indra/llui/utils/llui_libtest.cpp
  • indra/newview/CMakeLists.txt
  • indra/newview/tests/aldaycyclelandmarks_test.cpp
  • indra/newview/tests/lllogininstance_test.cpp
  • indra/newview/tests/llsecapi_test.cpp
  • indra/newview/tests/llsechandler_basic_test.cpp
  • indra/newview/tests/llslurl_test.cpp
  • indra/newview/tests/lltextureinfo_test.cpp
  • indra/newview/tests/llviewercontrollistener_test.cpp
  • indra/newview/tests/llviewernetwork_test.cpp
  • indra/newview/tests/llvocache_test.cpp
  • indra/newview/tests/llworldmap_test.cpp
  • indra/newview/tests/llworldmipmap_test.cpp
  • indra/newview/tests/newview_test_fakes.h
  • indra/test/CMakeLists.txt
  • indra/test/catch_and_store_what_in.h
  • indra/test/chained_callback.h
  • indra/test/debug.h
  • indra/test/lltut.cpp
  • indra/test/lltut.h
  • indra/test/setenv.h
  • indra/test/sync.h
  • indra/test/test.cpp
💤 Files with no reviewable changes: 20
  • indra/llimage/tests/llimageworker_test.cpp
  • indra/externals/README.md
  • indra/externals/tut/tut/tut_assert.hpp
  • indra/externals/tut/LICENSE.txt
  • indra/CMakeLists.txt
  • indra/test/lltut.h
  • indra/externals/tut/tut/tut_result.hpp
  • indra/cmake/LLAddBuildTest.cmake
  • indra/test/lltut.cpp
  • indra/externals/tut/tut/tut_reporter.hpp
  • indra/externals/tut/tut/tut_restartable.hpp
  • indra/externals/tut/tut/tut_posix.hpp
  • indra/test/chained_callback.h
  • indra/test/setenv.h
  • indra/llui/tests/alheadlessui_fixture.h
  • indra/externals/tut/tut/tut.hpp
  • indra/test/test.cpp
  • indra/externals/tut/tut/tut_exception.hpp
  • indra/externals/tut/tut/tut_runner.hpp
  • indra/newview/tests/newview_test_fakes.h

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Summary

Summary by CodeRabbit

  • Testing

    • Unit tests now use Catch2 v3, with tests grouped into one executable per library and selectable as separate CTest entries.
    • The default test run excludes manual tests; individual libraries and test files can be selected, and benchmark runs are documented.
    • Test output is displayed when an individual test case fails.
  • Documentation

    • Expanded testing guidance covers running and writing tests, test selection, reruns, benchmarks, and converting existing tests.

The test build and shared test support are updated for Catch2 v3. TUT infrastructure and its test runner are removed. Existing test sources, build instructions, and testing guidance are updated.

Changes: Catch2 Test Infrastructure Migration

Layer / File(s) Summary
Replace TUT test infrastructure
indra/CMakeLists.txt, indra/cmake/*, indra/externals/tut/*, indra/test/*, doc/TESTING.md, doc/BUILD.md, CLAUDE.md, .github/workflows/build.yaml
The build switches to ALAddTestBinary and removes TUT libraries, runner code, and utilities. The documentation describes Catch2 test layout, configuration, filtering, authoring, and benchmark commands. The workflow comment describes when captured test output is displayed.
Unify shared test support
indra/cmake/ALAddTestBinary.cmake, indra/test/sync.h, indra/llrender/tests/llheadlessgl_fixture.h, indra/llui/tests/alheadlessui_fixture.h, indra/newview/CMakeLists.txt, indra/newview/tests/newview_test_fakes.h
The test binary configuration adds staged-library paths to the loader environment on Windows and Linux. Shared synchronization, fixture, state-guard, teardown, and recovery-writer support no longer depends on AL_TEST_CATCH2.
Adapt existing test suites
indra/alscript/*, indra/llcommon/*, indra/llcorehttp/tests/*, indra/llimage/tests/*, indra/llmath/*, indra/llmessage/tests/*, indra/llrender/*, indra/llui/*, indra/newview/tests/*
Test namespaces and framework references are updated across existing suites. The summaries report no test-logic changes except for the separately described llkdu conversion.
Convert llkdu tests
indra/llkdu/CMakeLists.txt, indra/llkdu/tests/llimagej2ckdu_test.cpp
The llkdu test target uses al_add_test_binary, compiles llimagej2ckdu.cpp, and links llmath. Three TUT tests become Catch2 cases for metadata, decode, and encode methods.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to 3186e

No identified issue blocks merging. The KDU-only tests should be compiled and run when a KDU build is available.

Pre-merge checks: Passed 4 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Docstring Coverage Inconclusive Docstring coverage is 23.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 30 functions across 50 files. (11 skipped… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely identifies the main change: removing TUT in stage 5 of the Catch2 migration.
Description check Passed The description is detailed and follows the template. It explains the migration, affected components, testing results, known limitations, related work, and documentation updates. The issue link is mar…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 23.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 30 functions across 50 files. (11 skipped: 9 unsupported, 2 over the file limit.)

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the test list twice,
Then hops through Catch2, neat and nice.
Old TUT leaves the burrow door,
New cases gather by the score.
Logs wait for a failing case,
While carrots cheer the testing place.

Comment @coderabbitai help to get the list of available commands.

@RyeMutt
RyeMutt merged commit a70a039 into develop Oct 14, 2026
22 checks passed
@RyeMutt
RyeMutt deleted the rye/tests-tut-removed branch October 14, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant