Skip to content

fix(input): keep the order of uses in the cache entry of a .ll input - #280

Draft
SizzleUnrlsd wants to merge 6 commits into
mainfrom
fix/ll-input-cache-use-order
Draft

SizzleUnrlsd wants to merge 6 commits into
mainfrom
fix/ll-input-cache-use-order

Conversation

@SizzleUnrlsd

@SizzleUnrlsd SizzleUnrlsd commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

With the compile IR cache, a .ll given as input was parsed from its text by the run that fills the cache, and from bitcode written without the order of uses by the runs that read it (#279). The two modules could list the users of a value in another order. The entry's bitcode now holds that order, and the cache schema moves to v5, so that the entries written before are not reused: the first run after the update writes each entry again.

Closes #279

Change

  • fix(input): keep the order of uses in the cache entry of a .ll input: the entry is written with ShouldPreserveUseListOrder. One argument, and a comment that says why.
  • test(unit): check that a .ll input read from the cache keeps the order of uses: testCompileIRCacheKeepsUseOrderOfIRInput, described below.
  • fix(input): move the compile IR cache to schema v5: an entry written before this change holds no order, and is no longer looked up. See The cache schema.
  • test(unit): check that a .ll entry of the previous cache schema is not reused: two more checks in the test.
  • docs(changelog), twice: the order of uses, then the cache filled again once.

The test

testCompileIRCacheKeepsUseOrderOfIRInput, in the unit tests, takes the path of a run that fills the cache, then of a run that reads it:

  1. It loads a .ll of three functions through loadModuleForAnalysis, with a cache directory. This gives the module parsed from the text, and an entry in the cache.
  2. It loads the file again, which gives the module parsed from that entry.
  3. It compares the callers of callee in the two modules: they must come in the same order.
  4. It writes the entry again from the first module, without the order of uses, and loads once more: the callers must now come in another order.
  5. It records that entry, which holds no order, under the schema compile-ir-cache-v4, and loads again: the callers must come in the order of the parsed text, and the entry must be back under the current schema.

In the file, callee is declared after its two callers, as Clang prints declarations. The text parser meets its uses before its declaration. A bitcode reader knows every function before it reads a body, and lists the callers another way unless it is told the order.

Step 4 keeps the test from passing for nothing: it fails if the file stops telling the two orders apart, or if a load stops reading the entry.

On main at 1fadba2, the test of steps 1 to 4 fails at step 3, the module read from the cache lists the callers of a function as the parsed text does, and its three other checks pass. With the order written but the schema still at v4, step 5 fails at an entry of the schema that held no order is not reused, and the five other checks pass. At the head, the six pass.

What the order in the bitcode restores

Counted with a probe build that is not in this branch: it compares the module parsed from the text with the module parsed from its bitcode, value by value. It takes the users of every global variable, function, argument, block and instruction, with the operand of each use.

.ll files Values with several uses Lists in another order, order left out With the order
The file of the test 1 1 0
Lua 5.4.6, 34 files printed by Clang 20 at -O0 -g 18,640 898: 883 functions, 15 global variables 0
The 12 .ll files of test/ 32 0 0

This count does not tell constants apart: a user that is a constant counts as a constant and an operand number, and the use lists of the constants themselves are not compared.

LLVM's own check covers them. verify-uselistorder of LLVM 20.1.2 maps every value of a module, constants included, writes the module as bitcode with the order of uses, reads it back and compares each use list. It then does the same through text. Run with --num-shuffles=0 on the file of the test and on the 34 Lua files:

  • Its bitcode step passes on the 35 files: the module parsed from the text and the module read back from its bitcode have the same use lists.
  • Its text step reports use-list order changed on lvm.ll and onelua.ll, the two files that hold blockaddress constants, for the jump table of Lua's interpreter. The cache does not go through that round trip.

Reports

On the 34 Lua files, --jobs=1, one compile IR cache directory per round, empty before the first run, three rounds:

Build First run Second run Reports
main at 1fadba2 34 cache misses 34 cache hits the same content: 1008 diagnostics and 2170 functions
This branch 34 cache misses 34 cache hits the same content: 1008 diagnostics and 2170 functions

The content of the diagnostics is the same in every run, with either build. Their order varies from one run to the next, from an empty cache as from a filled one, so these reports are not the same byte for byte.

No diagnostic was seen to change: this pull request restores the order of the lists, and no report shows it.

The runs of this branch were made at 5f71208, before the schema change, which changes no analysis.

The cache schema

An entry that an earlier build wrote for a .ll input holds no order of uses, and stayed valid as long as the file, the coretrace-compiler revision and the LLVM version did. The schema moves from compile-ir-cache-v4 to compile-ir-cache-v5, with the existing mechanism: the cache key and the metadata of each entry hold the schema, so a build of v5 does not look up an entry of v4.

The schema is one for the whole cache. The entries of C and C++ sources are written again too, once.

Checked with two builds on one cache directory, main at 1fadba2 and this branch, --jobs=1:

Run 35 .ll files: the file of the test and Lua's 34 Lua's 34 C sources
main, empty cache 35 misses, 35 entries written 0 hits, 34 entries written
main again 35 hits 34 hits
This branch, first run 0 hits, 35 misses, 35 entries written 0 hits, 34 entries written
This branch again 35 hits 34 hits
  • The earlier entries are left as they were: after the branch's first run on the .ll files, the 105 files that main wrote have the same checksums, and the directory holds 70 entries, 35 under each schema. main still reuses its own 35.
  • They stay in the directory, unused. Nothing removes them.
  • The existing functional check, check_compile_ir_cache_rebuilds_previous_schema, covers the metadata of an entry under the current key, for a C source. It passes unchanged.

Tests

Local runs, on macOS arm64 with AppleClang and LLVM 20.1.2, this branch built as a user builds it, coretrace-compiler fetched at the pinned revision. They ran on the tree of a3edac9; the commit after it changes the changelog only.

  • Functional tests, run_test.py --jobs 2 --no-cache, which runs the analyzer on the fixtures: 2760/2760. The script counts each unit test program below as one of the 2760.
  • Unit tests, two programs built with -DBUILD_ANALYZER_UNIT_TESTS=ON. Both were executed: stack_usage_analyzer_unit_tests ends with All analyzer module unit tests passed. after 402 checks, the 6 of the new test among them, and ownership_engine_unit_tests with All ownership engine unit tests passed. after 116 checks.
  • The new test before each fix: on main, 1 check fails and the 399 others pass. With the order written and the schema at v4, 1 check fails and the 401 others pass.
  • scripts/format-check.sh with clang-format 20, scripts/check-license-compliance.sh and the commit checker pass.
  • Not built on their own: the commits of the two fixes, 6ebba0d and c982506, without the test that follows each.

CI at 5a32343

  • Build run 43691722987: the three jobs pass, on ubuntu-24.04, macos-15 and macos-latest. In each, the functional tests give 2760/2760, with an entry of the previous schema is compiled again and rewritten among them, and both unit test programs are executed: All analyzer module unit tests passed., with the six checks of the new test, and All ownership engine unit tests passed.
  • Other runs: the release binaries for Linux amd64 and arm64, the CI integration (the GitHub Action, the Docker image, the consumer project), the concurrency analysis, clang-format, license-compliance and the commit conventions pass.

Merging

This branch starts from main and does not depend on #272 or #274. git merge-tree finds no conflict with either branch, at e6ae53e and dbe9c03. #274 writes the order of uses in the entry of a compiled source; this pull request writes it in the entry of a .ll input.

A .ll input is parsed from its text by the run that fills the compile
IR cache, and from the bitcode of its entry by the runs that read it.
That bitcode held no order of uses, so the two modules could list the
users of a value another way. The entry is now written with
ShouldPreserveUseListOrder.

Refs #279
…r of uses

The test loads a .ll whose callee is declared after its two callers,
then loads it again from the cache, and compares the callers of the
callee in the two modules. A third load, on the entry written again
without the order of uses, gives another order: the file tells the two
apart, and a load reads the entry.

Refs #279
@SizzleUnrlsd SizzleUnrlsd added the bug Something isn't working label Nov 15, 2026
@SizzleUnrlsd SizzleUnrlsd self-assigned this Nov 15, 2026
The entry of a .ll input now holds the order of each value's uses. An
entry that a build of schema v4 wrote does not, and would be reused as
long as its source is unchanged. The key and the metadata of an entry
hold the schema: under v5, an earlier entry is not looked up, and the
first run writes a new one.

Refs #279
…t reused

The entry written without the order of uses is recorded under schema
v4, as the builds that wrote no order left it. The next load must give
the callers in the order of the parsed text, and leave the entry under
the current schema.

Refs #279
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The compile IR cache of a .ll input does not keep the order of uses

1 participant