Repository navigation
fix(input): keep the order of uses in the cache entry of a .ll input - #280
Draft
SizzleUnrlsd wants to merge 6 commits into
Draft
SizzleUnrlsd wants to merge 6 commits into
SizzleUnrlsd wants to merge 6 commits into
Conversation
A .ll input is parsed from its text by the run that fills the compile IR cache, and from the bitcode of its entry by the runs that read it. That bitcode held no order of uses, so the two modules could list the users of a value another way. The entry is now written with ShouldPreserveUseListOrder. Refs #279
…r of uses The test loads a .ll whose callee is declared after its two callers, then loads it again from the cache, and compares the callers of the callee in the two modules. A third load, on the entry written again without the order of uses, gives another order: the file tells the two apart, and a load reads the entry. Refs #279
The entry of a .ll input now holds the order of each value's uses. An entry that a build of schema v4 wrote does not, and would be reused as long as its source is unchanged. The key and the metadata of an entry hold the schema: under v5, an earlier entry is not looked up, and the first run writes a new one. Refs #279
…t reused The entry written without the order of uses is recorded under schema v4, as the builds that wrote no order left it. The next load must give the callers in the order of the parsed text, and leave the entry under the current schema. Refs #279
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
With the compile IR cache, a
.llgiven as input was parsed from its text by the run that fills the cache, and from bitcode written without the order of uses by the runs that read it (#279). The two modules could list the users of a value in another order. The entry's bitcode now holds that order, and the cache schema moves to v5, so that the entries written before are not reused: the first run after the update writes each entry again.Closes #279
Change
fix(input): keep the order of uses in the cache entry of a .ll input: the entry is written withShouldPreserveUseListOrder. One argument, and a comment that says why.test(unit): check that a .ll input read from the cache keeps the order of uses:testCompileIRCacheKeepsUseOrderOfIRInput, described below.fix(input): move the compile IR cache to schema v5: an entry written before this change holds no order, and is no longer looked up. See The cache schema.test(unit): check that a .ll entry of the previous cache schema is not reused: two more checks in the test.docs(changelog), twice: the order of uses, then the cache filled again once.The test
testCompileIRCacheKeepsUseOrderOfIRInput, in the unit tests, takes the path of a run that fills the cache, then of a run that reads it:.llof three functions throughloadModuleForAnalysis, with a cache directory. This gives the module parsed from the text, and an entry in the cache.calleein the two modules: they must come in the same order.compile-ir-cache-v4, and loads again: the callers must come in the order of the parsed text, and the entry must be back under the current schema.In the file,
calleeis declared after its two callers, as Clang prints declarations. The text parser meets its uses before its declaration. A bitcode reader knows every function before it reads a body, and lists the callers another way unless it is told the order.Step 4 keeps the test from passing for nothing: it fails if the file stops telling the two orders apart, or if a load stops reading the entry.
On main at 1fadba2, the test of steps 1 to 4 fails at step 3,
the module read from the cache lists the callers of a function as the parsed text does, and its three other checks pass. With the order written but the schema still at v4, step 5 fails atan entry of the schema that held no order is not reused, and the five other checks pass. At the head, the six pass.What the order in the bitcode restores
Counted with a probe build that is not in this branch: it compares the module parsed from the text with the module parsed from its bitcode, value by value. It takes the users of every global variable, function, argument, block and instruction, with the operand of each use.
.llfiles-O0 -g.llfiles oftest/This count does not tell constants apart: a user that is a constant counts as a constant and an operand number, and the use lists of the constants themselves are not compared.
LLVM's own check covers them.
verify-uselistorderof LLVM 20.1.2 maps every value of a module, constants included, writes the module as bitcode with the order of uses, reads it back and compares each use list. It then does the same through text. Run with--num-shuffles=0on the file of the test and on the 34 Lua files:use-list order changedonlvm.llandonelua.ll, the two files that holdblockaddressconstants, for the jump table of Lua's interpreter. The cache does not go through that round trip.Reports
On the 34 Lua files,
--jobs=1, one compile IR cache directory per round, empty before the first run, three rounds:The content of the diagnostics is the same in every run, with either build. Their order varies from one run to the next, from an empty cache as from a filled one, so these reports are not the same byte for byte.
No diagnostic was seen to change: this pull request restores the order of the lists, and no report shows it.
The runs of this branch were made at 5f71208, before the schema change, which changes no analysis.
The cache schema
An entry that an earlier build wrote for a
.llinput holds no order of uses, and stayed valid as long as the file, the coretrace-compiler revision and the LLVM version did. The schema moves fromcompile-ir-cache-v4tocompile-ir-cache-v5, with the existing mechanism: the cache key and the metadata of each entry hold the schema, so a build of v5 does not look up an entry of v4.The schema is one for the whole cache. The entries of C and C++ sources are written again too, once.
Checked with two builds on one cache directory, main at 1fadba2 and this branch,
--jobs=1:.llfiles: the file of the test and Lua's 34.llfiles, the 105 files that main wrote have the same checksums, and the directory holds 70 entries, 35 under each schema. main still reuses its own 35.check_compile_ir_cache_rebuilds_previous_schema, covers the metadata of an entry under the current key, for a C source. It passes unchanged.Tests
Local runs, on macOS arm64 with AppleClang and LLVM 20.1.2, this branch built as a user builds it, coretrace-compiler fetched at the pinned revision. They ran on the tree of a3edac9; the commit after it changes the changelog only.
run_test.py --jobs 2 --no-cache, which runs the analyzer on the fixtures: 2760/2760. The script counts each unit test program below as one of the 2760.-DBUILD_ANALYZER_UNIT_TESTS=ON. Both were executed:stack_usage_analyzer_unit_testsends withAll analyzer module unit tests passed.after 402 checks, the 6 of the new test among them, andownership_engine_unit_testswithAll ownership engine unit tests passed.after 116 checks.scripts/format-check.shwith clang-format 20,scripts/check-license-compliance.shand the commit checker pass.CI at 5a32343
an entry of the previous schema is compiled again and rewrittenamong them, and both unit test programs are executed:All analyzer module unit tests passed., with the six checks of the new test, andAll ownership engine unit tests passed.Merging
This branch starts from main and does not depend on #272 or #274.
git merge-treefinds no conflict with either branch, at e6ae53e and dbe9c03. #274 writes the order of uses in the entry of a compiled source; this pull request writes it in the entry of a.llinput.