Repository navigation
feat(compilerlib): compile to an llvm::Module in the caller's context - #265
Merged
Merged
Conversation
compile() built its context and ran the whole compilation itself. The compilation now runs in runCompilation, from the context it is given, which compileToModule will share. Refs #263
The analyzers compile each source to textual IR or bitcode, then read it back into a module of their own: the module was written and parsed again for nothing. compileToModule(args, context, instrument) gives the module itself, created in the caller's context, with the arguments of an LLVM IR compilation (-S -emit-llvm or -c -emit-llvm). The output file is not written; the other files the arguments ask for, such as a dependency file, are. The module is the one compile() writes, instrumented or not. Code generation sets whether the context discards value names; the context's own setting is restored, since it applies to the modules read into it later, and so is the optimization record that Clang's action leaves on it before LLVM 22. Refs #263
compileToModule gives the module that compile() writes as text, at -O0 and -O2, instrumented or not, and the module of a bitcode compilation, whose output file is not written and whose dependency file is. It fails on a compilation that is not to LLVM IR, reports the errors of a broken source, and leaves the context's setting of value names. Refs #263
With LLVM 16, destroying a module of compileToModule freed memory twice
when the compilation named values and the context, restored after it,
discarded names: LLVM 16's Value::setName("") does nothing in such a
context, so Function::clearArguments leaves each argument's name in
the function's symbol table, which frees it again. LLVM 17 removes the
name. With LLVM 16, a context that discards names now keeps them after
a compilation that names values. The Linux / LLVM 16 job of the CI
aborted in ModuleLeavesTheValueNamesSettingOfTheContext.
Refs #263
compileToModule restored the host's context from the job itself. Once the host enabled crash recovery, a crash leaves the job where it stood: the context kept the job's setting of value names. The context is now restored in runToModule, which the job always returns to. Refs #263
Clang's backend replaces the context's diagnostic handler with one that refers to the compilation, and returns without putting the previous one back when it cannot set up the optimization record or link a module in. After such a compileToModule, the host's handler was destroyed, and the next diagnostic of the context went to the compilation that had ended: a segmentation fault. The host's handler is now set aside for the job, which runs with LLVM's default one, and put back when the job ends. Refs #263
The compilation fails on an optimization record whose directory does not exist. The context must still have the host's handler, report to it, and take another compilation. Before the fix, the context was left with the handler of Clang's backend. Refs #263
Instrumented, compilerlib sets up the optimization record itself. This path already put the handler back. Refs #263
The other early return of Clang's backend: a module given with -mlink-bitcode-file defines the unit's symbols again. Refs #263
The source crashes Clang once the host enabled crash recovery. The context must keep the host's setting of value names and its diagnostic handler. Before the fix, it kept the job's setting of value names. Refs #263
main brings the working directory of each compilation (#262) and the note about an allocation in a do-while loop (#260). Conflicts: - src/compilerlib/compiler.cpp: this branch runs compile() through runCompilation(). What #262 changed in the body of compile(), the move of the process for a compilation with a job that runs as a program, is applied there. - test/unit/compile_test.cpp: both includes are kept. - README.md and CHANGELOG.md: both sections and both entries are kept.
compileToModule from eight threads, each with a context and a project of its own, whose headers have the same name: each module comes from its own header, and the process's working directory stays where it is, instrumented or not. The test fails when the compilations share the process's file system, as they did before #262. Refs #263
SizzleUnrlsd
marked this pull request as ready for review
November 16, 2026 15:30
Merged
SizzleUnrlsd
added a commit
to CoreTrace/coretrace-stack-analyzer
that referenced
this pull request
Nov 17, 2026
The pinned commit was one of the branch of CoreTrace/coretrace-compiler#265. v0.13.0 is the release that holds compileToModule. Refs #273
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The CoreTrace analyzers compile each source to textual IR or bitcode, then parse it into a module of their own (#263).
compileToModule(args, context, instrument), in a new public headercompilerlib/compile_module.hpp, gives the module itself, created in anllvm::LLVMContextof the host. The output file is not written; the other files the arguments ask for, such as a dependency file, are.Closes #263
Change
refactor(compilerlib): run compile() through runCompilation: the compilation runs from the context it is given.compiledoes not change.feat(compilerlib): compile to an llvm::Module in the caller's context:compileToModuletakes the arguments of an LLVM IR compilation,-S -emit-llvmor-c -emit-llvm, and fails on any other;compilealready uses for in-memory output; withinstrument, the module is instrumented first;test(compilerlib): check the modules compileToModule gives: the module prints as the textcompilegives, at -O0 and -O2, instrumented or not; a bitcode compilation writes its dependency file and no output; a compilation to an object and a broken source fail; the context keeps its setting of value names.docs(readme): describe compiling to a module.fix(compilerlib): keep value names in an LLVM 16 context given some, with its README and changelog notes. The Linux / LLVM 16 job of this pull request aborted inModuleLeavesTheValueNamesSettingOfTheContext:double free or corruption. With LLVM 16,Value::setName("")does nothing in a context that discards names, so destroying a module whose values have names, after the context was set back to discarding them, left each argument's name in its function's symbol table, which freed it again. LLVM 17 removes the name (16.0.6 against 17.0.6,Value::setNameImpl). With LLVM 16, a context that discards names now keeps them after a compilation that names values; the test expects that with LLVM 16, and the restored setting from LLVM 17 on.fix(compilerlib): give a module's context its diagnostic handler back, found in review, with its header, README and changelog notes. Clang's backend replaces the context's diagnostic handler with one that refers to the compilation, and returns without putting the previous one back when it cannot set up the optimization record of-fsave-optimization-record, or cannot link in a module of-mlink-bitcode-file(BackendConsumer::HandleTranslationUnit; 16.0.6, 18.1.8 and 22.1.0 do the same). The host's context kept the handler of a compilation that had ended, and the next diagnostic the host emitted crashed: a segmentation fault on macOS arm64.compileToModulenow sets the host's handler aside before the job and puts it back once the job ends, however it ends. The job runs with LLVM's default handler, as a compilation in a context of its own does.fix(compilerlib): restore a module's context after a recovered crash: the scope that restores the context is on the frame the job returns to, whatever happens to the job. After a crash that the host recovered, the context kept the job's setting of value names.test(compilerlib), six tests of the host's handler: it is the context's, and is called for the host's next diagnostic, after a compilation that succeeds; after an optimization record file that cannot be opened, an unknown record format, an instrumented compilation that fails and a module that cannot be linked in, each followed by a compilation that succeeds in the same context; and after a crash that the host recovers, where the context has its setting of value names too.Two limits of the handler fix:
LLVMContext::setDiagnosticHandleralso resets whether the context respects diagnostic filters, which LLVM gives no way to read. A host that had set it finds it reset, as it already did after a compilation that succeeded, since Clang's backend puts the handler back the same way.This branch comes from commit 2c1c83b of the experimental branch, without what it took from the working directory change, which #262 brought separately.
Integration of #262. #262 is merged, and main (e6d5c6c) is merged into this branch (be9bd28). The two changed
compileincompiler.cpp:src/compilerlib/compiler.cpp: this branch runscompile()throughrunCompilation(). What feat(compilerlib): give each compilation a working directory of its own #262 changed in the body ofcompile(), the move of the process for a compilation with a job that runs as a program, is applied there.compileToModulecreates its compiler instance ascompiledoes for an output in memory, so it reads and writes through the file system of its compilation.test/unit/compile_test.cpp,README.md,CHANGELOG.md: both sides are kept.test(compilerlib): check modules compiled from several threads at once:compileToModulefrom eight threads, each with a context and a project of its own, whose headers have the same name. Each module comes from its own header, and the process's working directory stays where it is, instrumented or not. The test fails when the compilations share the process's file system, as they did before feat(compilerlib): give each compilation a working directory of its own #262.docs: the header, the README and the changelog say that-working-directoryand compilations from several threads are as forcompile, each compilation with a context of its own.A compilation that
compileToModulerefuses for its jobs, with-working-directory, leaves the process in that directory, ascompiledoes for an output in memory, and as main did before #262: #267.Measurements
coretrace-stack-analyzer db7fb25, with coretrace-compiler built from main (f0a22d9), against the same analyzer taking each module from
compileToModulein its bitcode mode, instead of compiling to a temporary.bcfile, reading it and parsing it. That analyzer change is CoreTrace/coretrace-stack-analyzer#274, a draft; the measured build predates the log line it adds and the LLVM 16 fix of this branch, which LLVM 20 does not compile. macOS arm64 (M3, 8 cores), LLVM 20.1.2,--format=json --timing, no IR cache, an empty resource summary cache. Medians of 5 runs, 3 for the 53 sources; the builds alternate within each repetition.--jobs--jobs=1, takes 8 % less time on both sets of C++ sources and 15 % less on Lua's. The analyzer still compiles one source at a time here, whatever--jobssays; feat(compilerlib): give each compilation a working directory of its own #262 is the change that lets it compile in parallel.--jobs=1).Tests
Local runs, on macOS arm64 with AppleClang:
ctest -j2ModuleLeavesTheValueNamesSettingOfTheContextaborts:pointer being freed was not allocated, in~ValueSymbolTablefrom~Module, as on the CI's Linux / LLVM 16 job.ctestctest -j2307/307; the 213 unit tests of compilerlib pass in a shuffled order.ctest -j2309/309: the 293 of main plus 16 tests of this branch. The 215 unit tests of compilerlib pass in a shuffled order, and the new test five times in a row.Each commit builds and passes the unit tests with LLVM 20; of the 12 commits of the review fixes, the 9 that change code or tests were each built and tested. With LLVM 16,
ModuleLeavesTheValueNamesSettingOfTheContextfails from the commit that adds it until the fix. clang-format 18,scripts/check-license-compliance.shand the commit checker pass.CI at 6eb4646