Adds support for special magic LDC_inline_mlir pragma support by asindarov · Pull Request #5295 · ldc-developers/ldc

asindarov · 2026-09-28T16:47:35Z

No description provided.

…tin llvm to mlir type translator and add tests for inlineMLIR feature

… inlinemlir.cpp

… declaration in inlinemlir

asindarov · 2026-10-07T15:22:23Z

Hello, I've been working on adding MLIR as a dependency to LDC while working on implementing inlineMLIR feature. I use macOS and so far locally it worked fine but when I pushed the changes, some of the pipeline workflows failed with linker errors. Then I looked up the cmake and pipeline code and found out that in the CI it tries to build LDC with LDC_LINK_MANUALLY=OFF and on my macOS this flag will be ON by default and so far everything worked fine until I set LDC_LINK_MANUALLY=OFF.

MLIR is now opt-in (LDC_WITH_MLIR defaults to OFF), so regular CI builds are green. The failure only happens with -DLDC_WITH_MLIR=ON and -DLDC_LINK_MANUALLY=OFF.

It appears when LDC_LINK_MANUALLY=ON, cmake uses built-in target_link_libraries function to link the target libraries. Otherwise with LDC_LINK_MANUALLY=OFF it seem to use custom linker logic.

During a call with @thewilsonator to investigate and fix it, we replaced the target names that we got from MLIR with plain -l flags built from the global properties set in MLIRConfig.cmake(The properties MLIR_DIALECT_LIBS, MLIR_TRANSLATION_LIBS, MLIR_CONVERSION_LIBS come from MLIRConfig.cmake).

Unfortunately, it still caused errors as follows:

 "mlir::FloatType::getFloatSemantics() const", referenced from:
      mlir::arith::ExtFOp::fold(mlir::arith::ExtFOpGenericAdaptor<llvm::ArrayRef<mlir::Attribute>>) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::TruncFOp::fold(mlir::arith::TruncFOpGenericAdaptor<llvm::ArrayRef<mlir::Attribute>>) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::BitcastOp::fold(mlir::arith::BitcastOpGenericAdaptor<llvm::ArrayRef<mlir::Attribute>>) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::getIdentityValueAttr(mlir::arith::AtomicRMWKind, mlir::Type, mlir::OpBuilder&, mlir::Location, bool) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::getIdentityValueAttr(mlir::arith::AtomicRMWKind, mlir::Type, mlir::OpBuilder&, mlir::Location, bool) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::getIdentityValueAttr(mlir::arith::AtomicRMWKind, mlir::Type, mlir::OpBuilder&, mlir::Location, bool) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::getIdentityValueAttr(mlir::arith::AtomicRMWKind, mlir::Type, mlir::OpBuilder&, mlir::Location, bool) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      ...
  "mlir::OpaqueLoc::getFallbackLocation() const", referenced from:
      mlir::LLVM::detail::DebugTranslation::translateLoc(mlir::Location, llvm::DILocalScope*, llvm::DILocation*) in libMLIRTargetLLVMIRExport.a[2](DebugTranslation.cpp.o)
  "mlir::TypedAttr::getType() const", referenced from:
      mlir::arith::ConstantOp::verify() in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::ConstantOp::verify() in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::ConstantOp::isBuildableWith(mlir::Attribute, mlir::Type) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::AddIOp::fold(mlir::arith::AddIOpGenericAdaptor<llvm::ArrayRef<mlir::Attribute>>) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::AddIOp::fold(mlir::arith::AddIOpGenericAdaptor<llvm::ArrayRef<mlir::Attribute>>) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::AddUIExtendedOp::fold(mlir::arith::AddUIExtendedOpGenericAdaptor<llvm::ArrayRef<mlir::Attribute>>, llvm::SmallVectorImpl<mlir::OpFoldResult>&) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::AddUIExtendedOp::fold(mlir::arith::AddUIExtendedOpGenericAdaptor<llvm::ArrayRef<mlir::Attribute>>, llvm::SmallVectorImpl<mlir::OpFoldResult>&) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      mlir::arith::AddUIExtendedOp::fold(mlir::arith::AddUIExtendedOpGenericAdaptor<llvm::ArrayRef<mlir::Attribute>>, llvm::SmallVectorImpl<mlir::OpFoldResult>&) in libMLIRArithDialect.a[2](ArithOps.cpp.o)
      ...
  "vtable for mlir::DialectExtensionBase", referenced from:
      mlir::DialectExtension<bool mlir::DialectRegistry::addExtension<mlir::affine::AffineDialect>(void (*)(mlir::MLIRContext*, mlir::affine::AffineDialect*))::Extension, mlir::affine::AffineDialect>::clone() const in libMLIRAffineDialect.a[5](ValueBoundsOpInterfaceImpl.cpp.o)
      mlir::DialectExtension<bool mlir::DialectRegistry::addExtension<mlir::BuiltinDialect>(void (*)(mlir::MLIRContext*, mlir::BuiltinDialect*))::Extension, mlir::BuiltinDialect>::clone() const in libMLIRMemRefDialect.a[3](MemRefMemorySlot.cpp.o)
      mlir::DialectExtension<bool mlir::DialectRegistry::addExtension<mlir::arm_neon::ArmNeonDialect>(void (*)(mlir::MLIRContext*, mlir::arm_neon::ArmNeonDialect*))::Extension, mlir::arm_neon::ArmNeonDialect>::clone() const in libMLIRArmNeonToLLVMIRTranslation.a[2](ArmNeonToLLVMIRTranslation.cpp.o)
      mlir::DialectExtension<bool mlir::DialectRegistry::addExtension<mlir::arm_sme::ArmSMEDialect>(void (*)(mlir::MLIRContext*, mlir::arm_sme::ArmSMEDialect*))::Extension, mlir::arm_sme::ArmSMEDialect>::clone() const in libMLIRArmSMEToLLVMIRTranslation.a[2](ArmSMEToLLVMIRTranslation.cpp.o)
      mlir::DialectExtension<bool mlir::DialectRegistry::addExtension<mlir::arm_sve::ArmSVEDialect>(void (*)(mlir::MLIRContext*, mlir::arm_sve::ArmSVEDialect*))::Extension, mlir::arm_sve::ArmSVEDialect>::clone() const in libMLIRArmSVEToLLVMIRTranslation.a[2](ArmSVEToLLVMIRTranslation.cpp.o)
      mlir::DialectExtension<bool mlir::DialectRegistry::addExtension<mlir::amx::AMXDialect>(void (*)(mlir::MLIRContext*, mlir::amx::AMXDialect*))::Extension, mlir::amx::AMXDialect>::clone() const in libMLIRAMXToLLVMIRTranslation.a[2](AMXToLLVMIRTranslation.cpp.o)
      mlir::DialectExtension<bool mlir::DialectRegistry::addExtension<mlir::gpu::GPUDialect>(void (*)(mlir::MLIRContext*, mlir::gpu::GPUDialect*))::Extension, mlir::gpu::GPUDialect>::clone() const in libMLIRGPUToLLVMIRTranslation.a[2](GPUToLLVMIRTranslation.cpp.o)
      ...
   NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "vtable for mlir::SimpleAffineExprFlattener", referenced from:
      canonicalizeMapExprAndTermOrder(mlir::AffineMap&) in libMLIRAffineDialect.a[3](AffineOps.cpp.o)
   NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "vtable for mlir::Pass", referenced from:
      mlir::Pass::~Pass() in libMLIRArithTransforms.a[8](ExpandOps.cpp.o)
   NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
ld: symbol(s) not found for architecture arm64

It appears some of the dependencies are not linked properly when -DLDC_WITH_MLIR=ON and LDC_LINK_MANUALLY=OFF flag is used while building the compiler, Nicholas suggested to get your opinion @kinke. Thanks in advance!

asindarov · 2026-10-07T15:24:42Z

+        set(MLIR_LIBS ${MLIR_DIALECTS} ${MLIR_TRANSLATION_LIBS}
+            ${MLIR_CONVERSION_LIBS})
+
+        list(TRANSFORM MLIR_LIBS PREPEND "-l")


This is the workaround we tried for linking MLIR with LDC_LINK_MANUALLY=OFF (plain -l flags instead of CMake target names). It makes ldmd2 link, but ldc2 still fails with undefined symbols. Full explanation and linker errors: here

kinke · 2026-10-07T23:41:22Z

MLIR is now opt-in (LDC_WITH_MLIR defaults to OFF), so regular CI builds are green.

Hmm, do you still have results for the autodetection? As I guess most CI jobs use an LLVM without MLIR anyway, so I'm wondering if it is actually autodetected/available anywhere, so that we can actually CI-test the MLIR stuff in at least one job.

It appears some of the dependencies are not linked properly when -DLDC_WITH_MLIR=ON and LDC_LINK_MANUALLY=OFF flag is used while building the compiler

Any transitive-only CMake deps would e.g. not be linked with LDC_LINK_MANUALLY=OFF, and the ordering might not account for dependencies inbetween these libs either (if the Apple linker is order-sensitive; I don't know) - since with LDC_LINK_MANUALLY, we just forward these MLIR_LIBRARIES to the D compiler, prefixed with -L (=> pass to the linker). So the 'manual' -l prefix as tentative workaround is needed, on Posix at least, if what we get are CMake target names (corresponding to the lib names). Does MLIR_ALL_LIBS get past the linker errors (since it's most likely the complete list, and potentially even in order)? If not, you could e.g. compare the ldc2 link command-line differences between LDC_LINK_MANUALLY=OFF and LDC_LINK_MANUALLY=ON modes, which should give the answer.

Note that I'd be fine with requiring LDC_LINK_MANUALLY=ON for MLIR support for now (and bailing out in CMake accordingly), it's the default mode anyway.

…ng ldmd2 compiler linker

asindarov · 2026-10-09T16:17:07Z

Hmm, do you still have results for the autodetection? …

From what I can see, only the FreeBSD x86_64 job has an LLVM with MLIR. With AUTO, MLIR was detected there and the build went through with LDC_LINK_MANUALLY=OFF. As you guessed, none of the other jobs' LLVMs include MLIR.

Does MLIR_ALL_LIBS get past the linker errors? …

Not directly: I found out that MLIR_ALL_LIBS also contains target names that have no static library behind them (obj libraries like obj.MLIRCAPIIR, and a few shared runtime libs) because otherwise I was getting linker errors, so -l failed for those. After filtering it down to STATIC_LIBRARY targets and prefixing those with -l, ldc2 built and linked with success in both LDC_LINK_MANUALLY=OFF and ON modes locally and the MLIR tests passed. But I have not verified with GNU ld though.

On FreeBSD ldc2 builds and the MLIR tests pass too, but some of lit tests are failing with: Undefined symbol "_ZTVN4llvm2cl6OptionE". They passed on master branch's last CI run, but I am yet not sure if it is because of my MLIR changes or not. I am still investigating the root cause. Thanks for your input.

…efineBuildJitRT.cmake by llvm_set_libs macro which tries to call executable

asindarov · 2026-10-09T17:40:59Z

I found the root cause, it turns out when we use find_package with config mode, it overwrites <PackageName>CONFIG.cmake cmake variable which in our case LLVM_CONFIG which in turn was used in llvm_set_libs macro that is used in 2 places, in FindLLVM.cmake and DefineBuildJitRT.cmake. When Jit was turned on, it would try to execute llvm-config but fail to do so because ${LLVM_CONFIG} would be set to the MLIRConfig.cmake file path. So I saved ${LLVM_CONFIG} value before calling find_package(MLIR...) and set it back to the old value that points to the llvm-config executable. Afterwards CI was successfull. Turns out in some of the Vanilla LLVM CI jobs LLVM with MLIR built is used and they are failing, most likely because I used different version of LLVM and the one in the CI has different version that includes broken changes or something, I will look into them.

…g used

… tests

asindarov · 2026-10-09T22:45:23Z

It all seem to work on CI jobs with LLVM that is built with MLIR. I will better split this PR into 2 parts 1) Adds CMake changes and revive old MLIR stubs, 2) Adds inlineMLIR feature.

asindarov added 8 commits September 28, 2026 18:42

Add special magic LDC_inline_mlir pragma support

1040ad1

Fix return void issue and fix mlir ir type issue by using mlir's buil…

f7dab2d

…tin llvm to mlir type translator and add tests for inlineMLIR feature

remove unnecessary comment in FindMLIR.cmake

3ca7b64

move mlir include under #if LDC_MLIR_ENABLED directive

f16ff1e

add more codegen tests for __mlir runtime function feature

2099c16

Add license header and wrap the code in #if MLIR_ENABLED directive in…

ef40e67

… inlinemlir.cpp

Make mlir optional in the cmake build and fix the wrong #if directive…

dfdac41

… declaration in inlinemlir

Add failing cmake changes when -DLDC_LINK_MANUALLY=OFF

c9b2668

asindarov commented Oct 7, 2026

View reviewed changes

make MLIR auto in cmake and only load mlir static libraries while usi…

90eb4c7

…ng ldmd2 compiler linker

restore the overwritten variable to the executable as it is used in D…

00a6d17

…efineBuildJitRT.cmake by llvm_set_libs macro which tries to call executable

asindarov added 2 commits October 9, 2026 23:11

Fix name of mlir pass conversion api break based on llvm version bein…

08f4f5d

…g used

relax the ordering of phi node arguments in lit inline_mlir.d codegen…

3fcd877

… tests

Navigation Menu

Repository navigation

Adds support for special magic LDC_inline_mlir pragma support - #5295

Adds support for special magic LDC_inline_mlir pragma support#5295
asindarov wants to merge 12 commits into
ldc-developers:masterfrom
asindarov:inline-mlir-support

asindarov commented Sep 28, 2026

Uh oh!

asindarov commented Oct 7, 2026

Uh oh!

asindarov Oct 7, 2026

Uh oh!

kinke commented Oct 7, 2026

Uh oh!

asindarov commented Oct 9, 2026 •

edited

Loading

Uh oh!

asindarov commented Oct 9, 2026 •

edited

Loading

Uh oh!

asindarov commented Oct 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

asindarov commented Sep 28, 2026

Uh oh!

asindarov commented Oct 7, 2026

Uh oh!

asindarov Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

kinke commented Oct 7, 2026

Uh oh!

asindarov commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asindarov commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asindarov commented Oct 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

asindarov commented Oct 9, 2026 •

edited

Loading

asindarov commented Oct 9, 2026 •

edited

Loading