Skip to content

Fix native lib resolution for cross-arch publish (#205)#206

Merged
CoreyKaylor merged 1 commit into
CoreyKaylor:mainfrom
JamestsaiTW:upstream-cross-arch-publish
Jul 13, 2026
Merged

Fix native lib resolution for cross-arch publish (#205)#206
CoreyKaylor merged 1 commit into
CoreyKaylor:mainfrom
JamestsaiTW:upstream-cross-arch-publish

Conversation

@JamestsaiTW

Copy link
Copy Markdown
Contributor

Problem

LightningDB.targets's LightningDBIncludeNativeDll target selects the native lmdb/liblmdb binary based on the build machine's OS/architecture (RuntimeInformation.OSArchitecture / MSBuild.IsOsPlatform), instead of the target RuntimeIdentifier. This breaks dotnet publish -r <rid> when publishing for a different architecture/OS than the host (issue #205).

Fix

The .NET SDK already resolves native binaries correctly per the target RID via the standard NuGet runtimes/{rid}/native package layout + .deps.json runtimeTargets, including in transitive PackageReference scenarios (App -> ProjectReference -> Library -> PackageReference(LightningDB)), where the custom target's host-arch copy could otherwise win the SDK's asset conflict resolution and copy the wrong binary.

To fix this while preserving compatibility for legacy packages.config / non-SDK-style consumers (the original reason this target was added, per be15808, for net45 support), the LightningDBIncludeNativeDll target now only runs when:

  • $(LightningDBEnableNativeCopyTarget) is explicitly set to true (opt-in, used by this repo's own ProjectReference consumers: LightningDB.Tests, SecondProcess, LightningDB.Benchmarks), or
  • The consuming project is NOT using PackageReference (i.e. $(RestoreProjectStyle) != 'PackageReference' and no @(PackageReference) items) — preserving the original copy-to-output behavior for packages.config consumers.

Modern PackageReference consumers now rely entirely on the SDK's built-in RID-based native asset resolution, which correctly handles the target RID (not the build machine's) in all publish scenarios, including transitive references.

Verification

  • Local repro (App -> Library(ProjectReference) -> PackageReference(LightningDB)), published with -r win-x86, -r win-arm64, -r linux-arm64, -r osx-x64 on a win-x64 host — SHA256 hash comparison confirmed each publish output contained the correct RID-specific native binary (not the host's).
  • RID-less local build/run scenario verified native lib loads successfully.
  • dotnet pack verified the packed nupkg still contains build/LightningDB.targets and all native runtime assets unchanged.
  • Simulated legacy/packages.config consumer (-p:RestoreProjectStyle=PackagesConfig) confirmed the copy-to-output behavior still triggers correctly.
  • dotnet test on LightningDB.Tests (net8.0): 137/138 passing (1 pre-existing unrelated failure that also reproduces on main).
  • Full repo solution build confirmed LightningDB.Tests/SecondProcess/LightningDB.Benchmarks (which opt in via LightningDBEnableNativeCopyTarget=true) still get native DLLs copied for local execution.
  • Additionally validated via an automated CI workflow on my fork (not included in this PR) that packed, published, and SHA256-verified the native asset for 7 RIDs (win-x86/x64/arm64, linux-x64/arm64, osx-x64/arm64) in the transitive PackageReference scenario — all passed.
  • Existing .NET Tests CI workflow (ubuntu-latest/windows-latest/macOS-latest) passes on this branch.

Fixes #205

Copilot AI review requested due to automatic review settings July 12, 2026 10:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts how native LMDB binaries are copied/resolved so dotnet publish -r <rid> uses the correct target-RID native asset instead of accidentally copying a host-architecture binary via a custom MSBuild target.

Changes:

  • Gate LightningDBIncludeNativeDll so it only runs for legacy non-PackageReference consumers, or when explicitly opted in.
  • Add LightningDBEnableNativeCopyTarget=true to internal ProjectReference consumers so local execution still gets native binaries copied.
  • Update LightningDB.targets with guidance about relying on SDK RID-based native asset resolution for PackageReference consumers.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
src/SecondProcess/SecondProcess.csproj Opts into legacy native copy target for ProjectReference-based local execution.
src/LightningDB/LightningDB.targets Gates the legacy copy target to avoid overriding SDK RID-based native asset resolution for PackageReference consumers.
src/LightningDB.Tests/LightningDB.Tests.csproj Opts into legacy native copy target for tests when using ProjectReference.
src/LightningDB.Benchmarks/LightningDB.Benchmarks.csproj Opts into legacy native copy target for benchmarks when using ProjectReference.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/LightningDB/LightningDB.targets Outdated
<LightningDBTargetRuntimeRelativePath Condition=" '$(LightningDBTargetRuntimeRelativePath)' == '' ">\..\</LightningDBTargetRuntimeRelativePath>
</PropertyGroup>
<Target Name="LightningDBIncludeNativeDll" AfterTargets="BeforeResolveReferences">
<!-- For PackageReference consumers, the SDK resolves runtimes/{rid}/native assets for the app's RID; set LightningDBEnableNativeCopyTarget=true to force the legacy copy behavior (e.g. packages.config or ProjectReference builds). -->
Comment thread src/SecondProcess/SecondProcess.csproj Outdated
<OutputType>Exe</OutputType>
<TargetFrameworks>net8.0;net9.0;net10.0</TargetFrameworks>
<LangVersion>13</LangVersion>
<LightningDBEnableNativeCopyTarget>true</LightningDBEnableNativeCopyTarget>
<PackageId>LightningDB.Tests</PackageId>
<GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
<IsTestProject>true</IsTestProject>
<LightningDBEnableNativeCopyTarget>true</LightningDBEnableNativeCopyTarget>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>13</LangVersion>
<LightningDBEnableNativeCopyTarget>true</LightningDBEnableNativeCopyTarget>
@CoreyKaylor

Copy link
Copy Markdown
Owner

Thanks for the clear diagnosis and verification work — I agree this is a real bug. The custom target selects the host OS/arch and can win over the SDK’s runtimes/{rid}/native assets during dotnet publish -r …, including the transitive case (App → ProjectReference → PackageReference(LightningDB)).

I’m less inclined to disable LightningDBIncludeNativeDll for all PackageReference consumers, though. We multi-target netstandard2.0, so .NET Framework apps still consume this package. Those often have no RuntimeIdentifier, may be AnyCPU (sometimes with Prefer32Bit), and historically relied on a build/*.targets copy to get lmdb next to the output. Turning the target off for every PackageReference project fixes Core/SDK publish, but risks regressing that Framework path.

I’d prefer to fix the targets file so one mechanism stays correct in both worlds. Local checks suggest the selection policy needs two parts:

  1. Prefer the right architecture (priority order)

  2. $(RuntimeIdentifier) when set → publish / -r / project RID (this is the build/LightningDB.targets copies native lib based on build-machine architecture, not the target RuntimeIdentifier (breaks cross-arch dotnet publish) #205 fix)

  3. Else concrete $(PlatformTarget) (x86 / x64 / ARM64) → VS / Framework platform configs

  4. Else Windows + Prefer32Bit → win-x86 (classic Framework AnyCPU on 64-bit OS)

  5. Else host (NETCoreSdkRuntimeIdentifier and/or IsOsPlatform + OSArchitecture) for RID-less local runs

Copy only runtimes/$(effectiveRid)/native/* (with a small alias for osx-x64 → our existing runtimes/osx/ layout). Drop or hard-opt-in the “Windows Arm64 host → win-x64” Parallels convenience path so a real win-arm64 RID isn’t overridden.

  1. Don’t host-copy from RID-less class libraries

RID-first alone is not enough for the transitive repro.

The targets import into the direct package consumer. Intermediate class libraries usually build with empty RuntimeIdentifier, so a host-arch flat CopyToOutputDirectory still runs and can poison the final app’s publish output even when the app is published with -r .

Select the native runtime asset using RuntimeIdentifier, then PlatformTarget,
Prefer32Bit, then the build host. Map osx-x64 to the package's osx layout.
Skip host-native flat copies for RID-less class libraries so they cannot
poison cross-RID publish of a consuming app.

Fixes CoreyKaylor#205
@CoreyKaylor CoreyKaylor force-pushed the upstream-cross-arch-publish branch from fc213fa to 4924dac Compare July 13, 2026 19:33
@CoreyKaylor CoreyKaylor merged commit 70ca262 into CoreyKaylor:main Jul 13, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

build/LightningDB.targets copies native lib based on build-machine architecture, not the target RuntimeIdentifier (breaks cross-arch dotnet publish)

3 participants