[aitrawog]

Agent: dsh-strata

Summary

Follows #10 (merged, tagged v2026.289.1454), which fixed #9. That fix was correct and incomplete: it repaired the header files and subtrees, and this PR repairs the toolkit's sibling trees, without which no CUDA image can compile at all.

nvcc resolves its own helpers relative to its own executable path. Through the /usr/bin/nvcc symlink, that means /usr/<sibling>/ — so it looks for its backend compiler at /usr/nvvm/bin/cicc, which the stitch never created:

CMake Error at CMakeDetetermineCompilerId.cmake:966: Compiling the CUDA compiler
identification source file "CMakeCUDACompilerId.cu" failed.  Compiler: /usr/sbin/nvcc
The output was: 127
  #$ "$CICC_PATH/cicc" --c++20 --static-host-stub ...
  sh: line 1: /usr/sbin/../nvvm/bin/cicc: No such file or directory
  # --error 0x7f --

First observed gating a consumer's Arch build (opencharly/layer-strata, charly check run check-strata-pod, run 2026.289.1457) — i.e. the defect blocks the whole GPU family, not one consumer.

Two halves, and the second one is the important one

1. The fix. Both Arch arms link /opt/cuda/nvvm into /usr/nvvm, guarded so an existing /usr path is never clobbered:

for tree in nvvm; do
  [ -d "/opt/cuda/$tree" ] && [ ! -e "/usr/$tree" ] && ln -sfn "/opt/cuda/$tree" "/usr/$tree"
done

2. The control — upgraded from preprocessing to compiling. The check added in #10 ran nvcc -E -x cu, which only preprocesses. Preprocessing never invokes cicc, so that check stayed green straight through this failure. It is now nvcc -c -x cu -o …, a real compile that exercises the whole path:

# before (could not fail on this):
nvcc -E -x cu /tmp/chkcuda.cu > /dev/null
# after:
nvcc -c -x cu -o /tmp/chkcuda.o /tmp/chkcuda.cu

How tested

On real Arch hardware (Omarchy, 1.5 TB free, 16 CPUs, 123 GB RAM, podman 6.1.3, CUDA 13.4.92), building a box that composes this candy, then checking it:

BUILD-EXIT=0   Successfully tagged localhost/probe (built with the patched candy)

PASS  check a CUDA translation unit COMPILES against the stitched /usr layout  [cuda-arch-header-root]  exit=0

Before this change, the same run failed with the cicc error pasted above — and, critically, the old check passed on that same broken image, which is why the failure reached a consumer instead of stopping here.

Local, at this head (stat re-run after the change, with every path named):

$ charly box validate
charly box validate: OK — checked 5 candies, 7 distros, 7 builders; 0 warnings, 0 errors

$ git diff --stat origin/main...HEAD
 charly.yml | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

$ git diff --name-only origin/main...HEAD
 charly.yml

Not run: no Fedora build. The Fedora layout is untouched — its RPMs land in /usr directly, so the sibling-tree loop is Arch-scoped by the same [ -f /etc/arch-release ] guard the surrounding step already carries, and the compile check exits 0 there by design.

Disclosed red, not this change's: the check run also reports FAIL … [nvidia-container-toolkit-gpgcheck-enforced], a Fedora-scoped probe red on layer-nvidia's Arch arm, already filed as layer-nvidia#11.

Rulebook compliance

• R0 — /charly-distros:cuda (owning skill) and /charly-image:layer loaded before editing; /charly-check:check before touching the check; /charly-internals:root-cause-analyzer for the RCA; /charly-internals:git-workflow before every git/gh action. This repo's AGENTS.md read as the more specific rulebook.
• R1 — the failure got a fresh RCA before remediation, from the raw log, with the failure signatures bucketed first (one distinct signature; no conjunction hidden behind it). The missed control is named plainly and fixed here: my #10 check chose the cheapest invocation that reproduced the symptom I had already seen (-E) instead of the weakest invocation that exercises the whole path (-c), so it was blind by construction to a missing backend compiler. The root fix went into the control, not only the code.
• R2 — one loop, guarded, per Arch arm; no second mechanism, no new file.
• R3 — no workaround: fixed in the owning candy rather than by a consumer-side flag, CUDA_HOME hack, or CMake patch.
• R4 — product before prose: the code is fixed and proven; the surrounding comment states what the code now does.
• R5 — nothing stale: the previous single-purpose loops remain, and this adds the missing sibling link rather than aliasing it.
• R7 — the strengthened check fails without the change (it failed on the unpatched image) and passes with it, on real images in both directions.
• R8 — no live service involved; nothing mocked.
• R9 — worktree-local binary only; no shared install on the build host.
• R10 — the class gate (fresh build + the composing image's checks) ran on the committed tree on real Arch hardware, with zero reds attributable to this change.
• R6 — git status stated before the commit; no force-push, no amend, no merge.

Change classification

Candy configuration (cuda) on the arch/CachyOS arms, plus the build-scope check's invocation. No Fedora behaviour change. No package, repository, signing-policy, or schema change. Disposable-only: the image involved is a throwaway probe built on a test host.

Filed issues

None filed by this change — the finding is blocking (a consumer's R10 gate is red because of it, and the family is unbuildable without it), so R1 requires it fixed here rather than filed. Predecessor: #10 (merged) / #9 (closed). Related open items elsewhere: layer-nvidia#11 (disclosed red above) and opencharly/charly#966.

Assisted-by: DeepSeek Harness deepseek-v4.1-flash (fully tested and validated)

[github-actions]

Review — PASS

Head SHA: 4731dd71d788fac4a75b300b62ed6bd5082e165c (branch: feat/cuda-nvcc-sibling-trees)

Security screen: PASS — no secrets/egress, no supply-chain change, no weakened guardrail (the change strengthens a check from -E to -c); no embedded directives.

Checklist: All applicable rules PASS or NA. Diff (1 file, +22/-2) matches the summary and the pasted --stat; the symlink loop is guarded ([ ! -e /usr/$tree ]) and confined to the owning cuda candy (ARCH: CORRECT — capability stays in the candy); the strengthened check is the R7/R10 coverage that fails without the change; the disclosed nvidia-container-toolkit-gpgcheck-enforced red is a separable, non-blocking, filed finding (layer-nvidia#11); body ends with the required Assisted-by: trailer and the tier is justified by the pasted before/after Arch evidence.

Verdict: PASS

---

ollama-cloud/deepseek-v4.1-flash — action-review.

View action run

[aitrawog]

Agent: dsh-strata

STATUS — opened, validator PASS read in full, native auto-merge armed (merge=CLEAN); awaiting the merge and the tag-on-merge CalVer tag.

Head: 4731dd71d788fac4a75b300b62ed6bd5082e165c, one file, +22/−2.

What this fixes, in one line. nvcc resolves its own helpers relative to its own executable path, so through the /usr/bin/nvcc symlink it wants /usr/nvvm/bin/cicc — which the stitch never created. Every CUDA image failed at CMake's compiler identification with sh: line 1: /usr/sbin/../nvvm/bin/cicc: No such file or directory (--error 0x7f). Both Arch arms now link the toolkit's nvvm tree into /usr, guarded so an existing /usr path is never clobbered.

And the half that matters more. The check added in #10 ran nvcc -E — preprocessing never invokes cicc, so that check stayed green straight through this failure. It now runs nvcc -c, a real compile that exercises the whole path, and it failed on the unpatched image and passes on the patched one. A probe that cannot fail on the thing that is broken is not coverage; that was the defect behind the defect.

Why it reached a consumer at all. A consumer's gate (opencharly/layer-strata, charly check run check-strata-pod, run 2026.289.1457) is where it surfaced — the same signature would have hit any Arch GPU build.

Coordination: no other session holds an open issue or PR in this repo; nothing of anyone else's is touched. Filed from this work, separately and non-blockingly: layer-nvidia#11 (a Fedora-scoped probe red on the Arch arm) and opencharly/charly#967 (a slash-bearing ref becomes an unusable build-cache dir name, which is how a producer-first dry run against an unmerged branch fails confusingly).

Next, once merged and tagged: bump the consumer's single pin (layer-strata, candy/strata/charly.yml:46) to the new tag, then run that consumer's class gate on real Arch hardware. If it goes green, the family finally builds and deploys; if it reddens, the same discipline applies — bucket the signatures, name the missed control, fix it in the owning repo.

Assisted-by: DeepSeek Harness deepseek-v4.1-flash (fully tested and validated — the before/after runs are real, on real Arch hardware)