feat(pkgs): add compat.opencv (source-built via install() CMake) + floor 0.0.87#67
Merged
Conversation
…a install() CMake) OpenCV 4.13.0 as a curated-module static build, driven by its own CMake through an install() hook (compat.openblas pattern, CMake instead of Make). mcpp's "list the .cpp" model can't express OpenCV's generated SIMD-dispatch / OpenCL-kernel / config headers, so CMake produces them; the recipe is host-free (ecosystem xim:cmake/make/ gcc only, bundled 3rdparty zlib/png/jpeg, WITH_ADE=OFF kills the sole download — verified offline under `unshare -rn`). - pkgs/c/compat.opencv.lua — build_dep-resolved tools, install() curated CMake build, linux/macos link flags; Windows deferred (anchor stub). Plain-string GLOBAL url (no mcpp-res write access this session; CN mirror is a maintainer follow-up). - tests/examples/opencv — Mat + cvtColor(BGR2GRAY) + imencode/imdecode PNG round-trip assertion; linux-gated workspace member. - Floor + CI pin → 0.0.87: this package needs the mcpp 0.0.87 fix (global tools no longer mis-scoped into the project store); on ≤0.0.86 the build-dep cmake's ELF interpreter dangles. index.toml min/latest_mcpp + validate.yml MCPP_VERSION. - Design/impl/root-cause docs under .agents/docs/. Verified end-to-end with mcpp 0.0.87: `mcpp test` → `test result ok. 1 passed`. Claude-Session: https://claude.ai/code/session_01Y3KE7MAE8yV7BG35eXy7FV
…0.0.87 The env MCPP_VERSION was bumped but the per-OS matrix archive/root (lines 138-151) still said mcpp-0.0.85-*, so the download URL became v0.0.87/mcpp-0.0.85-* → 404. Claude-Session: https://claude.ai/code/session_01Y3KE7MAE8yV7BG35eXy7FV
…ack pattern) The member BUILD phase (mcpp test --workspace runs build+test per member) compiled tests/roundtrip.cpp with -DHAVE_OPENCV from [target.cfg(linux).build] cxxflags but WITHOUT the cfg(linux) opencv dep's include path -> opencv2/core.hpp not found. Match the gui-stack precedent: guard the test on the builtin __linux__ and drop the custom define + [target.build] section, so the build phase is a clean no-op and the test phase compiles with the dep includes. Verified from a clean state (fresh toolchain + fresh opencv source build, released mcpp 0.0.87): mcpp build RC=0, mcpp test roundtrip ok.
CI workspace(linux) failed: opencv install() produced no headers (consumer compile hit 'opencv2/core.hpp: No such file'), yet only 16s elapsed after the tarball download — the CMake build never ran to completion, and xim's interface mode suppressed the subprocess output so the real error was invisible. Root cause (per this descriptor's own header comment): the raw xim:cmake binary is glibc-DYNAMIC; its ELF interpreter points at xim:glibc's loader, wired only through xim's PATH launcher. Invoking it by ABSOLUTE path (the old dep_bin) works on a warm host but fails 'cannot execute: required file not found' on a cold CI runner. Fix: invoke cmake/make/gcc/g++ by BARE name so the loader-wired launchers are used (xim:make is musl-static so it tolerated absolute paths — that's why compat.openblas never hit this). Also: verify installed headers (include/opencv4/opencv2/*.hpp), not just libs, so a partial install fails loudly here instead of at the consumer; and dump mcpp_opencv_build.log (or PATH) on any failure so CI shows ground truth. Verified locally from a clean MCPP_HOME + vendored xlings + GLOBAL mirror: fresh opencv build, headers materialise, roundtrip ok.
…failure xim's interface mode suppresses install() subprocess + log.error output, so the CMake hook's real failure is invisible in CI (only symptom: fetch exit 1). Add an if:failure() Linux step that cats mcpp_opencv_build.log and inspects the xim:cmake binary/interpreter + xim:glibc loader on the runner. Remove before merge.
…ct cmake install) Prior diag looked only under $HOME/.mcpp, but the vendored xlings installs into its own home ($root/registry). Search the whole runner for xim-x-cmake + opencv build log, and directly run the vendored xlings 'install cmake@4.0.2' with visible output to capture the real xim:cmake failure (its install error is otherwise suppressed). Remove before merge.
…t/libm) Ground truth from CI (via the temp diagnostic): xim:cmake installs fine; the real failure is cmake's compiler check cannot LINK a test exe — ld: cannot find crt1.o / crti.o / -lm All three live in <xim:glibc>/lib. xim:gcc's specs wire xim:glibc only for RUNTIME (rpath/dynamic-linker), not the link-time startfile/library search — mcpp's build supplies that via LIBRARY_PATH, which an install() subprocess doesn't inherit. On a dev host gcc silently fell back to /usr/lib crt/libc (works but NOT host-free); the minimal CI runner has no libc-dev so it hard-fails. openblas dodged it (archives a .a, never links an exe). Fix: resolve xim:glibc (+ xim:gcc) install dirs via pkginfo and export LIBRARY_PATH=<glibc>/lib:<gcc>/lib64 for the cmake configure/build/install steps — truly host-free. Declare xim:glibc@2.39 explicitly so pkginfo can resolve it.
With the glibc LINK env fixed, cmake configure advances but errors in OpenCVDetectPython: it finds xlings' python3 shim, reads an EMPTY PYTHON_VERSION_STRING, and calls find_package with an invalid 'OFF' argument -> hard configure error. We build no python bindings, so skip detection entirely (the module return()s early).
…free headers) With LIBRARY_PATH fixed, the build advances to compiling 3rdparty (libjpeg-turbo, zlib) then fails: 'stdlib.h / limits.h: No such file'. Same gap as the LINK env but for HEADERS — gcc's specs don't add xim:glibc's include (nor xim:linux-headers uapi) to the search path; mcpp's build supplies it via CPATH, not inherited by install(). Export CPATH=<glibc>/include:<linux-headers>/include for the cmake steps and declare xim:linux-headers@5.11.1 so pkginfo can resolve it. Fully host-free.
Member
Author
待评估 / pending evaluation — 暂不合入CI xlings 执行
正确的修复层级是生态(xlings/xim),不是每个源码构建 descriptor 各自硬接 决定:标记待评估,暂不合入。opencv 的落地依赖 xlings 侧补齐「install() hook 继承工具链构建环境」这一能力后再回来清理。分支上还有一个临时 CI 诊断步骤(commit |
Root-cause writeup of the ecosystem gap behind opencv PR #67 (待评估): xlings runs install() hooks in a subprocess that does NOT inherit the toolchain BUILD-time env (LIBRARY_PATH/CPATH), so xim:gcc can't find glibc's crt1.o/libm (link) or stdlib.h/limits.h (compile) on a host-free runner. Distinct from D1 (runtime loader resolution). Correct fix is in xlings/xim (inject build-env, or extend gcc specs), not per-descriptor hand-wiring with hardcoded versions.
Read xim-x-gcc-specs-config/gcc-specs-config.lua: it injects only runtime wiring (rpath + dynamic-linker), confirming the build-time gap. Add implementation-ready injection points for both fix options (A: xlings install-hook env export; B: extend the gcc specs generator). Notes both need maintainer C++/Lua + their verify loop.
…hain Turns the install()-env gap design (doc ④) into an executable, dependency-ordered plan: S1 libxpkg hook-env → S2 xlings toolchain-dir resolution + pin → S3 cold verify → S4 descriptor/CI cleanup → S5 opencv un-draft+merge → S6 optional CN mirror. Documents the version-carry rule and why the critical path is inherently serial (each step's artifact feeds the next), not multi-agent parallelizable.
Update the step plan to the verified reality: the install()-env fix is a single xlings change (extend the existing per-hook build-dep env injection to feed build-deps' lib/->LIBRARY_PATH and include/->CPATH), NOT the two-repo libxpkg+xlings plan. Compile-verified against pinned libxpkg 0.0.42 (mcpp build exit 0) + code-path confirmed via env marker. Branch fix/install-hook-toolchain-build-env (unpushed). S4 corrected: keep the xim:glibc/linux-headers build-dep DECLARATIONS (they drive the injection), drop only the descriptor's env hand-wiring.
…hardcoded versions)
Per architecture review: the C build env belongs in the hook itself (the scenario
tool), NOT in xlings (generic infra, PR #355 closed) nor mcpp. The install() hook
already sets LIBRARY_PATH/CPATH before invoking cmake; the only wart was the
hardcoded versions in pkginfo.install_dir('xim:glibc','2.39').
Fix: resolve each build-dep's dir via pkginfo.build_dep('glibc'/'gcc'/'linux-headers')
— xlings resolves the version from the build_deps DECLARATION (via
XLINGS_BUILDDEP_*_PATH), returning {path,bin,include,lib}. No version is repeated in
install() code. xlings/mcpp untouched.
Verified locally: fresh opencv build → compiler check passes (crt/headers found via
build_dep), full core+imgproc+imgcodecs build, roundtrip ok.
…wrapper
CI got past glibc headers (stdlib.h found) but failed on <linux/limits.h> that
glibc's local_lim.h pulls: xim:linux-headers is a WRAPPER (empty install_dir; real
kernel headers live in scode:linux-headers, its dep). So build_dep('linux-headers')
off the xim wrapper gave an empty include/ — locally the host's /usr/include/linux
masked it, on the CI runner it hard-failed. Declare scode:linux-headers@5.11.1 so
XLINGS_BUILDDEP_LINUX_HEADERS_PATH points at the real payload and build_dep resolves
its include/ (has linux/limits.h).
…<stdlib.h>)
Build advanced (glibc + kernel headers found, 21% built) then hit:
gcc/include/c++/16.1.0/cstdlib: #include_next <stdlib.h>: No such file.
CPATH adds glibc/include early (like -I); gcc's C++ cstdlib does
#include_next <stdlib.h>, which searches dirs AFTER gcc's own C++ include — so it
skips the early CPATH entry, and on the CI runner the sysroot lacks stdlib.h.
Switch header injection from CPATH to CFLAGS/CXXFLAGS='-idirafter <dir>' so glibc/
kernel includes land at the END of the search order (verified via g++ -E -v: glibc
sits after the C++ dirs, before nothing — #include_next resolves). Matches mcpp's
linkmodel ('GCC needs -idirafter'). LIBRARY_PATH (libs) unchanged.
…er needed) The if:failure() diagnostic (added while root-causing the host-free build) surfaced the build log + cmake/loader state. opencv now builds host-free green via the descriptor-side env (pkginfo.build_dep + scode:linux-headers + -idirafter), so the debug step is removed.
Multi-platform step 1 (macOS). Per-platform compiler in install(): macOS uses clang/clang++ (xim:llvm, libc++) to match the macOS default consumer ABI (gcc would clash). The Linux-only glibc/-idirafter host-free env is gated off for macOS (clang finds the system SDK itself). Test project + roundtrip ungated for __APPLE__. Windows still stubbed. First CI probe to reveal macOS build reality.
… debug macOS probe: xim:llvm/clang resolves but install() fails fast (9s, no build) — invisible under xim's interface mode. Surface the build log + clang/SDK sanity on failure (non-Windows). Remove before merge.
macOS install() failed before writing a build log (my diag searched $HOME but the project store is under $GITHUB_WORKSPACE). Likely cause: xim:llvm's slim clang has no baked-in sysroot (unlike Xcode's /usr/bin/clang), so the compiler check can't find <stdio.h>. Set SDKROOT=$(xcrun --show-sdk-path) for the macOS build; also fix the diagnostic to search the workspace so the real error surfaces if this isn't it.
macOS install() dies before cmake (no build log) and the error is invisible under xim interface mode. MCPP_VERBOSE should surface the hook's error detail (fetch failure reason). Temp, for macOS multi-platform debug.
…cOS failure) install() dies invisibly on macOS before cmake (no build log; MCPP_VERBOSE doesn't surface hook errors). Write a step-by-step trace to $HOME/ocv_trace.txt and cat it in the failure diagnostic to see exactly where it dies (find_srcroot / os.mv / before cmake). Temp.
…boundary)
$HOME differed between the install() (xlings) process and the diagnostic step, so
the trace was unfindable ('no trace'). Write to /tmp to definitively answer whether
install() runs at all on macOS.
Root cause of the macOS 'no build, no install() trace' mystery: mcpp's cfg() predicate for macOS is 'macos' (prepare.cppm:106), NOT 'macosx' (that's the xpm descriptor platform key). So [target.'cfg(macosx)'.dependencies] never matched → opencv was never pulled on macOS → install() never ran → consumer failed on missing headers. Fix the test project to cfg(macos). Also align build xim:llvm to 20.1.7 (the macOS default toolchain the consumer resolves) for libc++ ABI consistency.
…build) macOS fetch failed before install() because xim:make is linux-only (no macosx build) — so build-dep install aborted, invisibly. Switch to Ninja (xim:ninja is cross-platform: linux/macosx/windows) for ALL platforms, dropping xim:make. Also unpin the compiler (xim:llvm, no version) per review — libc++ ABI is stable. Sets up Windows too (Ninja + clang-cl later).
…d log.error) Per review: mcpp/xlings interface mode swallows the hook's log.* output, so log.error was useless for surfacing the build log. Write the failure detail (build log / raised error / PATH) to /tmp/ocv_diag.txt and cat it in the CI diagnostic step — the file+external-cat pattern, same as the trace.
…macOS link On arm64 imgproc emitted undefined carotene_o4t::* symbols at consumer link time (libtegra_hal.a not in the curated link set). Disabling the carotene HAL routes imgproc through its baseline C++ path, no extra lib. Linux green already; Ninja generator + this fix take macOS to a full host-free source build + link + run.
… HAL Carotene fix surfaced the next default-ON Apple-Silicon HAL: KleidiCV (kleidicv::hal::*) undefined at consumer link. Disable both arm64 HALs so core/imgproc use the baseline C++ path — no extra static HAL lib to link.
…CK dep Apple Silicon auto-found the Accelerate framework and emitted cblas_*/ lapack (sgesv_/dgetrf_/…) calls the consumer doesn't link. Off => core uses its built-in fallback; no external BLAS/LAPACK to carry (host-free, consistent with the IPP/TBB/Eigen-off minimal profile).
…dylib skew xim:llvm's macOS clang++.cfg supplies LLVM-20 libc++ HEADERS but links the SDK's Apple system libc++ DYLIB. OpenCV's unordered_map<string,…> code emits the LLVM-20 out-of-line std::__1::__hash_memory, absent in the runner's older Apple libc++ -> undefined symbol at consumer link. --no-default-config drops xim's cfg so OpenCV builds against the SDK's own libc++ headers (matching the dylib everything links); SDKROOT still supplies the sysroot.
…mory skew --no-default-config alone wasn't enough: at the SDK's default target (15.0) Apple libc++'s availability-guarded headers still emit the out-of-line std::__1::__hash_memory (added at 14.4), but the runtime libc++ linked at the consumer's min (14.0) lacks it. Pinning CMAKE_OSX_DEPLOYMENT_TARGET=14.0 (< 14.4) makes the guard select the inline hash path -> no external symbol; also silences the object-version-newer-than-target link warnings.
…c++) Previous --no-default-config didn't switch headers: clang finds its bundled LLVM-20 libc++ (<llvm>/include/c++/v1) toolchain-relative, where __hash_memory is unconditional, so deployment-target guards can't help. Add -nostdinc++ and -isystem $SDKROOT/usr/include/c++/v1 to build OpenCV against Apple's availability-guarded libc++ headers; with the 14.0 target pin the guard inlines the hash -> no external std::__1::__hash_memory the system dylib lacks.
Replace the Windows generated_files stub with a from-source build, matching Linux/macOS. clang-cl (xim:llvm) targets x86_64-pc-windows-msvc (msvc-stl) to match mcpp's Windows consumer ABI and autodetects MSVC + the Windows SDK from the registry; Ninja generator (xim:ninja is cross-platform). Driven through cmd (no bash on a bare runner) with a > redirect for the log; dflags values pass bare (single quotes would reach cmake literally under cmd). Install layout normalised to the linux/macOS shape (OPENCV_INSTALL_BINARIES_ PREFIX empty, lib/ + lib/opencv4/3rdparty + include/opencv4, DLLVERSION empty so libs are opencv_core.lib not opencv_core4130.lib) and BUILD_WITH_STATIC_CRT =OFF for the dynamic CRT (/MD) that mcpp's clang-cl consumer uses. windows ldflags link the static libs (-Llib -lopencv_core …; clang-cl maps -lfoo -> foo.lib, the compat.openblas precedent). Test project + roundtrip now wired on Windows (cfg(windows), _WIN32 guard). Diagnostics moved to $RUNNER_TEMP so the failure step works on Windows (Git Bash + cygpath) too.
First Windows source build was a near-complete success: clang-cl autodetected MSVC, cmake configured, all of core+imgproc+imgcodecs compiled, install laid the normalised layout (opencv_core.lib etc). Only the consumer link failed LNK2038: OpenCV built /MD (dynamic CRT) but mcpp compiles its objects /MT (roundtrip.o = MT_StaticRelease). Flip BUILD_WITH_STATIC_CRT to ON (/MT, also OpenCV's Windows default) so the RuntimeLibrary matches.
/MT fix cleared the CRT mismatch; the last consumer-link error was a single unresolved external __imp_CoCreateGuid (opencv_core cv::tempfile), a Win32 COM API in ole32.lib. Add -lole32 to the Windows ldflags (clang-cl resolves it off the Windows SDK lib path). Everything else — clang-cl build, install, layout, static libs — already links.
…rm test All three platforms build green, so remove the debug scaffolding and the per-platform gating that only existed while opencv was Linux-only: - descriptor: drop the trace()/diag_path/_dump_diagnostics on-disk-file scaffolding; on failure just log.error the build-log path (compat.openblas pattern). Refresh the stale header comment (macOS/Windows are done, not follow-ups). - validate.yml: drop MCPP_VERBOSE=1 and the 'opencv build-log on failure' step. - test project: collapse the three identical cfg(linux|macos|windows) dependency blocks into one unconditional [dependencies.compat]; drop roundtrip.cpp's #if defined(__linux__|__APPLE__|_WIN32) guard + #else no-op main.
Mirrored opencv-4.13.0.tar.gz to gitcode mcpp-res/opencv via local gtc
(byte-identical to the GitHub tag archive, sha256 verified 200 + match).
Descriptor url strings -> { GLOBAL, CN } fallback tables on all three
platforms, so CN users get a fast mirror and GLOBAL behaviour is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
compat.opencv 4.13.0 — curated static build of
core + imgproc + imgcodecs, driven by OpenCV's own CMake through aninstall()hook (thecompat.openblaspattern, CMake instead of Make). mcpp's "list the .cpp" model can't express OpenCV's generated SIMD-dispatch / OpenCL-kernel / config headers, so CMake produces them.Host-free: ecosystem
xim:cmake/make/gcconly; bundled3rdparty/zlib+png+jpeg (BUILD_*=ON);WITH_ADE=OFFkills the sole configure-time download — verified offline underunshare -rn.Requires mcpp 0.0.87
This package builds a glibc-dynamic build-dep tool (
xim:cmake) ininstall(). On ≤0.0.86 that tool was mis-scoped into the project store and its ELF interpreter dangled. Fixed by mcpp-community/mcpp#205 (0.0.87): global tools resolve to the shared registry. Floor + CI pin bumped to 0.0.87 accordingly (index.toml,validate.yml).Contents
pkgs/c/compat.opencv.lua—pkginfo.build_dep-resolved tools, curated CMakeinstall(), per-OS link flags; Windows deferred (anchor stub). Plain-string GLOBAL url (no mcpp-res write access this session → CN mirrormcpp-res/opencvis a maintainer follow-up;mirror-cn-reachablehas nothing to check for opencv).tests/examples/opencv/—cv::Mat+cvtColor(BGR2GRAY)+imencode/imdecodePNG round-trip; linux-gated workspace member..agents/docs/.Verified
End-to-end with mcpp 0.0.87:
mcpp test→test result ok. 1 passed; cmake installs to the global registry, project store holds onlycompat.opencv.https://claude.ai/code/session_01Y3KE7MAE8yV7BG35eXy7FV