Repository navigation
Conversation
The default definitions and samples paths are absolute and baked in at build time, so a relocated installation (CI staging, tarballs, containers) cannot find its own data unless ECCODES_DEFINITION_PATH and ECCODES_SAMPLES_PATH are set by hand. Look for the data relative to the directory of the loaded ecCodes library first (<libdir>/../share/eccodes/...), falling back to the compiled-in path. The environment variables keep precedence. The build tree has the same layout, so tests no longer need the paths exported either; drop that from the HPC CI script. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #581 +/- ##
========================================
Coverage 87.66% 87.67%
========================================
Files 857 857
Lines 64110 64122 +12
Branches 11409 11412 +3
========================================
+ Hits 56205 56216 +11
- Misses 7905 7906 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Why not stick with MEMFS in these situations? rather than adding extra options and hence complexity |
|
Is it really necessary to introduce relative paths for the definition and sample files? I'm not sure if the added complexity is justified. Why not just set ECCODES_DEFINITION_PATH and ECCODES_SAMPLES_PATH to the new locations? |
|
Yeah, It's a draft PR and I wanted to play around. Probably I will close it again. |
No problem. I just wanted to share my thoughts. |
Problem
The default definitions and samples paths are absolute and fixed at build time (
ECCODES_DEFINITION_PATH/ECCODES_SAMPLES_PATHineccodes_config.h). If an installation is moved aftermake install, ecCodes can no longer find its own data unless the environment variables are set by hand.This hit the new CI: ecCodes is installed to
github-ci/install/eccodes-<sha>…, then staged for downstream builds intogithub-ci/staging/<pkg>-…/deps/N. In metkit (ecmwf/metkit#293)metkit_test_codes_apifailed with:The same applies to tarballs, containers and any other relocated install. The CMake package is already relocatable (
eccodes-import.cmakederives the paths fromeccodes_BASE_DIR); only the library isn't.Change
ECCODES_{DEFINITION,SAMPLES}_RELPATH, e.g.../share/eccodes/definitions) and puts them ineccodes_config.h.grib_context.cclocates the loaded ecCodes library withdladdrand uses<libdir>/<relpath>if that directory exists. Otherwise it falls back to the compiled-in path.${CMAKE_DL_LIBS}fordladdr(needed on glibc < 2.34).build/lib,build/share/eccodes/…), so build-tree tests find their data without environment variables. This PR removes the exports from.ci/hpc/build.sh.j2, so CI exercises the new lookup.Testing
Tested locally on macOS (shared build):
codes_info -d/-sreport<moved>/lib/../share/eccodes/…, andgrib_lsdecodesGRIB2.tmplwith noECCODES_*variables set.build/lib/../share/eccodes/….ECCODES_SAMPLES_PATHset explicitly still wins.The full test suite was not run locally; this relies on CI.
Open points
dladdrresolves to the executable, so the lookup becomes<bindir>/../share/eccodes/…. That works for the installed tools and otherwise falls back to the compiled-in path because of the existence check.lib/x86_64-linux-gnu): the install relpath is../../share/…, which works for the install but not the build tree (alwaysbuild/lib). The build tree then falls back to the current behaviour.GetModuleHandleEx+GetModuleFileName) could follow if wanted.🤖 Generated with Claude Code