From ce6aa7154de0716db4af3199a215c5503673843a Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 9 Nov 2026 18:21:53 +0000 Subject: [PATCH 1/4] [BUILD] CI: take the OpenMS core from the nightly SDK by default The first OpenMS nightly without the GUI is published (2026.11.09, OpenMS 9eddc6d3): its tool registry no longer lists ExecutePipeline and ImageCreator, so the GUI layer builds against the SDK and the installer of that nightly. Pull requests and releases download them (minutes) instead of building the core (hours, per new OpenMS commit). nightly.yml keeps building OpenMS develop from source; mode 'source' remains available for testing against an OpenMS change that is not in a nightly yet. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0149aXma24dXL4j67seUsJyp --- .github/actions/openms-core/action.yml | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/.github/actions/openms-core/action.yml b/.github/actions/openms-core/action.yml index de2cd059..3e3b3204 100644 --- a/.github/actions/openms-core/action.yml +++ b/.github/actions/openms-core/action.yml @@ -17,12 +17,13 @@ description: >- no TOPP tools and no example data). THE DEFAULT MODE (used when 'mode' is empty) is set in the first step below, - DEFAULT_MODE: 'source', because the nightlies and releases of OpenMS up to now are - built WITH_GUI=ON: their tool registry lists ExecutePipeline and ImageCreator, - which cmake/find_openms.cmake rejects (both registries in one installation make - every TOPP tool fail). Once OpenMS no longer has the WITH_GUI option and a nightly - without the GUI is published, change it to 'sdk'. That is the only place to - change: the workflows pass a mode only where they need a specific one. + DEFAULT_MODE: 'sdk', the published nightly (or release) of OpenMS. Nightlies from + before the GUI moved here (before 2026-11-09) are built WITH_GUI=ON: their tool + registry lists ExecutePipeline and ImageCreator, which cmake/find_openms.cmake + rejects (both registries in one installation make every TOPP tool fail); build the + core with mode 'source' to test against such a commit, or against an OpenMS change + that is not in a nightly yet. That is the only place to change: the workflows pass + a mode only where they need a specific one. inputs: mode: @@ -101,9 +102,8 @@ runs: shell: bash env: MODE: ${{ inputs.mode }} - # The default mode: 'source' while the OpenMS nightlies still contain the GUI, - # then 'sdk' (see the description of this action). - DEFAULT_MODE: source + # The default mode (see the description of this action). + DEFAULT_MODE: sdk run: | mode=${MODE:-$DEFAULT_MODE} case "$mode" in From 22e058ce544b41a76f5e91bf7c86627350495046 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 9 Nov 2026 18:25:13 +0000 Subject: [PATCH 2/4] [FIX] resolve-sdk.sh: no mapfile (macOS has bash 3.2) The SDK mode failed on macOS with 'mapfile: command not found': GitHub's macOS runners run steps with /bin/bash 3.2. Read the folder listing with a while-read loop instead (which also avoids expanding an empty array under set -u, an error in bash 3.2). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0149aXma24dXL4j67seUsJyp --- .github/actions/openms-core/resolve-sdk.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/actions/openms-core/resolve-sdk.sh b/.github/actions/openms-core/resolve-sdk.sh index 25436fb4..b0ab5002 100755 --- a/.github/actions/openms-core/resolve-sdk.sh +++ b/.github/actions/openms-core/resolve-sdk.sh @@ -93,13 +93,13 @@ find_in_folder() { if [[ "$version" == latest ]]; then # newest first; a folder of a release or nightly that is still being uploaded, # or whose build failed on this platform, is passed over - mapfile -t folders < <(listing "$base/$channel" | sed 's#/$##' | grep -E "$folder_regex" | sort -rV | head -n 10) - for folder in "${folders[@]}"; do + # (a read loop, not mapfile: macOS has bash 3.2; fd 3 keeps the loop's stdin free) + while IFS= read -r folder <&3; do if find_in_folder "$folder"; then exit 0 fi echo >&2 "note: $base/$channel/$folder has no complete set for $os-$arch, trying the one before" - done + done 3< <(listing "$base/$channel" | sed 's#/$##' | grep -E "$folder_regex" | sort -rV | head -n 10) echo >&2 "ERROR: none of the newest folders of $base/$channel has the SDK ($sdk_suffix) and the installer ($installer_suffix)" exit 1 fi From 93a5333e2da4af5e369a93520eab626b560f2573 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 9 Nov 2026 18:32:01 +0000 Subject: [PATCH 3/4] [FIX] CI: explicit core_ref builds from source; SDK and installer from one build; wait for the nightly Review of OpenMS/OpenMS-TOPP-GUI#2 (CodeRabbit): - test.yml, release.yml: a core_ref given without core_mode builds that ref from source; with the SDK default it was ignored silently (the SDK has no ref to choose). - openms-core: refuse an SDK whose OpenMS_VERSION_FULLSTRING differs from the installer's version (a folder of the archive can mix two builds while a later run of the same day is uploading). - resolve-sdk.sh / openms-core input 'wait': the nightly installers of the GUI layer wait up to 120 minutes for the OpenMS nightly folder of their date instead of failing while it is still being uploaded. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0149aXma24dXL4j67seUsJyp --- .github/actions/openms-core/action.yml | 18 ++++++++++++++++++ .github/actions/openms-core/resolve-sdk.sh | 20 +++++++++++++++----- .github/workflows/release.yml | 5 ++++- .github/workflows/test.yml | 3 ++- 4 files changed, 39 insertions(+), 7 deletions(-) diff --git a/.github/actions/openms-core/action.yml b/.github/actions/openms-core/action.yml index 3e3b3204..de321135 100644 --- a/.github/actions/openms-core/action.yml +++ b/.github/actions/openms-core/action.yml @@ -52,6 +52,12 @@ inputs: release X.Y.Z required: false default: "latest" + wait: + description: >- + sdk: minutes to wait for the folder given by 'version' while it is not complete yet + (it is still being uploaded); not used for 'latest' + required: false + default: "0" sdk: description: >- sdk: 'false' installs only the OpenMS installer (prefix_path is then empty), for @@ -391,6 +397,7 @@ runs: CHANNEL: ${{ inputs.channel }} VERSION: ${{ inputs.version }} NEED_SDK: ${{ inputs.sdk }} + OPENMS_SDK_WAIT_MINUTES: ${{ inputs.wait }} run: | bash "$GITHUB_ACTION_PATH/resolve-sdk.sh" "$CHANNEL" "$VERSION" "$RUNNER_OS" "$RUNNER_ARCH" "$NEED_SDK" | tee -a "$GITHUB_OUTPUT" @@ -424,6 +431,7 @@ runs: CORE_DIR: ${{ steps.setup.outputs.core_dir }} DOWNLOAD_DIR: ${{ steps.setup.outputs.download_dir }} SDK_NAME: ${{ steps.sdk-resolve.outputs.sdk_name }} + FULLSTRING: ${{ steps.sdk-resolve.outputs.version_fullstring }} run: | set -euo pipefail rm -rf "$CORE_DIR" @@ -440,6 +448,16 @@ runs: ls "$CORE_DIR" exit 1 fi + # The SDK and the installer must come from one build of OpenMS: the GUI layer is + # compiled against the SDK and run against the installer. A folder of the archive + # can hold both from different builds while a later run of the same day is + # uploading. The SDK's CMake package names its full version string. + sdk_fullstring=$(sed -n 's/^set(OpenMS_VERSION_FULLSTRING "\(.*\)")[[:space:]]*$/\1/p' \ + "$sdk_dir/lib/cmake/OpenMS/OpenMSConfig.cmake" "$sdk_dir/CMake/OpenMSConfig.cmake" 2>/dev/null | head -n 1 || true) + if [[ -n "$sdk_fullstring" && "$sdk_fullstring" != "$FULLSTRING" ]]; then + echo >&2 "ERROR: the SDK is OpenMS $sdk_fullstring, the installer OpenMS $FULLSTRING: they come from different builds (an upload in progress?)" + exit 1 + fi echo "sdk_dir=$sdk_dir" >>"$GITHUB_OUTPUT" - name: Install the OpenMS Installer (Linux) diff --git a/.github/actions/openms-core/resolve-sdk.sh b/.github/actions/openms-core/resolve-sdk.sh index b0ab5002..264f5fd8 100755 --- a/.github/actions/openms-core/resolve-sdk.sh +++ b/.github/actions/openms-core/resolve-sdk.sh @@ -22,7 +22,8 @@ # # Prints shell assignments (version, sdk_name, sdk_url, installer_name, # installer_url, version_fullstring), e.g. for >> "$GITHUB_OUTPUT". -# OPENMS_ARCHIVE_URL overrides the base URL. +# OPENMS_ARCHIVE_URL overrides the base URL. OPENMS_SDK_WAIT_MINUTES (default 0) waits for a +# given version (not latest) whose folder is not complete yet. set -euo pipefail if [[ $# -lt 4 || $# -gt 5 ]]; then @@ -111,7 +112,16 @@ if ! grep -qE "$folder_regex" <<<"$version"; then echo >&2 "ERROR: '$version' is not a version of the $channel channel" exit 2 fi -if ! find_in_folder "$version"; then - echo >&2 "ERROR: $base/$channel/$version has no SDK OpenMS-SDK-$version-$sdk_suffix and installer OpenMS--$installer_suffix" - exit 1 -fi +# A given folder may still be uploading (the nightly of the GUI layer can start before +# OpenMS's nightly is published): wait up to OPENMS_SDK_WAIT_MINUTES for it. +wait_minutes=${OPENMS_SDK_WAIT_MINUTES:-0} +waited=0 +until find_in_folder "$version"; do + if (( waited >= wait_minutes )); then + echo >&2 "ERROR: $base/$channel/$version has no SDK OpenMS-SDK-$version-$sdk_suffix and installer OpenMS--$installer_suffix" + exit 1 + fi + echo >&2 "note: $base/$channel/$version is not complete for $os-$arch yet, waiting ($waited of $wait_minutes minutes)" + sleep 300 + waited=$((waited + 5)) +done diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 74124343..e77c1b65 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -136,11 +136,14 @@ jobs: id: core uses: ./.github/actions/openms-core with: - mode: ${{ inputs.core_mode }} + # an explicit core_ref is built from source (the SDK has no ref to choose) + mode: ${{ inputs.core_mode || (inputs.core_ref != '' && 'source') || '' }} ref: ${{ needs.version.outputs.core_ref }} exact: "true" channel: ${{ needs.version.outputs.core_channel }} version: ${{ needs.version.outputs.version }} + # the nightly of OpenMS may still be uploading when the GUI nightly starts + wait: ${{ needs.version.outputs.core_channel == 'nightly' && '120' || '0' }} ssh_key: ${{ secrets.ARCHIVE_RRSYNC_SSH }} ssh_host: ${{ secrets.ARCHIVE_RRSYNC_HOST }} ssh_port: ${{ secrets.ARCHIVE_RRSYNC_PORT }} diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 2d36e268..768f8903 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -98,7 +98,8 @@ jobs: id: core uses: ./.github/actions/openms-core with: - mode: ${{ inputs.core_mode }} + # an explicit core_ref is built from source (the SDK has no ref to choose) + mode: ${{ inputs.core_mode || (inputs.core_ref != '' && 'source') || '' }} ref: ${{ inputs.core_ref || env.OPENMS_CORE_DEFAULT_REF }} exact: ${{ inputs.core_exact && 'true' || 'false' }} channel: ${{ inputs.core_channel || 'nightly' }} From c62da423931eec169b1def639a6b6d1128bcbbe4 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 9 Nov 2026 18:34:34 +0000 Subject: [PATCH 4/4] [FIX] openms-core: fail on an SDK without OpenMS_VERSION_FULLSTRING Without it the check that SDK and installer come from one build was skipped; every SDK this layer can use (OpenMS 3.7 without the GUI) exports it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0149aXma24dXL4j67seUsJyp --- .github/actions/openms-core/action.yml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/.github/actions/openms-core/action.yml b/.github/actions/openms-core/action.yml index de321135..3bc44a08 100644 --- a/.github/actions/openms-core/action.yml +++ b/.github/actions/openms-core/action.yml @@ -454,7 +454,11 @@ runs: # uploading. The SDK's CMake package names its full version string. sdk_fullstring=$(sed -n 's/^set(OpenMS_VERSION_FULLSTRING "\(.*\)")[[:space:]]*$/\1/p' \ "$sdk_dir/lib/cmake/OpenMS/OpenMSConfig.cmake" "$sdk_dir/CMake/OpenMSConfig.cmake" 2>/dev/null | head -n 1 || true) - if [[ -n "$sdk_fullstring" && "$sdk_fullstring" != "$FULLSTRING" ]]; then + if [[ -z "$sdk_fullstring" ]]; then + echo >&2 "ERROR: the SDK's OpenMSConfig.cmake names no OpenMS_VERSION_FULLSTRING (an OpenMS from before 3.7, which this GUI layer cannot use)" + exit 1 + fi + if [[ "$sdk_fullstring" != "$FULLSTRING" ]]; then echo >&2 "ERROR: the SDK is OpenMS $sdk_fullstring, the installer OpenMS $FULLSTRING: they come from different builds (an upload in progress?)" exit 1 fi