Skip to content

[CI] Verify checksums of binary downloads - #1325

Open
Mark Hildebrand (hildebrandmw) wants to merge 3 commits into
mainfrom
mhildebr/sha
Open

[CI] Verify checksums of binary downloads#1325
Mark Hildebrand (hildebrandmw) wants to merge 3 commits into
mainfrom
mhildebr/sha

Conversation

@hildebrandmw

Copy link
Copy Markdown
Contributor

Validate the checksums of binary artifacts downloaded in CI. There is one more that I'm aware of in setup-disk-benchmark that is tracked in #1324.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens DiskANN CI supply-chain security by adding SHA-256 verification for externally downloaded binary artifacts (datasets for disk benchmarks and Intel SDE for emulation-based CI jobs).

Changes:

  • Add expected dataset archive SHA-256 values to the disk benchmark workflow matrices and pass them into the setup action.
  • Extend the setup-disk-benchmark composite action to verify the downloaded dataset archive checksum before extraction.
  • Add an Intel SDE SHA-256 env var and verify the downloaded SDE tarball checksum in CI before extracting.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
.github/workflows/disk-benchmarks.yml Adds dataset archive SHA-256 entries to the benchmark matrix and passes them to the setup action.
.github/workflows/disk-benchmarks-aa.yml Adds dataset archive SHA-256 entries to the A/A benchmark matrix and passes them to the setup action.
.github/workflows/ci.yml Adds SDE_SHA256 and verifies the Intel SDE tarball checksum during download.
.github/actions/setup-disk-benchmark/action.yml Adds sha256 input and verifies dataset archive checksum before extraction.
Suppressed comments (1)

.github/workflows/ci.yml:407

  • Same as the other SDE job: checksum verification only runs on cache-miss, so cached contents aren’t validated. If the goal is supply-chain hardening, add integrity validation for the cached payload as well.
      - name: Download Intel SDE
        if: steps.cache-sde.outputs.cache-hit != 'true'
        run: |
          set -euxo pipefail
          SDE_URL="https://downloadmirror.intel.com/915934/${SDE_VERSION}.tar.xz"
          wget -qO intel-sde.tar.xz "$SDE_URL"
          echo "${SDE_SHA256}  intel-sde.tar.xz" | sha256sum --check
          mkdir -p intel-sde

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/actions/setup-disk-benchmark/action.yml
Comment thread .github/workflows/ci.yml
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.55%. Comparing base (2882c75) to head (63eb9cc).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1325      +/-   ##
==========================================
+ Coverage   91.56%   92.55%   +0.98%     
==========================================
  Files         522      522              
  Lines       99541    99541              
==========================================
+ Hits        91144    92128     +984     
+ Misses       8397     7413     -984     
Flag Coverage Δ
miri 92.55% <ø> (+0.98%) ⬆️
unittests 92.51% <ø> (+1.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 43 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants