You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Qualify same-run installed wheels on WSL2 and Linux ARM64 across Python 3.11–3.14, retaining experimental native Windows CPU checks. Extend the same single-fit smoke with CPU bundle reload and prediction parity on the identical population/snapshot; entity IDs, counts and values must match (atol=1e-4), with no additional optimizer steps. Enable Linux ARM64 release attachment and publication only behind installed-wheel qualification. The three release artifacts are Linux x86-64 (also used by WSL2), Linux ARM64 and macOS ARM64. Native Windows wheels remain CI-only and are excluded from both release attachment and publication.
Base dm-platform-wheels; 9 incremental files. Merge #91 → #140 → #92. Windows release support uses WSL2 with the Linux x86-64 wheel.
Rebased onto main 836e343c; all 19 stack commits are signed. Local checks passed: 188 packaging/workflow tests plus four subtests, 94 core tests, 158 datamodule tests (one required a subprocess-permission rerun), 656 Rust tests (two ignored), 250-document validation, artifact manifests, 410-component notices and flake8/isort. The workflow fixture now uses local pathlib operations instead of application dependencies; all 156 tests in the exact minimal repository-check environment passed without pydantic.
Fresh run 41570848349 at 8f5471712d5b36312b1ec899166da409ff9b1454 completed successfully: four producers, 144 dependency resolutions, twelve CPU reload jobs and the inherited Linux x86-64 GPU gate (eight tasks) passed. Release attachment and Kitmaker publication were skipped. The final stack retains both release gates; native Windows remains CI-only. No ARM/Windows GPU coverage or package publication is claimed.
Separate automatic connector checks fail only two restricted BigQuery tests missing GCP_BIGQUERY_NO_TABLE_SERVICE_ACCOUNT and GCP_BIGQUERY_NO_JOB_SERVICE_ACCOUNT; 1,911 other connector tests passed on each PR. That credential configuration is owned elsewhere and is not fixed here.
Hotfix Context
Release preparation.
Checklist
Breaking changes listed (none intended).
Examples and documentation updated.
Focused tests added or updated.
Release attachment and Kitmaker publication jobs were skipped in this manual run; no package was published.
It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.
This change adds Windows support across DiskGraph and Kumo Core, including platform-specific file operations, executable selection, progress forwarding, and path handling. It adds a CPU-or-CUDA engine smoke test and a workflow for Linux ARM64, Windows WSL, and native Windows runners. The change also adds platform regression tests and documentation for smoke-test setup and coverage.
Sequence Diagram(s)
sequenceDiagram
participant Workflow as GitHub Actions workflow
participant Runner as Platform runner
participant SmokeTest as Kumo engine smoke test
participant Results as JUnit results
Workflow->>Runner: Start platform-specific job
Runner->>SmokeTest: Install packages and run CPU or CUDA checks
SmokeTest->>Results: Write test results
Workflow->>Results: Upload test results
Loading
Merge Risk: Moderate · previous review 095d2
The change adds Windows and ARM64 platform support and a smoke workflow. Before merging, gate the Unix-only Windows test, remove the temporary push trigger, fix the PowerShell example, and mark the reported runs as historical. Positive DataLoader worker counts on Windows are not supported by this change and should stay disabled or be handled with Windows-specific waiting. The zero-worker native Windows smoke path is not affected.
Important
Pre-merge checks failed
Please resolve all errors before merging. Addressing warnings is optional.
Failed checks: 1 error · 1 inconclusive
Check name
Status
Explanation
Resolution
Public Resource Access
The pull request adds an inaccessible external reference at packages/kumo-engine/platform-smoke.md:76: https://github.com/nv-gha-runners/enterprise-runner-configuration/pull/549. An anonymous requ…
Replace the PR 549 link with an accessible public reference, or copy the required ARM64 runner-access status and configuration details into this repository.
Change Map Design Review
The required <!-- kumo-pr-change-map --> conversation comment is not available in the supplied review context or repository. The PR description links to that comment but does not include its Change …
Provide the complete change-map conversation comment, or an accessible export of it, then review each proposed change against the repository code and the stated problem or release requirement.
Passed checks: 1 passed
Check name
Status
Explanation
Pr Size
The pull request changes 25 files in the reviewed Git range. The limit is 40 files, so the PR passes this check.
Full details: Public Resource Access
Explanation
The pull request adds an inaccessible external reference at packages/kumo-engine/platform-smoke.md:76: https://github.com/nv-gha-runners/enterprise-runner-configuration/pull/549. An anonymous request returned HTTP 404. The added workflow action commit references, Windows image reference, and recorded run links returned HTTP 200.
Resolution
Replace the PR 549 link with an accessible public reference, or copy the required ARM64 runner-access status and configuration details into this repository.
Full details: Change Map Design Review
Explanation
The required <!-- kumo-pr-change-map --> conversation comment is not available in the supplied review context or repository. The PR description links to that comment but does not include its Change map. Repository guidance confirms that the comment must contain the per-change rationale, existing behavior, focused verification, and stopping condition. Without it, this design review cannot assess necessity, abstraction layer, or omitted behavior for each change.
Resolution
Provide the complete change-map conversation comment, or an accessible export of it, then review each proposed change against the repository code and the stated problem or release requirement.
Finishing Touches
Fix failing CI checks
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Tip
🎯 Customize your risk assessments
Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.
✅ Automatic approvals, with your rules
Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.
CodeRabbit Verify
Run CodeRabbit Verify to test the changes in this pull request.
Qualify same-run installed wheels on WSL2 and Linux ARM64 across Python 3.11–3.14, retaining experimental native Windows CPU checks. Enable Linux ARM64 release attachment and publication only behind installed-wheel qualification. The three release artifacts are Linux x86-64 (also used by WSL2), Linux ARM64 and macOS ARM64. Native Windows wheels remain CI-only and are excluded from both release attachment and publication.
Base dm-platform-wheels; 9 incremental files. Merge #91 → #140 → #92. Windows release support uses WSL2 with the Linux x86-64 wheel.
Head 8f5471712d5b36312b1ec899166da409ff9b1454.
File
Function / scope
What changed
Why
.github/workflows/kumo-engine-platform-smoke.yml
Installed-wheel CPU matrix
Runs Python 3.11–3.14 across native Windows, WSL and Linux ARM64 using exact same-run artifacts, clean environments and retained results.
Qualifies installed binaries without checkout imports, GPU access or native rebuilding; removes the temporary push trigger.
.github/workflows/kumo-engine-wheel.yml
Build and release routing (including inherited GPU gate)
Calls installed-wheel CPU qualification and gates release attachment; enables Linux ARM64 in attachment/publication allowlists while native Windows remains commented out.
Preserves main’s Linux x86-64 wheel-smoke gate alongside the platform CPU gate; separates CI validation from release eligibility; Windows release support is WSL2-only.
docs/development/releasing-kumo-engine.md
Release scope and prerequisites
Documents WSL2 as the Windows release path, native Windows as experimental CI-only, and qualified ARM64 release eligibility and applicable CPU/runtime prerequisites.
Distinguishes supported release installation from preparatory native Windows validation.
packages/kumo-engine/README.md
Release scope and prerequisites
Documents WSL2 as the Windows release path, native Windows as experimental CI-only, and qualified ARM64 release eligibility and applicable CPU/runtime prerequisites.
Distinguishes supported release installation from preparatory native Windows validation.
packages/kumo-engine/platform-smoke.md
Qualification contract and release scope
Documents same-fit CPU bundle reload and prediction parity, plus WSL2-only Windows release support and experimental native Windows checks.
Explains the expanded installed-wheel proof without implying native Windows publication.
Checks runtime platform/Python, virtual-environment import origins and packaged ingest; checks the installed local-wheel URL, records independent wheel digest and environment evidence; rejects Faiss on Windows/ARM.
Detects checkout shadowing and wrong or incomplete runtime installation before qualification.
packages/kumo-engine/test/smoke_platform.py
Installed-wheel CPU round trip
Runs one fit, predicts, loads that fit’s bundle on CPU and predicts the same population/snapshot; compares IDs, counts and finite values at atol=1e-4 while requiring exactly four optimizer steps throughout.
Closes saved-model portability coverage without additional training, data or matrix jobs.
test/scripts/test_engine_wheel_workflow.py
Workflow policy regressions
Tests the twelve-job CPU matrix, same-run artifacts, qualification gate and three-artifact release allowlists.
Prevents experimental Windows publication and premature ARM64 release eligibility.
test/scripts/test_installed_wheel.py
Runtime audit regression tests
Tests accepted installed origins and platforms and rejects checkout imports or non-WSL substitutes.
Makes environment identity requirements enforceable.
Focused verification: Rebased onto main 836e343c; all 19 stack commits are signed. Local checks passed: 188 packaging/workflow tests plus four subtests, 94 core tests, 158 datamodule tests (one required a subprocess-permission rerun), 656 Rust tests (two ignored), 250-document validation, artifact manifests, 410-component notices and flake8/isort. The workflow fixture now uses local pathlib operations instead of application dependencies; all 156 tests in the exact minimal repository-check environment passed without pydantic.
Fresh run 41570848349 at 8f5471712d5b36312b1ec899166da409ff9b1454 completed successfully: four producers, 144 dependency resolutions, twelve CPU reload jobs and the inherited Linux x86-64 GPU gate (eight tasks) passed. Release attachment and Kitmaker publication were skipped. The final stack retains both release gates; native Windows remains CI-only. No ARM/Windows GPU coverage or package publication is claimed.
Separate automatic connector checks fail only two restricted BigQuery tests missing GCP_BIGQUERY_NO_TABLE_SERVICE_ACCOUNT and GCP_BIGQUERY_NO_JOB_SERVICE_ACCOUNT; 1,911 other connector tests passed on each PR. That credential configuration is owned elsewhere and is not fixed here.
Stopping condition: reviewed portable runtime, strict wheel assembly and installed-wheel CPU qualification with WSL2-only Windows release support. No native Windows publication, GPU or multi-worker support claim. No upload, release or version bump is authorized.
libc 0.2.186 does not export rlimit, setrlimit, or RLIMIT_NOFILE for Windows. The ungated test prevents cargo test -p diskgraph-ingest and the workspace test workflow from compiling on Windows. Excluding this crate avoids the failure but also omits its tests; non-test builds and runtime use are unaffected.
The workflow still starts all smoke jobs when code is pushed to dm-platform-smoke. The documented delivered configuration must remain manual-only through workflow_dispatch.
Suggested fix
- # Temporary pre-merge qualification on the upstream development branch.- # Remove this branch-specific trigger before merging; retain manual dispatch.- push:- branches: [dm-platform-smoke]
workflow_dispatch:
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.
If this trigger remains, a push to dm-platform-smoke starts all three jobs after merge. The required rollout is manual dispatch only. Remove push and retain workflow_dispatch. (docs.github.com)
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.
If a native Windows user copies line 16 into PowerShell, PowerShell does not accept the Bash-style KUMO_SMOKE_DEVICE=cpu prefix. The CPU smoke test does not start. Keep the Bash example and add a separate PowerShell example that sets $env:KUMO_SMOKE_DEVICE before running pytest. (learn.microsoft.com)
Reviewing files that changed from the base of the PR and between e46cae7 and 12108bf.
⛔ Files ignored due to path filters: 2
packages/diskgraph/Cargo.lock is excluded by !**/*.lock, !**/*.lock
uv.lock is excluded by !**/*.lock, !**/*.lock
📒 Files selected for processing: 4
packages/kumo-core/pyproject.toml
packages/kumo-engine/platform-smoke.md
packages/kumo-engine/setup.py
test/scripts/test_engine_wheel.py
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Note
Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.
Other comments: 1
packages/kumo-engine/platform-smoke.md: 1
48-50: Minor · Maintainability & Code Quality · Quick win
Label the reported runs as historical.
The WSL and native Windows results use revision 29b255e, not the reviewed head 095d27941a87e756544ec69bb9a46065388e7d9a. State that these runs do not qualify the current head, and require current-head reruns before treating these platform paths as qualified.
Suggested documentation update
-The latest test passed in 25.98 seconds at revision `29b255e`, the same-revision as the native Windows pass below. Observed source installation took+The latest historical test passed in 25.98 seconds at revision `29b255e`, the+same revision as the historical native Windows pass below. These runs do not+qualify the current head; rerun them at the current head before qualification.+Observed source installation took
241-241: Trivial · Stability & Availability · Heavy lift
Handle positive DataLoader worker counts on Windows.
The native Windows smoke path uses zero DataLoader workers, so it bypasses _WorkerPoolDataLoader and does not reach _WorkerHandle.open().
However, a configured positive worker count reaches _WorkerHandle.open(). On Windows, _pidfd_open() returns None, so the code duplicates process.sentinel with os.dup() and later passes it to select.poll(). Windows provides the sentinel as a handle, not a POSIX file descriptor, and select.poll() is not available as a Windows wait API. That configuration can fail during worker initialization or exit polling.
Keep positive DataLoader worker counts disabled on Windows, or add Windows-specific handle duplication, waiting, and cleanup with multiprocessing.connection.wait() or another Windows wait API before supporting them.
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.
Reuse exact build artifacts across Python 3.11–3.14 on native Windows, WSL, and Linux ARM64. Test outside the checkout in clean CPU environments, bind installed-package identity to the wheel, and require real ingestion, optimizer work, and finite predictions before release attachment.
## Summary
Prepare native ingestion portability through platform-specific file I/O,
executable resolution, process handling, local URIs and artifact
ownership. Reject unsupported Windows profiling before modifying
published output. This is preparatory runtime work: native Windows
remains experimental and CI-only, with no native Windows release/support
commitment. Existing optional-Faiss metadata is unchanged.
Base `main`; 22 files. Merge #91 → #140 → #92. Windows release support
uses WSL2 with the Linux x86-64 wheel.
Head `6f52003cd86e9c68989975499ef4276243c0e0e5`.
## Change map
[Maintained file-by-file
map](#91 (comment)).
## Testing Done
Rebased onto main `836e343c`; all 19 stack commits are signed. Local
checks passed: 188 packaging/workflow tests plus four subtests, 94 core
tests, 158 datamodule tests (one required a subprocess-permission
rerun), 656 Rust tests (two ignored), 250-document validation, artifact
manifests, 410-component notices and flake8/isort. The workflow fixture
now uses local pathlib operations instead of application dependencies;
all 156 tests in the exact minimal repository-check environment passed
without pydantic.
Fresh [run
41570848349](https://github.com/NVIDIA/kumo-engine/actions/runs/41570848349)
at `8f5471712d5b36312b1ec899166da409ff9b1454` completed successfully:
four producers, 144 dependency resolutions, twelve CPU reload jobs and
the inherited Linux x86-64 GPU gate (eight tasks) passed. Release
attachment and Kitmaker publication were skipped. The final stack
retains both release gates; native Windows remains CI-only. No
ARM/Windows GPU coverage or package publication is claimed.
Separate automatic connector checks fail only two restricted BigQuery
tests missing `GCP_BIGQUERY_NO_TABLE_SERVICE_ACCOUNT` and
`GCP_BIGQUERY_NO_JOB_SERVICE_ACCOUNT`; 1,911 other connector tests
passed on each PR. That credential configuration is owned elsewhere and
is not fixed here.
## Hotfix Context
Release preparation.
## Checklist
- [x] Breaking changes listed (none intended).
- [x] Examples and documentation updated.
- [x] Focused tests added or updated.
## Summary
Build Linux ARM64 and experimental native Windows wheels through the
shared assembly and native/legal verification pipeline, retaining Linux
x86-64 and macOS ARM64. Keep four CI build artifacts, but explicitly
allow only existing Linux x86-64 and macOS ARM64 wheels in release
attachment and publication. ARM64 release eligibility is deferred to
#92; native Windows stays disabled. Build checkout no longer persists
credentials.
Base `dm-platform-native`; 16 incremental files. Merge #91 → #140 → #92.
Windows release support uses WSL2 with the Linux x86-64 wheel.
Head `f87071eda8cf48f6cc138e65ca1857721e4a552b`.
## Change map
[Maintained file-by-file
map](#140 (comment)).
## Testing Done
Rebased onto main `836e343c`; all 19 stack commits are signed. Local
checks passed: 188 packaging/workflow tests plus four subtests, 94 core
tests, 158 datamodule tests (one required a subprocess-permission
rerun), 656 Rust tests (two ignored), 250-document validation, artifact
manifests, 410-component notices and flake8/isort. The workflow fixture
now uses local pathlib operations instead of application dependencies;
all 156 tests in the exact minimal repository-check environment passed
without pydantic.
Fresh [run
41570848349](https://github.com/NVIDIA/kumo-engine/actions/runs/41570848349)
at `8f5471712d5b36312b1ec899166da409ff9b1454` completed successfully:
four producers, 144 dependency resolutions, twelve CPU reload jobs and
the inherited Linux x86-64 GPU gate (eight tasks) passed. Release
attachment and Kitmaker publication were skipped. The final stack
retains both release gates; native Windows remains CI-only. No
ARM/Windows GPU coverage or package publication is claimed.
Separate automatic connector checks fail only two restricted BigQuery
tests missing `GCP_BIGQUERY_NO_TABLE_SERVICE_ACCOUNT` and
`GCP_BIGQUERY_NO_JOB_SERVICE_ACCOUNT`; 1,911 other connector tests
passed on each PR. That credential configuration is owned elsewhere and
is not fixed here.
## Hotfix Context
Release preparation.
## Checklist
- [x] Breaking changes listed (none intended).
- [x] Examples and documentation updated.
- [x] Focused tests added or updated.
Design discussion: [ARM64 documentation
follow-up](#92 (comment)),
[confirmed Python
range](#92 (comment)).
## Summary
Make the DGX Spark guide wheel-first for Python 3.11–3.14, with optional
CPU or GPU Faiss for link prediction and a link to the engine's
connector/text extras. Keep CUDA tooling and a source checkout specific
to the optional GPU Faiss build.
Document prebuilt Linux x86_64/ARM64 and macOS Apple silicon wheels as
available for this release. Preserve WSL2-only Windows support.
Documentation only; no packaging, runtime or publication changes.
## Change map
[Maintained file-by-file change
map](#214 (comment))
## Testing Done
Passed: repository documentation checker (259 files), 36 documentation
unit tests, catalog documentation check, and `git diff --check`. Shell
examples are unchanged from the prior successful `bash -n` verification.
Reviewed commands and version claims against package metadata, build
scripts and public wheel indexes. Flake8/isort are not applicable: no
Python files changed.
Installation examples and DGX Spark GPU/Faiss workloads were not
executed. The existing CPU wheel qualification covers Python 3.11–3.14;
this docs-only follow-up adds no runtime validation.
## Hotfix Context
Release documentation follow-up to merged #92; ships with the release
publishing macOS and ARM64 wheels and assumes those wheels are
available.
## Checklist
- [x] Breaking changes listed: none.
- [x] Installation examples and documentation updated.
- [x] Documentation and reference checks completed; no runtime changes
require new tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GitHub Actions
Inactive
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Summary
Qualify same-run installed wheels on WSL2 and Linux ARM64 across Python 3.11–3.14, retaining experimental native Windows CPU checks. Extend the same single-fit smoke with CPU bundle reload and prediction parity on the identical population/snapshot; entity IDs, counts and values must match (atol=1e-4), with no additional optimizer steps. Enable Linux ARM64 release attachment and publication only behind installed-wheel qualification. The three release artifacts are Linux x86-64 (also used by WSL2), Linux ARM64 and macOS ARM64. Native Windows wheels remain CI-only and are excluded from both release attachment and publication.
Base
dm-platform-wheels; 9 incremental files. Merge #91 → #140 → #92. Windows release support uses WSL2 with the Linux x86-64 wheel.Head
8f5471712d5b36312b1ec899166da409ff9b1454.Change map
Maintained file-by-file map.
Testing Done
Rebased onto main
836e343c; all 19 stack commits are signed. Local checks passed: 188 packaging/workflow tests plus four subtests, 94 core tests, 158 datamodule tests (one required a subprocess-permission rerun), 656 Rust tests (two ignored), 250-document validation, artifact manifests, 410-component notices and flake8/isort. The workflow fixture now uses local pathlib operations instead of application dependencies; all 156 tests in the exact minimal repository-check environment passed without pydantic.Fresh run 41570848349 at
8f5471712d5b36312b1ec899166da409ff9b1454completed successfully: four producers, 144 dependency resolutions, twelve CPU reload jobs and the inherited Linux x86-64 GPU gate (eight tasks) passed. Release attachment and Kitmaker publication were skipped. The final stack retains both release gates; native Windows remains CI-only. No ARM/Windows GPU coverage or package publication is claimed.Separate automatic connector checks fail only two restricted BigQuery tests missing
GCP_BIGQUERY_NO_TABLE_SERVICE_ACCOUNTandGCP_BIGQUERY_NO_JOB_SERVICE_ACCOUNT; 1,911 other connector tests passed on each PR. That credential configuration is owned elsewhere and is not fixed here.Hotfix Context
Release preparation.
Checklist
Release attachment and Kitmaker publication jobs were skipped in this manual run; no package was published.