Skip to content

Build ARM64 and experimental native Windows engine wheels - #140

Merged
danielmawhirter merged 1 commit into
dm-platform-nativefrom
dm-platform-wheels
Nov 3, 2026
Merged

danielmawhirter merged 1 commit into
dm-platform-nativefrom
dm-platform-wheels

Conversation

@danielmawhirter

@danielmawhirter danielmawhirter commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Build Linux ARM64 and experimental native Windows wheels through the shared assembly and native/legal verification pipeline, retaining Linux x86-64 and macOS ARM64. Keep four CI build artifacts, but explicitly allow only existing Linux x86-64 and macOS ARM64 wheels in release attachment and publication. ARM64 release eligibility is deferred to #92; native Windows stays disabled. Build checkout no longer persists credentials.

Base dm-platform-native; 16 incremental files. Merge #91 → #140 → #92. Windows release support uses WSL2 with the Linux x86-64 wheel.

Head f87071eda8cf48f6cc138e65ca1857721e4a552b.

Change map

Maintained file-by-file map.

Testing Done

Rebased onto main 836e343c; all 19 stack commits are signed. Local checks passed: 188 packaging/workflow tests plus four subtests, 94 core tests, 158 datamodule tests (one required a subprocess-permission rerun), 656 Rust tests (two ignored), 250-document validation, artifact manifests, 410-component notices and flake8/isort. The workflow fixture now uses local pathlib operations instead of application dependencies; all 156 tests in the exact minimal repository-check environment passed without pydantic.

Fresh run 41570848349 at 8f5471712d5b36312b1ec899166da409ff9b1454 completed successfully: four producers, 144 dependency resolutions, twelve CPU reload jobs and the inherited Linux x86-64 GPU gate (eight tasks) passed. Release attachment and Kitmaker publication were skipped. The final stack retains both release gates; native Windows remains CI-only. No ARM/Windows GPU coverage or package publication is claimed.

Separate automatic connector checks fail only two restricted BigQuery tests missing GCP_BIGQUERY_NO_TABLE_SERVICE_ACCOUNT and GCP_BIGQUERY_NO_JOB_SERVICE_ACCOUNT; 1,911 other connector tests passed on each PR. That credential configuration is owned elsewhere and is not fixed here.

Hotfix Context

Release preparation.

Checklist

  • Breaking changes listed (none intended).
  • Examples and documentation updated.
  • Focused tests added or updated.

@copy-pr-bot

copy-pr-bot Bot commented Oct 30, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@danielmawhirter
danielmawhirter temporarily deployed to gha-kumo-workflow-tests October 30, 2026 16:34 — with GitHub Actions Inactive
@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Walkthrough

The workflow adds Linux ARM64 and Windows x86_64 wheel builds alongside Linux x86_64 and macOS ARM64. The wheel builder configures native Windows builds and architecture-specific Linux tags. DiskGraph manifests and notices now include Windows artifacts. A new audit checks assembled wheel metadata, payloads, and platform-specific binaries. The workflow runs the audit and updates its build, requirements-check, and release-attachment matrices.

Sequence Diagram(s)

sequenceDiagram
  participant GitHubActions
  participant build_wheel.sh
  participant DiskGraphArtifacts
  participant verify_platform_wheel.py
  GitHubActions->>build_wheel.sh: Build wheel for platform and machine
  build_wheel.sh->>DiskGraphArtifacts: Select platform-specific artifacts
  DiskGraphArtifacts-->>build_wheel.sh: Provide native artifacts
  build_wheel.sh-->>GitHubActions: Return assembled wheel
  GitHubActions->>verify_platform_wheel.py: Verify wheel and platform tag
Loading
Merge Risk: Minimal · up to f8707

ARM64 and Windows wheels remain CI-only; existing release targets are unchanged. Normal CI checks remain necessary.

Pre-merge checks: Passed 2 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Change Map Design Review Warning (previously flagged, unaddressed) The new Windows and Linux ARM64 release rows can publish without installed-wheel qualification. The workflow only runs static payload verification and dependency resolution before attach-to-release … Add an installed-wheel qualification job for the native Windows and Linux ARM64 rows, covering CPU ingestion, training, and prediction, and add that job to attach-to-release.needs before publishing. If qualification is supplied by #92, ke…
Passed Passed checks: 2 passed
Check name Status Explanation
Pr Size Passed The pull request changes 16 files. The limit is 40 files, so the check passes.
Public Resource Access Passed PASS. The diff adds no external references in the workflow or release documentation. Generated legal and artifact files add public Apache license URLs, public GitHub origins for Windows Rust crates, a…
Full details: Public Resource Access

Explanation

PASS. The diff adds no external references in the workflow or release documentation. Generated legal and artifact files add public Apache license URLs, public GitHub origins for Windows Rust crates, and the public crates.io index (packages/diskgraph/third_party/artifact-manifest.json:18). Anonymous GET checks returned HTTP 200 for all unique added resource URLs, including the new origins in 3rdparty/notices/inventory.json and the license URLs in THIRD-PARTY.txt. No added or modified reference was confirmed inaccessible.

Full details: Change Map Design Review

Explanation

The new Windows and Linux ARM64 release rows can publish without installed-wheel qualification. The workflow only runs static payload verification and dependency resolution before attach-to-release (.github/workflows/kumo-engine-wheel.yml:160-166, 221-227, 229-248). kitmaker then publishes all four rows after environment approval (:250-293). The release documentation states that installed-wheel CPU ingestion, training, and prediction are a separate qualification requirement (docs/development/releasing-kumo-engine.md:190-194). The new rows therefore introduce a release path that can publish unqualified native wheels. The required marked change-map conversation comment is not present in the supplied context or Git refs, so its stated stopping condition cannot be verified.

Resolution

Add an installed-wheel qualification job for the native Windows and Linux ARM64 rows, covering CPU ingestion, training, and prediction, and add that job to attach-to-release.needs before publishing. If qualification is supplied by #92, keep the new release and Kitmaker rows disabled until #92 is merged. Update the change map with the release risk, the qualification gate, focused verification, and the stopping condition.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

ast-grep (0.51.1)

ast-grep could not parse rule config: /ast-grep-rules/ast-grep-essentials/rules/ruby/security/weak-hash-sha1-ruby.yml

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.
Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 322eea4 and f87071e.

Files selected for processing: 4
  • .github/workflows/kumo-engine-wheel.yml
  • docs/development/releasing-kumo-engine.md
  • packages/diskgraph/third_party/artifact-manifest.json
  • test/scripts/test_engine_wheel_workflow.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@danielmawhirter

danielmawhirter commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator Author

Change map

Build Linux ARM64 and experimental native Windows wheels through the shared assembly and native/legal verification pipeline, retaining Linux x86-64 and macOS ARM64. Keep four CI build artifacts, but explicitly allow only existing Linux x86-64 and macOS ARM64 wheels in release attachment and publication. ARM64 release eligibility is deferred to #92; native Windows stays disabled. Build checkout no longer persists credentials.

Base dm-platform-native; 16 incremental files. Merge #91 → #140 → #92. Windows release support uses WSL2 with the Linux x86-64 wheel.

Head f87071eda8cf48f6cc138e65ca1857721e4a552b.

File Function / scope What changed Why
.github/workflows/kumo-engine-wheel.yml Build and release routing (including inherited GPU gate) Builds four CI artifacts, preserves LF bytes, runs native regressions, disables persisted checkout credentials and limits release allowlists to Linux x86-64/macOS ARM64. Preserves main’s Linux x86-64 wheel-smoke gate; separates CI validation from release eligibility; Windows release support is WSL2-only.
3rdparty/notices/inventory.json Artifact-scoped legal inventory Adds separate Windows graphs and 23 Windows-only locked components with canonical BSL backfills. Windows licenses must describe its actual Cargo graph, not the Linux graph.
THIRD-PARTY.txt Generated legal notices Regenerates notices from the reviewed platform inventory. Includes Windows dependency obligations with original license text.
docs/development/releasing-kumo-engine.md Release scope and prerequisites Documents WSL2 as the Windows release path, native Windows as experimental CI-only, and ARM64 as CI-only until dependent qualification. Distinguishes supported release installation from preparatory native Windows validation.
packages/diskgraph/scripts/generate_artifact_manifest.py Cargo artifact graph generation Resolves distinct Windows extension and ingest graphs while retaining Linux IDs. Avoids representing a target union as a shipped artifact.
packages/diskgraph/third_party/README.md Generated graph documentation Documents separate Windows and Linux artifact identities. Keeps regeneration instructions accurate.
packages/diskgraph/third_party/artifact-manifest.json Generated artifact graphs Regenerates source and lock fingerprints against current main while retaining target-specific dependency graphs. Keeps provenance consistent with merged native RDL/history and index-clock changes.
packages/kumo-engine/scripts/build_wheel.sh Shared platform wheel assembly Adds Windows-native interpreter/extension/executable handling and Windows notice selection. Reuses the merged-wheel packaging policy across release platforms.
packages/kumo-engine/scripts/verify_platform_wheel.py Final static wheel validation Checks tags, RECORD, payload and native architecture; recognizes Windows pdh.dll as a system library and lists complete imports when rejecting unknown dependencies; versioned Python DLLs remain rejected. Audit real native wheels without falsely rejecting the Windows performance-counter library.
scripts/gen_third_party.py Manifest/inventory validation Requires Windows artifact graphs alongside existing Linux graphs. Rejects missing or mismatched Windows license inventories.
scripts/refresh_diskgraph_notice_inventory.py Notice inventory synchronization Refreshes all four DiskGraph artifact scopes. Keeps target membership and license provenance reproducible.
test/scripts/test_engine_wheel.py Builder regression coverage Tests Windows platform/binary and notice selection while retaining shared packaging checks. Prevents incorrect Windows wheel assembly.
test/scripts/test_engine_wheel_workflow.py Workflow policy regressions Tests builder release exclusions, matching attachment/publication allowlists and non-persisted checkout credentials. Prevents experimental Windows publication and premature ARM64 release eligibility.
test/scripts/test_gen_third_party.py Target-specific notice regressions Checks Windows-only legal coverage, canonical BSL checksums and rejection of a Linux graph substituted for Windows. Prevents omitted or falsely attributed legal obligations.
test/scripts/test_platform_wheel.py Native payload and DLL audit regressions Adds pdh.dll acceptance and complete import diagnostics regressions; retains rejection of invalid ABI, architecture, dependency, tag and RECORD fixtures. Preserves strict audit failures while testing the actual Windows system-library dependency.
test/scripts/test_refresh_diskgraph_notice_inventory.py Expanded artifact fixture Preserves version-refresh assertions across all artifact scopes. Tests inventory refresh after adding Windows graphs.

Focused verification: Rebased onto main 836e343c; all 19 stack commits are signed. Local checks passed: 188 packaging/workflow tests plus four subtests, 94 core tests, 158 datamodule tests (one required a subprocess-permission rerun), 656 Rust tests (two ignored), 250-document validation, artifact manifests, 410-component notices and flake8/isort. The workflow fixture now uses local pathlib operations instead of application dependencies; all 156 tests in the exact minimal repository-check environment passed without pydantic.

Fresh run 41570848349 at 8f5471712d5b36312b1ec899166da409ff9b1454 completed successfully: four producers, 144 dependency resolutions, twelve CPU reload jobs and the inherited Linux x86-64 GPU gate (eight tasks) passed. Release attachment and Kitmaker publication were skipped. The final stack retains both release gates; native Windows remains CI-only. No ARM/Windows GPU coverage or package publication is claimed.

Separate automatic connector checks fail only two restricted BigQuery tests missing GCP_BIGQUERY_NO_TABLE_SERVICE_ACCOUNT and GCP_BIGQUERY_NO_JOB_SERVICE_ACCOUNT; 1,911 other connector tests passed on each PR. That credential configuration is owned elsewhere and is not fixed here.

Stopping condition: reviewed portable runtime, strict wheel assembly and installed-wheel CPU qualification with WSL2-only Windows release support. No native Windows publication, GPU or multi-worker support claim. No upload, release or version bump is authorized.

@danielmawhirter
danielmawhirter temporarily deployed to gha-kumo-workflow-tests October 30, 2026 16:48 — with GitHub Actions Inactive
@danielmawhirter
danielmawhirter temporarily deployed to gha-kumo-workflow-tests October 30, 2026 16:58 — with GitHub Actions Inactive
@danielmawhirter
danielmawhirter temporarily deployed to gha-kumo-workflow-tests October 30, 2026 17:20 — with GitHub Actions Inactive
@danielmawhirter
danielmawhirter temporarily deployed to gha-kumo-workflow-tests October 30, 2026 17:47 — with GitHub Actions Inactive
@danielmawhirter
danielmawhirter changed the base branch from main to dm-platform-native October 30, 2026 18:24
@danielmawhirter
danielmawhirter marked this pull request as ready for review October 30, 2026 18:31

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

Other comments: 1
.github/workflows/kumo-engine-wheel.yml: 1

78-82: Minor · Security & Privacy · Quick win

Set persist-credentials: false on the build job checkout.

This step uses the default checkout settings, so the git token stays in .git/config for the rest of the job. The same job then runs build_wheel.sh, cargo, and pip. These tools run third-party build scripts that can read the persisted token. The Windows and ARM64 rows add more of this build tooling. The step also receives GITHUB_TOKEN: ${{ secrets.KUMO_GITHUB_BOT_TOKEN }} in the build step. The check-requirements and kitmaker jobs already set this option. zizmor reports the same gap (artipacked).

Confirm that no later build step needs the persisted credential. The build only uses git rev-parse on the local checkout.

🔒 Proposed fix
         env:
           # Notice hashes describe the committed bytes, including on Windows.
           GIT_CONFIG_COUNT: '1'
           GIT_CONFIG_KEY_0: core.autocrlf
           GIT_CONFIG_VALUE_0: 'false'
+        with:
+          persist-credentials: false

Based on learnings: jobs that run build tooling able to execute third-party code should call actions/checkout with persist-credentials: false.

Sources: Learnings, Linters/SAST tools

Nitpick comments: 1
docs/development/releasing-kumo-engine.md: 1

123-123: Trivial · Maintainability & Code Quality · Quick win

Update the stale platform count in step 5.

The text says "Both platform wheels are published". The workflow now publishes four wheels: Linux x86_64, Linux ARM64, Windows x86_64, and macOS ARM64. Update the sentence so the release approver knows how many publications to expect. This line is unchanged, but this PR makes it wrong.

📝 Proposed fix
-   platform wheels are published. A publication cannot be recalled, and PyPI never
+   platform wheels (Linux x86_64/ARM64, Windows x86_64, macOS ARM64) are
+   published. A publication cannot be recalled, and PyPI never

Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 2e2c1a3 and 0ce7028.

Files selected for processing: 16
  • .github/workflows/kumo-engine-wheel.yml
  • 3rdparty/notices/inventory.json
  • THIRD-PARTY.txt
  • docs/development/releasing-kumo-engine.md
  • packages/diskgraph/scripts/generate_artifact_manifest.py
  • packages/diskgraph/third_party/README.md
  • packages/diskgraph/third_party/artifact-manifest.json
  • packages/kumo-engine/scripts/build_wheel.sh
  • packages/kumo-engine/scripts/verify_platform_wheel.py
  • scripts/gen_third_party.py
  • scripts/refresh_diskgraph_notice_inventory.py
  • test/scripts/test_engine_wheel.py
  • test/scripts/test_engine_wheel_workflow.py
  • test/scripts/test_gen_third_party.py
  • test/scripts/test_platform_wheel.py
  • test/scripts/test_refresh_diskgraph_notice_inventory.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Comment thread .github/workflows/kumo-engine-wheel.yml Resolved
@danielmawhirter
danielmawhirter temporarily deployed to gha-kumo-workflow-tests October 30, 2026 19:20 — with GitHub Actions Inactive
@danielmawhirter danielmawhirter changed the title Build native Windows and Linux ARM64 engine wheels Build ARM64 and experimental native Windows engine wheels Oct 30, 2026
devanj08 added a commit that referenced this pull request Oct 31, 2026
…#134)

## Summary

Add `Graph.visualize()` to inspect declared tables and relationships
directly in notebooks.
Standard and Compact layouts include light and dark themes, schema
details, relationship highlighting and optional sample values.
Export SVG, vector PDF or Markdown on the machine running Kumo Engine,
with selectable save commands in the notebook toolbar and a static SVG
fallback for saved HTML output.
Notebook display automatically uses environment-provided
anywidget/ipywidgets in Snowflake Workspace and the existing HTML path
elsewhere, without changing package requirements.

See the [change
map](#134 (comment))
for file-by-file rationale.

## What Happens and Why

Schema definitions alone make it difficult to inspect relationships
across multiple tables before training.
Hover-driven camera movement shifts rows beneath the pointer, while
fixed light colors make schema details unsuitable for dark notebook
displays.
Notebook host controls can also overlap graph actions placed at the top
right.
Snowflake Workspace blocks the scripted HTML path, saved Jupyter outputs
can strip scripts and inline SVG, and VS Code relocates executable
scripts outside their original graph element.

## The Fix

Render declared schemas in deterministic Standard/Compact layouts with
stationary hover details, coordinated light/dark styling, manual camera
controls, optional samples and sample-free SVG/PDF/Markdown exports.

### Public API

Options and examples:

```python
graph.visualize(
    path=None,             # str, pathlib.Path, or None
    show_cols=False,
    show_samples=False,
    layout="auto",         # "auto", "standard", or "compact"
)
```

| Option | Behavior |
| --- | --- |
| `path=None` | Return SVG text with interactive notebook display
support, without writing a file. |
| `path="graph.svg"` | Write UTF-8 SVG of the initial layout, including
the displayed columns. |
| `path="graph.pdf"` | Write the vector diagram and complete paginated
schema/connection index. |
| `path="graph.md"` or `path="graph.markdown"` | Write a Mermaid
relationship diagram and all-column Markdown schema tables. |
| `show_cols=False` | Display PK, FK and declared time (`TS`) and
end-time (`TE`) columns. |
| `show_cols=True` | Include other columns, up to 10 rows per table, or
5 above 10 tables; key/time/end-time columns always remain visible even
above that limit. |
| `show_samples=False` | Read no source data. |
| `show_samples=True` | Add up to six non-null samples per visible
column to notebook tooltips from a preview of up to 30 rows per table. |
| `layout="auto"` | Choose Standard or Compact from graph structure,
independently of viewport size. |
| `layout="standard"` | Select the layered relationship layout. |
| `layout="compact"` | Select the compact arrangement. |
| Returned SVG: `.as_widget()` | Optional explicit widget conversion;
`Graph.visualize()` still returns SVG text and selects the notebook
transport at display time. |

```python
from pathlib import Path
from IPython.display import display

# Notebook display; no source-data reads by default.
graph.visualize()
graph.visualize(show_cols=True)
graph.visualize(show_samples=True)
graph.visualize(show_cols=True, show_samples=True, layout="compact")

# The same API automatically uses widget MIME in Snowflake Workspace.
display(graph.visualize(show_cols=True))

# Explicit layout selection.
graph.visualize(layout="auto")
graph.visualize(layout="standard")
graph.visualize(layout="compact")

# Save on the machine running Engine, including a training machine.
graph.visualize("graph.svg", show_cols=True, layout="standard")
graph.visualize(path=Path("graph.pdf"), show_cols=True, layout="compact")
graph.visualize("graph.md")
graph.visualize("graph.markdown")

# The return value remains SVG text, including when a file is written.
view = graph.visualize()
svg_text = str(view)
```

File suffix matching is case-insensitive; any suffix other than `.pdf`,
`.md` or `.markdown` receives SVG.
Paths are relative to the current working directory unless absolute,
parent directories must already exist, and existing files are
overwritten.
PDF and Markdown indexes include every declared column regardless of
`show_cols`; Markdown uses the viewer's Mermaid layout rather than the
notebook arrangement, and its schema tables remain readable without
Mermaid support.
The Export commands button reveals `_ = graph.visualize("graph.pdf",
layout="standard")` first (using the selected layout) and `_ =
graph.visualize("graph.md")` below it; executing a command writes to the
notebook filesystem without a browser download or eager PDF generation.
The default HTML view follows browser appearance, the widget view
inherits a declared host color scheme when mounted, and SVG/PDF files
and the saved static image use the light palette with sample values
excluded.

### Notebook hosts and saved outputs

| Context | Display and persistence contract |
| --- | --- |
| Databricks, local Jupyter and other HTML hosts | `graph.visualize()`
or `display(graph.visualize())`; existing self-contained HTML path with
Databricks `nodarkmode`, no anywidget import or comms |
| Snowflake Workspace | The same `graph.visualize()` and
`display(graph.visualize())` automatically publish actual widget MIME
when the Workspace marker and active Snowflake notebook extension are
present; one cached model supports independent frontend views, with no
scripted HTML competing for display |
| Workspace packages | [Runtime 2.6+ preinstalls
anywidget/ipywidgets](https://docs.snowflake.com/en/user-guide/ui-snowsight/notebooks-in-workspaces/notebooks-in-workspaces-interactive-widgets#installation),
including the documented GPU 2.9 target; older runtimes need compatible
packages under their environment policy; missing or incompatible widget
packages produce a static graph and actionable diagnostic |
| Connector extras | `kumo-engine[snowflake]` adds connector
dependencies; package installation, connector choice and credentials do
not select the notebook renderer |
| Saved HTML output | Complete, sample-free SVG image in the
Python-requested initial layout plus **“Rerun this cell to activate
animation.”**; hidden interactive controls until initialization
succeeds; rerun once after upgrading and save to replace old output; no
saved browser camera or layout-toggle state |
| Widget restoration and trust | Snowflake widget state is not
persistent across sessions; HTML fallback does not repair a missing
widget model; VS Code Restricted Mode can refuse HTML/SVG entirely |
| File retrieval and fullscreen | PDF/Markdown commands save in the
notebook working directory for notebook-side retrieval; fullscreen
remains host-permission-dependent with a clear denial message |

### Mac and Windows compatibility

| Target | Scope and evidence |
| --- | --- |
| Mac/Windows browser connected to a hosted notebook | Native Engine
dependencies run on the kernel host; no client-side Engine port needed,
but actual macOS/Windows browser sessions remain untested |
| Native Apple Silicon Mac | [macOS support already merged in
#47](#47); matching macOS
wheel and native notebook smoke still required for this candidate; local
validation artifact is Linux-only |
| Native Windows | Public `Graph` import reaches the compiled DiskGraph
extension before rendering; [native portability
#91](#91) and [experimental
Windows wheel work #140](#140)
remain prerequisites, not support added by this visualization PR |
| Validation limits | Linux Chromium/Firefox/WebKit tests and
cross-platform resolver checks; no native Mac/Windows installation,
Safari-on-macOS or Windows browser certification; WSL is a separate
Linux target |

## Blast Radius

Dependency declarations and `uv.lock` are unchanged from the PR base;
graphs remain limited to 30 tables with valid declared schemas and no
self-relations, PDF glyphs unavailable in the bundled fonts use explicit
Unicode escapes, and notebook interaction depends on host
HTML/widget/fullscreen permissions.
Sample previews can run warehouse queries and incur costs, their row
limit does not bound bytes scanned, and saved notebook HTML or widget
state can contain sampled values even though file exports and the static
fallback image do not.

## Verification

The [graph/export
tests](https://github.com/NVIDIA/kumo-engine/blob/1d63aed16f8f36d24585e63460c63c03a9275921/packages/kumo-engine/test/test_graph_validation.py)
and [layout
tests](https://github.com/NVIDIA/kumo-engine/blob/1d63aed16f8f36d24585e63460c63c03a9275921/packages/kumo-engine/test/test_visualization_layout.py)
pass together (99 tests), including temporal-role retention beyond
column limits, actual PDF/Markdown exports, PDF badge-spacing checks
with bundled fonts, browser tooltip/visibility checks and scoped
lint/import/whitespace checks.
Prior verification covers the complete installed wheel, actual
IPython/kernel Workspace dispatch, Jupyter save/reopen, repeated-output
VS Code renderer replay and genuine Databricks strict/theme/fullscreen
checks; this end-time correction is checked against source without a new
wheel or hosted deployment; host replays do not certify authenticated
frontends or native Mac/Windows systems, and the existing
restricted-BigQuery credential failures remain reproduced on the [exact
base commit's CI
job](https://github.com/NVIDIA/kumo-engine/actions/runs/40782418127/job/123147227893).

---------

Co-authored-by: Haowei Yu <howyu@nvidia.com>
Victsm
Victsm previously approved these changes Nov 2, 2026

@Victsm Victsm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the incremental changes over #91. Platform assembly, binary/DLL verification, dependency checks and generated licensing updates look sound. Linux ARM64 and native Windows remain excluded from publication in this PR; #92 separately enables qualified Linux ARM64, and native Windows remains experimental/CI-only.

The four-platform build matrix can extend the release critical path (the retained Windows build took 28m13s versus 12m51s for Linux x86_64). That is an operational cost, not an observed Linux runtime regression. Preserve the explicit publication allowlists and required gates when rebasing after #91; clear required CI before merging.

Victsm
Victsm previously approved these changes Nov 3, 2026

@Victsm Victsm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the rebased wheel-building change. Four producers and same-run installed-wheel qualification passed in run 41519588041. Native Windows remains experimental and excluded from publication. Suitable after #91 lands and the stack is refreshed.

@danielmawhirter
danielmawhirter merged commit 3151f94 into main Nov 3, 2026
129 of 130 checks passed
@danielmawhirter
danielmawhirter deleted the dm-platform-wheels branch November 3, 2026 18:42
danielmawhirter added a commit that referenced this pull request Nov 3, 2026
## Summary

Prepare native ingestion portability through platform-specific file I/O,
executable resolution, process handling, local URIs and artifact
ownership. Reject unsupported Windows profiling before modifying
published output. This is preparatory runtime work: native Windows
remains experimental and CI-only, with no native Windows release/support
commitment. Existing optional-Faiss metadata is unchanged.

Base `main`; 22 files. Merge #91 → #140 → #92. Windows release support
uses WSL2 with the Linux x86-64 wheel.

Head `6f52003cd86e9c68989975499ef4276243c0e0e5`.

## Change map

[Maintained file-by-file
map](#91 (comment)).

## Testing Done

Rebased onto main `836e343c`; all 19 stack commits are signed. Local
checks passed: 188 packaging/workflow tests plus four subtests, 94 core
tests, 158 datamodule tests (one required a subprocess-permission
rerun), 656 Rust tests (two ignored), 250-document validation, artifact
manifests, 410-component notices and flake8/isort. The workflow fixture
now uses local pathlib operations instead of application dependencies;
all 156 tests in the exact minimal repository-check environment passed
without pydantic.

Fresh [run
41570848349](https://github.com/NVIDIA/kumo-engine/actions/runs/41570848349)
at `8f5471712d5b36312b1ec899166da409ff9b1454` completed successfully:
four producers, 144 dependency resolutions, twelve CPU reload jobs and
the inherited Linux x86-64 GPU gate (eight tasks) passed. Release
attachment and Kitmaker publication were skipped. The final stack
retains both release gates; native Windows remains CI-only. No
ARM/Windows GPU coverage or package publication is claimed.


Separate automatic connector checks fail only two restricted BigQuery
tests missing `GCP_BIGQUERY_NO_TABLE_SERVICE_ACCOUNT` and
`GCP_BIGQUERY_NO_JOB_SERVICE_ACCOUNT`; 1,911 other connector tests
passed on each PR. That credential configuration is owned elsewhere and
is not fixed here.

## Hotfix Context

Release preparation.

## Checklist

- [x] Breaking changes listed (none intended).
- [x] Examples and documentation updated.
- [x] Focused tests added or updated.
danielmawhirter added a commit that referenced this pull request Nov 3, 2026
## Summary

Qualify same-run installed wheels on WSL2 and Linux ARM64 across Python
3.11–3.14, retaining experimental native Windows CPU checks. Extend the
same single-fit smoke with CPU bundle reload and prediction parity on
the identical population/snapshot; entity IDs, counts and values must
match (atol=1e-4), with no additional optimizer steps. Enable Linux
ARM64 release attachment and publication only behind installed-wheel
qualification. The three release artifacts are Linux x86-64 (also used
by WSL2), Linux ARM64 and macOS ARM64. Native Windows wheels remain
CI-only and are excluded from both release attachment and publication.

Base `dm-platform-wheels`; 9 incremental files. Merge #91 → #140 → #92.
Windows release support uses WSL2 with the Linux x86-64 wheel.

Head `3bac19078321174c7dcb1be01a1aff0b06291bfc`.

## Change map

[Maintained file-by-file
map](#92 (comment)).

## Testing Done

Rebased onto main `524a2ce5`. Local checks passed: 127 Python tests plus
four subtests, 105 Rust pybind tests, 35 runtime tests, 410-component
notice and artifact-manifest checks, 248-document validation,
flake8/isort, actionlint and workflow security. All 19 stack commits
have verified SSH signatures.

Fresh [run
41519588041](https://github.com/NVIDIA/kumo-engine/actions/runs/41519588041)
at `3bac19078321174c7dcb1be01a1aff0b06291bfc` passed four producers,
twelve installed-wheel CPU reload jobs and 144 ADLS-inclusive dependency
resolutions. Release attachment and Kitmaker were skipped; nothing was
published.

Separate connector CI still fails two tests missing
`GCP_BIGQUERY_NO_TABLE_SERVICE_ACCOUNT` and
`GCP_BIGQUERY_NO_JOB_SERVICE_ACCOUNT`; 1,875 other connector tests
passed. Credential configuration is owned elsewhere and is not fixed
here.

## Hotfix Context

Release preparation.

## Checklist

- [x] Breaking changes listed (none intended).
- [x] Examples and documentation updated.
- [x] Focused tests added or updated.

Release attachment and Kitmaker publication jobs were skipped in this
manual run; no package was published.

This branch was previously deployed

1 inactive deployment
gha-kumo-workflow-tests — f87071ed Deployed Nov 3, 2026 by danielmawhirter via import #1327
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants