Skip to content

Keep anchor and history loading in native model batches - #282

Open
Victsm wants to merge 4 commits into
codex/prepared-scalar-execution-20261110from
codex/anchor-history-native-20261110
Open

Victsm wants to merge 4 commits into
codex/prepared-scalar-execution-20261110from
codex/anchor-history-native-20261110

Conversation

@Victsm

@Victsm Victsm commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Use DG-owned anchor batches and native model results directly instead of rebuilding per-anchor Python requests/results in the supported fixed-size model route. Sequence loading keeps native component merging inside iteration; graph loading preserves topology/features together. Preserve split/shuffle/padding policies, contract validation, and existing Windows/reference fallbacks. Encoder fitting remains unchanged.

Stack slice 4/5. Targets the preceding stack branch; merge in dependency order.

Change map

#282 (comment)

Testing

64 focused native-anchor/model-batch tests passed at this checkpoint; package Ruff check and formatting pass. The complete stack additionally passed 1,951 public tests (3 CUDA skips; 47 notebook deselections).

Dependency order: #279 → #280 → #281 → #282 → #283.

@Victsm

Victsm commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

Change map

User-requested aggregation consolidation, slice 4/5.

Observed requirement and smallest sufficient change: Use DG-owned anchor batches and native model results directly instead of rebuilding per-anchor Python requests/results in the supported fixed-size model route. Sequence loading keeps native component merging inside iteration; graph loading preserves topology/features together. Preserve split/shuffle/padding policies, contract validation, and existing Windows/reference fallbacks. Encoder fitting remains unchanged.

Focused verification / stopping condition: 64 focused native-anchor/model-batch tests passed at this checkpoint; package Ruff check and formatting pass. The complete stack additionally passed 1,951 public tests (3 CUDA skips; 47 notebook deselections). Stop at this slice’s verified behavior; fitting redesign, new temporal/LP capabilities and live sweeps are separate.

File Function / scope What changed Why
packages/experimental/kumo-pytorch/conformance/properties.py Native work and split invariants Instrument batch builds; verify deterministic shuffle and exact coverage Apply the documented engine-order contract to native loaders
packages/experimental/kumo-pytorch/src/kumo/pytorch/assembly.py PreparedBatchPlan model-column assembly Accept batch-level native values and enforce model contracts Keep tensor encoding/layout policy without reconstructing records
packages/experimental/kumo-pytorch/src/kumo/pytorch/data/model.py ModelSourceBatch Carry native ordered-event/model value metadata Adapt native batches while preserving existing model-facing structure
packages/experimental/kumo-pytorch/src/kumo/pytorch/engines/diskgraph.py DiskGraphSource / native model and scalar adapters Connect declared contracts to prepared execution and native column batches Reuse the common backend while preserving model layout and validation
packages/experimental/kumo-pytorch/src/kumo/pytorch/engines/native_anchors.py NativeAnchors / fitting and batch iteration Own native seed views and consume native model batches Preserve identity/order/partitioning without a second Python anchor store
packages/experimental/kumo-pytorch/src/kumo/pytorch/preparation.py Plan.materialize / materialize_values Bind native anchors and discover persisted scalar bindings Preserve explicit fitting/split policy while exposing precompute workflow
packages/experimental/kumo-pytorch/src/kumo/pytorch/task.py Task batch iteration Delegate eligible selections to native anchor iteration Preserve caller ordering, shuffle, rank and padding policy
packages/experimental/kumo-pytorch/tests/conformance/test_functional_harness.py Conformance negative control Inject growing work on build_model_batch Ensure native work regressions still fail the property
packages/experimental/kumo-pytorch/tests/test_model_column_hotpath.py Focused regression coverage Verify numerical, clock, identity, lifecycle or route behavior of this slice Catch wrong answers and unintended fallback/reconstruction, not merely successful calls
packages/experimental/kumo-pytorch/tests/test_native_anchor_handoff.py Focused regression coverage Verify numerical, clock, identity, lifecycle or route behavior of this slice Catch wrong answers and unintended fallback/reconstruction, not merely successful calls

Verified repair — 2026-11-11

Head: 98fa94ecadccc81b4f377df600c54e116a6522d9. Native-only tests skip when DiskGraph is absent. Conformance counts native batch work and checks the documented shuffle contract: determinism and exact coverage, with unshuffled order preserved.

Validation: repaired combined stack passed 146 Rust TT integration tests and 579 Python native/conformance tests with a freshly rebuilt extension. Public Python suite: 1,951 passed; optional-native import checks: two modules skipped as expected without DiskGraph. Generated artifact manifest and 403-component notice checks passed. No live sweeps or merges.

@copy-pr-bot

copy-pr-bot Bot commented Nov 10, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@Victsm
Victsm force-pushed the codex/prepared-scalar-execution-20261110 branch from eccadec to b59d299 Compare November 11, 2026 03:47
@Victsm
Victsm force-pushed the codex/anchor-history-native-20261110 branch from 33a82dd to 98fa94e Compare November 11, 2026 03:47
@Victsm
Victsm had a problem deploying to gha-kumo-workflow-tests November 11, 2026 03:47 — with GitHub Actions Error
@Victsm
Victsm deployed to gha-kumo-workflow-tests November 11, 2026 03:47 — with GitHub Actions Active

This branch was successfully deployed

1 active deployment
gha-kumo-workflow-tests — 98fa94ec Deployed Nov 11, 2026 by Victsm via import #2046
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant