[Victsm]

Summary

Add materialize_values and MaterializedValues so scalar features/generated targets are computed once into one enriched AnchorTable. Read projected source chunks through the existing custom-TT reader, reuse one prepared native plan, and stream typed output while preserving IDs, splits, metadata and external labels. Later native batches consume cached scalar columns while graph/history retrieval stays dynamic. The unchanged inspection/fitting route reuses the common kernel; there is no separate serving aggregation backend.

Stack slice 5/5. Targets the preceding stack branch; merge in dependency order.

Change map

#283 (comment)

Testing

Complete stack: 1,951 public tests passed (3 CUDA skips; 47 notebook deselections), 567 bounded native tests, and 221 Rust tests (3 existing ignores). Post-format executable documentation/native-cache tests: 30 passed. Package Ruff check and formatting pass. Matched warm-cache CPU benchmark (same typed scalar output writer, 65,536 anchors, 256-row chunks, one thread): original 244.1s, round1 171.5s, round2 164.0s, revised 153.1s; all output digests match. This is scalar compute+write, not model training. Anonymous peak was about 288 MiB plus about 52.45 GiB file-backed RSS; no memory-reduction claim. Final publication build: 400 Rust library tests, 567 native Python tests and 44 public materialization/cache/type/docs tests passed. Its scale rerun took 152.7s with the same exact output digest. strict Clippy passes for all targets in the three modified Rust crates. No model training or live BP sweep.

Dependency order: #279 → #280 → #281 → #282 → #283.

[Victsm]

Change map

User-requested aggregation consolidation, slice 5/5.

Observed requirement and smallest sufficient change: Add materialize_values and MaterializedValues so scalar features/generated targets are computed once into one enriched AnchorTable. Read projected source chunks through the existing custom-TT reader, reuse one prepared native plan, and stream typed output while preserving IDs, splits, metadata and external labels. Later native batches consume cached scalar columns while graph/history retrieval stays dynamic. The unchanged inspection/fitting route reuses the common kernel; there is no separate serving aggregation backend.

Focused verification / stopping condition: Complete stack: 1,951 public tests passed (3 CUDA skips; 47 notebook deselections), 567 bounded native tests, and 221 Rust tests (3 existing ignores). Post-format executable documentation/native-cache tests: 30 passed. Package Ruff check and formatting pass. Matched warm-cache CPU benchmark (same typed scalar output writer, 65,536 anchors, 256-row chunks, one thread): original 244.1s, round1 171.5s, round2 164.0s, revised 153.1s; all output digests match. This is scalar compute+write, not model training. Anonymous peak was about 288 MiB plus about 52.45 GiB file-backed RSS; no memory-reduction claim. Final publication build: 400 Rust library tests, 567 native Python tests and 44 public materialization/cache/type/docs tests passed. Its scale rerun took 152.7s with the same exact output digest. strict Clippy passes for all targets in the three modified Rust crates. No model training or live BP sweep. Stop at this slice’s verified behavior; fitting redesign, new temporal/LP capabilities and live sweeps are separate.

File	Function / scope	What changed	Why
packages/experimental/kumo-pytorch/docs/04-engines-and-data.md	API and workflow documentation	Describe supported routes, preparation boundaries and explicit limitations	Keep usage and performance claims aligned with the implementation
packages/experimental/kumo-pytorch/docs/api/README.md	API and workflow documentation	Describe supported routes, preparation boundaries and explicit limitations	Keep usage and performance claims aligned with the implementation
packages/experimental/kumo-pytorch/docs/api/lifecycle.md	API and workflow documentation	Describe supported routes, preparation boundaries and explicit limitations	Keep usage and performance claims aligned with the implementation
packages/experimental/kumo-pytorch/src/kumo/pytorch/aggregate_materialization.py	materialize_values / MaterializedValues / cached binding	Stream and reopen one typed enriched anchor file	Compute scalars once and preserve durable feature/target bindings
packages/experimental/kumo-pytorch/src/kumo/pytorch/api.py	Public exports	Export MaterializedValues	Expose the durable materialization handle without changing fitting policy
packages/experimental/kumo-pytorch/src/kumo/pytorch/engines/diskgraph.py	DiskGraphSource / native model and scalar adapters	Connect declared contracts to prepared execution and native column batches	Reuse the common backend while preserving model layout and validation
packages/experimental/kumo-pytorch/src/kumo/pytorch/preparation.py	Plan.materialize / materialize_values	Bind native anchors and discover persisted scalar bindings	Preserve explicit fitting/split policy while exposing precompute workflow
packages/experimental/kumo-pytorch/tests/api_reference.py	Focused regression coverage	Verify numerical, clock, identity, lifecycle or route behavior of this slice	Catch wrong answers and unintended fallback/reconstruction, not merely successful calls
packages/experimental/kumo-pytorch/tests/test_aggregate_materialization.py	Focused regression coverage	Verify numerical, clock, identity, lifecycle or route behavior of this slice	Catch wrong answers and unintended fallback/reconstruction, not merely successful calls
packages/experimental/kumo-pytorch/tests/test_materialized_native_batches.py	Focused regression coverage	Verify numerical, clock, identity, lifecycle or route behavior of this slice	Catch wrong answers and unintended fallback/reconstruction, not merely successful calls

Verified repair — 2026-11-11

Head: 3e686bd336fd0062dab9a217f66b86c3b6177b65. Native composition tests skip without DiskGraph; inherits the verified repairs from the preceding slices.

Validation: repaired combined stack passed 146 Rust TT integration tests and 579 Python native/conformance tests with a freshly rebuilt extension. Public Python suite: 1,951 passed; optional-native import checks: two modules skipped as expected without DiskGraph. Generated artifact manifest and 403-component notice checks passed. No live sweeps or merges.

Follow-up verified repair: dfa67ae599d1902018c03b5fea93659f92e0e598 preserves NULL profile values in the in-memory scalar writer. The new statistics regression exposed a missing dictionary entry; seven focused materialization tests pass, including a NULL profile field. This does not publish either local follow-up.

Current head c2bbeebd48eb491655bdfeae11fdef5348dde46c also gates the native-only function inside the mixed materialization test module. Locally: 6 passed/1 skipped without DG, and all 7 passed with DG. This addresses the remaining optional-package CPU failure.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: NVIDIA/kumo-engine
• Branch: codex/enriched-anchor-values-20261110
• File: .coderabbit.yaml
• Commit: c2bbeeb
• Layers: Repository YAML
• Review profile: QUIET
• Plan: Enterprise
• Run ID: fa6b7cd6-2994-4d70-8800-b17cf9eea190

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts