Skip to content

⚡️ Improve DD table hashing and capacity growth - #2561

Merged
burgholzer merged 2 commits into
mainfrom
codex/dd-table-adaptation
Sep 17, 2026
Merged

burgholzer merged 2 commits into
mainfrom
codex/dd-table-adaptation

Conversation

@burgholzer

@burgholzer burgholzer commented Sep 14, 2026

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

Description

Aligned node addresses can leave compute-table buckets unused, while fixed
unique-table directories waste memory on sparse levels and build long chains
on crowded levels. Mix pointer hashes with the existing murmur64 function in
both binary and unary compute tables. Let populated unique-table levels grow
independently without moving canonical nodes.

Vector and matrix unique tables now start with 1,024 buckets per level and
grow up to 1,048,576, using the general profile selected in the table study.
DDPackageConfig::utMaxNumBucket sets the ceiling. Initial and maximum
capacities must be powers of two; the maximum must cover both initial sizes.
Equal initial and maximum capacities give fixed sizing without a zero sentinel.

Grown directories survive collection and reset. Bucket views can be invalidated
by insertion, but canonical nodes and owned roots remain stable.
UniqueTable::hash requires an allocated level. Custom initial capacities above
the default ceiling require a correspondingly larger maximum. No dependencies
were added.

For a 4,096-qubit zero state, default unique bucket storage falls from 2,048 to
64 MiB. The matrix-vector compute cache remains at 16,384 entries: the tested
262,144-entry option helped some reuse-heavy workloads but had mixed timing
results. Real-number canonicalization is unchanged; its sorted buckets and
tolerance-based boundary matching require a separate growth implementation.

Validation: all 196 native DD tests pass, including automatic default growth,
fixed sizing, invalid ceilings, canonical identity, owned roots, collection/reset,
and unary cache lookup/invalidation with aligned keys. On this host, the focused
unary check occupied 1 of 64 buckets before mixing and 39 of 64 afterward.
Repository lint and C++ lint pass. Existing upstream build diagnostics remain.

Codex assisted with implementation, review, validation, and this description.

Checklist

  • The pull request only contains commits that are focused and relevant to this change.
  • I have added appropriate tests that cover the new/changed functionality.
  • I have updated the documentation to reflect these changes.
  • The changes follow the project's style guidelines and introduce no new warnings.
  • The changes are fully tested and pass the CI checks.
  • I have reviewed my own code changes.

If PR contains AI-assisted content:

  • Any agent that created, edited, or submitted GitHub content was explicitly authorized for that scope, as required by our AI Usage Guidelines.
  • Every agent-authored or agent-edited public text body begins with the visible disclosure 🤖 *AI text below* 🤖 (titles are exempt).
  • I have disclosed AI assistance in the PR description.
  • I confirm that I have personally reviewed and understood all AI-generated content, and accept full responsibility for it.

Mix pointer operands before masking binary compute-table hashes so
aligned node addresses do not waste cache buckets. Allow unique-table
levels to grow independently without moving canonical nodes.

Keep growth opt-in through a per-level bucket ceiling and retain grown
capacities across reset. Reuse level initialization when constructing a
unique table, and cover rehashing, roots, collection, and fixed hashing.

Assisted-by: GPT-6 via Codex
@burgholzer burgholzer added enhancement Improvement of existing feature DD Anything related to the DD package c++ Anything related to C++ code labels Sep 14, 2026
@burgholzer burgholzer self-assigned this Sep 14, 2026
@codecov

codecov Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Start unique-table levels with 1,024 buckets and grow them to a ceiling
of 1,048,576. Fixed capacity uses equal initial and maximum sizes, so
hashing and validation no longer need a zero-capacity sentinel.

Mix pointer hashes in unary compute tables as well as binary tables.
Cover default growth, fixed sizing, invalid ceilings, and aligned-key
cache lookup and invalidation.

Assisted-by: GPT-6 via Codex

@burgholzer burgholzer left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting this in now. This is quite a measurable improvement in memory and runtime performance across the board.

@burgholzer
burgholzer enabled auto-merge (squash) September 17, 2026 20:29
@burgholzer
burgholzer merged commit f7566b2 into main Sep 17, 2026
25 of 26 checks passed
@burgholzer
burgholzer deleted the codex/dd-table-adaptation branch September 17, 2026 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Anything related to C++ code DD Anything related to the DD package enhancement Improvement of existing feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants