Skip to content

[2/n] Add target-first offline mask reuse calibration - #2235

Draft
kaix-nv wants to merge 1 commit into
kaix/vllm_skip_calib_upstreamfrom
kaix/mask-reuse-01-offline-calibration
Draft

[2/n] Add target-first offline mask reuse calibration#2235
kaix-nv wants to merge 1 commit into
kaix/vllm_skip_calib_upstreamfrom
kaix/mask-reuse-01-offline-calibration

Conversation

@kaix-nv

@kaix-nv kaix-nv commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Type of change: new feature

Adds the first mask-reuse sub-PR: target-first offline calibration, compact capture selection, checkpoint identity binding, and fail-closed candidate publication. This slice intentionally excludes vLLM capture and topology discovery so its calibration contract can be reviewed independently.

This PR is stacked on #1992.

Usage

python examples/vllm_serve/create_checkpoint_manifest.py <CKPT> --model-id <MODEL_ID>
python examples/vllm_serve/calibrate_mask_reuse.py \
  --checkpoint <CKPT> \
  --compact-captures <CAPTURES.jsonl> \
  --capture-manifest <CAPTURES.manifest.json> \
  --vanilla-config <CKPT>/config.json \
  --topology <TOPOLOGY.json> \
  --calibration-plan <PLAN.json> \
  --family-registry <FAMILIES.json> \
  --grouped-fit <GROUPED_FIT.json> \
  --outer-report <OUTER_REPORT.json> \
  --max-anchor-dropped-mass 0.02 \
  --reuse-dropped-mass-report-threshold 0.02 \
  --target-bmm1-skip-ratio 0.50 \
  --output-policy <POLICY.json> \
  --output-report <REPORT.json>

Testing

  • python -m pytest -q on the four checkpoint, selector, compact-selector, and CLI test modules: 36 passed.
  • pre-commit run --files ...: passed.
  • Commit signature verified.

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded trust_remote_code=True, torch.load(..., weights_only=False), pickle, etc.).

  • Is this change backward compatible?: ✅
  • If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A — no copied code or new dependency.
  • Did you write any new necessary tests?: ✅
  • Did you update Changelog?: ❌ — deferred until the complete stack is ready for review.
  • Did you get Claude approval on this PR?: ❌ — pending before ready for review.

Additional Information

Stack 1 of 3. The complete three-branch tree is source-equivalent to kaix/mask-reuse-calib for all mask-reuse implementation and feature-test files.

@copy-pr-bot

copy-pr-bot Bot commented Aug 24, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@kaix-nv kaix-nv changed the title [Sparse Attention] Add target-first offline mask reuse calibration [2/n] Add target-first offline mask reuse calibration Aug 24, 2026
@kaix-nv
kaix-nv force-pushed the kaix/mask-reuse-01-offline-calibration branch from 64bdaf1 to 606592e Compare August 25, 2026 04:47
Signed-off-by: Kai Xu <kaix@nvidia.com>
@kaix-nv
kaix-nv force-pushed the kaix/mask-reuse-01-offline-calibration branch from 606592e to 2229a8c Compare August 25, 2026 06:32
@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://NVIDIA.github.io/Model-Optimizer/pr-preview/pr-2235/

Built to branch gh-pages at 2026-08-25 06:37 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@codecov

codecov Bot commented Aug 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.65451% with 240 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.00%. Comparing base (f361582) to head (2229a8c).

Files with missing lines Patch % Lines
...rsity/attention_sparsity/calibration/mask_reuse.py 86.27% 101 Missing ⚠️
...tention_sparsity/calibration/mask_reuse_compact.py 87.00% 89 Missing ⚠️
...ention_sparsity/calibration/checkpoint_manifest.py 79.91% 50 Missing ⚠️
Additional details and impacted files
@@                        Coverage Diff                        @@
##           kaix/vllm_skip_calib_upstream    #2235      +/-   ##
=================================================================
+ Coverage                          78.82%   79.00%   +0.18%     
=================================================================
  Files                                524      527       +3     
  Lines                              60979    62652    +1673     
=================================================================
+ Hits                               48066    49499    +1433     
- Misses                             12913    13153     +240     
Flag Coverage Δ
unit 56.40% <85.65%> (+0.80%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant