Extend PyTorch RTN weight quantization to MoE experts by Copilot · Pull Request #2584 · microsoft/Olive

Copilot · 2026-07-21T23:13:59Z

Summary

Extends Olive's native PyTorch RTN quantization (olive/common/quant/,
olive/passes/pytorch/rtn.py) to cover MoE fused-expert weights, in addition
to the existing nn.Linear / nn.Embedding support. Produces a standard HF
safetensors checkpoint with MoE experts already quantized, so downstream
consumers (Mobius / ORT GenAI ModelBuilder) don't need to run their own
inline quantization pass.

Design (carried over from `jambayk/moe-quant`, unchanged)

Storage-only MoE quantization. No QuantExperts module re-implementing
per-architecture forwards (that design was considered and rejected — see
the original design notes). Instead, a torch.Tensor wrapper subclass,
QuantTensor (modelled on Quark's quark.qtensor.QTensor), replaces the
quantized parameter in-place. Host model forwards run unchanged.
Uniform by parameter, not by layer type. Target selection
(selection.py) treats every nn.Parameter the same way regardless of
rank — 2D linear/embedding weights and 3D fused-expert weights both flow
through the same WeightQuantizer, which quantizes along the last dim
regardless of rank.
Mobius owns ONNX export of MoE experts. Olive never attempts
torch.onnx.export on a 3D QuantTensor; this is an explicit, permanent
design decision, documented in-code and via onnxruntime/mobius#427.

What changed in this PR on top of `jambayk/moe-quant`

selection.py (new) — single generator (iter_quant_targets)
unifying target selection for both the HF quantizer and RTN/GPTQ passes.
Adds MoE-aware routing detection (_collect_experts,
_layers_missing_experts, _config_indicates_moe) with fail-closed
behavior: if the config looks like an MoE architecture but the experts
subtree can't be resolved, quantization refuses to proceed rather than
silently skipping the expert weights.
wrapper.py — LayerWrapper.get_experts() / get_router()
accessors, generalizing the existing per-layer-type accessor pattern to
MoE sub-modules.
Round-1 remediation (8 items from the first adversarial review pass)
— assorted correctness/robustness fixes to selection, patterns, and
QuantTensor construction.
Round-2 remediation (4 items):
- patterns.py: reject nested-group alternation ((a|b) inside a
  repeated group) at any nesting depth — closes a ReDoS bypass of the
  skip-pattern regex safety check. Docstrings demoted from "prevents
  ReDoS" to "best-effort UX check, not a security boundary" (decision:
  no regex third-party dependency added).
- tensor.py / hf_utils.py / state_dict.py: explicit
  is_placeholder flag threaded through QuantTensor's lifecycle so
  init-style ops (zero_, normal_, ...) only no-op on real
  placeholders and raise otherwise (previously any QuantTensor
  silently no-op'd on these ops, which could mask real bugs).
- tensor.py: reject rank>1 boolean-mask indexing instead of
  misclassifying it as a safe leading-dim integer index.
- selection.py / defaults.yaml: rewrite _config_indicates_moe to
  reuse the existing resolve_alias() nested-config mechanism (already
  used for HF I/O config resolution) plus a bounded sub-config sweep —
  fixes DBRX-style nested MoE config detection (ffn_config.moe_num_experts).

All changes verified: 314 tests passing (test/common/quant/,
test/passes/pytorch/test_rtn.py), lintrunner clean.

Known follow-ups (tracked separately, not blocking this PR)

MoE quantization: 5 unresolved edge-case bugs in QuantTensor indexing / regex safety checks #2598 — 5 edge-case bugs found in round-3 adversarial review
(QuantTensor indexing safety, patterns.py regex safety). None are
reachable via the currently-supported RTN pipeline (round-to-nearest,
no live forward pass), so they don't block merging this PR, but should
be fixed before any pass that runs a live forward through a quantized
MoE QuantTensor (see MoE quantization: extend native GPTQ pass (gptq.py) to support MoE experts #2599).
MoE quantization: extend native GPTQ pass (gptq.py) to support MoE experts #2599 — extend the native GPTQ pass to support MoE experts (requires
redesigning the calibration forward-hook mechanism for fused-3D expert
tensors — not a simple flag flip).
MoE quantization: investigate AWQ (autoawq) support for MoE experts #2600 — investigate whether the autoawq pass can support MoE
experts (depends on upstream autoawq library capability, not just
Olive-side plumbing).

Testing

pytest test/common/quant/ test/passes/pytorch/test_rtn.py — 314 passed.
lintrunner — clean.
Full forward-parity tests (test_forward_parity.py) included, comparing
quantized vs. unquantized model outputs for both 2D and 3D (MoE) targets.

…ent) This commit checkpoints the in-progress MoE quantization work before a larger refactor that deletes QuantLinear/QuantEmbedding in favour of storing every quantized weight (2D linear, 2D embedding, 3D MoE experts) as a QuantTensor nn.Parameter on the original host module. Included so far: - New olive/common/quant/patterns.py for re: prefix matching in modules_to_not_convert / overrides. - New olive/common/quant/tensor.py with QuantTensor wrapper subclass (_make_wrapper_subclass + __torch_function__ + __torch_dispatch__), supporting 2D and 3D layouts. - LayerWrapper.get_experts() / get_router() accessors. - 3D quantize helpers in olive/common/quant/utils.py. - moe field on OliveHfQuantizationConfig. - _process_model_before_weight_loading skips ModuleList(Expert) subtrees when moe=False, fixing a latent silent-quantization bug for Mixtral / PhiMoE / Qwen2/3-MoE. - Fused-3D MoE support in prepare_model / finalize via QuantTensor parameters; current save layout uses _qweight buffer suffixes — to be replaced in the upcoming refactor with the canonical <param>.qweight/.scales/.qzeros layout. - ModelBuilder raises NotImplementedError for Olive-quantized MoE checkpoints (Mobius is the intended consumer). - Test additions: test/common/quant/test_patterns.py, test/common/quant/test_tensor.py, TestOliveHfQuantizerMoE / TestRegexOverrides in test_hf_utils.py, test/passes/pytorch/test_quant_utils.py for flatten helper, test_olive_quantized_model_raises_for_moe in test_model_builder.py. - 294 tests pass; lintrunner clean (--skip PYLINT). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Switch Olive's native quantization representation to a single design: every quantized weight is an nn.Parameter(QuantTensor) on the original host nn.Linear / nn.Embedding / fused-3D experts module, with sibling <pname>_qweight / _scales / _qzeros buffers aliasing the QuantTensor's inner tensors. Save: a state-dict hook drops the QuantTensor parameter entry; the buffers already carry the data (plain Tensors, safetensors-friendly). Load: HF's loader fills the buffers natively via dotted paths; a post-load helper re-binds the QuantTensor inner refs to the freshly loaded buffer storage. QuantLinear / QuantEmbedding (olive/common/quant/nn.py) are kept only as ONNX-exportable wrappers used by make_export_compatible_quant; they are no longer the runtime representation. * New olive/common/quant/state_dict.py with install_quant_tensor_param and refresh_quant_tensor_refs helpers. * OliveHfQuantizer rewritten for the new layout (placeholder install before weight load + ref refresh after). * finalize() in passes/pytorch/quant_utils.py installs QuantTensor params via install_quant_tensor_param (replaces the old flatten_quant_tensor_params helper). * prepare_model skips modules whose weight is already a QuantTensor, so composing multiple Rtn passes on top of a partially quantized model works. * make_export_compatible_quant detects nn.Linear / nn.Embedding whose weight is a QuantTensor and swaps them with QuantLinear / QuantEmbedding wrappers before any model dtype casting, preserving the existing com.microsoft::MatMulNBits / com.microsoft::GatherBlockQuantized symbolic export path. * OliveQuantizedModel (model_builder.py) normalizes the new <dotted>.weight_qweight key layout back to the legacy <dotted>.qweight layout for the existing genai loader, and raises NotImplementedError for moe=True checkpoints. * Tests updated to assert against QuantTensor weight instead of isinstance(module, QuantLinear); legacy tie_quant_modules tests removed; new install_quant_tensor_param test suite added. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…er to N-D * Remove olive/common/quant/nn.py (QuantModule, QuantLinear, QuantEmbedding) entirely. The only purpose of those modules was ONNX export, which is now handled by reusing the existing QuantLinearNbit from olive/common/hf/quant.py and a new parallel QuantEmbeddingNbit (com.microsoft::GatherBlockQuantized symbolic) in the same file. * Add QuantLinearNbit.from_quant_tensor / QuantEmbeddingNbit.from_quant_tensor factories so make_export_compatible_quant can swap any nn.Linear / nn.Embedding whose weight is a QuantTensor into the export wrappers. * Generalize WeightQuantizer (get_num_groups, get_qparam_shape, find_qparams, quantize, dequantize, _reshape_tensor) and pack_to_uint8 / unpack_from_uint8 to operate on any N-D tensor; quantization is always along the last dim, leading dims are preserved. * Drop quantize_along_leading_dim / pack_to_uint8_along_last / unpack_from_uint8_along_last and the explicit 3D leading-dim loops in QuantTensor.from_float and _dequantize. * Delete test/common/quant/test_nn.py; add N-D tests for the generalized quantizer + pack helpers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Instead of pre-walking the safetensors dict to rewrite ``<dotted>.weight_qweight`` -> ``<dotted>.qweight``, derive the destination attribute name inside ``set_tensor`` once we already know ``submodule`` is a ``QuantizedTensorModule``. Strip any of the known Olive buffer suffixes (``QWEIGHT_SUFFIX``, ``SCALES_SUFFIX``, ``QZEROS_SUFFIX`` from ``olive.common.quant.state_dict``) from the last path component to produce the bare ``qweight`` / ``scales`` / ``qzeros`` attribute that the genai ``QuantizedTensorModule`` expects. Also drops internal dev-iteration version labels from comments and docstrings in olive/common/quant and olive/passes/onnx. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Both QuantTensor's 2D layout and QuantLinearNbit's MatMulNBits buffer layout pack the quantization axis as uint8 with the same in-byte order (low nibble = elem[2j], high nibble = elem[2j+1] for 4-bit, etc.). They differ only in qweight rank: QuantTensor uses (out, in / pack_factor), QuantLinearNbit uses (out, n_blocks, blob_size) where n_blocks * blob_size == in / pack_factor. So the conversion is a pure reshape; the previous unpack -> .t() -> from_tensors round-trip is unnecessary. scales and qzeros buffer shapes also match exactly between the two layouts, so they are copied as-is. For symmetric weights (QuantTensor.qzeros is None) we fill the QuantLinearNbit.qzeros buffer with the packed midq pattern that the contrib op expects. Verified numerically: F.linear via QuantTensor and the dequantize-from-buffers path through QuantLinearNbit produce bit-identical outputs across {4,8} bits, {symmetric, asymmetric}, {groupwise, per-channel}. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The ORT contrib MatMulNBits / GatherBlockQuantized ops treat a missing zero-points input as midq for unsigned quantization, matching Olive's symmetric-quantization convention. Drop the synthetic packed-midq buffer that was previously emitted for symmetric weights and instead omit the input entirely: * QuantLinearNbit gains a has_qzeros flag (default True for back-compat); pack/from_tensors/from_quant_tensor pass through None as needed. * QuantLinearTorchFunction (TorchScript + dynamo) skips the qzeros input when None, inserting an empty placeholder only when g_idx must be positionally aligned. * QuantEmbeddingTorchFunction.symbolic gains the missing dynamo arg exposed by the new symmetric-embedding export path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

g_idx alongside a missing qzeros is not a real combination in Olive (GPTQ always produces qzeros), so skip the empty-tensor placeholder and just omit qzeros from the input list entirely. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Replace duplicated model walks in hf_utils._process_model_before_weight_loading and quant_utils.prepare_model with a shared iter_quant_targets helper that returns a list of (module, dotted_name, param_name, shape, dtype, device, kind) entries. Selection rules (lm_head/embeds/moe category flags, skip patterns, extra_skip_modules, already-quantized) live in one place. - QuantLinearNbit/QuantEmbeddingNbit: raise instead of synthesising a placeholder when g_idx is supplied alongside symmetric quantization. - tie_quant_word_embeddings: require both input and output embeddings to already be QuantTensor-backed with matching shape/dtype before tying. - Fix CodeQL mismatched-assignment false positives in QuantTensor dispatch (index args directly), fix ruff D205/D401/PLW0108/A002 warnings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Neither attribute is read anywhere — the post-walk loop that produced the literal skip-name list was a leftover from before the refactor. The configured patterns already live on quantization_config. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Stop tagging modules with quant_info / quant_info_3d and stop branching on 2D vs 3D in the quantization passes. The quantizer already operates along the last dim regardless of rank, so a single iteration over parameters that carry a quant_info attribute is enough. - QuantTarget slims to (module, module_name, pname, full_name) with a .param property; the caller reads shape/dtype/device from the parameter directly. No more 'kind' field. - prepare_model writes target.param.quant_info in one pass — both 2D linear/embedding weights and fused experts parameters use the same code path. The quant_info_3d dict-stash on experts modules is gone. - finalize iterates every parameter that has quant_info, calls QuantTensor.from_float (already rank-generic), and installs in place. - GPTQ and AutoClip read module.weight.quant_info; module discovery uses hasattr(module.weight, 'quant_info') instead of a module-level attribute. - HF placeholder install pulls shape/dtype/device off target.param and the placeholder builder is now rank-generic. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The dataclass had four fields and a one-line .param property, used by two callers. A plain tuple is shorter, matches how the layerwise quantization loop already iterates over (module, pname, param, info) tuples, and removes the unused module_name field and dead for_each_target helper. QuantTarget remains as a type alias for the public signature. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Filter fused-MoE params to 2D/3D ranks in iter_quant_targets so a 1D bias-like parameter fails at selection time instead of much later in finalize. * refresh_quant_tensor_refs: also check isinstance(param.data, QuantTensor) for forward-compat with future torch versions that may not return the underlying subclass from nn.Parameter(). * OliveHfQuantizationConfig: replace bare '# pylint: disable' with the specific super-init-not-called rule; use output.get(k) in to_dict. * finalize: log a warning when moe=True that the resulting checkpoint isn't directly ONNX-exportable via the Olive conversion pass — it must be consumed by an MoE-aware model builder. * Add regression test that _module_weight_has_quant_info ignores nn.LayerNorm / nn.Conv2d / unmarked nn.Linear (defends GPTQ/AutoClip discovery against future drift). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- __torch_dispatch__ clone/contiguous now forwards extra args/kwargs. - iter_quant_targets skips ALL nn.Embedding when embeds=False (positional / token-type embeddings like GPT-2 wpe are no longer silently quantized). - WeightQuantizer assertion message: 2/4/8-bit (was 4/8-bit). - tie_quant_word_embeddings: mark dst aliased buffers non-persistent so safetensors save emits one copy of qweight/scales/qzeros. - finalize: group selected params by host module so each module's to(device)/to(cpu) cycle runs once for MoE experts modules carrying multiple 3D weight params. - state_dict: add ensure_state_dict_hooks(model) defensive walk that installs the save hook on every host module that owns a QuantTensor parameter (idempotent). - Add test_forward_parity.py: bit-exact eager parity for full models (embedding + linears) and fused 3D MoE forwards, plus end-to-end ONNX export -> onnxruntime numerical parity for Olive-quantized nn.Linear via make_export_compatible_quant. - Enable pylint by adding file-level protected-access disables on the files that intentionally touch nn.Module._parameters. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

azure-pipelines · 2026-07-21T23:14:09Z

Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run.

…main Resolve conflicts in quant_utils.py by combining the param-level iter_quant_targets walk with main's QKV-aware override renormalization and QuantTensor-based already-quantized detection. Adapt kquant.py and its tests to the storage-only (param-level quant_info) API.

…x validator + tests

…iles

Copilot

Pull request overview

This PR extends Olive’s native PyTorch RTN-style weight quantization to support Mixture-of-Experts (MoE) expert weights by moving from module-swapping (QuantLinear/QuantEmbedding) to a parameter-level, storage-only representation (QuantTensor) that can also handle fused 3D expert parameters. It also centralizes quantization target selection and pattern matching, updates ONNX export/model-builder integration boundaries, and adds extensive regression and parity tests.

Changes:

Introduces QuantTensor + state-dict helpers to quantize weights (including fused 3D MoE expert tensors) without swapping parent modules, and removes olive.common.quant.nn.
Adds centralized quantization target selection (iter_quant_targets) and pattern matching helpers (re: regex support with safety validation; insertion-order override precedence).
Updates PyTorch passes (RTN/GPTQ/KQuant/AutoClip), ONNX ModelBuilder behavior, documentation, and adds broad test coverage (selection, tensor behavior, regex safety, parity, and model-builder rejection for MoE).

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
test/passes/pytorch/test_rtn.py	Updates RTN tests to assert quantization via `QuantTensor`-backed weights instead of `QuantLinear`/`QuantEmbedding`.
test/passes/pytorch/test_quant_utils.py	Adjusts quant-utils tests for parameter-level `quant_info` and adds new state-dict helper coverage.
test/passes/pytorch/test_kquant.py	Updates KQuant tests to use `QuantTensor` checks and bit assertions.
test/passes/pytorch/test_gptq.py	Updates GPTQ tests to validate `QuantTensor`-backed weights and composition behavior.
test/passes/onnx/test_model_builder.py	Adds coverage ensuring ModelBuilder rejects Olive-quantized MoE checkpoints.
test/common/quant/test_utils.py	Extends quant utils tests to validate N-D quantization and pack/unpack helpers.
test/common/quant/test_tensor.py	New tests for `QuantTensor` 2D/3D behavior, indexing, movement guards, and ONNX-export guards.
test/common/quant/test_selection.py	New tests for `iter_quant_targets` selection rules including MoE gating, fail-closed behavior, and already-quantized skipping.
test/common/quant/test_patterns.py	New tests for override/skip matching semantics and regex safety validation.
test/common/quant/test_nn.py	Removes tests for deprecated `QuantLinear`/`QuantEmbedding` module wrappers.
test/common/quant/test_hf_utils.py	Updates HF quantizer tests for `QuantTensor`-based layout, adds MoE and regex-override coverage.
test/common/quant/test_forward_parity.py	New numerical parity tests (eager vs dense reference) plus ONNX export parity for the export-compatible wrappers.
olive/passes/pytorch/rtn.py	Enables MoE support in RTN pass config via `allow_moe=True`.
olive/passes/pytorch/quant_utils.py	Refactors quantization to parameter-level selection, adds MoE config/options, and installs `QuantTensor` params during finalize.
olive/passes/pytorch/kquant.py	Adjusts KQuant to use weight-level `quant_info` and shared `_module_weight_has_quant_info`.
olive/passes/pytorch/gptq.py	Migrates GPTQ calibration/processing to store metadata on `module.weight.quant_info`.
olive/passes/pytorch/autoclip.py	Migrates AutoClip input caching and processing to use `module.weight.quant_info`.
olive/passes/onnx/model_builder.py	Rejects Olive-quantized MoE checkpoints; adds suffix mapping so Olive’s `_qweight/_scales/*_qzeros` buffers load into expected ModelBuilder attributes.
olive/common/quant/utils.py	Generalizes quantization and packing utilities to N-D tensors (quantize/pack along last dim).
olive/common/quant/tensor.py	New `QuantTensor` tensor-subclass implementing storage-only quantization with dispatch, 3D MoE indexing behavior, and ONNX/movement guards.
olive/common/quant/state_dict.py	New state-dict utilities for installing `QuantTensor` parameters + buffer aliases and refreshing references after load.
olive/common/quant/selection.py	New shared quantization target selection logic with MoE-aware rules and fail-closed behavior.
olive/common/quant/patterns.py	New pattern matching helpers for overrides/skip patterns with `re:` support and regex safety validation.
olive/common/quant/nn.py	Removes deprecated `QuantLinear`/`QuantEmbedding` module implementations.
olive/common/quant/hf_utils.py	Updates HF quantization config and quantizer implementation to use `QuantTensor` placeholders + state-dict refresh; adds regex overrides.
olive/common/hf/wrapper.py	Adds MoE experts/router conventions and accessors on `LayerWrapper`.
olive/common/hf/quant.py	Adds export-compatible wrappers creation from `QuantTensor` and improves symmetric handling (optional qzeros) for contrib ops.
docs/source/features/quantization.md	Documents PyTorch Native RTN, MoE flag behavior, regex semantics/safety, precedence rules, and migration away from `QuantLinear`/`QuantEmbedding`.

transformers>=5.x defaults save_pretrained(save_original_format=True), which for Mixtral-family MoE architectures round-trips the on-disk state dict through a legacy per-expert nn.Linear-shaped layout (splitting the fused-3D experts.gate_up_proj/down_proj into experts.{i}.w1/w2/w3.weight and back). That reshape/(un)fuse machinery assumes plain float weight tensors and silently drops the trailing group-size dimension of our quantized _scales/_qzeros buffers, which crashes real forward() calls on the reloaded model. Request the new non-legacy on-disk format (save_original_format=False) when supported, so the fused-3D quantized buffers round-trip byte-for- byte as-is. Falls back to the default for older transformers versions that don't accept this kwarg (they also predate the legacy-format conversion machinery, so there's nothing to opt out of). Add test_rtn_moe_real_forward_after_reload, a regression test that quantizes a real MoE model, saves via the actual pass output, reloads from disk, and calls the model's real forward() -- asserting no crash, no NaN/Inf, and that the fused-3D scales buffer keeps its group-size dimension. Verified this test fails at the exact corrupted-shape assertion without the fix, and passes with it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85549b60-0fb9-4d65-a4e8-7a8995939d68

Two related silent-data-integrity bugs in the placeholder lifecycle: 1. copy_() did not propagate/clear is_placeholder. Copying real data into a placeholder QuantTensor left it flagged as a placeholder, so a later in-place initializer could still silently no-op and discard the just-copied real data. Fix: mirror the source's is_placeholder state after copy_. 2. refresh_quant_tensor_refs() unconditionally cleared is_placeholder for every QuantTensor parameter in the model, even though it is invoked once for the whole model with no per-parameter signal about whether that parameter's checkpoint key was actually present. A parameter with a missing key keeps its original placeholder buffer objects untouched, so we can detect an actual load per-parameter by comparing buffer object identity: only clear is_placeholder if at least one buffer object was actually swapped by the loader. Added regression tests for both fixes plus the missing-key case in test_tensor.py.

Item 1: treat torch.uint8 tensor indices as legacy boolean masks in _is_bool_index, matching PyTorch's own semantics for that dtype (same bug class as the earlier boolean-mask fix, different dtype variant). Item 3 (strengthened): refresh_quant_tensor_refs now accepts an optional checkpoint_keys set and, when provided, uses exact full-dotted-key membership as the authoritative is_placeholder determination instead of the weaker buffer-identity heuristic (which in-place .copy_() loaders can fool). OliveHfQuantizer now captures checkpoint_files (a kwarg HF's preprocess_model already passes to _process_model_before_weight_loading) via a new _read_checkpoint_keys() helper that reads .safetensors headers through safe_open().keys(), and forwards it to _process_model_after_weight_loading. Item 4: QuantTensor.__getitem__ now rejects rank>=2 integer-tensor indices (e.g. an un-flattened (tokens, k) top-k routing tensor) instead of silently producing a >3D QuantTensor that can never be dequantized or re-indexed. Callers needing a multi-dim batch of expert ids should flatten to 1-D first and reshape the dense output afterward; this restriction (Option A) is chosen over generalizing arbitrary-rank support (Option B) because there is no validated caller or design for relaxing _maybe_dense's rank-based OOM guard, and no real consumer uses rank>=2 indexing today (confirmed via repo-wide search; even GPTQModel's reference MoE calibration uses per-expert scalar indexing, not batched rank>=2 gather). Item 5: documented as a known, deferred issue rather than fixed. Found a concrete working ReDoS-scanner bypass via `(?#...)` inline-comment regex syntax (the 3rd consecutive bypass of this blacklist-enumeration check). Per discussion, this is not treated as a security vulnerability under Olive's current trust model -- re: patterns are trusted, user-authored config running in the user's own process, not adversarial input crossing a trust boundary. Added a NOTE/TODO in patterns.py recording the bypass mechanism, the decision not to patch it now, and a sketched runtime-timeout-based alternative to revisit if this config path is ever exposed to untrusted input. Tests: 317 tests in test/common/quant/ pass (up from 306), plus 44 tests across test/passes/pytorch/test_rtn.py, test_gptq.py, test_kquant.py. lintrunner clean.

- Fix Critical: tied lm_head/embed_tokens embeddings corrupted after checkpoint reload (refresh_quant_tensor_refs rewritten to dedupe shared QuantTensor objects, pick one canonical source site, and alias all hosting modules' buffers back to it). - Fix Major: refresh_quant_tensor_refs silently left placeholder (zero) weights when a checkpoint was missing expected keys; now fails closed with a RuntimeError, requiring ALL mandatory buffers (qweight+scales, +qzeros if asymmetric) to show complete load evidence (AND logic, not OR) to avoid false negatives on partial buffer loads. - Fix Major: ModelBuilder ignored regex `overrides` for per-layer bits/group_size, now resolved via match_override. - Fix Major: QuantEmbeddingNbit had no ORT block_size validation (GatherBlockQuantized requires power-of-2, >=16); added _validate_onnx_block_size to both QuantEmbeddingNbit and QuantLinearNbit. - Fix Major: QuantEmbeddingNbit.from_quant_tensor scales/qzeros shape mismatch; now reshaped like QuantLinearNbit. - Add torch.equal override for QuantTensor: transformers 5.4's tie_weights() calls torch.equal on tied meta-device params before Olive's postprocess_model hook runs, which previously crashed. Found via a full-PR review pass (readability, code, critical, deep, integration reviewers + qa-tester) requested to confirm mergeability. Fixed across two rounds: a comprehensive fix for all findings, then a targeted fix for a partial-buffer false-negative in the fail-closed check that the round-1 targeted re-review (code + critical reviewers) caught. 448 tests passing (test/common/quant, test/common/hf/test_quant.py, test/passes/pytorch/test_rtn.py, test/passes/pytorch/test_gptq.py, test/passes/pytorch/test_kquant.py, test/passes/pytorch/test_quant_utils.py, test/passes/pytorch/test_autoclip.py, test/passes/onnx/test_model_builder.py), lintrunner clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85549b60-0fb9-4d65-a4e8-7a8995939d68

Ti-Tai Wang (titaiwangms) · 2026-08-01T01:38:49Z

Full-PR review pass (readability, code, critical, deep, integration reviewers + QA)

Per request, ran the full 6-agent review team over the entire PR diff (not just the recent commits), to confirm mergeability. This surfaced pre-existing bugs unrelated to #2598's original 5 items:

1 Critical

Tied lm_head/embed_tokens embeddings got corrupted after checkpoint save→reload. Root cause: refresh_quant_tensor_refs visited every module hosting the shared QuantTensor and did last-write-wins re-binding, picking up a stale placeholder object instead of the reloaded one. Verified end-to-end (diff was ~2.26e18 before fix).

4 Major

refresh_quant_tensor_refs silently left placeholder (zero) weights if a checkpoint was missing expected keys instead of failing. Fixed to fail closed with a RuntimeError, requiring all mandatory buffers (qweight+scales, +qzeros if asymmetric) to individually show complete load evidence — an initial round-1 fix used OR logic across buffers, which a targeted re-review (code + critical reviewers) caught as a false negative on partial buffer loads; fixed in round 2 with AND logic.
ModelBuilder ignored regex overrides for per-layer bits/group_size (only did a plain dict lookup). Fixed to use match_override.
QuantEmbeddingNbit had no ORT block_size validation; GatherBlockQuantized requires power-of-2 and >=16. Added validation to both QuantEmbeddingNbit and QuantLinearNbit.
QuantEmbeddingNbit.from_quant_tensor had a scales/qzeros shape mismatch; fixed to reshape like the Linear equivalent.

Also fixed: a torch.equal override for QuantTensor — transformers 5.4's tie_weights() calls torch.equal on tied meta-device params before Olive's postprocess_model hook runs, which previously crashed.

All fixed across two rounds (comprehensive fix, then a targeted fix for the partial-buffer false negative found in re-review), each verified with fail-before/pass-after tests. Added 2 new test files (test/common/hf/test_quant.py, test/common/quant/test_state_dict.py) plus a real end-to-end tied-embedding save→reload regression test in test/passes/pytorch/test_rtn.py.

Current status: 448 tests passing across the affected test modules, lintrunner clean, working tree clean. Commits 3ba5b683 (item 1/4-strengthening/5-doc from #2598) and 6baed50e (this review's fixes) pushed.

Two Minor findings remain deferred as non-blocking follow-ups (not filed as issues yet): torch.equal override implements encoding-equality rather than value-equality for quantized tensors; named_modules(remove_duplicate=True) could theoretically hide a dual-registered module in an edge case not currently exercised.

…k-moe-quant-extend-rtn-weight # Conflicts: # test/passes/onnx/test_model_builder.py

Ti-Tai Wang (titaiwangms) · 2026-08-04T23:53:39Z

cc Jambay Kinley (@jambayk) This PR adds QuantTensor infra that GPTQ will be using and RTN MoE support. Most of lines are for testing only. It's ready for review.

…weight

Xiaoyu (xiaoyu-work) · 2026-08-05T16:03:04Z

/azp run

azure-pipelines · 2026-08-05T16:03:20Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Xiaoyu (xiaoyu-work) · 2026-08-06T23:38:15Z

is this tested for real model?

Ti-Tai Wang (titaiwangms) · 2026-08-07T00:43:19Z

is this tested for real model?

Yes, I will present a comprehensive perf table with GPTQ in a new pr and link it to here.

Ti-Tai Wang (titaiwangms) · 2026-08-07T17:55:19Z

Xiaoyu (@xiaoyu-work) #2612

Jambay Kinley (jambayk)

Thanks! the changes look good to me. I haven't looked very closely at the specific implementation details but the high level concepts and flow look good to me. the improvement to hf loading and function call dispatches on the quant tensor is great as well.

## Problem PR #2584 merged RTN MoE quantization (`moe=True`) support without a layout check on fused-expert weights. RTN's `WeightQuantizer` groups unconditionally along a tensor's last dimension, which is only correct when the fused expert weight is stored `(num_experts, out_features, in_features)` -- K last. Architectures such as gpt-oss store the transposed `(num_experts, in, out)` layout instead, and are silently mis-quantized (wrong axis grouped, no error) today on `main` when run through `moe=True`. ## Fix Adds `olive/passes/pytorch/moe_support.py` with `check_moe_layout_support`, gated into RTN's `_run_for_config` after `prepare_model()` and before `finalize()`. The check trusts `transformers`'s own `is_transposed` attribute (set by the `use_experts_implementation` decorator, not derived from config/checkpoint data) directly: - Accept only experts modules that report `is_transposed is False`. - Reject anything where `is_transposed` is missing or not a `bool` (covers older transformers releases, undecorated architectures such as llama4/aria, and unrecognized implementations). - Reject `is_transposed=True` (e.g. gpt-oss). - Exempt classic per-expert `nn.ModuleList` experts (e.g. Mixtral/PhiMoE on older transformers) that carry no direct 3D parameter. A `trust_remote_code` custom experts implementation that misreports its own `is_transposed` is out of scope: that is treated as user-introduced misuse of an explicitly opted-in trust boundary, not a layout Olive can independently verify. No architecture allow-list is used. Only affects the `moe=True` path -- gated behind `if qcfg.moe:` and only runs when experts modules are actually detected, so non-MoE quantization is unaffected. ## Testing 28/28 relevant tests pass (`test/passes/pytorch/test_moe_support.py`, `test/passes/pytorch/test_rtn.py`); lintrunner clean. Note: PR #2610 (GPTQ MoE) will stack on top of this branch to reuse `check_moe_layout_support` and avoid duplicating the layout logic. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e10674c1-6909-4b09-9f5a-d41b28c89d2d

## Describe your changes Adds GPTQ support for Mixture-of-Experts (MoE) models whose fused expert weights are stored natively as `(num_experts, out_features, in_features)` — i.e. K (the reduction dim) is already the last dimension, matching GPTQ's existing `(OUT, K)` assumption with no layout transpose required ("K-last" architectures). This builds on #2584 (RTN MoE quantization / `QuantTensor` / `ModelWrapper` infrastructure, not yet merged) and targets this exact PR's branch as its base. **Layout gating (updated):** rather than a fixed `model_type` allow-list, GPTQ now delegates fused-expert layout validation to the same shared, strict `is_transposed`-based guard used by RTN (`olive/passes/pytorch/moe_support.py::check_moe_layout_support`) — trusting the metadata transformers' own `@use_experts_implementation` decorator attaches, rather than a static per-architecture list. Any experts module reporting (or defaulting to) a transposed `(E, K, OUT)` layout, or missing/unverifiable `is_transposed` metadata (e.g. `gpt_oss`, `llama4`, `aria`, or any architecture whose experts haven't adopted the fused-experts decorator), fails closed with an actionable error message — those require a layout-normalization step before GPTQ's Hessian accumulation and are intentionally out of scope here (planned as a follow-up PR). GPTQ additionally requires the experts module to be forward-interceptable (`.config` present, not a bare `nn.ModuleList`), since — unlike RTN — it must record per-expert Hessians via a forward-hook swap. ### Key design points - **Per-expert Hessian collection** uses transformers' `ALL_EXPERTS_FUNCTIONS` registry (`transformers >= 5.0`), not monkey-patching: a single generic calibration function is registered once and swapped in per-model via `set_experts_implementation`, so every decorated Experts module routes through it uniformly — no per-architecture branching. Each expert gets its own independent `(K, K)` Hessian (no cross-expert pooling), and an explicit record on/off switch prevents double-recording on GPTQ's true-sequential second pass. - **Per-expert RTN fallback** for cold/low-coverage experts, gated by a percentage-of- calibration-set threshold (`moe_fallback_threshold`, default 0.5%), following GPTQModel's convention. Guarantees GPTQ+fallback is never worse than plain RTN for a given expert, including the zero-sample case (no Hessian exists at all). - **Routing coverage report**, logged per layer and at a run summary, derived from the forward call's own routing-index argument (never re-implements top-k routing, which would be actively wrong for architectures like DeepSeek-V3 that apply grouped, bias-corrected scoring before top-k). - **MoE routers are now unconditionally excluded from quantization** (any form, including bare `nn.Linear` routers such as Jamba's), matching GPTQ Model/AWQ/vLLM convention. This is a shared-infrastructure change in `iter_quant_targets` and affects the existing RTN MoE path too, not just GPTQ. - `LayerWrapper` (`olive/common/hf/wrapper.py`) gained MLP/router mapping entries for `granitemoe` (`block_sparse_moe`) and `jamba` (`feed_forward`), fixing a pre-existing crash for these two architectures that affected RTN as well as GPTQ. - Fail-closed gating: `transformers` version + shared `is_transposed`-based layout guard (see above) + a forward-interceptability check (`.config` presence, rejecting bare `nn.ModuleList` experts) — verified before any weight mutation. - A preflight warning estimates per-layer Hessian memory from the experts module's actual tensor shapes and warns above a threshold (DeepSeek-V3-scale configs can require tens of GB per layer); this is a known v1 limitation, not solved here. This PR went through a 5-reviewer fan-out (readability, correctness, adversarial/critical, deep spec-adherence, cross-module integration) plus a QA pass that executed concrete repros for the highest-risk claims (exception safety, layout-gate bypass, RTN-fallback boundary). All Critical/ Major findings from that round were fixed in a follow-up commit, including: exception-safe calibration lifecycle (state restoration on error), a re-entrancy guard and registry-identity check on the experts-implementation swap, scoping the `get_attention_inputs()` partial-resolution change back to only the call sites that need it (to avoid silently breaking `rotate.py`'s positional QKV assumption), the Hessian-memory preflight warning, on-grid RTN-fallback weights before the true-sequential re-run, and coverage counts no longer hardcoded to a specific parameter name. The allow-list + class-identity cross-check originally used for layout gating was later replaced with the shared `is_transposed`-based guard described above, to align with RTN's approach. A subsequent full-PR review pass (same 5-reviewer + QA fan-out, but against the whole diff rather than incrementally) found a few additional pre-existing issues predating this round — most notably that `get_mlp_inputs`/`get_mlp_outputs` need the same `partial_ok` opt-in treatment `get_attention_inputs` already received, to avoid `rotate.py` silently skipping MoE MLP rotation. Fixes for these are being applied as a follow-up commit. ## Checklist before requesting a review - [x] Add unit tests for this change. - [x] Make sure all tests can pass. - [ ] Update documents if necessary. - [x] Lint and apply fixes to your code by running `lintrunner -a` - [ ] Is this a user-facing change? If yes, give a description of this change to be included in the release notes. This is user-facing (new `Gptq` pass capability for MoE models via `allow_moe`/`moe=True` + `moe_fallback_threshold`) — release note: "Added GPTQ quantization support for Mixture-of- Experts models with native (K-last) expert weight layouts: Qwen2-MoE, Qwen3-MoE, Phi-MoE, Mixtral, DeepSeek-V3, Granite-MoE, OLMoE, and Jamba." ## (Optional) Issue link Builds on #2584. Related to #2599. --------- Co-authored-by: Copilot CLI <223556219+Copilot@users.noreply.github.com> Co-authored-by: Jambay Kinley <jambaykinley@microsoft.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: copilot <copilot@github.com> Copilot-Session: 85549b60-0fb9-4d65-a4e8-7a8995939d68 Copilot-Session: e10674c1-6909-4b09-9f5a-d41b28c89d2d

## Describe your changes Stacked on top of #2610 (this PR targets `b1-gptq-moe`, not `main`). Adds a manual validation script for comparing perplexity/size before and after quantizing a real (downloaded) HF checkpoint, plus three onboarding docs under `skills/olive/references/` for quantization work in this repo: - `scripts/quantize_and_compare_perplexity.py` — generic, pass-agnostic script (works for any registered Olive PyTorch quantization pass, not just GPTQ/MoE) that loads a real model, quantizes it, and reports weights-size and WikiText-2 perplexity deltas. This is the same style of real-model validation tool that surfaced real RTN bugs in #2584 after synthetic-model unit tests had already passed. - `skills/olive/references/quantization-onboarding.md` — general RTN/GPTQ pass onboarding: shared config surface, when to use RTN vs. GPTQ, calibration split hygiene. - `skills/olive/references/moe-gptq.md` — MoE-GPTQ-specific onboarding: why MoE needs its own calibration path, the K-last layout allow-list, the dual fallback-threshold design (#2610), and what real-model benchmarking showed about fallback rates and quantization wall-time. - `skills/olive/references/profiling-benchmark-example.md` — worked example of running the benchmark script and interpreting its output. ### Three-model benchmark (bits=4, group_size=128, sym=true, full WikiText-2 `train` calibration, full `test` eval) | Model | Baseline PPL | RTN PPL (Δ, time) | GPTQ PPL (Δ, time) | KQuant PPL (Δ, time) | Fallback experts | | --- | --- | --- | --- | --- | --- | | granite-3.0-1b-a400m-base | 6.2877 | 7.5861 (+1.2984, 8.0s) | 6.9560 (+0.6683, 658.7s) | 7.5162 (+1.2286, 12.1s) | 2/768 (0.3%) | | OLMoE-1B-7B-0924 | 6.6182 | 7.1091 (+0.4909, 52.6s) | 6.8966 (+0.2784, 1499.3s) | 7.0507 (+0.4325, 71.8s) | 10/1024 (1.0%) | | Qwen1.5-MoE-A2.7B | 6.4246 | 6.9251 (+0.5005, 85.8s) | 6.6117 (+0.1872, 2475.6s) | 6.9318 (+0.5072, 148.2s) | 0/1440 (0.0%) | GPTQ consistently beats RTN on perplexity delta across all three models, at a real (but model-size/expert-count-correlated, not cleanly separable) wall-time cost. See `moe-gptq.md` for the full discussion, including the OLMoE layer-2/expert-5 case that empirically validates the dual fallback-threshold design from #2610. KQuant (#2618) numbers added for comparison: KQuant is data-free (no calibration set, no per-expert fallback concept — the "Fallback experts" column doesn't apply to it) and its quantization time is close to RTN's (both are cheap, uncalibrated passes), but its perplexity delta tracks RTN's rather than GPTQ's on all three models. All three KQuant runs used `moe=true` and forced `experts_implementation="eager"` at inference (`grouped_mm` cannot run against `QuantTensor`-wrapped experts; see #2619). ### Notes - This PR depends on `b1-gptq-moe` (#2610): `capture_moe_fallback_counts()` in the script unconditionally imports `olive.passes.pytorch.moe_calib`, which only exists on that branch. Please review/merge #2610 first. - Went through a full internal review pass (readability/correctness/adversarial/spec-adherence/ cross-module) before opening; findings incorporated include: fixing pass-name resolution to use the actual pass registry (`OlivePackageConfig.import_pass_module`) instead of guessing module paths, several docstring/arithmetic corrections in the reference docs, and hedging a couple of causal claims that the 3-data-point benchmark can't fully support. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e10674c1-6909-4b09-9f5a-d41b28c89d2d

Copilot AI and others added 14 commits May 15, 2026 22:01

Initial plan

fbd60bd

Copilot AI assigned Copilot and Ti-Tai Wang (titaiwangms) Jul 21, 2026

Copilot started work on behalf of Ti-Tai Wang (titaiwangms) July 21, 2026 23:14 View session

Copilot AI linked an issue Jul 21, 2026 that may be closed by this pull request

Resume jambayk/moe-quant: extend RTN weight quantization to MoE experts #2583

Closed

14 tasks

copilot added 5 commits July 21, 2026 23:30

Override precedence: insertion-order first-match; add ReDoS-safe rege…

4c50d8b

…x validator + tests

Add MoE selection/tensor regression tests + PyTorch Native RTN docs

a0aa4cc

Central 3D movement-op guard + real-HF Mixtral round-trip test + docs

c41a618

Add Gptq+Rtn composition test; make lintrunner clean across changed f…

8035bb9

…iles

Copilot AI changed the title ~~[WIP] Extend RTN weight quantization to MoE experts~~ Extend PyTorch RTN weight quantization to MoE experts Jul 22, 2026

Copilot finished work on behalf of Ti-Tai Wang (titaiwangms) July 22, 2026 00:08

Copilot AI requested a review from Ti-Tai Wang (titaiwangms) July 22, 2026 00:08

Ti-Tai Wang (titaiwangms) requested a review from Copilot July 22, 2026 16:31

Copilot started reviewing on behalf of Ti-Tai Wang (titaiwangms) July 22, 2026 16:32 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

Comment thread olive/common/quant/hf_utils.py Outdated

Comment thread olive/common/quant/hf_utils.py Outdated

Comment thread olive/common/quant/selection.py

Copilot finished work on behalf of Ti-Tai Wang (titaiwangms) July 30, 2026 23:14

Copilot started work on behalf of Ti-Tai Wang (titaiwangms) July 30, 2026 23:15 View session

Copilot finished work on behalf of Ti-Tai Wang (titaiwangms) July 30, 2026 23:16

Ti-Tai Wang (titaiwangms) and others added 2 commits July 31, 2026 00:41

Ti-Tai Wang (titaiwangms) linked an issue Jul 31, 2026 that may be closed by this pull request

MoE quantization: 5 unresolved edge-case bugs in QuantTensor indexing / regex safety checks #2598

Closed

Ti-Tai Wang (titaiwangms) requested a review from Jambay Kinley (jambayk) August 1, 2026 00:42

Ti-Tai Wang (titaiwangms) requested a review from Xiaoyu (xiaoyu-work) August 3, 2026 19:03

Merge remote-tracking branch 'origin/main' into copilot/resume-jambay…

2225683

…k-moe-quant-extend-rtn-weight # Conflicts: # test/passes/onnx/test_model_builder.py

Merge branch 'main' into copilot/resume-jambayk-moe-quant-extend-rtn-…

7521071

…weight

Ti-Tai Wang (titaiwangms) mentioned this pull request Aug 5, 2026

Add GPTQ quantization support for K-last MoE architectures #2610

Merged

5 tasks

Ti-Tai Wang (titaiwangms) mentioned this pull request Aug 7, 2026

Add MoE GPTQ benchmark script and quantization onboarding docs #2612

Merged

Jambay Kinley (jambayk) approved these changes Aug 11, 2026

View reviewed changes

Ti-Tai Wang (titaiwangms) merged commit 96fb44d into main Aug 11, 2026
12 checks passed

Ti-Tai Wang (titaiwangms) deleted the copilot/resume-jambayk-moe-quant-extend-rtn-weight branch August 11, 2026 20:55

Ti-Tai Wang (titaiwangms) mentioned this pull request Aug 11, 2026

Guard RTN MoE quantization by expert layout #2616

Merged

This was referenced Aug 12, 2026

MoE-quantized models crash on default grouped_mm inference path (QuantTensor transpose not supported) #2619

Open

Design: support quantizing is_transposed=True MoE architectures (gpt-oss, llama4, aria) #2621

Open

Navigation Menu

Uh oh!

Extend PyTorch RTN weight quantization to MoE experts - #2584

Extend PyTorch RTN weight quantization to MoE experts#2584
Ti-Tai Wang (titaiwangms) merged 28 commits into
mainfrom
copilot/resume-jambayk-moe-quant-extend-rtn-weight

Copilot AI commented Jul 21, 2026 •

edited by titaiwangms

Loading

Uh oh!

azure-pipelines Bot commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Ti-Tai Wang (titaiwangms) commented Aug 1, 2026

Uh oh!

Ti-Tai Wang (titaiwangms) commented Aug 4, 2026

Uh oh!

Xiaoyu (xiaoyu-work) commented Aug 5, 2026

Uh oh!

azure-pipelines Bot commented Aug 5, 2026

Uh oh!

Xiaoyu (xiaoyu-work) commented Aug 6, 2026

Uh oh!

Ti-Tai Wang (titaiwangms) commented Aug 7, 2026

Uh oh!

Ti-Tai Wang (titaiwangms) commented Aug 7, 2026

Uh oh!

Jambay Kinley (jambayk) left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

Copilot AI commented Jul 21, 2026 • edited by titaiwangms Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Design (carried over from jambayk/moe-quant, unchanged)

What changed in this PR on top of jambayk/moe-quant

Known follow-ups (tracked separately, not blocking this PR)

Testing

Uh oh!

azure-pipelines Bot commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Ti-Tai Wang (titaiwangms) commented Aug 1, 2026

Full-PR review pass (readability, code, critical, deep, integration reviewers + QA)

Uh oh!

Ti-Tai Wang (titaiwangms) commented Aug 4, 2026

Uh oh!

Xiaoyu (xiaoyu-work) commented Aug 5, 2026

Uh oh!

azure-pipelines Bot commented Aug 5, 2026

Uh oh!

Xiaoyu (xiaoyu-work) commented Aug 6, 2026

Uh oh!

Ti-Tai Wang (titaiwangms) commented Aug 7, 2026

Uh oh!

Ti-Tai Wang (titaiwangms) commented Aug 7, 2026

Uh oh!

Jambay Kinley (jambayk) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Copilot AI commented Jul 21, 2026 •

edited by titaiwangms

Loading

Design (carried over from `jambayk/moe-quant`, unchanged)

What changed in this PR on top of `jambayk/moe-quant`