Repository navigation

[feat] Tensor parallelism for MiniMax-H3 - #14609

Open

JingyaHuang wants to merge 24 commits into

huggingface:mainfrom

JingyaHuang:add-h3-tp-support

Open

[feat] Tensor parallelism for MiniMax-H3#14609
JingyaHuang wants to merge 24 commits into
huggingface:mainfrom
JingyaHuang:add-h3-tp-support

Commits on Oct 9, 2026
- Add tensor-parallel support for MiniMax-H3
  JingyaHuang
  committed
  764e7c8
- Shard MiniMax-H3's adaln_proj to fit tensor parallelism on one device
  
  JingyaHuang
  and
  claude
  committed
  58d9b51
- Build MiniMax-H3's row timestep plan on CPU, as its caller expects
  
  JingyaHuang
  and
  claude
  committed
  c766b88
- feat:combine tp+cp
  JingyaHuang
  committed
  4162784
- Allow tensor parallelism and context parallelism in one ParallelConfig
  
  authored and
  JingyaHuang
  committed
  7f649f9
- feat: validate the support on TPU
  JingyaHuang
  committed
  673a3f1
- fix: put tp first in the combined TP+CP mesh
  JingyaHuang
  committed
  3db653d
- fix: style
  JingyaHuang
  committed
  9c0f7aa
- fix: apply context parallelism on the sharded-load path
  JingyaHuang
  committed
  bbc55a7
- revert: attention head padding
  JingyaHuang
  committed
  6ce34c7
Commits on Oct 10, 2026
- revert: drop the Neuron all-gather padding and restore the TPU pre-shard path
  JingyaHuang
  committed
  986ea9a
- refactor: share the CPU pre-shard path between Neuron and TPU tensor parallelism
  JingyaHuang
  committed
  6845f33
Commits on Oct 13, 2026
Commits on Oct 14, 2026
- Merge branch 'main' into add-h3-tp-support
  JingyaHuang
  authored
  d2c362e
- Merge branch 'main' into add-h3-tp-support
  JingyaHuang
  authored
  3cd26a9
- Merge branch 'main' into add-h3-tp-support
  JingyaHuang
  committed
  6b3b3e8
- fix: fall back to one device when no accelerator is present
  
  JingyaHuang
  and
  claude
  committed
  cd61166
- Move TP+CP composition to #14725
  
  JingyaHuang
  and
  claude
  committed
  cc85110
Commits on Oct 15, 2026
- Merge branch 'main' into add-h3-tp-support
  JingyaHuang
  committed
  b1076e6
- review: use `config._device` for the Neuron TP device
  JingyaHuang
  committed
  8526d94
- review: replace ReplicatedInputRowwiseParallel with a "rowwise_split_input" style
  
  JingyaHuang
  and
  claude
  committed
  ed452ef
- review: explain when to use "rowwise_split_input"
  
  JingyaHuang
  and
  claude
  committed
  bc35ec2

Navigation Menu

Repository navigation

[feat] Tensor parallelism for MiniMax-H3 - #14609

[feat] Tensor parallelism for MiniMax-H3#14609
JingyaHuang wants to merge 24 commits into
huggingface:mainfrom
JingyaHuang:add-h3-tp-support

Commits on Oct 9, 2026

Add tensor-parallel support for MiniMax-H3

Shard MiniMax-H3's adaln_proj to fit tensor parallelism on one device

Build MiniMax-H3's row timestep plan on CPU, as its caller expects

feat:combine tp+cp

Allow tensor parallelism and context parallelism in one ParallelConfig

feat: validate the support on TPU

fix: put tp first in the combined TP+CP mesh

fix: style

fix: apply context parallelism on the sharded-load path

revert: attention head padding

Commits on Oct 10, 2026

revert: drop the Neuron all-gather padding and restore the TPU pre-shard path

refactor: share the CPU pre-shard path between Neuron and TPU tensor parallelism

Commits on Oct 13, 2026

Merge branch 'main' into add-h3-tp-support

fix: new device count API for TPU since deprecation

fix: pad uneven all-gathers on Neuron and TPU in _all_gather_anything

Commits on Oct 14, 2026

Merge branch 'main' into add-h3-tp-support

Merge branch 'main' into add-h3-tp-support

Merge branch 'main' into add-h3-tp-support

fix: fall back to one device when no accelerator is present

Move TP+CP composition to #14725

Commits on Oct 15, 2026

Merge branch 'main' into add-h3-tp-support

review: use `config._device` for the Neuron TP device

review: replace ReplicatedInputRowwiseParallel with a "rowwise_split_input" style

review: explain when to use "rowwise_split_input"