Skip to content

chore(deps): update loader dependencies non-major - #291

Merged
dreadnode-renovate-bot[bot] merged 1 commit into
mainfrom
renovate/loader-deps
Sep 16, 2026
Merged

chore(deps): update loader dependencies non-major#291
dreadnode-renovate-bot[bot] merged 1 commit into
mainfrom
renovate/loader-deps

Conversation

@dreadnode-renovate-bot

@dreadnode-renovate-bot dreadnode-renovate-bot Bot commented Sep 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package Change Age Confidence
accelerate ==1.14.0 → ==1.15.0 age confidence
transformers ==5.16.1 → ==5.17.0 age confidence

Release Notes

huggingface/accelerate (accelerate)

v1.15.0: : FSDP2 activation memory, dtensor improvements

Compare Source

v1.15.0: FSDP2 activation memory, dtensor improvements

FSDP2

A large batch of FSDP2 work this release: two fixes that cut activation memory at long sequence lengths, tied-embedding support on torch >= 2.13, and a round of checkpointing correctness and scale fixes.

Activation checkpointing was wrapping each child of the matched layer (self_attn, mlp, the norms) instead of the layer itself, so every inter-child activation stayed saved for backward. It now wraps the layer.

There's also a new FSDP2-only activation_checkpointing_offload, which moves the remaining per-layer checkpoint inputs to pinned CPU memory. Gradients are exactly those of plain activation checkpointing:

# fsdp2.yaml
fsdp_config:
  fsdp_version: 2
  fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP
  fsdp_activation_checkpointing: true
  fsdp_activation_checkpointing_offload: true
accelerate launch --config_file fsdp2.yaml train.py
  • FSDP2 activation checkpointing: wrap the matched transformer layer itself, not each of its children by @​qgallouedec in #​4172
  • Add FSDP2 activation_checkpointing_offload: offload checkpointed layer inputs to pinned CPU memory by @​qgallouedec in #​4175
  • Fix FSDP2 tied-embedding models on torch >= 2.13: put the output embedding in the same fully_shard group by @​qgallouedec in #​4171
  • Fix FSDP2/PEFT/FULL_STATE_DICT dropping every rank's adapter shard except rank 0 by @​AmineDiro in #​4206
  • FSDP2: per-rank torch.save/load for SHARDED_STATE_DICT to fix 2800+ NPU checkpoint timeout by @​gygdh-001 in #​4105
  • Fix FSDP sharded checkpoint path resolution by @​HaomingSong in #​4119
  • Raise a clear error when FSDP is enabled on a mesh with no shard dimension by @​qgallouedec in #​4180

DTensor

Two fixes for DTensor-sharded models, which you hit with FSDP2, tensor parallelism, or any N-D parallelism setup: gradient clipping no longer fails on the foreach op when plain tensors and DTensors are mixed, and prepare_model leaves an already-sharded model where it is:

Offloading & Quantization

An entire model can now be dispatched to disk, including tied weights — useful for tools like llm-compressor that compress large models on machines that can't hold them:

Trackers

Custom trackers can be registered by name and then selected from log_with= like any built-in one:

from accelerate import Accelerator
from accelerate.tracking import register_tracker_class

register_tracker_class(MyTracker)  # MyTracker.name == "my_tracker"
accelerator = Accelerator(log_with="my_tracker")

Device support

Neuron gains a torch dynamo backend (so --torch-compile works with the Transformers Trainer) and MPS is now reported and handled properly by accelerate env and find_executable_batch_size.

CLI

Minor fixes

New Contributors

Full Changelog: huggingface/accelerate@v1.14.0...v1.15.0

huggingface/transformers (transformers)

v5.17.0: Release 5.17.0

Compare Source

Release v5.17.0

New Model additions

HYV4
image

Hy4-Preview is a 780B-parameter mixture-of-experts language model that activates 49B parameters per
token. Each MoE layer holds 256 routed experts plus one always-active shared expert and routes every
token to 8 of them. The context window is 1M tokens.

The architecture combines four features:

  • Multi-head Latent Attention (MLA) compresses keys and values into a low-rank latent
    (kv_lora_rank) that kv_b_proj expands back to one key/value per query head.
  • DeepSeek Sparse Attention (DSA) selects index_topk keys per query with a lightweight indexer.
    Following IndexShare, only the layers marked "full"
    in indexer_types run an indexer; "shared" layers reuse the previous full layer's selection.
  • Gated MLA with learnable attention sinks, where each head owns a sink logit that participates
    in the softmax and contributes no value, as in GPT-OSS.
  • Independent Hyper-Connections (iHC) replace the plain residual path with hc_mult parallel
    residual streams that are collapsed before, and redistributed after, every sublayer.

The implementation does not execute the multi-token prediction (MTP) layers. Released checkpoints
keep those weights so that other runtimes can use them for speculative decoding; they are ignored
at load time.

Links: Documentation

VibeVoice
image

VibeVoice is a novel framework for synthesizing high-fidelity, long-form speech with multiple speakers by employing a next-token diffusion approach within a Large Language Model (LLM) structure. It's designed to capture the authentic conversational "vibe" and is particularly suited for generating audio content like podcasts and multi-participant audiobooks.

Links: Documentation

NeoMME

NeoMME is a family of efficient 260M and 800M parameter multimodal-native multilingual foundation encoders from H Company. It processes multilingual text tokens and raw image patches in a single bidirectional Transformer encoder, without a separately pretrained vision tower or causal language model.

NeoMME-Retriever is a model fine-tuned from the NeoMME backbone for visual document retrieval with joint late-interaction and dense objectives. It takes text queries and documents (text or page screenshots) and produces multi-vector embeddings for MeanMaxSim scoring (late-interaction) and mean-pooled embeddings for cosine similarity (dense).

Links: Documentation

Fun-ASR-Nano

Fun-ASR-Nano is an 800M-parameter end-to-end speech recognition model developed by Alibaba DAMO Academy's FunAudioLLM team. It achieves state-of-the-art performance on Chinese, English, and Japanese ASR benchmarks while being significantly smaller than comparable models.

Key features are

  • Chinese, English, and Japanese, including 7 Chinese dialects and 26 regional accents
  • Hotword customization for domain-specific vocabulary
  • Native punctuation output (no separate punctuation model needed)

Links: Documentation

KimiLinear

Kimi Linear is a hybrid linear attention architecture from Moonshot AI, introduced in
Kimi Linear: An Expressive, Efficient Attention Architecture.

At its core is Kimi Delta Attention (KDA), a refinement of Gated DeltaNet
that gives each key channel its own forget gate, so the recurrent state decays per channel instead of per head. KDA is
used in most layers; every fourth layer keeps a full-attention block that reuses DeepSeek-V3's Multi-head Latent
Attention (MLA), and the feed-forward blocks are DeepSeek-V3-style MoE with a shared expert.

Links: Documentation

Canary

Canary-1B-v2, a fast, robust multilingual model for Automatic Speech Recognition (ASR) and Speech-to-Text Translation (AST):

Canary reuses the Fast Conformer encoder from Parakeet (loaded through [ParakeetEncoder] / [ParakeetEncoderConfig]) and pairs it with a Transformer decoder that uses fixed sinusoidal positional embeddings, cross-attention to the encoder outputs and tied input/output embeddings. The task is selected through a decoder prompt prefix built by [CanaryProcessor] of the form <|startofcontext|> <|startoftranscript|> <|emo:undefined|> <source_lang> <target_lang> <pnc|nopnc> <|noitn|> <|notimestamp|> <|nodiarize|>, where source_lang == target_lang selects transcription and otherwise selects translation.

Links: Documentation

NeuCodec

The NeuCodec model was proposed in Finite Scalar Quantization Enables Redundant and Transmission-Robust Neural Audio Compression at Low Bit-rates.

NeuCodec is a neural audio codec extending on XCodec2. It takes advantage of the following features:

  • Finite Scalar Quantization (FSQ) quantisation resulting in a single codebook, making it ideal for downstream modeling with Speech Language Models.
  • Trained with CC data such that there are no Non-Commercial data restrictions.
  • At 50 tokens/sec and 16 bits per token, the overall bit-rate is 0.8kbps.
  • The codec takes in 16kHz input and outputs 24kHz using an upsampling decoder.
  • The FSQ encoding scheme allows for bit-level error resistance suitable for unreliable and noisy channels.

Links: Documentation

Breaking changes

Vision rotary embeddings (2D/3D) have been standardized into a unified RoPE frequency computation module, so users with custom vision models relying on attention-layer-level or model-specific RoPE grid interleaving logic must migrate to the new centralized modeling_rope_utils.py implementation.

Generation

Generation improvements include a performance optimization that avoids unnecessary accelerator synchronization on every decode step (reducing per-step overhead), and a fix to prevent unconditional downloading of remote hub files during generation. Several correctness fixes were also applied, including enforcing auto-compile cache checks for encoder-decoder models, standardizing past_key_values naming in AfMoE, and resolving flaky export and integration test failures.

Cache

Fixed several cache-related bugs, including a quantized cache issue in VibeVoice, incorrect rejection of non-static cache implementations in VoxtralRealtime, missing auto-compile cache checks for encoder-decoder models, and a silent failure when paged attention is called without a cache. Documentation was also updated to clarify ContinuousBatchingConfig usage and sliding window model limitations.

Kernels

Kernel support was improved with fixes for nested FLA kernel imports when only fla-core is installed, a warning when hub-kernel functions silently fall back to slower pure-PyTorch reference implementations, and the ability to register standalone functions (e.g., RoPE) in KernelConfig with optional non-inheritance of default mappings. Additional fixes include corrected repository paths for ESMFold2 kernels and updated documentation for KernelConfig customization.

Quantization

Fixed several quantization bugs, including a quant cache issue in VibeVoice, incorrect FP8 embedding handling for Qwen models, missing FP8 tensor parallelism layer overrides, and unnecessary MXFP4 weight dequantization on XPU devices.

Bugfixes and improvements

❗ Important

✂ PR body was truncated to here.

Configuration

📅 Schedule: (UTC)

  • Branch creation
    • At any time (no schedule defined)
  • Automerge
    • At any time (no schedule defined)

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

  • If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate CLI.

@dreadnode-renovate-bot
chore(deps): update loader dependencies non-major
ebfe26e 
| datasource | package      | from   | to     |
| ---------- | ------------ | ------ | ------ |
| pypi       | accelerate   | 1.14.0 | 1.15.0 |
| pypi       | transformers | 5.16.1 | 5.17.0 |
@dreadnode-renovate-bot dreadnode-renovate-bot Bot added the type/digest Dependency digest updates label Sep 13, 2026
@dreadnode-renovate-bot
dreadnode-renovate-bot Bot merged commit 1c2364a into main Sep 16, 2026
7 checks passed
@dreadnode-renovate-bot
dreadnode-renovate-bot Bot deleted the renovate/loader-deps branch September 16, 2026 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

type/digest Dependency digest updates

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

0 participants