add GLM-5.3-Flash (GLM5-Next) support by timkhronos · Pull Request #27773 · ggml-org/llama.cpp

timkhronos · 2026-08-26T23:27:55Z

Overview

Add support for GLM -5.3-flash a 320B hybrid model, supporting both text and vision.

Additional information

Architecture

GLM 5.3 flash mixes 34 KDA linear layers with 11 DSA laters, with mHC and Deepseek style Moe. Most of the parts are already in llama.cpp so I reused whatever I could:

KDA layers reuse the Kimi-K3 implementation
Attention layers are nope only MLA
For mHC I reused the Deepseek V4 implementation. I moved the build_hc helpers from the DSV4 graph into graph_context so both models can share them.
Moe and swiglu clamping follow DSV4.

What I implemented new:

Here, the DSA indexer scores pools of 4 consecutive token, and always keeps the incomplete tail. I implemented this on top of the existing DSA cache. The indexer cache stores key | gate per token and pooling happens in the graph. No new backend ops have been added.
~~llama_memory_hybrid_dsa: recurrent state + DSA cache, cloned from hybrid ISWA.~~ Rebased onto llama_memory_hybrid_idx instead of the earlier ISWA clone.
Vision Tower: ~~the encoder is the same family as glmv4 with per head qk-norm, clamped Swiglu and no post conv norm. It reuses glm4v projector with a swiglu_limit key and an optional image token budget.~~ Added as glm5v as GLM 5.3 Flash requires a different pre processing method than what glm4v uses.
Small, precision sensitive tensors (indexer, mHC mixers, KDA gates, MLA low rank paths, roughly 1GB total) are kept unquantized.

Tests

Logits match transformers on a small random model across full prefill, small ubatches and single token decode while sparse selection is active, covering both scatter and gather.
Vision embeddings match to ~1e-5.
The converted model generates coherently and correctly, and vision is working as expected.

Limitations

MTP is in a separate pr (Add MTP for GLM-5.3-Flash #27917), but NextN tensors are already preserved here.
Multiple sequences require --kv-unified

Quantized GGUFs converted with this PR are available here.

Requirements

I have read and agree with the contributing guidelines
AI usage disclosure: YES, AI was used in an assistive capacity, and helped figure out and solve several conversion issues, and helped validate the correctness of the implementation.

…id-idx

danielhanchen · 2026-08-28T06:28:04Z

Hey @timkhronos great work on the PR! A few requests if possible:

You're using glm5-next for general.architecture, whilst model: add GLM-5-Next (GLM-5.3-Flash) #27754 and model : add GLM-5.3-Flash (glm5next) #27752 uses glm5next - Qwen3-Next for eg does qwen3next - I'm unsure what the convention is @ngxson but adopting glm5next might be more generalized? If glm5-next is accepted, a simple first shard rewrite for our uploads should suffice.
The bigger issue is blk.N.indexer.kpool_ape / kpool_gate vs blk.N.indexer_compressor_ape / _gate. deepseek4 mainline already uses blk.N.indexer_compressor_ape and blk.N.indexer_compressor_gate but your PR changes it - the ones we uploaded uses deepseek4's convention. If this PR is accepted, we have to provide a script to rewrite all tensor names or folks have to re-download. ~~If not, can you add aliases so the ones we published works - thanks in advance.~~ Seems like it's more complex than I expected.

Tagging @ngxson for visibility as well.

I re-checked and if (1) + (2) is applied, the quants we uploaded work fine (+ the small shard-1 rewrite) and KLD / PPL are correct under this PR.

Seems like a simple alias isn't possible actually :( It breaks the quants made with this PR

danielhanchen · 2026-08-28T08:05:50Z

Hmmm https://github.com/timkhronos/llama.cpp/pull/9/changes would alias the tensors but it looks a bit problematic hmmm

Sciguy429 · 2026-08-28T13:12:32Z

Throwing up some performance numbers here from the lower end of consumer hardware (128GB DDR5 + 24GB VRAM (4090)).

avar6 has some freshly converted imatrix quants from this PR up as of now if anyone else wants to give them a go: https://huggingface.co/avar6/GLM-5.3-Flash-BF16-gguf

For the IQ3_S, I am getting roughly 300t/s prefill at 256K context and 2048 b/ub size. Generation speed starts off at around 9t/s and drops down considerably by mid window (~128K) to around 6t/s. This seems to track with the 'pooled indexer keys' issue. The model is fully coherent and seems to be working fine. I don't have PPL/KL numbers at the moment as I still need to generate a logit dump.

I have noticed an interesting memory quirk, which I haven't seen before. This is the only model I have ever seen have inconsistent checkpoint sizes. As the context fills the checkpoints grow alarmingly fast in size. At ~90K they are already up to nearly 1.6GB. I don't know if this is an expected behavior for this model arch, or if this is a something which needs to be looked into.

Also, something of note for you @danielhanchen which I found last night while looking over the three PRs for this arch. The vision towers between this PR and yours differ as well. This PR reuses the name clip.vision.projector_type = "glm4v" while you built a new one clip.vision.projector_type = "glm5next". Likely not much of an issue given how easy it is to regenerate mmproj files, but it will need to be delt with as well.

danielhanchen · 2026-08-28T15:50:16Z

Yes I'll re-do the vision! This is fine!

@timkhronos I confirmed timkhronos#9 works fine and does not break your GGUFs. We will however need to do a cheap shard-1 update so that should be fine

…ed up long context decode, fla, and slight MTP improvements.

danielhanchen · 2026-08-28T23:13:33Z

@timkhronos I saw you changed the tensor naming - but my solution I provided was to allow everyone's quants to work - now your own ones you uploaded don't work haha.

We still need to provide the shard rewrite for the naming (glm5-next) which we're fine with, but now the DeepSeek convention means you yourself have to reupload all shards or do a tensor rename inplace with a script - was this your intention?

timkhronos · 2026-08-28T23:23:03Z

@danielhanchen Hey!

I ended up going with the the indexer_compressor naming scheme, as it is closer to what's already there, and I was meaning to ask Avar to reconvert anyways, as his ggufs were made when we were missing quantization protection for some crucial tensors so they are not ideal.

Your vision projectors will need reconverting though most likely, and your main model ggufs might be missing the index_share_for_mtp_iteration key as well.

danielhanchen · 2026-08-29T05:23:36Z

@timkhronos Hey! I made some shard rewrites to https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF/tree/main/Shard_Rewrite for in preparation!

…ed heads (unslothai#242) Drop ggml-org#27754, unslothai#144 and unslothai#137: upstream merged GLM-5-Next (ggml-org#27773), Qwen4Exp MTP (ggml-org#29761) and row prefetch (ggml-org#29599), and the old pins no longer merge. Old-GGUF loading moves to unslothai#239, shared MTP heads to unslothai#240, and what the dropped pins had beyond upstream to unslothai#241. Repin ggml-org#24423, ggml-org#25731, unslothai#61 and unslothai#176 to their conflict-fixed heads.

* Rebase GLM-Next support onto master, and migrate to llama-memory-hybrid-idx * Add initial MTP support * Merge branch optimizations. Reduce allocated compute buffer size, speed up long context decode, fla, and slight MTP improvements. * Review driven changes, remove env vars, protect tensors * Strip MTP for initial PR * Clean up after mtp strip * Clean up after mtp strip * Update speculative.cpp * Update llama-context.h * Clean up after mtp strip * Fix tokenizer ignore merges * Improve quantization protection selection * Refactor mhc helpers, graph base * Lint Fixes * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * Skip glm5-next in model saver, fix CRLF * Skip glm5-next in sweep * Remove T4 fallback * Review cleanup * Review suggestions * Defer separate MTP gguf handling to MTP PR, drop filter * Repad n_head_kv * kpool init apply * Order by descending score * Drop guard * read kpool from hparams, clarify kpool cache flags, remove kpool_build_state(nullptr) * Add glm5-next support to model saver and add arch test fixture * Review cleanup * Kpool pooled caching clarify * Add multi stream support * Finish Rebase * Sparse FA fir DSA prefill * Const * Update llama-model.cpp to fix rebase error * gguf-py : merge tensor map entries for HC tensors * model : use build_gdn_l2_norm in GLM5_NEXT implementation * chore : remove trailing whitespace * model : use new OP precision setting API in GLM5_NEXT implementation * mtmd : use ggml_swiglu_clamp in GLM5V and apply the image token limit The two clamps around swiglu_split are what ggml_swiglu_clamp already does, so the clamp bounds collapse back to one value. GLM5V also never called set_limit_image_tokens(), so --image-max-tokens had no effect. Assisted-by: Claude Opus 5 (cherry picked from commit 46d18e1) * llama : keep the GLM5-Next k-pool layout across ubatches The layout was rebuilt from a full cell scan on every ubatch. Pools are fixed by the positions relative to the sequence's first one, so the layout now lives on the memory and a ubatch only appends to it. A sequence edit no longer stales every pooled key either, only the ones at or after the edited position, which makes a tail seq_rm free. The pooling subgraph is built unconditionally so the graph shape no longer changes every kpool tokens, and the pool axis is folded into rows before soft_max, which otherwise exceeds the CUDA gridDim.y limit past n_kv 262144. Assisted-by: Claude Opus 5 (cherry picked from commit 5d1c40b) * model : write the GLM5-Next recurrent rollback checkpoints The conv state and the delta net state were only written to the live row, so a rollback restored whatever the checkpoint rows happened to hold. Take the same route as kimi-k3: build_recurrent_attn for the state, and write all K_rs conv groups. That also drops a state view that assumed contiguous rows. Enroll the arch in test-recurrent-state-rollback, which catches this under its garbage-filled cache pass. Assisted-by: Claude Opus 5 (cherry picked from commit 5ace37e) * llama: fix PR ggml-org#27773 test-save-load-state restore failure Clear the attention and indexer cache data after a failed hybrid state restore so restored NaNs cannot affect a later sequence. Assisted-by: Codex * llama: fix PR ggml-org#27773 gpu-rocm graph reallocation Reserve the full GLM5-Next pool capacity and dirty pool count. The gpu-rocm Test step aborts when n_new grows while the graph node count stays fixed; CUDA, Vulkan, Metal, and WebGPU checks report the same error. Assisted-by: Codex * llama : fix GLM5-Next k-pool layout staleness after edits and shared teardown Two defects in the cross-ubatch k-pool layout added by the k-pool commit: 1. Wrong results. An edited sequence only rebuilt its pool layout when its cell count changed, so if the first ubatch after an edit added back exactly as many cells as were removed, the stale position-to-cell list survived. With a unified cache and more than one sequence, where another sequence takes the freed cells, the reused layout points at the wrong cells (CPU: large logit drift, CUDA: NaN). Rebuild whenever the sequence is stale, not only on a size mismatch. 2. Slowdown. "shared" mode was assumed to end only with an edit that forces a rebuild, but sharing also ends when the other sequence is removed. The survivor kept shared = true, pinning cache_safe off and re-pooling every pool on every ubatch (server trigger: n>1 completions with -kvu, via the seq_cp in copy_state_to). In seq_rm, if the layout has shared cells, stale every sequence so one rebuild re-derives sharing and cache_safe returns to 1. Assisted-by: Claude Opus 5 * llama : fix build_attn_mha stream stride for non-contiguous q build_attn_mha split the batch into streams with a stream stride of q->nb[3]/n_stream. That only equals one stream's span, (ne[2]/n_stream)*nb[2], when q is contiguous. GLM5-Next is nope-only, so it does not concat a rope part and passes the permuted q_absorbed straight in, where nb[3] != ne[2]*nb[2]; the stride was then n_head times too large and every stream s >= 1 read another head's queries. Split-KV (-np N without --kv-unified) multi-stream prefill was wrong for every stream past the first. Unified KV and decode were unaffected (n_stream == 1, and decode takes the gather path). Other MLA models concat rope so q is contiguous and the computed value is unchanged for them. Compute the stride from the token dimension, which is identical for a contiguous q. Assisted-by: Claude Opus 5 * llama : re-derive GLM5-Next k-pool sharing on state_read/state_drop The shared-cell teardown added to seq_rm (stale every sequence when the layout has shared cells, so a survivor does not keep shared = true and pin cache_safe off) was missing from the other paths that can free shared cells: state_read and state_drop staled only the one sequence. Apply the same re-derivation there and correct the comment that claimed sharing ends only via an edit or seq_rm. Assisted-by: Claude Opus 5 * quant : drop duplicate GLM5-Next hc_ filter The hc_ name filter was listed twice in the GLM5_NEXT protection block. Assisted-by: Claude Opus 5 * glm5-next: scope K-pool cache access to indexed operations * glm5-next: keep K-pool access in hybrid index memory * glm5-next: keep mHC graph builders model-local * glm5-next: mark only touched pools per ubatch --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com> Co-authored-by: Piotr Wilkin <ilintar@gmail.com> (cherry picked from commit 649dcb1)

ggml-org#27773 adds the glm5-next arch without its rows in the Metal fusion baseline, so test-fusion --check fails on it. The rows come from test-fusion --record on an M5 Max, and --check passes 270/270. (cherry picked from commit db00347)

halo-box/llama.cpp is archived, so its last master comes in once: the 2026-10-07 upstream sync (halo halo-box#34) with its resolutions, plus the archival README of that repository is not taken (strix keeps its own README). Resolution rules: MTP follows upstream's design; everywhere else strix's side wins where the two collide, and upstream's non-conflicting changes stay. - qwen4exp: strix's QSA stack (selected-key / WMMA block selection, incremental pooled keys, PLE prefetch, HIP prefill work) is kept whole in qwen4exp.cpp and the qwen4exp struct of models.h. strix's graph_mtp is kept: it builds the same draft as upstream's ggml-org#29761 (eh_proj of [enorm(e); hnorm(h)], the draft's own nextn.hc_head_* mixer, sparse QSA attention) and also loads shared sidecars without embeddings / LM head. The converter, gguf-py tensor maps and arch tables are upstream's, so new conversions follow upstream's layout; strix loads both. - llama-memory-hybrid-idx: both APIs kept side by side - strix's qsa_* for qwen4exp and upstream's k-pool for the new GLM5Next (ggml-org#27773 and follow-ups). strix's context set_input_qsa, which upstream deleted, is restored. - llama-graph: strix's in-place gated-delta-net state path is kept; upstream's single recurrent-state gather (ggml-org#29856, avoids a realloc abort under GGML_SCHED_NO_REALLOC) is reverted on this path and needs porting. - vulkan: strix's flash_attn.comp and lightning_indexer.comp are kept whole; upstream's FA shmem out-of-bounds fix (ggml-org#29988) is ported into strix's shader. Upstream's lightning indexer rewrite (ggml-org#29825) is reverted on the Vulkan host side to match strix's shader; its CUDA/Metal parts stay. Upstream's sparse FA path (ggml-org#29639) stays out: strix deleted its shader. - ggml-alloc: upstream's alloc_buffer_n refactor (ggml-org#23671) replaces strix's backport of the view-init fix; strix's regression test is kept. - cuda: strix's RDNA3.5 paths kept; upstream's shared-expert MoE fusion added next to strix's fused GDN decode, with both sets of fusion args. - server: speculative accept keeps strix's replay branch, then upstream's rejection sampling at temp > 0. - tests: both sides' new tests kept (rollback table gets both columns, save-load-state test 10 is strix's, 11 upstream's); strix's rollback test migrated to common_batch / llama_process. - fusion.yml kept (upstream removed it), flash_attn_sparse_compact.comp stays deleted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Rebase GLM-Next support onto master, and migrate to llama-memory-hybrid-idx * Add initial MTP support * Merge branch optimizations. Reduce allocated compute buffer size, speed up long context decode, fla, and slight MTP improvements. * Review driven changes, remove env vars, protect tensors * Strip MTP for initial PR * Clean up after mtp strip * Clean up after mtp strip * Update speculative.cpp * Update llama-context.h * Clean up after mtp strip * Fix tokenizer ignore merges * Improve quantization protection selection * Refactor mhc helpers, graph base * Lint Fixes * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * Skip glm5-next in model saver, fix CRLF * Skip glm5-next in sweep * Remove T4 fallback * Review cleanup * Review suggestions * Defer separate MTP gguf handling to MTP PR, drop filter * Repad n_head_kv * kpool init apply * Order by descending score * Drop guard * read kpool from hparams, clarify kpool cache flags, remove kpool_build_state(nullptr) * Add glm5-next support to model saver and add arch test fixture * Review cleanup * Kpool pooled caching clarify * Add multi stream support * Finish Rebase * Sparse FA fir DSA prefill * Const * Update llama-model.cpp to fix rebase error * gguf-py : merge tensor map entries for HC tensors * model : use build_gdn_l2_norm in GLM5_NEXT implementation * chore : remove trailing whitespace * model : use new OP precision setting API in GLM5_NEXT implementation * mtmd : use ggml_swiglu_clamp in GLM5V and apply the image token limit The two clamps around swiglu_split are what ggml_swiglu_clamp already does, so the clamp bounds collapse back to one value. GLM5V also never called set_limit_image_tokens(), so --image-max-tokens had no effect. Assisted-by: Claude Opus 5 (cherry picked from commit 46d18e1) * llama : keep the GLM5-Next k-pool layout across ubatches The layout was rebuilt from a full cell scan on every ubatch. Pools are fixed by the positions relative to the sequence's first one, so the layout now lives on the memory and a ubatch only appends to it. A sequence edit no longer stales every pooled key either, only the ones at or after the edited position, which makes a tail seq_rm free. The pooling subgraph is built unconditionally so the graph shape no longer changes every kpool tokens, and the pool axis is folded into rows before soft_max, which otherwise exceeds the CUDA gridDim.y limit past n_kv 262144. Assisted-by: Claude Opus 5 (cherry picked from commit 5d1c40b) * model : write the GLM5-Next recurrent rollback checkpoints The conv state and the delta net state were only written to the live row, so a rollback restored whatever the checkpoint rows happened to hold. Take the same route as kimi-k3: build_recurrent_attn for the state, and write all K_rs conv groups. That also drops a state view that assumed contiguous rows. Enroll the arch in test-recurrent-state-rollback, which catches this under its garbage-filled cache pass. Assisted-by: Claude Opus 5 (cherry picked from commit 5ace37e) * llama: fix PR ggml-org#27773 test-save-load-state restore failure Clear the attention and indexer cache data after a failed hybrid state restore so restored NaNs cannot affect a later sequence. Assisted-by: Codex * llama: fix PR ggml-org#27773 gpu-rocm graph reallocation Reserve the full GLM5-Next pool capacity and dirty pool count. The gpu-rocm Test step aborts when n_new grows while the graph node count stays fixed; CUDA, Vulkan, Metal, and WebGPU checks report the same error. Assisted-by: Codex * llama : fix GLM5-Next k-pool layout staleness after edits and shared teardown Two defects in the cross-ubatch k-pool layout added by the k-pool commit: 1. Wrong results. An edited sequence only rebuilt its pool layout when its cell count changed, so if the first ubatch after an edit added back exactly as many cells as were removed, the stale position-to-cell list survived. With a unified cache and more than one sequence, where another sequence takes the freed cells, the reused layout points at the wrong cells (CPU: large logit drift, CUDA: NaN). Rebuild whenever the sequence is stale, not only on a size mismatch. 2. Slowdown. "shared" mode was assumed to end only with an edit that forces a rebuild, but sharing also ends when the other sequence is removed. The survivor kept shared = true, pinning cache_safe off and re-pooling every pool on every ubatch (server trigger: n>1 completions with -kvu, via the seq_cp in copy_state_to). In seq_rm, if the layout has shared cells, stale every sequence so one rebuild re-derives sharing and cache_safe returns to 1. Assisted-by: Claude Opus 5 * llama : fix build_attn_mha stream stride for non-contiguous q build_attn_mha split the batch into streams with a stream stride of q->nb[3]/n_stream. That only equals one stream's span, (ne[2]/n_stream)*nb[2], when q is contiguous. GLM5-Next is nope-only, so it does not concat a rope part and passes the permuted q_absorbed straight in, where nb[3] != ne[2]*nb[2]; the stride was then n_head times too large and every stream s >= 1 read another head's queries. Split-KV (-np N without --kv-unified) multi-stream prefill was wrong for every stream past the first. Unified KV and decode were unaffected (n_stream == 1, and decode takes the gather path). Other MLA models concat rope so q is contiguous and the computed value is unchanged for them. Compute the stride from the token dimension, which is identical for a contiguous q. Assisted-by: Claude Opus 5 * llama : re-derive GLM5-Next k-pool sharing on state_read/state_drop The shared-cell teardown added to seq_rm (stale every sequence when the layout has shared cells, so a survivor does not keep shared = true and pin cache_safe off) was missing from the other paths that can free shared cells: state_read and state_drop staled only the one sequence. Apply the same re-derivation there and correct the comment that claimed sharing ends only via an edit or seq_rm. Assisted-by: Claude Opus 5 * quant : drop duplicate GLM5-Next hc_ filter The hc_ name filter was listed twice in the GLM5_NEXT protection block. Assisted-by: Claude Opus 5 * glm5-next: scope K-pool cache access to indexed operations * glm5-next: keep K-pool access in hybrid index memory * glm5-next: keep mHC graph builders model-local * glm5-next: mark only touched pools per ubatch --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com> Co-authored-by: Piotr Wilkin <ilintar@gmail.com> (cherry picked from commit 649dcb1) Conflict: tools/mtmd/mtmd-image.h, kept the downstream Qwen3VL tiling preprocessor next to glm5v. Assisted-by: Claude Opus 5.5

(cherry picked from commit c879310) (cherry picked from commit c23700b) (cherry picked from commit 4d3c915) b11514 rebase: ported onto upstream GLM5-Next (ggml-org#27773); tensor names match upstream; kept the upstream segment comment. Split rules validated by the GLM5 Meta row once pick 435 lands. b11514 squash: folded 1 commit. - squash! glm5-next: support tensor-parallel execution --- From f0382a969f9c: ggml-backend-meta: drop the unconditional mirrored x batch-split mul_mat rule A mirrored src0 with several batches against a src1 split on a batch axis does not keep the batch mapping. Each device holds every src0 batch but only its slice of src1, so ggml broadcasts the wrong src0 batch onto the local src1 batches. Two CPU-backed Meta devices return [1,20,3,40] for a product whose correct result is [1,2,30,40]. No abort, no warning. Keep the singleton rule below it (src0 ne[axis] == 1). It covers the GLM5-Next pooled-key matmul, k_p [head_dim, n_pool, 1] against q_p split on axis 2. Every other combination falls through to the existing GGML_ABORT instead of computing silently wrong results. Checked: the 2-device CPU repro aborts for ne02=2 and matches for ne02=1; test-llama-archs on Vulkan0 (RX 7900 XTX) without glm5-next (C.7 ARANGE abort) gives the same 160 Meta rows, 102 OK, before and after; llama-finetune -sm tensor -ngl 99 on Vulkan0 trains with losses identical to -sm layer. Assisted-by: Claude Opus 5.5 Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Squashed-with: f0382a969f9c Co-authored-by: Guilherme Gallo <guilherme.gallo@collabora.com> Assisted-by: Claude Code

github-actions Bot added model Model specific mtmd Related to multimodal functionality (video/image/audio) conversion labels Aug 26, 2026

ngxson reviewed Aug 27, 2026

View reviewed changes

Comment thread tools/mtmd/clip.cpp Resolved

ngxson reviewed Aug 27, 2026

View reviewed changes

Comment thread tools/mtmd/clip-model.h Resolved

ngxson reviewed Aug 27, 2026

View reviewed changes

Comment thread tools/mtmd/clip-impl.h Resolved

Rebase GLM-Next support onto master, and migrate to llama-memory-hybr…

9370c82

…id-idx

timkhronos force-pushed the GLM5.3-Flash branch from 106ece6 to 9370c82 Compare August 27, 2026 23:42

Add initial MTP support

d4ce825

This was referenced Aug 28, 2026

GLM-5.2 on DwarfStar — the engine already supports it, no new stack needed evanwtf/local-llm#17

Closed

GLM-5.3-Flash: build llama.cpp PR #27773 and load-test the 89.9 GiB Q2 evanwtf/local-llm#25

Closed

danielhanchen mentioned this pull request Aug 28, 2026

glm5-next: fall back to the deepseek4 spelling for the k-pool tensors timkhronos/llama.cpp#9

Open

timkhronos added 8 commits August 28, 2026 20:02

Merge branch optimizations. Reduce allocated compute buffer size, spe…

d2fc716

…ed up long context decode, fla, and slight MTP improvements.

Review driven changes, remove env vars, protect tensors

9855711

Strip MTP for initial PR

b93ed51

Clean up after mtp strip

ba436fd

Clean up after mtp strip

3ff8d92

Update speculative.cpp

f176c8a

Update llama-context.h

c34db58

Clean up after mtp strip

0aa327b

timkhronos mentioned this pull request Aug 28, 2026

Add MTP for GLM-5.3-Flash #27917

Draft

matteoscalabrini mentioned this pull request Aug 29, 2026

model : add GLM-5.3-Flash (glm5next) #27752

Closed

Fix tokenizer ignore merges

ec1cdbf

danielhanchen mentioned this pull request Oct 5, 2026

Carry what the dropped GLM-5-Next, Qwen MTP and readahead pins had beyond upstream unslothai/llama.cpp#241

Open

ggerganov mentioned this pull request Oct 5, 2026

llama.cpp : bump version to 0.6.0 #29997

Merged

This was referenced Oct 6, 2026

glm5-next : keep the indexer compute buffer bounded at long contexts unslothai/llama.cpp#243

Open

Studio: keep GLM-5.3-Flash's MTP head in Auto unslothai/unsloth#12833

Merged

paulp83 mentioned this pull request Oct 7, 2026

Misc. bug: CUDA illegal memory access on GLM-5.3-Flash (glm5next) long prefill at -ub 2048 (Blackwell/sm_120) #28282

Open

paulp83 mentioned this pull request Oct 7, 2026

Eval bug: glm5-next + --spec-type draft-dflash aborts on first request: GGML_ASSERT(t_layer_inp[il] != nullptr) (glm5-next never sets t_layer_inp) #30093

Open

evandhoffman mentioned this pull request Oct 7, 2026

Mainline llama.cpp serves GLM-5.3-Flash with MTP (#27773, #29928): a second GLM engine on the M5 Max, against ds4 evanwtf/local-llm#987

Open

DustinTrap mentioned this pull request Oct 9, 2026

Decode-speed options researched 2026-10-08: not pursued now; revisit when upstream fixes land DustinTrap/dgx-spark#30

Open

paulp83 mentioned this pull request Oct 9, 2026

feat(glm): dual avar6 Q2_K-Q3_K fast slug + setup post_download conversion for mainline-format GGUFs noonghunna/club-3090#1605

Closed

OverStruck mentioned this pull request Oct 10, 2026

GLM-5.3-Flash: support the upstream llama.cpp architecture name glm5-next (current Unsloth GGUFs) asher/gmlx#186

Closed

WildeTechSolutions mentioned this pull request Oct 12, 2026

GLM 5.3 Flash vision via strata-vision — mtmd already has glm5v upstream (llama.cpp#27773); ggml pin predates it VecSzn/PolyStrata#14

Closed

gaetan-puleo mentioned this pull request Oct 12, 2026

Merge upstream llama.cpp up to 4a096b8ff (2026-09-30) halo-box/strix-llama.cpp#174

Closed

2 of 5 tasks

christopherthompson81 mentioned this pull request Oct 22, 2026

GLM-5: support the DSA MoE (GLM-5 to 5.3), GLM-5.3-Flash and GLM-5V christopherthompson81/inference.rs#555

Open

ChristianHohlfeld mentioned this pull request Oct 29, 2026

GLM5-Next: safely load legacy 6block glm5next GGUF metadata ChristianHohlfeld/mag-portable-stack#217

Merged

Navigation Menu

Repository navigation

add GLM-5.3-Flash (GLM5-Next) support - #27773

add GLM-5.3-Flash (GLM5-Next) support#27773
am17an merged 61 commits into
ggml-org:masterfrom
timkhronos:GLM5.3-Flash

timkhronos commented Aug 26, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

danielhanchen commented Aug 28, 2026 •

edited

Loading

Uh oh!

danielhanchen commented Aug 28, 2026

Uh oh!

Sciguy429 commented Aug 28, 2026

Uh oh!

danielhanchen commented Aug 28, 2026

Uh oh!

danielhanchen commented Aug 28, 2026

Uh oh!

timkhronos commented Aug 28, 2026 •

edited

Loading

Uh oh!

danielhanchen commented Aug 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

timkhronos commented Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Additional information

Architecture

What I implemented new:

Tests

Limitations

Requirements

Uh oh!

Uh oh!

Uh oh!

Uh oh!

danielhanchen commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danielhanchen commented Aug 28, 2026

Uh oh!

Sciguy429 commented Aug 28, 2026

Uh oh!

danielhanchen commented Aug 28, 2026

Uh oh!

danielhanchen commented Aug 28, 2026

Uh oh!

timkhronos commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danielhanchen commented Aug 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

timkhronos commented Aug 26, 2026 •

edited

Loading

danielhanchen commented Aug 28, 2026 •

edited

Loading

timkhronos commented Aug 28, 2026 •

edited

Loading