[ICS] Unable to run llama-cli with MTP speculative decoding model

[haarika-madaka]

Customer failed to run MTP speculative decoding model with the llama-cli. With plain decode (same build, same model, no -md) it works fine.

GGML_OPENVINO_DEVICE=GPU

llama-cli -m gemma-4-12b-it.gguf -md gemma-4-12b-mtp-q8.gguf --spec-type draft-mtp -ngl 99 -ngld 99 -fa on -c 4096 -n 128 --temp 0 -p "test prompt"

We are able to reproduce the issue. MTP speculative decoding mode is not listed in the validated models.

ssh [pse@aisw-arl-ultra7-265-l.fm.intel.com](mailto:pse@aisw-arl-ultra7-265-l.fm.intel.com)

cd ~/llama.cpp

./build/ReleaseOV/bin/llama-cli -m ~/models/gemma-4-12B-it-Q4_K_M.gguf -md ~/models/mtp-gemma-4-12B-it-Q8_0.gguf --spec-type draft-mtp -ngl 99 -ngld 99 -fa on -c 4096 -n 128 --temp 0 -p "test prompt"

[haarika-madaka]

@ravi9 Can we have an update on this topic? Thanks

[ravi9]

MTP support will be added in OV in a later version - mostly mid-Sept.
Once it is added in OV, we will add it in llama.cpp by end of Sept.