llama.cpp 0.6.0 adds GLM-5.3-Flash support and a server API for decision models

The release notes cover everything since version 0.5.0. If you run models on your own hardware, four groups of changes matter: the newly supported models, the measured speedups for speculative decoding, the new server features, and the interface changes that other programs depend on.

New models, including one with 320 billion parameters

Version 0.6.0 loads GLM-5.3-Flash, which the release notes also call GLM5-Next. The author of the change describes it as a hybrid model for text and vision with 320 billion parameters. Converted GGUF files are linked from the pull request. According to the author, about 1 GB of small, precision-sensitive tensors is left unquantized in those files.

The same pull request lists two limits. MTP speculative decoding for this model is left to a separate pull request, and running several sequences at once requires the --kv-unified option.

The release also adds support for Ling 3.0 VL and for the Clef and Nimble decision models, which the server section below covers.

Two measured speedups for speculative decoding

Both measurements come from the authors of the changes and are published in their pull requests. They are not independent tests.

The first change adds MTP speculative decoding for the architecture that the release notes call Qwen4Exp. The author measured it on a DGX Spark, with the Qwen3.8-Flash-Next model in the iq4_xs quantization and on 24 prompts. Decoding speed rose from 28.36 to 43.88 tokens per second, which the author reports as a speedup of 1.55 times. The reported average acceptance rate for drafted tokens is 0.640. You turn the mode on with --spec-type draft-mtp.

The second change adds new kernels to the Metal backend for matrix multiplication with a small number of rows. Speculative decoding and decoding several sequences at once both produce such operations. The kernels are used on Apple GPUs that lack the tensor API, which the author gives as M1 through M4. On an M3 Ultra, with Qwen3.8-27B in the Q4_0 quantization, the author measured generation with eight parallel sequences rising from 38.6 to 119.7 tokens per second. Speed with one sequence stayed at 32.9 tokens per second.

The same pull request shows what this means for a draft model. Before the change, decoding with a DFlash2 draft model on that machine was slower than plain serial decoding: 30.2 against 32.1 tokens per second, on a code prompt at temperature 0. After the change the same test gives 110.0 tokens per second, and the speed of serial decoding did not change.

A server endpoint for decision models

The server gets a new endpoint, /v1/systemone, for what the project calls decision models. A request carries a state, such as a customer message with order data, and a set of questions. Each question asks for a choice among the given options, a yes or no, or a score on a scale, and the model returns probabilities. The endpoint supports the laya, julia-1, lev, openjev, and kev models, and the release extends it to Nimble and Clef.

The author describes these models as wrappers around ordinary embedding models, so the change to the core library is small. Converted GGUF files for the five named models are published under the ggml-org account on Hugging Face.

The other server changes are smaller. GET /v1/models now reports the input and output modalities of each model in a new architecture object. /v1/embeddings accepts typed content with images, audio, and video, and it returns HTTP 400 for invalid requests. The built-in web UI gets model downloads, a data layer for the Hugging Face Hub, and a check of whether a model fits in memory.

What changes for programs built on the library

The release notes list these interface changes:

• The include/llama.h header adds an extended batch API, llama_batch_ext, with a llama_process() call. The new API accepts batches that mix tokens and embeddings, and the examples, the server, and the multimodal library already use it.
• The saved session formats have new version numbers: LLAMA_SESSION_VERSION is 11, and LLAMA_STATE_SEQ_VERSION is 4.
• The mtmd_get_memory_usage() function now returns a mtmd_memory_usage struct, so code that calls it has to be adapted.
• The bundled ggml library is now version 0.26.0. It enables Windows ARM64 builds with the MSVC compiler and adds sparse flash attention kernels for the Vulkan, SYCL, and Metal backends.