llama.cpp 0.5.0 adds the HRM-Text architecture and lets its server listen on several addresses

Version 0.5.0 of llama.cpp came out on September 23, 2026, and its notes cover the changes since version 0.4.1. The parts that affect existing setups are model coverage and llama-server: one new architecture, a new conversion target, multi-address binding, and a fix for router deployments protected by API keys.

A looped architecture, with its costs stated

Version 0.5.0 can load HRM-Text models, including DFM Mimir 1B. HRM-Text alternates between two stacks of transformer blocks over the same tokens, so a single pass through the model runs the same blocks many times. The release also adds conversion support for MiMo-V2.6 and DFlash speculative decoding for HunyuanOCR.

The contributor who wrote the support calls it experimental and lists the costs in the pull request. According to the contributor, the KV cache for Mimir 1B holds 128 layers, four times a same-width model with 32 layers, or about 3072 MiB at a 4096-token context in F16. The contributor measured decoding at 2.65 tokens per second in BF16 on an 8-thread desktop CPU, roughly four times slower than a dense model of the same width. Only causal attention is implemented, and the prefix-LM mode of the original implementation is not.

One server, several addresses

Until now llama-server listened on a single address. Binding it to a VPN address made it unreachable through localhost, which broke clients that assume localhost. The --host option now accepts a comma-separated list of IP addresses and Unix socket paths, and the server opens one listener per address, all serving the same API and model. TCP listeners share the configured port. If any address fails to bind, startup stops and the listeners already opened are closed.

A fix for router mode with API keys

In router mode, one llama-server process starts separate child instances for individual models. When the router was started with --api-key-file, the children received the same file and checked requests against it alone. Clients that authenticated with a key passed through --api-key therefore got 401 errors on chat completions, and some internal router calls were rejected as well. Version 0.5.0 no longer passes the key file to child instances, so no API key ends up on their command line. A related change stops children from writing into the router's log file.

Other changes

• Function-call outputs sent to the server can now include an image through input_image.
• Default values for temperature, top-p, min-p, and repetition penalties can be set through environment variables.
• The multimodal code gained extra sanity checks, including a fix for a buffer overrun in SigLIP image handling for unusually tall or wide images.
• The notes also list CUDA and Metal optimizations but publish no measurements for them.