llama.cpp now runs Aleph Alpha's Kolibri-1, and the Q4_K_M file takes 49.8 GB

llama.cpp, the open-source runtime that runs language models on ordinary computers, supports Kolibri-1 as of build b11709, published on October 17, 2026. Aleph Alpha, a German AI company, released the model on October 3, 2026, under the Apache 2.0 license. Its own instructions describe serving on data-center GPUs, while GGUF files for llama.cpp start at 17.6 GB.

What Kolibri-1 is

Kolibri-1 is a mixture-of-experts model built for German and English. It has 78 billion parameters, but only 3.46 billion are active for each token. Each of its 50 layers holds 384 experts, which are smaller subnetworks, plus one shared expert, and a router picks six experts per token.

The native context length is 262,144 tokens, and the company says it has validated quality up to 1,048,576 tokens. On the GPQA Diamond science benchmark, Aleph Alpha reports a score of 84.3 in English and 81.3 in German. These are the vendor's own measurements.

What llama.cpp added

The change converts Aleph Alpha's weights into a GGUF file and describes the model's layer layout to llama.cpp. Most attention layers look only at the preceding 512 tokens, and every fifth layer sees the whole context. According to the author, the FP8 checkpoint Aleph Alpha published needs no special handling.

How large each file is

Quantization stores weights with fewer bits per value, which makes the file smaller at some cost in accuracy. The quantizer bartowski has published GGUF files of Kolibri-1 made with build b11716. The sizes below are as Hugging Face lists them.

• IQ1_S is the smallest file, at 17.6 GB.
• IQ2_M takes 28.1 GB.
• Q3_K_M takes 37.6 GB.
• Q4_K_M takes 49.8 GB.
• Q6_K takes 69.0 GB, and Q8_0 takes 83.1 GB.
• The unquantized BF16 copy takes 156.3 GB.

What fits on which machine

A model runs at a usable speed only when the file, plus room for the context and the operating system, fits in memory. This suggests that a 32 GB machine can hold only the smallest variants. At 64 GB, Q4_K_M fits, while Q8_0 needs more. For the FP8 weights of about 78 GB, Aleph Alpha lists two 80 GB A100 cards or one H200 as the minimum.

The author of the change measured 106.01 tokens per second of generation with their own Q4_K_M build, on a server with four A800 cards. No measurement on a home computer has been published, and no independent quality test of the quantized files exists yet.

Finding models from the web interface

Build b11714, published the same day, adds a Discover Models view to the web interface of llama-server. It searches GGUF models on Hugging Face, shows whether each variant fits in memory, and downloads it with pause and resume. The setting is off by default and has to be enabled in the interface.