llama.cpp now runs PrismML's ternary Bonsai 2 27B without the company's fork

Stock llama.cpp can now run Bonsai 2 27B, a model from PrismML whose weights take only three values: −1, 0, and +1. Build b11556, published on October 11, 2026, adds the runtime step those weights depend on. Until then the model needed PrismML's own fork of llama.cpp, and the standard build loaded the file without a warning and produced gibberish.

What changed in llama.cpp

PrismML stores the Bonsai 2 weights in a rotated form. Before the weights are reduced to three values, each matrix goes through a Hadamard rotation, a fixed mathematical mixing of values that makes them easier to compress. The rotation is baked into the stored weights. The runtime therefore has to apply the same transform to the activations, the intermediate values the model computes while it runs. Skip that step, and weights and activations no longer match, so the output is meaningless.

The change reads the rotation from the metadata in the GGUF model file and applies it during inference. The conversion script writes that metadata when a checkpoint comes with PrismML's description of the rotation. It refuses model architectures where the runtime would not apply the transform, so a converted file either runs correctly or fails to convert.

In the pull request, the author reports a perplexity ratio of 1.0001 on Apple's Metal and 0.9995 on CUDA, measured on the same file against PrismML's fork. Perplexity measures how well a model predicts a test text, so a ratio close to 1 means the two runtimes behave almost identically. Without the change, the same test gave a perplexity above 1.2 million.

Which file works, and which still does not

The change covers the Q2_0 packing, a 7.63 GB file that PrismML keeps in a separate testing repository on Hugging Face. The file name still says the fork is required, because it was published before upstream support existed.

PrismML's main repository offers two smaller packings, PQ2_0 at 7.21 GB and PTQ1_0 at 5.95 GB. They use quantization types that mainline llama.cpp does not know, and this change does not add them, so those files still need the fork. For comparison, the same model at 16-bit precision (FP16) takes about 54 GB.

Bonsai 2 rotates its weights in blocks of 1,024 values. CUDA kernels, the routines that run on the graphics processor, for block widths above 512 were merged into llama.cpp on October 7, 2026. The matching kernels for Vulkan are still in review.

What Bonsai 2 27B is

Bonsai 2 27B is derived from Qwen3.8-27B and has 27.36 billion parameters. Each weight is −1, 0, or +1, and every group of 128 weights shares one scale factor. PrismML puts the whole model at 1.72 bits per weight. The weights are released under the Apache 2.0 license.

PrismML says the model keeps 98.2% of the full-precision model's score across 14 benchmarks run in thinking mode. That is the vendor's own measurement, and it has not been confirmed independently.