Nemotron 3.5 Bolt squeezes 74 billion parameters into 24 GB, but Nvidia's llama.cpp recipes start at 32 GB
Nvidia released two open models for agents on October 15, 2026: the large Nemotron 3.5 Super VL and the smaller Nemotron 3.5 Bolt 74B-A9B, built to run on your own machine. Two-bit quantization brings Bolt down to a 24 GB package. Nvidia's own llama.cpp instructions, though, call for a GPU with at least 32 GB or an RTX Spark with 64 GB of memory.
Source
Open From Data Center to Desktop: NVIDIA Expands Nemotron Family With Highest Efficiency Super VL and Bolt ModelsNVIDIA Blog i NVIDIA Newsroom · Original published October 15, 2026
Nvidia released Nemotron 3.5 Bolt 74B-A9B on October 15, 2026, an open-weight model for local agents. Agents are programs that carry out multistep tasks on their own, and Bolt is meant to run them without sending the work to the cloud. It has 74 billion parameters, 9 billion of them active per token, packed into 24 GB. The larger Nemotron 3.5 Super VL for servers shipped the same day, and the smallest model, Bolt E4B, is due on October 23, 2026.
How the model gets down to 24 GB
Bolt was compressed from the larger Super VL. Like its parent, it is a mixture-of-experts model that uses only part of the network for each token, and Nvidia says that makes it faster than most popular dense models with a similar memory footprint. It accepts text, images, and video, with a context of up to 262,144 tokens.
Quantization stores weights with fewer bits per value. Nvidia cut the model's experts to about two bits per parameter, 2.0625 to be exact, and trained the model at that precision to recover lost accuracy. The GGUF file, the format llama.cpp uses, is 22.0 GB, and the image-processing add-on is another 1.7 GB.
How much memory it actually needs
The name says 24 GB, but Nvidia's llama.cpp instructions assume more memory, because the context needs room alongside the weights. A 24 GB card is not among the configurations Nvidia lists. This suggests the 24 GB figure describes the weights, not the smallest GPU that runs the model with the recommended settings.
Nvidia also lists support in Ollama, LM Studio, Unsloth, and vLLM. For llama.cpp, the model card gives two configurations:
- A 28 GB recipe for an RTX Spark with 64 GB or a GPU with at least 32 GB, with a 65,536-token context.
- A 32 GB recipe for an RTX Spark with 128 GB or a GPU with at least 48 GB, with the full 262,144-token context.
What Nvidia's own benchmarks show
In Nvidia's measurements, Bolt scores 50.98 on Terminal-Bench 2.1, against 44.38 for Qwen3.6-35B-A3B. The gap is wider on the BrowseComp search benchmark, 66.75 to 48.74. On PinchBench, the same table puts Qwen3.6-35B-A3B, run with unquantized BF16 weights, slightly ahead at 88.07 to 87.82.
No independent check of these results has been published, and Nvidia's Bolt figures apply to the two-bit version. Both models are released under the OpenMDW-1.1 license, which the model card says allows commercial and non-commercial use.
The larger model for servers
Nemotron 3.5 Super VL has 120 billion parameters, 12 billion of them active per token, and a context of up to one million tokens. For the unquantized BF16 weights, the model card lists four H100 GPUs or two GB200s. Nvidia claims Super VL delivers up to five times the throughput of other models in its class.
The model is available on Hugging Face, on ModelScope, and as a NIM microservice from Nvidia. On OpenRouter it cost $0.10 per million input tokens and $0.40 per million output tokens on October 19, 2026, and a free variant is also listed.
Sources
Related
llama.cpp now runs Aleph Alpha's Kolibri-1, and the Q4_K_M file takes 49.8 GB
llama.cpp build b11709, published on October 17, 2026, adds support for Kolibri-1, a German and English model from Aleph Alpha with 78 billion parameters. GGUF files range from 17.6 GB to 156.3 GB, so the model can run without the data-center GPUs its maker lists as the minimum. A later build adds a way to find and download models from the web interface.
llama.cppverified
llama.cpp now runs Reflection's Beam, and its smallest GGUF file is 103.1 GB
llama.cpp build b11701 adds support for Beam, Reflection's open model with 501 billion parameters, released on October 16, 2026. Unsloth has published GGUF files ranging from 103.1 GB to about 1 TB, so the model can now run outside server stacks such as vLLM and SGLang. No speed measurement has been published yet.
llama.cppverified
llama.cpp now runs PrismML's ternary Bonsai 2 27B without the company's fork
Build b11556 adds the weight rotation step that Bonsai 2 27B was missing in stock llama.cpp. The 7.63 GB Q2_0 file now gives correct output there, while PrismML's smaller PQ2_0 and PTQ1_0 packs still require the company's own build.
llama.cppverified