NVIDIA's free PAIR beta sends local inference to whichever machine on your network is free
NVIDIA published the Personal AI Router at IFA 2026 on September 3, 2026. It is a free, open source tool for Windows, macOS, and Linux that finds compatible machines on your local network and routes each independent inference request to one that has capacity, so parallel agent subtasks stop queueing behind a single GPU. It works with Ollama and LM Studio, and supports GeForce RTX 20 Series and newer, RTX PRO workstation cards, DGX Spark, and Apple M4 silicon. What it does not do is pool memory, so a model that does not fit on one machine still does not fit.
Source
Sparks Fly: NVIDIA Accelerates Local AI at IFA 2026NVIDIA Blog i NVIDIA Newsroom · Original published September 3, 2026
Much of the appeal of running models locally disappears when four agent subtasks all queue behind the same graphics card. NVIDIA's answer, published at IFA 2026 on September 3, 2026, is a free, open source tool called Personal AI Router, or PAIR. It discovers compatible machines on the local network and routes each independent inference request to a system with capacity, adapting as devices join and leave.
What it does
PAIR gives your apps and agents a single local endpoint and proxies requests behind it. At launch it supports two backends you probably already run, Ollama and LM Studio, and it ships with both a graphical and a terminal interface on Windows, macOS, and Linux. Setup, as NVIDIA describes it, is installing the tool on each machine, pairing them on the same network, and pointing your app at the endpoint.
The privacy argument follows from the design rather than from a policy. NVIDIA says prompts, files, and agent context stay on your local network instead of going to a cloud inference service, and the tool needs no internet connection to operate. You only need one to download models.
- GeForce RTX 20 Series GPUs and newer.
- RTX PRO workstation GPUs, Turing architecture and newer.
- NVIDIA DGX Spark.
- Apple M4 silicon or newer.
What it does not do
This is the part worth reading twice before you plan around it. The machines stay separate systems handling parallel tasks, and PAIR does not combine them into one virtual GPU. A model that does not fit in the memory of a single machine on your network still does not fit; adding a second PC buys you throughput on independent requests, not headroom for a bigger model.
The rest of the fine print is ordinary beta fine print. The tool is in beta, its interface is English only, it asks for 8 GB of system memory or more, and NVIDIA recommends 20 GB of disk space.
What shipped alongside it
The same post carries inference numbers, and they are the vendor's own measurements rather than an independent benchmark. They arrive through the backends most people use locally, so you get them by updating rather than by changing anything.
- llama.cpp: up to 1.9x higher throughput on a GeForce RTX 5090, from kernel optimizations, better speculative decoding, and faster prefill, according to NVIDIA.
- vLLM: 1.2x on an RTX PRO 6000 Blackwell Workstation Edition and up to 1.4x on two clustered DGX Spark systems, according to NVIDIA.
- NVIDIA RTX Spark Windows PCs arrive in October 2026.
„PAIR doesn't combine them into one virtual GPU.“
Sources
Related
Nemotron 3.5 Bolt squeezes 74 billion parameters into 24 GB, but Nvidia's llama.cpp recipes start at 32 GB
Nvidia released two open models for agents on October 15, 2026: the large Nemotron 3.5 Super VL and the smaller Nemotron 3.5 Bolt 74B-A9B, built to run on your own machine. Two-bit quantization brings Bolt down to a 24 GB package. Nvidia's own llama.cpp instructions, though, call for a GPU with at least 32 GB or an RTX Spark with 64 GB of memory.
NVIDIAverified
llama.cpp now runs Aleph Alpha's Kolibri-1, and the Q4_K_M file takes 49.8 GB
llama.cpp build b11709, published on October 17, 2026, adds support for Kolibri-1, a German and English model from Aleph Alpha with 78 billion parameters. GGUF files range from 17.6 GB to 156.3 GB, so the model can run without the data-center GPUs its maker lists as the minimum. A later build adds a way to find and download models from the web interface.
llama.cppverified
llama.cpp now runs Reflection's Beam, and its smallest GGUF file is 103.1 GB
llama.cpp build b11701 adds support for Beam, Reflection's open model with 501 billion parameters, released on October 16, 2026. Unsloth has published GGUF files ranging from 103.1 GB to about 1 TB, so the model can now run outside server stacks such as vLLM and SGLang. No speed measurement has been published yet.
llama.cppverified