Skip to content
What are you looking for?. Open searchSR

Tags

Benchmarks

13 items
Modelsmedium signal

Mistral Large 4 opens a 1-trillion-parameter preview API at $1.36 per million input tokens

Mistral launched a public preview of Mistral Large 4 on October 6, 2026. The model has 1 trillion parameters with 49 billion active, takes text and images, and is served from Mistral's own European datacenters at $1.36 per million input tokens and $4.18 per million output tokens. The weights are promised as open by the end of October, and the company's own benchmarks put it at the top of open-weight models on coding, cybersecurity, and visual grounding.

Mistral AIverified

Modelsstrong signal

Google announces Gemini 4 Argon for vetted cyber defenders only, at a $2 introductory input price

Google announced Gemini 4 Argon on September 30, 2026. The model is rolling out to a set of trusted cyber defenders through the Fairwind Program, and Google gives no date for developers, enterprises, or consumers. The introductory price is $2 per million input tokens and $10 per million output tokens, and it rises to $4 and $20 when the introductory period ends. The output limit grows to 1 million tokens from 64,000. The benchmark results in the announcement are Google's own reporting, and nobody outside the program can check them yet.

Googleverified

Left, the words Research preview and HydraFusion with the line Advanced runtime model orchestration in GitHub Copilot; right, a dark terminal model picker listing Auto, HydraFusion (Preview) highlighted in blue, GPT-5.6 Sol, GPT-5.6 Terra, Claude Opus 5, and Claude Sonnet 5.
Agentsmedium signal

GitHub Copilot's HydraFusion, which can run two models on one request, reaches VS Code and the Copilot app

HydraFusion sits in the model picker but is not a model. It chooses one of three workflows per request and can hand a task to two models in a row, one drafting and one checking. GitHub moved the research preview from Copilot CLI to Visual Studio Code and the Copilot app on September 30, 2026, for Pro, Pro+, Business, and Enterprise plans. Billing follows the tokens each model consumes, so a request can cost more or less than a single-model one.

GitHub Changelogverified

Modelsmedium signal

NVIDIA releases Kumo Tabular, an open model that predicts table rows without training and tops four benchmarks

Kumo Tabular reads a table of labeled rows and returns predictions for new rows in a single forward pass, with no training, no tuning, and no feature engineering. NVIDIA published it on September 29, 2026 in three sizes from 28 million to 215 million parameters, under the OpenMDW-1.1 license that allows commercial use. By NVIDIA's own measurement it ranks first on TabArena, BeyondArena, TALENT, and ScoringBench. It handles numeric and categorical columns only, and it needs a CUDA GPU.

Hugging Face Blogverified

Modelsstrong signal

OpenAI ships GPT-6.1 Sol at GPT-6 Sol prices and adds a $500 Pro plan with Ultrafast

GPT-6.1 Sol costs $2 per million input tokens and $10 per million output tokens, the same as GPT-6 Sol, and its cached input drops to $0.10. OpenAI says the model comes close to GPT-6 Astra on coding and computer use at a fifth of Astra's price. The larger GPT-6.1 Astra did not ship. A new Pro 500 plan at $500 a month is the only Pro tier that includes Ultrafast, and Pro 200 returns with a smaller allowance.

OpenAIverified

Modelsstrong signal

OpenAI releases GPT-6 Sol and Luna and halves the API price of both tiers

GPT-6 Sol costs $2 per million input tokens and $10 per million output tokens, and GPT-6 Luna costs $0.10 and $0.50. OpenAI measures the 50% cut against the promotional prices it charged for GPT-5.6. The launch also brings a reworked prompt cache with discounts of up to 90% on cached input. Every benchmark comparison with Claude comes from OpenAI and uses Opus 5, not Opus 5.5.

OpenAIverified

Industrystrong signal

OpenAI says an internal model resolved Navier–Stokes and sets up a mathematics advisory group

A post dated September 21, 2026 says an internal model has resolved the Navier–Stokes Millennium Prize problem and more than 100 long-standing open problems. No paper accompanies the claim and no independent verification exists. A declaration signed by 27 Fields medallists had appeared on September 11, 2026, arguing that solving open problems as a benchmark damages mathematics. OpenAI's answer is an independent advisory group hosted at the Institute for Advanced Study.

OpenAIverified

Modelsmedium signal

NeoMME's 260M encoder lands within 0.002 of a 3.75B model on ViDoRe v3

H company published NeoMME on September 3, 2026: a pair of multimodal encoders, at 260M and 800M parameters, released under Apache 2.0 and loadable through Hugging Face Transformers. On the ViDoRe v3 document retrieval benchmark the small one scores 0.523 nDCG@10, which is 0.002 behind ColQwen2.5 at roughly 14 times its parameter count. It also encodes about 51 pages per second on a single NVIDIA L40S, and the index it produces can be compressed from about 1.5 MB per page to 6 kB while keeping more than 95% of retrieval quality.

Hugging Faceverified

Modelsstrong signal

OpenAI ships GPT-6 Astra at $10 per million input tokens, with enterprise access off by default

GPT-6 Astra went out on September 3, 2026, first to a limited set of organizations and then, over the following days, to ChatGPT Plus, Pro, Business, and Enterprise. In the API it answers to the name gpt-6-astra at $10 per million input tokens and $50 per million output tokens, and it is also served through Microsoft Azure and Amazon Bedrock. Enterprise workspaces do not get it automatically, because access is off by default at launch and an administrator has to turn it on. OpenAI also says the extra safety checks around this model can pause work in ChatGPT and Codex and stop it outright in the API.

OpenAIverified

Modelsmedium signal

Ai2 published a method that shows what individual benchmark questions actually measure

BenchMIRT borrows item response theory from psychometrics and applies it to a benchmark one question at a time, instead of trusting the average. Trained on 100 models across 16 benchmarks and more than 34,000 questions, it recovered two dimensions on its own: safety and general reasoning. Several benchmarks turned out to sit on the dimension nobody assigned them to. Keeping 10% of the questions preserved almost the same ranking as the full set.

Ai2verified

Modelsstrong signal

OpenAI puts Astra in its critical cyber tier, and the safeguards will stop legitimate work too

OpenAI says its unreleased Astra model meets the Critical cybersecurity threshold in its Preparedness Framework, the first model it has designated that way. In testing the model found previously unknown flaws in a hardened browser and operating system and chained them into working compromises. Parts of the model's development and release were delayed while protections were rebuilt. The part that matters outside the lab is the deployment note: the new monitoring can pause or stop work that has nothing to do with security.

OpenAIverified

Modelsstrong signal

Anthropic ships Claude Fable 5.1 and cuts the price of cache reads

Claude Fable 5.1 and Claude Mythos 5.1 are one model at two levels of safeguards, and only the first is generally available. The change that shows up on a bill is the cache read, which drops to $0.25 per million tokens. Anthropic estimates that as roughly 25% off a typical workload and up to about 45% off a heavily agentic one. The model is also allowed to look for software vulnerabilities now, while exploit work stays behind the older restrictions.

Anthropicverified