Kimi K3.1 Pro: How An Open-Weight Giant Got Near The Frontier

🔍 Read the full analysis: Kimi K3.1 Pro: How An Open-Weight Giant Got Near The Frontier on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on office and shipping supplies

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Moonshot AI launched Kimi K3.1 Pro on October 23, and evaluations by Artificial Analysis placed it fourth on the firm’s Intelligence Index, three points behind the top-ranked model. The release is notable because Moonshot says it will publish the model’s weights by November 4, which could bring a near-frontier system to users able to run it themselves; for now, however, the weights are unavailable and the licence has not been published.

Artificial Analysis gave K3.1 Pro an Intelligence Index score of 55.1, using each model at its highest reasoning setting. Claude Opus 5.5 scored 57.6, Claude Sonnet 5.5 scored 56.0 and Muse Wonder Full scored 55.8. The evaluator said K3.1 Pro would lead open-weight models once its weights are released. That ranking is conditional: the model is not yet available for independent deployment or examination of its weights.

The benchmark results show a particularly large gain in agentic coding. Artificial Analysis measured K3.1 Pro at 67% on Terminal-Bench 4.0, compared with 13% for the earlier Kimi K3. On GDPval-AA, a test of simulated knowledge work, it scored 1,846 Elo, below Opus 5.5 at 1,866 and Muse Wonder Full at 1,857. Its score on Humanity’s Last Exam was 52%, up from 47% for K3 and below Opus 5.5 at 61%.

Artificial Analysis listed the model at $2 per million input tokens and $10 per million output tokens, or about $2.11 per Intelligence Index task in its comparison. It estimated Opus 5.5 at about $5.98 per task. Those figures reflect the evaluator’s pricing and test setup, not a guarantee that every workload will cost the same amount.

A Narrower Gap for Open Weights

If Moonshot publishes usable weights under a licence that permits the intended forms of use, K3.1 Pro could give companies and researchers a self-hosting option with benchmark performance close to leading closed models. That could matter to organizations seeking more control over data handling, customization or deployment than an API alone provides. It would not automatically make the model inexpensive to operate: Artificial Analysis lists 2.8 trillion total parameters and 104 billion active parameters, a scale that points to substantial infrastructure needs.

The measured results are strongest in agentic tasks, where a model uses tools or performs multi-step work. The improvement from K3’s 13% to K3.1 Pro’s 67% on Terminal-Bench 4.0 is a large change on that specific evaluation. It should not be read as proof of equal performance across all coding jobs, or as a measure of reliability in production. The more modest knowledge-test gains and the remaining gap on Humanity’s Last Exam show why a single overall ranking cannot settle whether the model suits a particular task.

Price comparisons also need care. The listed API rates match Claude Sonnet 5.5’s input and output prices, while the estimated cost per benchmark task is lower than Opus 5.5’s under Artificial Analysis’s test. Actual bills can differ with prompt length, reasoning settings and usage patterns. For buyers, the development is a reason to evaluate the model against real workflows, not evidence of guaranteed savings or performance.

K3.1 Builds on the Earlier Kimi

K3.1 Pro is described by Moonshot as a more heavily post-trained version of Kimi K3. The new model was made available through the Kimi apps, Kimi Work, Kimi Code and Kimi API on October 23. Artificial Analysis lists a one-million-token context window and text and image input. These product and specification details describe access through Moonshot’s services; they do not mean the weights are already open for download.

Before this launch, Artificial Analysis listed Xiaomi’s MiMo-V2.6-Pro at 46.3 as the highest-scoring open model in the comparison, while Opus 5.5 scored 57.6. Kimi K3.1 Pro’s 55.1 narrows that measured gap to 2.5 points against the listed Opus score. Moonshot’s launch materials also promote game development, 3D and CAD work, finance tasks and fewer hallucinations. Those claims draw on vendor demonstrations and internal tests, so they are distinct from the evaluator’s independent benchmark results.

“The Kimi Hallu Bench Hard test is “deliberately crafted” to provoke errors, and its rates “do not represent overall error rates in daily use.””

— Moonshot AI

Weights, Licence and Real-World Reliability

The central outstanding detail is whether Moonshot will publish the weights by its stated November 4 target, and under what licence. Until the release, outside parties cannot verify the model’s weights, reproduce the claimed deployment conditions or assess whether the licence permits commercial use, modification or redistribution. Moonshot’s description of K3.1 Pro as open-weight does not resolve those questions.

Some performance claims also remain unverified outside Moonshot’s tests. The company says hallucinations fell from 23.4% for K3 to 8.3% on its internal Kimi Hallu Bench Hard test, a roughly 65% reduction in that test’s claim-level rate. Artificial Analysis reported a smaller change on its separate AA-Omniscience evaluation: from 53% for K3 to 44% for K3.1 Pro. The results use different evaluations and should not be treated as directly interchangeable or as a general error rate.

Moonshot’s claims about end-to-end game creation, CAD and financial work also rely on company demonstrations or internal measurements. The source material does not establish how consistently the model performs those tasks across independent users, varied prompts or production environments. Nor does the available information specify the hardware and operating costs required for organizations to self-host it.

The November Weights Release

The next milestone is Moonshot’s promised publication of K3.1 Pro’s weights by November 4. The release date and licence terms will determine whether the model can be independently inspected and how broadly it can be deployed. If the weights appear, independent researchers and developers will be able to test the model outside Moonshot’s hosted products, subject to the licence and computing requirements.

Readers should watch for confirmation that the files are available, the exact licence, and third-party evaluations of the released model. Until those details emerge, the clearest evidence is the current Artificial Analysis benchmark and API pricing, while broader capability claims remain partly based on Moonshot’s own tests.

Key Questions

Is Kimi K3.1 Pro open source now?

No. Moonshot offers access through its apps and API, but the weights have not yet been published. The company has promised them by November 4, and no licence has been published in the source material.

How did Kimi K3.1 Pro perform on independent benchmarks?

Artificial Analysis scored it 55.1 on its Intelligence Index, placing it fourth among the models in its comparison. It measured 67% on Terminal-Bench 4.0 and 1,846 Elo on GDPval-AA; these results apply to those specific evaluations.

What does Moonshot claim about hallucinations?

Moonshot reported a fall from 23.4% to 8.3% on its internal Kimi Hallu Bench Hard test. Artificial Analysis measured a decrease from 53% to 44% on a separate evaluation, AA-Omniscience. Neither figure establishes an overall error rate in everyday use.

How much does using the model cost?

Artificial Analysis listed API prices of $2 per million input tokens and $10 per million output tokens, and estimated about $2.11 per Intelligence Index task in its comparison. Actual costs vary by usage, and self-hosting would add infrastructure expenses.

Source: ThorstenMeyerAI.com