Most launch posts do not contain a paragraph admitting the new model is worse than one the company could have shipped. Moonshot’s does, with a number attached. On xdev-bench-hard, an exploit-development benchmark scored at pass@3, Kimi K3.1 Pro reaches 81.0%. A checkpoint trained without the company’s cyber risk-reduction measures scores 90.5% on the same test [VENDOR]. The 9.5-point difference is what Moonshot says it paid to keep offensive cyber capability below a threshold, by removing cyber-related data from both mid-training and reinforcement learning. The capped model is what OrcaRouter serves as Kimi K3.1, at Moonshot’s list price with no markup.
That is a policy decision with a measured cost, and it is directly relevant to anyone deciding whether to run this model. The capability figure lives in the vendor’s release material rather than on the card. What follows is what was removed, what was not, and which workloads are affected.
What the benchmark asks for
Knowing what xdev-bench-hard measures decides how much the 9.5 points matter.
The evaluation is built from recently disclosed vulnerabilities in real software — the Linux kernel, Node.js, JavaScriptCore, Exim and VirGL are the examples Moonshot names — with mitigations enabled. Each task supplies the vulnerable source, a short hint and a compiled binary. The agent has to develop an exploit, achieve arbitrary code execution inside a separate sandbox, and retrieve a secret string so success can be verified automatically. No hard timeouts, no output limits, pass@3 over a random subset.
That is a narrow, adversarial, exploit-writing test. It says very little about whether a model can write a scanner, triage an alert, or draft a patch — the work most security teams actually do — and a great deal about whether it can turn a known-vulnerable binary into working code execution.
Where the 9.5 points sit
The important context is that the cap is relative to Moonshot’s own pipeline, not to the market. The shipped model still scores 81.0%, against 33.3% for Kimi K3 on the same test and 57.1% for GLM-5.3 [VENDOR]. This is a frontier model moved down a rung, not a model removed from the category.
Second, the cap is in the weights. Cyber-related data was removed from training; the shipped checkpoint has that capability baked out. There is no server-side filter, no account-level gate and no silent redirect. If you call the model and it produces weaker exploit code than a competitor’s, that is the model, and it will behave the same way on any provider that serves the same checkpoint.
Third, Moonshot says it will “progressively raise” the cyber capability available in open weights [VENDOR] — which means this is a schedule, not a fixed line. A model card statement written today may not describe the same checkpoint at the next release.
How the three labs handle the same problem
The comparison is instructive because the mechanisms produce different surprises for a buyer.
Capability removal, as here. The model is weaker for everyone; behaviour is uniform and reproducible; your benchmark and your production calls agree. The cost is real capability you cannot buy back at any price.
Serving-side routing. Some labs serve the frontier capability to verified organisations and quietly answer unverified callers with an older model — no error, a different model’s response. For a verified security team that is the most capable arrangement available; for everyone else it is a model that is not the model on the card, on exactly the requests where it matters most.
Account-level gating. Access is decided before the request, by risk rating. Clean, knowable in advance, and useless if you are on the wrong side of the gate.
The distinction that matters for evaluation: with removal, a published benchmark describes what you get. With routing or gating, it may not.
Does this affect non-security work
Mostly no, and there is evidence rather than assertion for that.
The same release moved Terminal-Bench 4.0 from 12.63% to 67.17%, GDPval-AA by 312 Elo, and AutomationBench from 58% to 73% on independent harnesses [INDEPENDENT]. General coding, agentic tool use and professional deliverables all improved sharply in the release that also removed cyber capability. The removal was targeted; it did not cost the model its general competence.
If your work is defensive — detection engineering, patch drafting, alert triage, code review — the 9.5 points are concentrated in exploit development, which is a different task family from all four of those.
Three areas where the cap is worth checking before you commit:
Penetration testing and red-team tooling. Exploit development is the benchmark’s own subject, and this is the workload where the removed points live. If your firm writes proof-of-concept exploits, expect a lower ceiling than a model without the cap, and measure it on your own tasks rather than assuming the 9.5 transfers.
Vulnerability research. Turning a disclosed bug into a working demonstration is adjacent to the benchmark. Here the model is still strong — 81.0% — but a checkpoint trained for it is better, and Moonshot is not selling that checkpoint.
Security benchmarks. If you evaluate models on offensive capability, this model will rank lower than its general index suggests, and that is the intended behaviour rather than a measurement error.
What to tell a reviewer
Three sentences that survive scrutiny.
Kimi K3.1 Pro deliberately gives up about nine and a half points on an exploit-development benchmark to keep offensive cyber capability below a threshold, and the vendor published the number rather than hiding it.
The removal is in the weights, so it is uniform across providers and reproducible on your own evaluation — unlike a routing policy that changes answers silently.
General coding and agentic capability were not harmed; the same release produced the largest gains in the model’s own history on terminal, deliverable and automation evaluations.
That is a more defensible position than either of the alternatives: a model with no published policy, or one where the policy is invisible at request time. For a security-adjacent buyer, the published number is the feature — it tells you exactly which wall you are going to hit and when to re-measure.
Sourcing note: xdev-bench-hard figures (81.0% shipped, 90.5% unmitigated checkpoint, 33.3% Kimi K3, 57.1% GLM-5.3), the training-data removal description, and the “progressively raise” statement are Moonshot’s launch material and are unreproduced by us. Terminal-Bench, GDPval-AA and AutomationBench movement are Artificial Analysis measurements on index v4.3.2, read 26 October 2026. The three-lab comparison and the workload guidance are ours.