Kimi K3 for Self-Hosting: What It Actually Takes

K3's weights are genuinely downloadable, which is remarkable for a 2.8-trillion-parameter frontier model. It is also the part of the release most likely to be misread. Open weights at this scale do not mean you can run it — they mean a well-funded infrastructure team can, and that the option exists at all is the point.

This page is written for platform and infrastructure teams evaluating on-premises deployment. If you want the model overview first, start with the complete Kimi K3 guide.

Best effort level	max (the API default is max)
Context available	1M tokens — enough for real working material, not just snippets
Cost	$3.00 per million input tokens, $15.00 per million output, $0.30 cached input

Why Kimi K3 suits self-hosted deployment

• Open weights under the Kimi K3 License. Commercial use is permitted, subject to thresholds covered in our licence explainer.
• MXFP4 weights and MXFP8 activations, with quantisation-aware training applied from the fine-tuning stage onward — so the quantised form is what was trained, not a lossy afterthought.
• Only 104 billion of 2.8 trillion parameters activate per token, so compute per token is far below what the total suggests. Memory, not compute, is the binding constraint.
• Supported by vLLM, SGLang, and TokenSpeed, so you are not writing a serving stack from scratch.
• Full data residency and no per-token billing — the two reasons this is ever worth it.

A workflow that works

1. Do the memory arithmetic first. ~1.4 TB of weights is a multi-node problem before you add KV cache for a 1M-token window. If that sentence is a surprise, the API is your answer.
2. Start on the API. Validate that K3 solves your problem before you procure hardware for it. The model behaves the same either way.
3. Pick a serving engine early. vLLM, SGLang, and TokenSpeed differ in throughput, quantisation support, and long-context behaviour; the choice constrains your hardware.
4. Read the licence with your legal team. The revenue and user thresholds are specific, and they matter if you are building a product on top of this.
5. Model the break-even honestly. Against $3.00/$15.00 per million tokens, self-hosting has to clear a high bar on volume before it wins on cost alone.

Settings to start from

Set reasoning_effort to max. Self-hosting removes per-token billing from the decision, which changes the calculus: the marginal cost of reasoning is your own hardware utilisation rather than an invoice line. If you have paid for the cluster, run at max and let capacity planning be the constraint.

Remember that reasoning cannot be turned off on this model, and that the API default is max — the most expensive of the three levels. Setting it explicitly is the single highest-value line of configuration you will write. See reasoning effort explained if you want to tune it properly.

What to watch out for

• This is not a workstation model. No consumer GPU, and no single-GPU server, runs 2.8T parameters. Distilled or smaller models are the honest answer for local use.
• The licence is not MIT. Commercial use is allowed, but Model-as-a-Service businesses above $20M in revenue over any consecutive twelve months need a separate agreement with Moonshot, and large deployments carry an attribution requirement.
• Total cost of ownership includes people. Serving a frontier MoE model at scale is a specialist job, and that headcount is part of the comparison.
• You inherit the operational burden. Uptime, upgrades, and long-context memory tuning become yours. The API price includes someone else doing all of that.

The verdict

The right choice for a narrow set of organisations: high, steady volume plus a hard data-residency requirement plus an infrastructure team that already runs GPU fleets. Everyone else should use the API and treat the open weights as insurance against vendor lock-in rather than a deployment plan.

If cost is the constraint, read how to cut Kimi K3 costs before downgrading — caching and effort levels usually beat switching models.

More on Kimi K3

Start with the complete Kimi K3 guide for the overview, or go deeper:

• How to Cut Kimi K3 Costs Without Losing Quality
• Kimi K3 for Data Analysis: Spreadsheets, SQL and Charts
• How to Access Kimi K3: Every Route in 2026