The short answer, for almost everyone: no, you cannot run Kimi K3 locally. The weights are ~1.4 TB in their MXFP4 form, and every one of the 2.8 trillion parameters must be resident in memory to serve a single token. This page exists because "open weights" is widely read as "runs on my machine", and at this scale it does not.
The memory arithmetic
| Requirement | Reality |
|---|---|
| Weights | ~1.4 TB in MXFP4 |
| Must all be resident? | Yes — sparsity reduces compute, not memory |
| Plus KV cache | Substantial, and grows with context length |
| Realistic deployment | Multi-node GPU cluster |
| Consumer GPU | Not viable at any configuration |
This is the point most often misunderstood about Mixture-of-Experts models. Activating 16 of 896 experts means you compute less per token. It does not mean you can leave the other 880 experts on disk — the router may select any of them for the next token.
What you would actually need
A multi-node deployment of data-centre GPUs with high-bandwidth interconnect, sized so that ~1.4 TB of weights plus KV cache fits across the cluster with headroom for your target context length and concurrency.
Serving is supported by vLLM, SGLang, and TokenSpeed, which differ meaningfully in throughput, quantisation handling, and long-context behaviour. Pick the engine before you specify hardware — the choice constrains what configurations work.
Exact GPU counts depend on your accelerator, interconnect, and concurrency targets, and anyone quoting a single number without those inputs is guessing. Model it against your own requirements.
When self-hosting is actually the right call
All three of these should be true, not one:
- Hard data-residency or air-gap requirements that no third-party API can satisfy.
- High, steady volume — enough that fixed infrastructure cost beats $3.00/$15.00 per million tokens with margin to spare.
- An existing platform team that already operates GPU fleets in production.
If only the first is true, look at whether a smaller open model meets the requirement. If only the second is true, negotiate volume pricing before you procure hardware.
What to do instead
Use the API. It is the same model, priced at $3.00 per million input tokens with cached reads at $0.30, and someone else handles uptime, upgrades, and long-context memory tuning.
Treat the open weights as what they usefully are for most organisations: an exit option and a check on pricing power, rather than a deployment plan. See how to access Kimi K3 for the routes that do not involve buying GPUs.
More on Kimi K3
Start with the complete Kimi K3 guide for the overview, or go deeper:
Ready to go deeper?
Read the full Kimi K3 guideFrequently Asked Questions
Can I run Kimi K3 on my own computer?
No. The weights are roughly 1.4 TB and all 2.8 trillion parameters must be resident in memory to serve a token. No consumer GPU or single-GPU server can do this — it requires a multi-node data-centre deployment.
How many GPUs does Kimi K3 need?
It depends on your accelerator, interconnect, target context length, and concurrency — anyone quoting a single number without those inputs is guessing. The floor is set by fitting ~1.4 TB of weights plus KV cache across the cluster, which puts it firmly in multi-node territory.
Doesn't MoE sparsity mean I need less memory?
No — this is the most common misunderstanding. Activating 16 of 896 experts reduces compute per token, not memory. The router may select any expert for the next token, so all of them must stay resident. Sparsity makes K3 cheap to serve per token, not cheap to host.
