Preface¶

On July 27, 2026, Moonshot AI simultaneously released the full model weights and technical report supporting code repository for Kimi K3 on Hugging Face and GitHub. This is one of the largest open-source weight models publicly available for download to date, with a total parameter count of 2.8 trillion (2.8T), approximately 104 billion (104B) active parameters during inference, and a context window of up to 1 million tokens.

Prior to this, models of comparable scale were almost exclusively available via closed-source APIs. The release of K3 has once again made “whether open-source weights can catch up with closed-state-of-the-art models” a hot topic of discussion in the developer community. This article verifies key information based on the official repository MoonshotAI/Kimi-K3 and the Hugging Face model page, and sorts out the architecture, quantization, evaluation, and self-hosted deployment one by one.

What is Kimi K3?¶

Kimi K3 is Moonshot AI’s flagship native multimodal Agent model that supports text and image inputs, targeting long-context encoding, knowledge work, reasoning, and tool calling scenarios. The official positioning describes it as “the world’s first open 3T-class model”.

Compared to the previous generation Kimi K2 series, the main changes of K3 include:
- The sparse MoE scale has been significantly expanded (896 experts, 16 activated per token);
- The context window has been expanded from approximately 250,000 tokens in K2.6 to 1048576 tokens;
- Introduced Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), paired with the Stable LatentMoE framework. The official claims that the overall scaling efficiency is improved by approximately 2.5 times compared to K2.

If you cannot afford the self-hosting cost for the time being, you can also call the kimi-k3 model via the Kimi API. The official provides OpenAI / Anthropic compatible interfaces, with pricing: \(3 / million tokens** for cache misses, **\)0.30 / million tokens for cache hits, and $15 / million tokens for outputs.

Architecture and Key Parameters¶

K3 adopts the Mixture-of-Experts (MoE) sparse architecture, with core specifications as follows (data from the official README):

KDA (Kimi Delta Attention) is an attention mechanism designed by Moonshot for long contexts and large-scale MoE. KDA compresses the KV cache into fixed-size state vectors. The official technical blog states that it can reduce the KV cache usage to approximately 1/16 of the standard MLA at 128K context, with more obvious advantages in 1M token scenarios. The inference framework needs to implement the KDA kernel to load the model correctly, so it cannot be directly run with the old version of vLLM.

K3 enables thinking mode by default, and the API will return the reasoning_content field; for multi-turn conversations, you need to pass back the complete assistant message (including reasoning and tool_calls) as-is, which is slightly different from the usage of ordinary chat models.

Native MXFP4 Quantization¶

K3 adopted Quantization-Aware Training starting from the SFT phase, and finally released it in the form of MXFP4 weights + MXFP8 activations. This is the first mainstream model open-sourced in native 4-bit weight format at this scale.

The significance of MXFP4 is as follows: at approximately 2.8T parameters, if estimated at 4 bits per parameter, the pure weight data is approximately 1.4 TB; the actual MXFP4 weight package on Hugging Face is approximately 594 GB (including quantization metadata and shards). Compared to FP16 full precision, the download and storage costs have dropped significantly, but higher requirements are put forward for the low-precision computing power of GPUs and the support of inference frameworks.

During deployment, you need to explicitly specify --quantization mxfp4 and enable --trust-remote-code to load the KDA custom operators.

Coding and Agent Evaluation Performance¶

The official published comparison results with closed-source models such as Claude Fable 5 and GPT-5.6 Sol in the README (K3 uses reasoning_effort=max). The benchmarks related to coding and Agent are excerpted as follows:

As can be seen, K3 performs outstandingly in terminal and long-cycle coding tasks such as Terminal-Bench 2.1 and SWE-Marathon, and some indicators are comparable to closed-state-of-the-art models. It should be noted that the Agent harnesses used in different benchmarks are not uniform (Kimi Code, Codex, Claude Code, etc.), so you should understand the official footnotes when making horizontal comparisons, and it is not appropriate to simply conclude that it “comprehensively surpasses closed-source models”.

The community’s discussion of “open-source catching up with closed-source” mostly refers to: K3 has the ability to compete with closed-source flagships on several coding and Agent leaderboards, rather than leading in every benchmark.

Self-Hosted Inference: Key Points for vLLM Deployment¶

After the weights were made public, Moonshot simultaneously provided deployment guidelines for vLLM, SGLang, and TokenSpeed. The official vLLM recipe can be found at recipes.vllm.ai/moonshotai/Kimi-K3.

Hardware Threshold¶

This is the most important point to face for K3 self-hosting: it is not a desktop-level model that can be run on a single GPU or 8 GPUs.
- The official vLLM recipe recommends at least 8× GB300, and multiple nodes are required for production traffic;
- Moonshot documentation recommends a supernode configuration with 64 or more accelerators;
- For the ROCm path, 8× MI355X / MI350X and corresponding Docker images are required.

Even though MXFP4 has reduced the weight volume to approximately 594 GB, a single H100 (80 GB VRAM) still cannot accommodate the complete model; distributed inference across nodes with tensor parallel and expert parallel is the norm. Long contexts (such as 1M tokens) will additionally occupy a large amount of KV cache, so in actual deployment, you usually start with a smaller max-model-len (such as 131072) and then adjust it gradually according to the VRAM and throughput requirements.

vLLM Startup Example¶

The official recommends using the K3-specific Docker image vllm/vllm-openai:kimi-k3 (based on CUDA 13, the host driver needs to be r580+). The typical startup parameter ideas are as follows:

# Use the official K3 image (CUDA 13 / cu130)
docker pull vllm/vllm-openai:kimi-k3

# Single-machine 8-card example (refer to the official recipe for specific parameters)
python -m vllm.entrypoints.openai.api_server \
  --model moonshotai/Kimi-K3 \
  --trust-remote-code \
  --quantization mxfp4 \
  --tensor-parallel-size 8 \
  --max-model-len 131072 \
  --served-model-name kimi-k3

Some practical suggestions:
1. --trust-remote-code must be enabled, otherwise the KDA custom CUDA operators cannot be loaded;
2. --quantization mxfp4 corresponds to the native quantization format of the model. Blackwell hardware can directly support this format, while it is simulated by the framework on H100/H200;
3. Multi-node MoE communication: For cross-node communication, it is recommended to configure --all2all-backend deepep_v2 (RDMA) or flashinfer_nvlink_one_sided (NVLink);
4. After the service starts, it exposes the OpenAI-compatible API, and existing SDKs only need to modify the base_url to access.

If you do not have cluster conditions for the time being, you can first experience it via the Kimi API or third-party hosted inference (such as partners like Telnyx), and then migrate to self-hosting after the hardware and operation and maintenance capabilities are ready.

Open Source License and Usage Notes¶

Both the K3 weights and the code repository use the Kimi K3 License (a custom license, not an OSI-standard open source protocol). According to the community’s interpretation of the license terms:
- Research and most commercial uses can be used for free;
- If you operate as Model-as-a-Service and the cumulative revenue exceeds 20 million US dollars in 12 months, you need to sign a commercial agreement separately;
- When the product’s MAU exceeds 100 million or MRR exceeds 20 million US dollars, you need to prominently mark the “Kimi K3” brand.

Before downloading the weights, it is recommended to read the LICENSE file to confirm whether your scenario complies with the terms.

Summary¶

The release of the full Kimi K3 weights marks the first time that open-source models have directly competed with closed-state-of-the-art models at the 3T parameter scale. Native MXFP4 quantization, KDA long-context attention, a 1M token window, and multimodal Agent capabilities together form its technical labels; while the 594 GB-level weights and the recommended 64-card supernode configuration also clearly define the gap between “can download” and “can build a serving system”.

For most developers, a more realistic path is: first use the API to verify the business scenario, then evaluate the cluster cost and compliance requirements, and finally decide whether to self-host. In any case, the release of K3 provides a reproducible and auditable frontier-level baseline for the open-source ecosystem, and subsequent work such as fine-tuning, distillation, and vertical domain adaptation deserves continuous attention.