Model releases · 11 August 2026
Moonshot AI calls it the first open 3T-class model. Native text, image and video in one model, a million tokens of context, and an architecture built on two attention mechanisms that are specific to this release.
| Released by | Moonshot AI |
|---|---|
| Published | June 2026, last updated July 2026 |
| Parameters | 2.8T total, 104B activated per token |
| Architecture | Mixture-of-Experts (KimiK3ForConditionalGeneration) |
| Layers | 93 (1 dense) — 69 Kimi Delta Attention + 24 Gated MLA |
| Attention hidden size / heads | 7168 / 96 |
| Experts | 896 |
| Context length | 1,048,576 tokens (1M) |
| Modalities | Text, image and video in, text out |
| Licence | Kimi K3 License (custom, not MIT or Apache) |
| Weight format | safetensors — 1,561 GB across 96 shards |
| Hosted at | moonshotai/Kimi-K3 on Hugging Face |
Kimi K3 is Moonshot AI's largest open-weight release: a 2.8-trillion-parameter Mixture-of-Experts model that activates 104 billion parameters per token. Moonshot describes it as an open-weight native multimodal agentic model and "the world's first open 3T-class model", aimed at long-horizon coding, knowledge work and reasoning.
It is by a wide margin the most-liked model in this set on Hugging Face, with over ten thousand likes and more than 1.5 million downloads.
Two named mechanisms carry the release. Kimi Delta Attention (KDA) covers 69 of the 93 layers; the remaining 24 use Gated Multi-head Latent Attention. A single dense layer sits alongside the 92 MoE layers. Moonshot also names Attention Residuals (AttnRes) as part of the design.
Attention hidden size is 7168 across 96 heads, with an 896-expert pool and a 163,840-token vocabulary.
Unlike releases that pair a language model with a separately trained vision projector, Moonshot states that K3 understands text, images and video within the same model. The architecture name — ForConditionalGeneration rather than ForCausalLM — reflects that.
K3 ships under a custom "Kimi K3 License" rather than MIT or Apache 2.0. That is a meaningful difference from the other large releases of this period — Motif 3, DeepSeek V4 Flash and GLM-5.2 are all MIT. Anyone planning commercial use should read the licence file rather than assuming open weights mean open terms.
1,561 GB of weights across 96 shards puts this firmly in multi-node territory. There is no official quantised release from Moonshot, and no size in this family fits a single machine of any ordinary description.
| Runtime | Supported | Notes |
|---|---|---|
| transformers | Yes | Standard configuration published by the lab. |
| llama.cpp | No | kimi_k3 is a custom architecture and no official GGUF is published. |
| MLX | No | No conversion published by the lab. |
| Official quantised build | No | Weights are released in full precision only. |
2.8 trillion in total, with 104 billion activated per token across an 896-expert Mixture-of-Experts pool.
The weights are openly published, but under a custom Kimi K3 License rather than MIT or Apache 2.0. Read the licence file before commercial use.
Yes, natively. Moonshot states that text, images and video are understood within the same model rather than through a separate vision component.
1,048,576 tokens, or one million.
Not on a single machine. The weights are 1,561 GB across 96 shards with no official quantised build, which places it in multi-node serving territory.
OnDevice LLM is a private AI assistant that runs entirely on your iPhone — no account, no cloud, and nothing you type leaves the device.
Published 11 August 2026