Model releases  ·  11 August 2026

Motif 3: a 314B MIT-licensed Mixture-of-Experts model

A Korean lab has published a frontier-scale sparse model under one of the most permissive licences available, built on an attention mechanism that does not appear in any other open release.

At a glance

What it is

Motif 3 is a sparse Mixture-of-Experts model with 314 billion total parameters, of which about 13.2 billion are active for any given token. Motif Technologies describes it as built from the ground up on a fully in-house design, rather than continued training on someone else's base.

The licence is the headline for anyone building on it. MIT is more permissive than the custom community licences that usually accompany models at this scale, and more permissive than Apache 2.0 in practice, since it carries no patent or attribution-of-changes clauses.

The architecture is genuinely unfamiliar

Most open MoE releases are recognisable variations on a shared template. Motif 3 is not. Four elements are worth naming because they do not appear together anywhere else:

• Grouped Differential Latent Attention (GDLA) — grouped differential attention combined with the compressed key-value representation of multi-head latent attention, with a gated output. 80 query heads against 16 key-value heads.
• Expert-Specific PolyNorm activations, rather than a single activation shared across the expert pool.
• Modified manifold-constrained hyper-connections (mHC) in place of a conventional residual path.
• A built-in Multi-Token Prediction head, giving self-speculative decoding without a separate drafter model.

The expert configuration is fine-grained: 384 routed experts with only 8 active per token, plus one shared expert always on. That is a large pool at low per-token cost, and it is the reason a 314B model activates only 13.2B.

Training

Pretraining covers roughly 12.5 trillion tokens across web text, STEM, code, mathematics and multilingual content, with stated emphasis on Korean, reasoning-heavy, legal and financial data. Post-training is where the description gets unusual: general supervised fine-tuning, six separate RL-trained specialist teachers and a software-engineering teacher, merged into one model through what the lab calls Multi-teacher On-Policy Distillation.

The lab highlights long-horizon agentic tool use and terminal-based problem solving as particular strengths, and notes calibrated abstention on hallucination-sensitive evaluations — that is, the model declining to answer rather than inventing.

What's in the release

The main repository carries the bfloat16 weights across 155 shards totalling 629.7 GB, a tokenizer with a 220,160-token vocabulary, and a technical report PDF. Two sibling repositories exist: Motif-3-Base for the pretrained checkpoint and Motif-3-NVFP4 for an NVFP4-quantised version.

Community GGUF conversions have started appearing but were not published by the lab.

Runtime support

What we don't know yet

• No third-party benchmark verification yet; the published results are the lab's own.
• Serving requirements are not spelled out in the model card. At 629.7 GB in bfloat16, the practical floor is a multi-GPU node even before considering context.
• Whether the GDLA attention mechanism will get support in the common open-source inference runtimes, and on what timeline.

Sources

• Model card on Hugging Face
• Base checkpoint
• Technical report (arXiv)
• Motif Technologies

Frequently asked questions

Published 11 August 2026