Model releases · 11 August 2026
Thirty billion parameters with three billion active, a hybrid architecture that is not a pure transformer, and GGUF builds published on release day by the llama.cpp project itself.
| Released by | NVIDIA |
|---|---|
| Published | 11 August 2026 |
| Parameters | 30B total, 3B activated per token |
| Architecture | Hybrid MoE — interleaved Mamba-2 and MoE layers with select attention layers (nemotron_h) |
| Layers / hidden size | 52 / 2688 |
| Experts | 128 routed, top-6 per token |
| Context length | Up to 1,000,000 tokens; NVIDIA uses 256K for single-GPU deployment |
| Single-GPU target | 1× H100 80GB or 1× A100 80GB |
| Languages | English and code, plus Spanish, French, German, Italian, Japanese |
| Licence | OpenMDW 1.1 — not MIT or Apache |
| Weight formats | BF16 65.8 GB · NVFP4 21.6 GB · GGUF Q4_K_M 25.4 GB |
| Data cutoff | Pre-training September 2025, post-training May 2026 |
| Hosted at | nvidia on Hugging Face |
Nemotron 3.5 Lightning is NVIDIA's entry in the same category as the other large releases of this week: a model sized for agents that run continuously rather than answer one question at a time. NVIDIA's positioning is throughput — high-volume, specialised tasks completed faster, with a claim of up to four times the output speed of similarly sized models.
The Nemotron line is described as open weights, open training data and open recipes, and the reproducibility material is published alongside: evaluation recipes, containers, prompts, inference parameters and scoring settings, in NVIDIA's NeMo Gym repository.
This is what separates it from everything else released this week. Nemotron 3.5 Lightning interleaves Mamba-2 layers with MoE layers, keeping only select attention layers in the stack.
That choice is the most plausible source of the speed claim. Attention cost grows with the square of sequence length; state-space layers like Mamba-2 do not. At the million-token context the model advertises, that difference stops being academic.
The sparse side is conventional by comparison: 128 routed experts with the top 6 active per token, across 52 layers at hidden size 2688, giving 3B active out of 30B total.
NVIDIA is explicit that the BF16 repository is not the one to deploy. It is described as the reference weights, meant as a starting point for post-training, domain adaptation, and producing your own quantised or GGUF variants. For actual inference, NVIDIA points to the NVFP4 release, which is also where the W4A16 build and the DSpark recipe for DGX Spark live.
Day-one domain fine-tunes already exist from third parties, with finance and healthcare variants published within hours of release.
The GGUF builds are published under ggml-org — the llama.cpp project's own organisation, not a community re-uploader. That is a meaningful difference from Muse Glimmer the week before, where the architecture support arrived as a separate pull request after the weights and older builds simply refused to load the file.
Four GGUF builds are available: BF16 at 65.9 GB, Q8_0 at 35.0 GB, Q4_K_M at 25.4 GB and NVFP4 at 22.5 GB.
Lightning ships with several speculative decoding methods, including DSpark for low-concurrency data-centre deployments and a DFlash variant. That makes three separate labs — Meta, DeepSeek and now NVIDIA — publishing drafter-based decoding alongside their models in the same period.
| Runtime | Supported | Notes |
|---|---|---|
| llama.cpp | Yes | GGUF builds published on release day by ggml-org, the llama.cpp project's own account. |
| transformers | Yes | Reference BF16 weights published by NVIDIA. |
| NVFP4 / W4A16 | Yes | NVIDIA's recommended path for inference, with a DSpark recipe for DGX Spark. |
| MLX | Partial | Community MLX conversions appeared on release day; nothing published by NVIDIA. |
A 30-billion-parameter Mixture-of-Experts model from NVIDIA with 3 billion parameters active per token, built for high-volume agent workloads. It uses a hybrid architecture interleaving Mamba-2 layers with MoE layers.
OpenMDW version 1.1, which is neither MIT nor Apache 2.0. It is a newer open model licence and its terms should be read directly before commercial deployment.
Yes. GGUF builds were published on release day under ggml-org, the llama.cpp project's own Hugging Face account, rather than arriving later from the community.
NVIDIA targets a single H100 or A100 with 80 GB at 256K context. The quantised builds are 21.6 GB in NVFP4 and 25.4 GB as GGUF Q4_K_M.
NVIDIA states the BF16 repository is reference weights intended for customisation and further quantisation. For inference they point to the NVFP4 release.
OnDevice LLM is a private AI assistant that runs entirely on your iPhone — no account, no cloud, and nothing you type leaves the device.
Published 11 August 2026