Model releases · 11 August 2026
A sparse model small enough that the vendor quotes MacBook throughput rather than datacentre throughput — released under MIT, in three precisions, with llama.cpp support still an open pull request.
| Released by | inclusionAI |
|---|---|
| Published | 10–11 August 2026 |
| Parameters | 7.9B total, 1.3B activated per token |
| Architecture | Hybrid-linear MoE — 3 KDA layers to 1 MLA layer per 4-layer block (bailing_hybrid) |
| Layers / hidden size | 24 / 1536 |
| Experts | 128 routed, top-8 per token, plus 1 shared |
| Context length | 131,072 tokens |
| Vocabulary | 157,184 tokens |
| Licence | MIT |
| Official builds | BF16 15.8 GB · FP8 8.42 GB · INT4 5.82 GB |
| Measured throughput | 86–90 tok/s on an M4 Pro MacBook and 100–105 tok/s on DGX Spark, both at FP8 (vendor figures) |
| Peak memory | ~8.34 GiB at 8K context, FP8 (vendor figure) |
| Hosted at | inclusionAI/Ling-3.0-tiny on Hugging Face |
Ling-3.0-tiny is the small member of inclusionAI's Ling 3.0 family: 7.9 billion parameters in total, of which only 1.3 billion are active for any given token. The lab describes it as a lightweight hybrid reasoning MoE model, explicitly optimised for local deployment rather than served inference.
What makes the release unusual is where the performance numbers come from. Rather than quoting throughput on datacentre hardware, inclusionAI states the model was validated on NVIDIA DGX Spark, an Apple Silicon MacBook and a Mac mini — and publishes figures for each.
The architecture stacks in a 3:1 ratio: three Kimi Delta Attention layers followed by one Multi-head Latent Attention layer, per four-layer block. KDA is the mechanism Moonshot AI introduced with Kimi K3 — a 2.8-trillion-parameter model at the opposite end of the size range. Seeing it appear in a 7.9B release from a different lab, weeks later, is a reasonable indicator of how quickly attention designs now propagate between labs.
The sparse side is a 128-expert MoE FFN with 8 routed experts plus 1 shared expert per token, across 24 layers at hidden size 1536.
inclusionAI shipped BF16, FP8 and INT4 repositories itself rather than leaving quantisation to the community — 15.8 GB, 8.42 GB and 5.82 GB respectively. Community GGUF conversions appeared within a day, ranging from 8.41 GB at Q8_0 down to 2.70 GB at IQ2_M, with Q4_K_M at 4.82 GB.
This is the part worth checking before downloading anything. As of publication, convert_hf_to_gguf.py on llama.cpp master contains no entry for the Bailing architectures used here, and support is tracked in an open pull request, #26608 "BailingMoE3 Support", alongside an open model request for Ling-3.0-flash filed a week earlier.
The community GGUF files in circulation were therefore produced against that unmerged branch. One of them is explicitly labelled experimental. A stock llama.cpp build will reject them with an unknown-architecture error, and older builds — even recent ones — carry bailingmoe and bailingmoe2 but not the new bailing_hybrid type.
inclusionAI's announcement cites Artificial Analysis results of 25 on the Intelligence Index and 16 on the Agentic Index, with 772 Elo on GDPval-AA v2 and 20.80 on 𝜏³-Banking. Those are modest numbers in absolute terms, which is the expected trade for 1.3B active parameters; the relevant comparison is against other models of similar active size rather than against frontier releases.
| Runtime | Supported | Notes |
|---|---|---|
| transformers | Yes | Reference implementation published by the lab in BF16, FP8 and INT4. |
| llama.cpp | No | Not merged. Support is an open pull request (#26608). Community GGUFs are built from that branch and will not load in a stock build. |
| MLX | Partial | A community 4-bit MLX conversion appeared on release day; nothing published by the lab. |
| Apple Silicon | Yes | The lab reports validation on MacBook and Mac mini, with 86–90 tok/s at FP8 on an M4 Pro. |
7.9 billion parameters in total with 1.3 billion active per token. The official builds are 15.8 GB in BF16, 8.42 GB in FP8 and 5.82 GB in INT4.
Not yet in a stock build. Support for the Bailing hybrid architecture is an open pull request, #26608, and the GGUF files already circulating were built from that unmerged branch.
MIT, with no additional acceptable-use policy.
The lab reports validating it on an Apple Silicon MacBook and a Mac mini, quoting 86 to 90 tokens per second at FP8 on an M4 Pro with about 8.34 GiB peak memory at 8K context.
No. Sparse activation reduces compute per token, not resident memory. All 128 experts have to be held in memory regardless of how few are used for any given token.
OnDevice LLM is a private AI assistant that runs entirely on your iPhone — no account, no cloud, and nothing you type leaves the device.
Published 11 August 2026