Baidu on November 5, 2026 released the open weights of ERNIE-5.5-Flash-Base. The company's ERNIE blog describes a sparse Mixture-of-Experts language model with 280B total parameters and 14B activated per token, pre-trained from scratch on 33.5T tokens. Training context reached 128K tokens; fine-tuning and inference support a 1M-token context.
The released checkpoint sits at the end of pre-training, before mid-training or post-training. Baidu positions it as a shared starting point for domain adaptation, instruction tuning, and reinforcement-learning research, and says a post-trained ERNIE 5.5 Flash will follow without naming a date.
Key Facts
| Item | Detail |
|---|---|
| Release | November 5, 2026 (Baidu ERNIE blog) |
| Architecture | Sparse MoE; 43 layers (1 dense + 42 MoE) |
| Parameters | 280B total; 14B activated per token |
| Pre-training | 33.5T tokens; Muon optimizer; FP8 mixed precision |
| Context | Up to 128K in training; 1M for fine-tuning and inference |
| Weights | Hugging Face / ModelScope; FP8 and BF16 safetensors |
| Stage | Pre-training endpoint only (no mid-/post-training) |
| License | ERNIE 5.5 Flash Base Model License Agreement v1.0 |
Baidu's published comparison, run with the same harness against DeepSeek-V4-Flash-Base (284B-A13B), Hy3-preview-Base (295B-A21B), and MiMo-V2.5-Base (310B-A15B), lists ERNIE ahead on many of the shared exam, STEM, code, and long-context rows. Examples from that table: MMLU 93.3, MMLU-Pro 78.4, HumanEval+ 82.3, LiveCodeBench 40.1. These are Baidu's numbers; independent third-party reruns were not available at publication.
Background
ERNIE 5.5 Flash uses hybrid attention (Heavily Compressed Attention on most layers, Multi-head Latent Attention with sparse attention on a few), Multi-Router Latent MoE with 512 routed experts (top-10) plus one shared expert, and related training-system work Baidu describes as co-designed with the architecture. The Hugging Face model card matches the blog on size, activation, token count, and context.
IT Home and Zhidx both reported the same core specs on November 5, citing the official release. Serving notes point to SGLang and vLLM with a Baidu ERNIE plugin.
What We Don't Know Yet
- Release date for the post-trained ERNIE 5.5 Flash
- Independent benchmark reruns outside Baidu's unified pipeline
- How the custom license terms apply in practice for large commercial deployments outside China