Ultra-fast 309B MoE model for coding & agents
Open-source (Apache 2.0) LLM series 'born for reasoning.' Pre-trained & RL-tuned models (like the 7B) match o1-mini on math/code. Base/SFT/RL models released.
MiMo includes Multiple-Token Prediction (MTP) layers enabling speculative decoding with ~90% acceptance rate, reducing latency by 2.29× during training and 1.96× during validation. Supported in vLLM, SGLang, and HuggingFace transformers.
MiMo provides RL training recipes using rule-based accuracy rewards and test difficulty-driven code rewards, with 130K curated math and code problems. Users can apply the same post-training methodology to their own models using the open-source infrastructure.
MiMo-7B-RL surpasses much larger models on mathematics (AIME 2024: 68.2%) and code benchmarks (LiveCodeBench v5: 57.8%), proving that pre-training and post-training strategies matter more than scale for reasoning.
MiMo is Apache-2.0 licensed open source available on HuggingFace and ModelScope. Users can integrate via standard transformers library, vLLM, or SGLang, with full control over deployment, fine-tuning, and data.
MiMo-7B is pre-trained from scratch with reasoning-optimized data pipelines and achieves 95.8% on MATH500 and 57.8% on LiveCodeBench v5, matching OpenAI o1-mini performance. Users can download base, SFT, or RL-trained checkpoints to deploy locally.