VibeThinker-3B: 3B model matches giants on math, lags on facts

Sina has released VibeThinker-3B, a three-billion-parameter Chinese language model that on several hard math and coding benchmarks matches models hundreds of times larger. The technical report shows the 3B model sometimes matches DeepSeek V3.2 and Kimi K2.5 on competitive tests such as AIME26 while performing strongly on LeetCode contests held after training.

What did VibeThinker-3B achieve?

VibeThinker-3B, with 3 billion parameters, matches top models on structured math and coding tasks and even beats many larger models on some coding benchmarks. The report says the model performs on par with DeepSeek V3.2 and Kimi K2.5 on AIME26, falls within the range of five current top models including Gemini 3 Pro, GLM-5 and Claude Opus 4.5 across six math and coding benchmarks, and on LiveCodeBench beats every other model under 20 billion parameters.

The team also validated contest performance after training by entering LeetCode contests held between late April and late May 2026. VibeThinker-3B solved 123 out of 128 problems on the first try. That score placed it ahead of GPT-5.2, Qwen3-Max, Kimi K2.5, and Claude Opus 4.6, and behind only GPT-5.3-Codex, Gemini 3.1 Pro and Gemini 3 Flash in that comparison.

How was the 3B model trained to reach those results?

Sina built VibeThinker-3B on top of Alibaba’s Qwen2.5-Coder-3B and then applied a multi-stage post-training pipeline that the report credits for the performance gains. The pipeline begins with supervised fine-tuning across a broad range of tasks, adds a stage tailored for hard multi-step reasoning problems, then applies reinforcement learning sequentially for math, programming and STEM, follows with self-distillation to consolidate skills, and finishes with an instruction-tuning phase to improve prompt adherence.

During supervised fine-tuning the team deliberately accumulated many solution paths. Reinforcement learning then strengthened the successful patterns. The report argues these training methods, plus data quality and reliable validation signals, drive the leap in performance rather than parameter count alone.

How does the model perform on factual knowledge tasks?

VibeThinker-3B performs substantially worse on benchmarks that require broad factual coverage. The report contrasts its math and coding parity with a clear shortfall on knowledge-heavy tests such as GPQA-Diamond, where the small model falls well behind much larger competitors.

The authors use this contrast to propose the "Parametric Compression-Coverage Hypothesis," which states logical reasoning tasks compress into a compact set of recurring patterns while world knowledge requires broad coverage and therefore many parameters. They describe logical steps like searching, checking conditions, correcting errors and combining intermediate results as patterns that can be packed into a compact core, whereas open-ended factual knowledge demands parameter capacity to store many facts.

Why it matters

The split between reasoning and knowledge changes how teams might choose models for specific jobs. For verifiable, structured tasks such as math problems or coding competitions, a carefully post-trained small model can approach the performance of much larger systems. For tasks that require wide factual recall, large models still hold the advantage because of coverage rather than pattern complexity.

This reframes small models not just as cost-saving options but as a distinct research path: they can be engineered to excel at narrow, verifiable tasks through targeted post-training, while larger models remain necessary where breadth of information is the constraint.

What to watch

Watch whether other teams replicate the contest-style validation approach and whether small models built on compact bases plus multi-stage post-training match or exceed VibeThinker-3B on additional verifiable benchmarks. Also watch benchmarks that mix reasoning and open-domain facts to see where the compression advantage breaks down.

Sina made VibeThinker-3B openly available on Hugging Face and GitHub, and the report cites the model’s predecessor, VibeThinker-1.5B, which launched in November 2025, as the earlier step in this line of work.