Inference
EN
Yubo Wang et al.

Massive Multitask Language Understanding Professional

An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.

ModelsModels with a result.75
Top resultBest result on this benchmark.89.6%Qwen3.7 Max
Top-3 spreadPoints from first to third.1.1 pts
YearYear of release.2024

Result and price

28 of 75 results

Results

75 results
#ModelResultThe score from the source.
01Qwen3.7 Max89.6%
02Claude Opus 4.589.5%
03Qwen3.7 Plus88.5%
03Qwen3.6 Plus88.5%
05Qwen3.5 397B87.8%
06DeepSeek V4 Pro 081387.5%
07Kimi K2.5 (Reasoning)87.1%
07Kimi K2.587.1%
09Nemotron 3 Ultra86.8%
10Qwen3.5-122B-A10B86.7%
11Solar Pro 486.3%
12DeepSeek V4 Flash 073186.2%
12Solar Open 286.2%
12Qwen3.6-27B86.2%
15Qwen3.5-27B86.1%
16GLM-585.7%
17Qwen3.5-35B-A3B85.3%
18Qwen3.6-35B-A3B85.2%
18Gemma 4 31B85.2%
20MAI-Thinking-185.0%
20DeepSeek-R1-052885.0%
22MiMo-V2-Flash84.9%
23DeepSeek V3.1 (Reasoning)84.8%
24Qwen3 235B 2507 (Reasoning)84.4%
25GLM-4.784.3%
26Ripple-65B Preview84.2%
27DeepSeek-R184.0%
28Nemotron 3 Super 100B83.7%
28DeepSeek V3.183.7%
30K-EXAONE 2.083.5%
31Qwen3 235B 250783.0%
32Gemma 4 26B A4B82.6%
33Claude Opus 4.682.0%
34Exaone 4.0 32B81.8%
35Sarvam 105B81.7%
36Nemotron 3.5 Lightning 30B A3B NVFP481.6%
37Kimi K281.1%
38Sarvam 30B80.0%
39Grok 3 [Beta]79.9%
40LongCat-Flash-Lite-Sparse79.2%
40Claude Sonnet 4.679.2%
42Nemotron 3 Nano 30B78.3%
43Granite 4.2 30B77.6%
44Nemotron 3 Nano Omni 30B A3B77.3%
45Gemma 4 12B77.2%
46Qwen2.5-Max76.1%
47Phi-4-reasoning-plus76.0%
48Celeris-175.9%
48DeepSeek V375.9%
50Grok 275.5%
51Phi-4-reasoning74.3%
52ZAYA1-8B74.2%
53Granite 4.2 8B74.0%
54Grok 2 mini72.0%
55MiniCPM5-2B70.8%
56Gemma 4 E4B69.4%
57Llama 3.3 70B68.9%
58Qwen2.5-VL-32B68.8%
59ZAYA1-74B-Preview68.1%
60Granite 4.2 3B67.8%
61Mistral Small 3.166.8%
62Llama 3.1 70B66.4%
63Claude 3.5 Haiku65.0%
64Gemma 4 E2B60.0%
65Soofi S 30B-A3B51.4%
66MiniCPM5-1B48.9%
67Llama 3.1 8B48.3%
68Phi-3.5-mini47.4%
69Granite-4.0-1B34.0%
70Granite-4.0-H-1B32.9%
71LFM2.5-230M20.3%
72LFM2.5-350M20.0%
73LFM2.5-VL-450M19.3%
74Granite-4.0-H-350M14.4%
75Granite-4.0-350M12.1%