Inference
Vals AI

MMLU Pro

Academic multiple-choice benchmark covering 14 subjects including STEM, humanities, and social sciences.

ModelsModels with a result.128
Top resultBest result on this test.92.4%Claude Fable 5.1
Top-3 spreadPoints from first to third.0.9 pts
UpdatedDate the source changed the results.1 Sept 2026

Result and cost

020406080100$0.0001$0.0003$0.001$0.003$0.01$0.03$0.1$0.3RESULTUS DOLLARS PER TASK · LOG SCALE

Results

129 results
#ModelResultThe score from the source.CostUS dollars to run one task.TimeTime to run one task.
01Claude Fable 5.192.4%$0.09821 s
02Claude Opus 591.6%$0.02810 s
03Claude Fable 591.5%$0.08225 s
04Gemini 3.1 Pro Preview91.0%$0.02824 s
05Gemini 3.8 Flash90.2%$0.0228 s
06Gemini 3.7 Flash90.1%$0.0114 s
07Gemini 3 Pro90.1%$0.03724 s
08Claude Opus 4.789.9%$0.0623 min
09Claude Opus 4.889.6%$0.06123 s
10Gemini 3.5 Flash89.5%$0.02526 s
11Grok 4.689.4%$0.01555 s
12Qwen3.7 Max89.3%$0.01729 s
13Gemini 3.6 Flash89.3%$0.0137 s
14Grok 4.589.2%$0.0156 min
15Claude Opus 4.6 (Adaptive)89.1%$0.05249 s
16GPT-5.6 Sol89.1%$0.03216 s
17Muse Spark 1.188.7%$0.01121 s
18Qwen3.8 Max88.6%$0.01964 s
19Gemini 3 Flash Preview88.6%$0.01936 s
20Muse Spark 1.288.3%$0.01144 s
21GPT-5.588.1%$0.02842 s
22Claude Opus 4.1 Thinking87.9%$0.12428 s
23Qwen3.6 Plus87.7%<$0.0146 s
24Claude Sonnet 587.5%$0.04325 s
25GPT-5.487.5%$0.05129 s
26Claude Sonnet 4.5 Thinking87.4%$0.0438 s
27Claude Sonnet 4.687.3%$0.06655 s
28Muse Spark87.3%<$0.0146 s
29Claude Opus 4.5 Thinking87.3%$0.21539 s
30Claude Opus 4.187.2%<$0.0118 s
31Qwen3.5 Plus Thinking87.2%<$0.0177 s
32DeepSeek V4 Pro 042387.1%$0.02710 min
33MiniMax M2.187.0%<$0.0141 s
34GLM-5.186.9%$0.01269 s
35GLM-5.386.8%$0.01762 s
36GLM-5.286.7%$0.01452 s
37GPT-5.6 Terra86.7%<$0.018 s
38GPT-586.5%$0.02437 s
39GPT-5.186.4%$0.01923 s
40Inkling86.3%$0.0473 s
41Grok 4.20 (Reasoning)86.3%$0.01312 s
42Gemini 3.1 Flash Lite Preview86.2%<$0.0115 s
43GPT-5.286.2%$0.03735 s
44DeepSeek V4 Flash 042386.2%<$0.0117 s
45Claude Opus 486.2%$0.0510 s
46GLM-5.3-Flash86.1%<$0.0145 s
47GPT-5.6 Luna86.0%<$0.0117 s
48GLM-5 Thinking86.0%$0.01287 s
49Kimi K2.5 Thinking85.9%<$0.0139 s
50Grok 4.385.8%<$0.0111 min
51Gemini 3.5 Flash-Lite85.8%<$0.015 s
52o385.6%$0.01217 s
53Claude Opus 4.585.6%$0.0528 s
54Inkling-Small85.6%<$0.0172 s
55Grok 485.3%$0.0589 s
56DeepSeek V3.2 (Thinking)84.9%<$0.012 min
57Qwen3 Max84.7%$0.0272 min
58MiMo-V2.5-Pro84.6%<$0.0168 s
59GPT-5.4 mini84.6%<$0.0120 s
60Qwen3.8-27B84.3%$0.01170 s
61MiniMax M384.2%<$0.0141 s
62Grok 4.1 Fast (Reasoning)84.2%<$0.0125 s
63Qwen3.5 Flash84.1%<$0.0133 s
64Gemini 2.5 Pro Exp 03.2584.1%<$0.0118 s
65Claude Sonnet 4 Thinking83.9%$0.04647 s
66Gemini 2.5 Flash (Preview)83.7%<$0.019 s
67Gemini 2.5 Flash Preview Thinking83.7%<$0.0110 s
68Qwen3 Max83.5%<$0.0143 s
69o183.5%$0.11627 s
70DeepSeek-R183.2%$0.01227 s
71DeepSeek V3.283.1%<$0.0141 s
72MiMo-V2.582.9%<$0.0130 s
73GLM-4.782.7%<$0.012 min
74Claude 3.7 Sonnet Thinking82.7%$0.0432 s
75GPT-5 mini82.2%<$0.0122 s
76GLM-4.682.2%<$0.0147 s
77Ling 3.0 Flash82.0%<$0.0120 s
78Grok 3 Mini Fast High Reasoning81.4%<$0.0117 s
79Qwen3 235B A22B81.2%<$0.0152 s
80GLM-4.581.2%<$0.012 min
81Kimi K2 Thinking81.1%<$0.012 min
82Claude 3.7 Sonnet80.7%<$0.016 s
83o4-mini (high)80.6%<$0.0110 s
84GPT-4.180.5%<$0.018 s
85MiniMax M2.780.4%<$0.0150 s
86MiniMax M2.580.1%<$0.0148 s
87Grok 3 Mini Fast Low Reasoning80.0%<$0.017 s
88Grok 3 [Beta]79.9%$0.01712 s
89Grok 4 Fast (Reasoning)79.7%<$0.017 s
90Claude Sonnet 479.4%$0.0110 s
91Llama4 Maverick Instruct Basic79.4%<$0.017 s
92Kimi K2 Instruct79.4%<$0.0120 s
93GPT-OSS 120B79.2%<$0.0135 s
94Gemini 2.5 Flash Lite Preview Thinking79.1%<$0.0110 s
95Claude Haiku 4.5 Thinking78.7%$0.01826 s
96o3-mini78.7%$0.01322 s
97Gemini 2.5 Flash Lite Preview78.6%<$0.013 s
98Claude 3.5 Sonnet78.4%<$0.016 s
99Gemini 2.0 Flash 00177.4%<$0.014 s
100GPT-4.1 mini77.2%<$0.014 s
101GPT-5.4 nano77.2%<$0.018 s
102DeepSeek V376.6%<$0.0111 s
103GPT-5 nano76.1%<$0.0126 s
104Grok 275.5%<$0.019 s
105Mistral Medium 3.575.3%$0.04266 s
106Gemini 1.5 Pro 00275.3%<$0.013 s
107Mistral Medium75.3%<$0.019 s
108Grok 4.1 Fast Non Reasoning75.2%<$0.013 s
109Mistral Large74.8%<$0.0114 s
110GPT-4o73.3%<$0.019 s
111GPT-OSS 20B71.6%<$0.0130 s
112Llama 3.3 Nemotron Super 49B V1 42e8456170.8%—17 s
113Grok 4 Fast Non Reasoning70.3%<$0.012 s
114Llama 3.3 70B Instruct Turbo69.9%<$0.014 s
115Llama 4 Scout 17B 16E Instruct69.6%<$0.015 s
116Llama 3.3 Nemotron Super 49B V1 42e84561 Thinking69.6%—36 s
117Command A69.2%<$0.0110 s
118Laguna XS.269.0%<$0.0148 s
119Laguna M.168.8%<$0.012 min
120Magistral Medium68.7%$0.01532 s
121Gemini 1.5 Flash 00265.6%<$0.012 s
122Mistral Small65.2%<$0.014 s
123Claude 3.5 Haiku64.1%<$0.016 s
124GPT-4.1 nano63.5%<$0.012 s
125GPT-4o mini62.7%<$0.015 s
126Magistral Small62.1%<$0.0116 s
127Jamba Large 1.649.8%<$0.019 s
128Command R Plus44.0%<$0.014 s
129Jamba Mini 1.630.3%<$0.012 s