Inference
Vals AI

MATH 500

Academic math benchmark on probability, algebra, and trigonometry

ModelsModels with a result.57
Top resultBest result on this test.96.4%Gemini 3 Pro
Top-3 spreadPoints from first to third.0.4 pts
UpdatedDate the source changed the results.9 Jan 2026

Result and cost

020406080100$0.0001$0.0003$0.001$0.003$0.01$0.03$0.1$0.3RESULTUS DOLLARS PER TASK · LOG SCALE

Results

57 results
#ModelResultThe score from the source.CostUS dollars to run one task.TimeTime to run one task.
01Gemini 3 Pro96.4%$0.05132 s
02Grok 496.2%$0.03727 s
03GPT-596.0%$0.0232 s
04Claude Opus 4.1 Thinking95.4%$0.20246 s
05Gemini 2.5 Pro Exp 03.2595.2%<$0.0126 s
06GPT-5 mini94.8%<$0.0112 s
06GPT-OSS 120B94.8%<$0.0116 s
08o394.6%$0.01117 s
08Qwen3 235B A22B94.6%<$0.012 min
10o4-mini (high)94.2%<$0.0113 s
10Kimi K2 Instruct94.2%<$0.0137 s
10GPT-OSS 20B94.2%<$0.0131 s
10Grok 3 Mini Fast High Reasoning94.2%<$0.0123 s
14GLM-4.594.0%$0.01385 s
15Claude Sonnet 4 Thinking93.8%$0.06863 s
15GPT-5 nano93.8%<$0.0122 s
17Claude Opus 4.193.0%$0.04631 s
18DeepSeek-R192.2%$0.0113 min
19o3-mini91.8%<$0.0114 s
19Gemini 2.5 Flash Preview 04.17 Thinking91.8%$0.01124 s
21Claude 3.7 Sonnet Thinking91.6%$0.1252 min
21Gemini 2.5 Flash Preview 04.1791.6%<$0.019 s
23Llama 3.3 Nemotron Super 49B V1 42e84561 Thinking91.4%—43 s
24o190.4%$0.09524 s
24Claude Opus 490.4%$0.04515 s
26Claude Sonnet 490.3%$0.0110 s
27Grok 3 [Beta]89.8%$0.0139 s
28MiniMax M2.189.0%<$0.0160 s
28Gemini 2.0 Flash Exp89.0%<$0.013 s
30GPT-4.1 mini88.0%<$0.016 s
30Gemini 2.0 Flash 00188.0%<$0.013 s
32GPT-4.187.2%<$0.0113 s
33Mistral Medium87.0%<$0.0114 s
34Llama4 Maverick Instruct Basic85.2%<$0.017 s
35Gemini 2.0 Flash Thinking Exp 01.2184.6%<$0.0112 s
36DeepSeek V384.5%<$0.0113 s
37Gemini 1.5 Pro 00282.8%<$0.015 s
38GPT-4.1 nano80.2%<$0.013 s
39Llama 4 Scout 17B 16E Instruct79.2%<$0.0111 s
40Gemini 1.5 Flash 00278.8%<$0.013 s
41Grok 278.4%<$0.0120 s
42Claude 3.7 Sonnet76.8%<$0.016 s
43Command A76.2%<$0.019 s
44GPT-4o74.6%<$0.0113 s
45Mistral Large74.4%<$0.0110 s
46Llama 3.3 70B Instruct Turbo73.4%<$0.015 s
47GPT-4o mini72.6%<$0.016 s
48Claude 3.5 Sonnet72.4%<$0.015 s
49Meta Llama 3.1 405B Instruct Turbo71.4%<$0.0145 s
50Llama 3.3 Nemotron Super 49B V1 42e8456171.2%—13 s
51Grok 3 Mini Fast Low Reasoning70.2%<$0.0110 s
52Mistral Small69.5%<$0.015 s
53Meta Llama 3.1 70B Instruct Turbo65.0%<$0.019 s
54Claude 3.5 Haiku64.2%<$0.015 s
55Jamba Large 1.654.8%<$0.0113 s
56Meta Llama 3.1 8B Instruct Turbo44.4%<$0.016 s
57Jamba Mini 1.625.4%<$0.015 s