Inference
EN
Epoch AI

OTIS Mock AIME 2024-2025

Evaluated by Epoch AI on their own harness.

ModelsModels with a result.200
Top resultBest result on this benchmark.100.0%Gemini 4 Carbon
Top-3 spreadPoints from first to third.0.0 pts
YearYear of release.—

Result and price

122 of 200 results

Results

200 results
#ModelResultThe score from the source.
01Gemini 4 Carbonhigh effort100.0%
01Claude Opus 5.5max effort100.0%
01Gemini 4 Argonhigh effort100.0%
01GPT-6 Astramax effort100.0%
01GPT-6.1 Solmax effort100.0%
01Claude Sonnet 5.5max effort100.0%
01GPT-6.1 Terramax effort100.0%
01Claude Fable 5.1max effort100.0%
01GPT-5.5 Proxhigh effort100.0%
01GPT-5.6 Solmax effort100.0%
01GPT-6 Solmax effort100.0%
01GPT-5.5xhigh effort100.0%
01Qwen3.8 Max (0902)xhigh effort100.0%
14Claude Fable 5.2max effort99.7% ±0.3
14Claude Fable 5max effort99.7% ±0.3
14GPT-5.6 Terramax effort99.7% ±0.3
17Muse Wondermax effort99.4% ±0.4
17Qwen3.8 Maxxhigh effort99.4% ±0.4
19Muse Spark 1.3xhigh effort99.2% ±0.6
19Grok 4.6xhigh effort99.2% ±0.5
19Doubao Seed Evolving (Nov 2026)high effort99.2% ±0.5
22Claude Opus 5max effort98.9% ±1.1
22Gemini 3.8 Flashhigh effort98.9% ±0.9
22Claude Haiku 5.5xhigh effort98.9% ±0.5
22GPT-6 Lunamax effort98.9% ±0.7
22Mistral Large 4high effort98.9% ±0.4
27DeepSeek V4 Pro 0813max effort98.6% ±1.0
28DeepSeek V4.1 Flash 1017max effort98.3% ±0.6
28Claude Opus 4.8max effort98.3% ±1.4
28GPT-5.6 Lunamax effort98.3% ±1.2
31Grok 4.7xhigh effort98.1% ±1.4
31Muse Spark Openxhigh effort98.1% ±1.3
33Grok 4.5high effort97.8% ±1.3
34Gemini 3.7 Flashhigh effort97.2% ±1.9
34Kimi K3max effort97.2% ±1.1
36DeepSeek V4 Pro 0423max effort96.7% ±2.0
37Kimi K2.696.1% ±2.4
37Doubao Seed 2.1 Turbohigh effort96.1% ±2.5
37GPT-5.2xhigh effort96.1% ±2.7
40Gemini 3.5 Flashhigh effort95.6% ±2.7
40Gemini 3.1 Prohigh effort95.6% ±3.1
40Qwen3.7 Max95.6% ±3.1
40Kimi K2.7 Code95.6% ±3.1
40Gemini 3 Flashhigh effort95.6% ±3.1
40Doubao Seed 2.1 Prohigh effort95.6% ±2.9
46GPT-5.4xhigh effort95.3% ±3.2
47DeepSeek V4 Flash 0731max effort94.4% ±2.1
48Gemini 3.6 Flashhigh effort94.2% ±3.1
49GLM-5.3-Flashmax effort93.9% ±2.2
49Doubao-Seed-2.0 Prohigh effort93.9% ±2.7
51GLM-5.193.3% ±3.8
51Qwen3.6 Plus93.3% ±3.8
51Grok 4.3 Betahigh effort93.3% ±3.0
54Grok 4.2092.2% ±3.3
54Kimi K2.592.2% ±3.9
56Gemini 3 Pro91.4% ±3.7
56GPT-5high effort91.4% ±3.7
58GLM-5.3max effort91.1% ±3.5
58Claude Opus 4.6max effort91.1% ±4.3
58Qwen 3.6 Max (preview)91.1% ±4.3
58Doubao Seed 2.0 Lite (Apr 2026)high effort91.1% ±3.6
62Inkling-Smallxhigh effort90.0% ±4.0
63Muse Spark88.9% ±4.7
63Inklingxhigh effort88.9% ±3.9
63GPT-5.4 minixhigh effort88.9% ±4.7
63GPT-OSS 120Bhigh effortAbove fitFar above this model's index.88.9% ±4.5
67GPT-5.1high effort88.6% ±4.0
68GPT-5.4 nanohigh effort87.8% ±3.9
69Claude Opus 4.7max effort86.7% ±5.1
69GPT-5 minihigh effort86.7% ±4.3
69Qwen 3.5 Plus (hosted 397B-A17B)86.7% ±5.1
69Qwen3-235B-A22B (Jul 2025)86.7% ±5.1
73GLM-5.2max effort86.4% ±4.2
74Gemini 2.5 Pro (Jun 2025)84.7% ±4.8
75Qwen 3.5 Flash (hosted 35B-A3B)84.4% ±5.5
75Qwen3.6 Flash84.4% ±5.5
77Grok 484.0% ±5.0
78o3high effort83.9% ±4.4
79GLM-4.783.3% ±4.8
80Kimi K2 Thinking83.1% ±4.8
81Gemma 4 26B A4Bminimal effort82.2% ±5.8
81Qwen3.5 397B A17Bnone effort82.2% ±5.8
83Claude Opus 4.581.7% ±5.0
83o4-minihigh effort81.7% ±4.7
85GPT-5 nanohigh effort81.1% ±4.6
86Claude Sonnet 5max effort80.0% ±6.0
86Qwen3.7 Plusnone effort80.0% ±6.0
86Gemini 3.1 Flash-Litehigh effort80.0% ±6.0
86GLM-580.0% ±6.0
90Grok 3 Minihigh effort77.8% ±6.3
90Qwen3.7 Flashnone effort77.8% ±6.3
92o3-minihigh effort76.9% ±5.2
93Claude Sonnet 4.574.4% ±6.3
94Gemma 4 31Bminimal effort73.3% ±6.7
94Qwen3 Max73.3% ±5.9
94o1high effort73.3% ±6.7
97Gemini 2.5 Flash71.9% ±5.5
98Claude Sonnet 4.6max effort71.1% ±6.8
98Gemini 3.5 Flash-Litehigh effort71.1% ±6.8
100Qwen3.5-35B-A3Bnone effort70.0% ±6.0
101Doubao Seed 1.669.4% ±6.1
102Qwen3.6-35B-A3Bnone effort68.9% ±7.0
103DeepSeek V3.268.4% ±7.5
104GPT-5.5 Instant68.1% ±6.2
105Seed OSS 36B Instruct67.5% ±6.3
106Qwen3.6-27Bnone effort66.7% ±7.1
107DeepSeek-R1 (May 2025)66.4% ±4.9
108Qwen3-30B-A3B (Jul 2025)66.3% ±6.0
109Claude Opus 4.164.4% ±7.2
110Qwen3.5 9Bnone effort61.7% ±6.0
111Claude Sonnet 461.1% ±7.0
112Claude Opus 460.0% ±7.4
113QwQ-32B59.2% ±6.3
114Gemini 2.0 Flash Thinking57.8% ±7.4
115Qwen3.5-4Bnone effort55.8% ±6.2
116Grok 3 [Beta]55.6% ±7.5
116DeepSeek R1 Distill Qwen 32B55.6% ±6.2
118DeepSeek-R153.3% ±7.5
119Qwen3 4B Instruct52.2% ±6.6
120R1 Distill Llama 70B51.4% ±6.1
121Claude Haiku 4.551.3% ±7.1
122GPT-OSS 20Bhigh effort50.8% ±6.7
123DeepSeek R1 Distill Qwen 14B50.6% ±6.0
124Claude 3.7 Sonnet50.0% ±7.5
125o1-minihigh effort46.9% ±6.5
126GPT-4.1 mini44.7% ±6.1
127DeepSeek R1 Qwen3 8B43.9% ±6.8
128GPT-4.138.3% ±6.3
129GPT-4.537.8% ±7.3
129DeepSeek-V3 (Mar 2025)37.8% ±6.2
131Mistral Medium 332.2% ±6.0
132Gemini 2.0 Flash31.1% ±6.3
132o1-preview31.1% ±7.0
134Mistral Small 3.230.3% ±5.8
135Magistral Small 1.030.0% ±5.9
136GPT-4.1 nano28.9% ±6.0
137Magistral Small 1.228.1% ±5.6
138MiniMax M3none effort26.7% ±6.7
139Qwen3 14Bnone effort25.8% ±4.8
140Qwen3 30B A3Bnone effort25.6% ±5.2
141GLM-4.7-Flashnone effort25.0% ±4.9
142Qwen3 32Bnone effort23.1% ±5.1
143Gemma 3 27B22.5% ±5.3
144Qwen3 8Bnone effort22.2% ±4.5
145DeepSeek R1 Distill Qwen 1.5B21.4% ±5.3
146Llama 4 Maverick20.6% ±4.9
147Qwen Plus17.8% ±4.4
148Gemma 3 12B16.7% ±4.2
149Qwen2.5-Max16.1% ±4.1
150DeepSeek V315.8% ±4.3
151Gemini 1.5 Pro14.9% ±5.2
152Phi-413.8% ±3.7
153Grok 211.5% ±3.4
154Gemini 1.5 Flash10.1% ±4.1
155Llama 3.1 405B9.7% ±3.2
156Claude 3.5 Sonnet (October 2024)8.5% ±2.6
157Mistral Large 28.1% ±2.5
157Qwen2.5-72B8.1% ±2.6
157Qwen3 1.7Bnone effort8.1% ±2.8
160Llama 4 Scout7.8% ±3.0
161Gemma 3 4B7.5% ±2.8
162Qwen2.5 32B7.4% ±2.1
163GPT-4o mini6.9% ±2.7
164GPT-4 Turbo (Apr 2024)6.7% ±2.4
165Claude 3.5 Sonnet6.5% ±2.2
166GPT-4o6.3% ±2.3
167Qwen Turbo6.1% ±2.2
168Mistral Small 36.0% ±2.7
169Llama 3.3 70B5.1% ±2.4
170Mistral Small 3.14.9% ±1.5
171Claude 3 Opus4.7% ±2.0
172Gemini 1.5 Flash 8B4.6% ±2.0
173Tulu 3 (Tülu 3) 70B4.4% ±1.8
174Claude 3.5 Haiku4.3% ±1.6
174Llama 3 70B4.3% ±2.2
176Llama 3.1 70B3.6% ±1.7
177Granite 4.0 Micro2.8% ±1.4
178Llama 3.2 90B2.6% ±0.9
179Claude 22.5% ±1.5
179Claude 3 Sonnet2.5% ±2.0
179Hermes 2 Theta Llama-3 70B2.5% ±1.2
179Qwen2.5 7B2.5% ±0.9
183GPT-3.5 Turbo2.2% ±2.2
184Claude 2.11.9% ±1.0
184Llama 3-8B1.9% ±1.0
184Mistral Large1.9% ±0.9
187Claude 3 Haiku1.8% ±0.8
188Llama 3.1 8B1.7% ±0.9
189Gemma 2 27B1.4% ±0.9
190Gemini 1.0 Pro1.1% ±0.6
190Gemma 3 1B1.1% ±0.5
190GPT-4 (Jun 2023)1.1% ±0.5
193DeepSeek LLM 67B0.8% ±0.5
193Granite-4.0-1B0.8% ±0.5
195Gemma 2 9B0.6% ±0.4
195GPT-4 (Mar 2023)0.6% ±0.4
195Llama 3.2 1B0.6% ±0.4
198Granite-4.0-350M0.3% ±0.3
198Mistral 7B0.3% ±0.3
200Llama 2-70B0.0%