GPT-4o vs GPT-4o mini

Change models
GPT-4oOpenAI
IndexAll categories, out of 100.
28.7 ±3.3 best
26.2 ±4.3
GapIndex points below the top.
Top
−2.5 · no clear gap
CoverageIndex weight that has results.
100% best
95%
SpeedOutput tokens per second.
66/s best
56/s
EloLMArena rating and rank.
1300 best
1286
ReleasedMonth of release.
2024-05
2024-07
Input / 1MUS dollars per 1M input tokens.
$2.50
$0.15 best
Output / 1MUS dollars per 1M output tokens.
$10.00
$0.60 best
Batch output / 1MBatch price per 1M output tokens.
$5.00
—
1K chats2K tokens in, 500 out, each.
$10.00
$0.60 best
1K documents20K tokens in, 1K out, each.
$60.00
$3.60 best
1K agent tasks200K tokens in, 20K out, each.
$700.00
$42.00 best
ContextMaximum tokens in one request.
128K
128K
AgenticMulti-step tasks with tools.
26.7
34.0 best
CodingCode writing and repair.
22.8 best
19.5
ReasoningLogic problems and puzzles.
37.4
41.1 best
MultimodalTasks with images and text.
36.0 best
19.0
KnowledgeFacts and expert knowledge.
34.1 best
25.7
MultilingualTasks in many languages.
30.8
—
InstructionTasks with strict prompt rules.
21.1 best
18.0
MathMath problems.
23.4 best
22.0

Benchmarks

23 shared
Only GPT-4o: 19Only GPT-4o mini: 4
GPT-4o
GPT-4o mini
2.4%222 / 222
4.2% best194 / 222
48.9% best149 / 217
37.7%179 / 217
6.3%166 / 200
6.9% best163 / 200
54.3% best169 / 199
42.6%185 / 199
34.3% best125 / 146
31.0%134 / 146
TaxEval v2Finance
74.5% best35 / 143
60.5%127 / 143
43.4% best123 / 142
26.4%138 / 142
53.8% best117 / 135
44.2%127 / 135
MMLU ProKnowledge
72.6% best114 / 135
62.7%131 / 135
0.0%101 / 133
0.0%101 / 133
0.0%120 / 131
0.0%120 / 131
CorpFin v2Finance
45.9% best115 / 131
45.5%116 / 131
0.0%102 / 122
0.0%102 / 122
57.4% best75 / 97
54.5%79 / 97
AIMEMath
11.9% best86 / 95
11.5%87 / 95
MedQAHealthcare
88.2% best55 / 95
72.4%81 / 95
MMMU ProMultimodal
62.2% best81 / 92
56.6%87 / 92
26.0% best73 / 91
8.3%90 / 91
Chess PuzzlesReasoning
9.0% best81 / 90
3.0%88 / 90
0.4%89 / 90
0.7% best87 / 90