GPT-4o vs Grok 2

Change models
GPT-4oOpenAI
IndexAll categories, out of 100.
28.7 ±3.3
29.1 ±9.4 best
GapIndex points below the top.
−0.4 · no clear gap
Top
CoverageIndex weight that has results.
100% best
55%
SpeedOutput tokens per second.
66/s
—
EloLMArena rating and rank.
1300
1305 best
ReleasedMonth of release.
2024-05
2024-08
Input / 1MUS dollars per 1M input tokens.
$2.50
—
Output / 1MUS dollars per 1M output tokens.
$10.00
—
Batch output / 1MBatch price per 1M output tokens.
$5.00
—
1K chats2K tokens in, 500 out, each.
$10.00
—
1K documents20K tokens in, 1K out, each.
$60.00
—
1K agent tasks200K tokens in, 20K out, each.
$700.00
—
ContextMaximum tokens in one request.
128K
—
AgenticMulti-step tasks with tools.
26.7
—
CodingCode writing and repair.
22.8
25.2 best
ReasoningLogic problems and puzzles.
37.4
—
MultimodalTasks with images and text.
36.0 best
33.5
KnowledgeFacts and expert knowledge.
34.1 best
34.0
MultilingualTasks in many languages.
30.8
—
InstructionTasks with strict prompt rules.
21.1
—
MathMath problems.
23.4
25.0 best

Benchmarks

14 shared
Only GPT-4o: 28Only Grok 2: 5
GPT-4o
Grok 2
48.9%149 / 217
53.8% best139 / 217
6.3%166 / 200
11.5% best153 / 200
TaxEval v2Finance
74.5% best35 / 143
67.0%109 / 143
43.4% best123 / 142
38.7%130 / 142
53.8% best117 / 135
50.8%118 / 135
MMLU ProKnowledge
72.6%114 / 135
75.5% best108 / 135
CorpFin v2Finance
45.9%115 / 131
51.1% best104 / 131
AIMEMath
11.9%86 / 95
15.2% best84 / 95
MedQAHealthcare
88.2%55 / 95
92.3% best33 / 95
51.0%48 / 88
63.5% best39 / 88
MGSMMultilingual
90.4% best38 / 74
86.1%64 / 74
0.3%64 / 69
0.7% best62 / 69
0.0%33 / 64
0.0%33 / 64
74.0%46 / 59
78.4% best42 / 59