Llama 3.1 405B vs Mistral Medium 3

Change models
IndexAll categories, out of 100.
27.2 ±8.9
27.4 ±5.2 best
GapIndex points below the top.
−0.2 · no clear gap
Top
CoverageIndex weight that has results.
65%
95% best
SpeedOutput tokens per second.
29/s
45/s best
EloLMArena rating and rank.
—
1370
ReleasedMonth of release.
2024-07
2025-05
Input / 1MUS dollars per 1M input tokens.
—
$0.44
Output / 1MUS dollars per 1M output tokens.
—
$2.20
1K chats2K tokens in, 500 out, each.
—
$1.98
1K documents20K tokens in, 1K out, each.
—
$11.00
1K agent tasks200K tokens in, 20K out, each.
—
$132.00
ContextMaximum tokens in one request.
128K
131K best
AgenticMulti-step tasks with tools.
16.8
20.2 best
CodingCode writing and repair.
—
9.7
ReasoningLogic problems and puzzles.
30.8
32.8 best
MultimodalTasks with images and text.
—
24.3
KnowledgeFacts and expert knowledge.
26.2
31.6 best
MultilingualTasks in many languages.
—
—
InstructionTasks with strict prompt rules.
33.7 best
26.3
MathMath problems.
18.7
32.0 best

Benchmarks

9 shared
Only Llama 3.1 405B: 1Only Mistral Medium 3: 5
Llama 3.1 405B
Mistral Medium 3
50.9%146 / 220
59.5% best132 / 220
4.0%195 / 216
4.1% best193 / 216
25.3%183 / 211
31.3% best173 / 211
0.0%145 / 207
0.0%145 / 207
9.7%158 / 203
32.2% best134 / 203
51.5%164 / 193
57.8% best155 / 193
19.0%128 / 143
24.3% best118 / 143
39.0%102 / 140
39.3% best101 / 140
49.8%49 / 88
81.6% best27 / 88