Inference
EN
Anthropic

Claude 3.5 Haiku

Claude 3.5 Haiku is a standard model from Anthropic. 22 benchmarks count toward its index, in 8 categories.

  • Closed weights
  • Standard
  • Input: text
  • Released 2024-10
RankedEnough results for an index.
IndexAll categories, out of 100.24.1±3.4Median 51.0
CoverageIndex weight that has results.100%
SpeedOutput tokens per second.—Median 67/s
First tokenSeconds to the first output token.—Median 1.3 s
Input / 1MUS dollars per 1M input tokens.—Median $0.60
Output / 1MUS dollars per 1M output tokens.—Median $2.55
ContextMaximum tokens in one request.200KMedian 262K
EloLMArena rating and rank.1255±3 #298Median 1422

Capabilities

Index per category, out of 100.
Out of 100
AgenticMulti-step tasks with tools.2 counted26.6210 / 218
CodingCode writing and repair.3 counted28.0200 / 209
ReasoningLogic problems and puzzles.2 counted31.4204 / 227
MultimodalTasks with images and text.1 counted14.5—
KnowledgeFacts and expert knowledge.5 counted24.4258 / 268
MultilingualTasks in many languages.1 counted31.1—
InstructionTasks with strict prompt rules.2 counted32.652 / 63
MathMath problems.6 counted16.2163 / 168

Results

22 counted
BenchmarkThe benchmark name.ResultThe score from the source.PlacePlace among results on this benchmark.
Multilingual Grade School MathMultilingual85.6%6 / 13
Evaluating Large Language Models Trained on CodeCoding88.1%11 / 22
Instruction-Following EvalInstruction85.9%37 / 70
MATH level 5Math46.4% ±1.253 / 88
Artificial Analysis IFBenchInstruction42.8%97 / 146
Critical Physics TasksReasoning0.0%150 / 214
Massive Multitask Language UnderstandingKnowledge80.3%25 / 34
GDPval-AA normalizedAgentic0.0%101 / 133
τ²-Bench Tool-Agent-User EvaluationAgentic24.6%122 / 149
GPQA diamond · Epoch AIKnowledge38.1% ±2.6178 / 217
Massive Multitask Language Understanding ProfessionalKnowledge65.0%63 / 75
Artificial Analysis Long Context ReasoningReasoning27.3%186 / 217
OTIS Mock AIME 2024-2025Math4.3% ±1.6174 / 200
LiveCodeBenchCoding41.9% ±1.1127 / 142
FrontierMath-2025-02-28-PrivateMath0.3% ±0.364 / 69
Graduate-Level Google-Proof Q&AKnowledge41.6%131 / 141
Artificial Analysis MMMU-ProMultimodal45.6%106 / 113
Artificial Analysis Humanity's Last ExamKnowledge3.6%210 / 222
Software Engineering Benchmark VerifiedCoding40.6%108 / 114
MATH 500Math64.2% ±4.256 / 59
Artificial Analysis GPQA DiamondKnowledge40.8%190 / 199
MMLU ProKnowledge64.1% ±0.5129 / 135
GPQA Diamond · Vals AIKnowledge37.9% ±2.4129 / 135
Artificial Analysis Omniscience AccuracyKnowledge13.2%157 / 164
AIMEMath3.3% ±0.393 / 95
American Invitational Mathematics Examination 2024Math5.3%27 / 27
FrontierMath-2025-02-28-PublicMath0.0%33 / 64
CorpFin v2Finance50.8% ±1.0106 / 131
LegalBenchLegal70.3% ±0.6123 / 149
MGSMMultilingual84.6% ±2.268 / 74
TaxEval v2Finance57.4% ±1.0135 / 143
BIG-Bench HardReasoning86.6%—
Discrete Reasoning Over ParagraphsReasoning83.1%—

Sources

BenchLMCC BY-NC 4.0Vals AINo licence statedEpoch AICC BY

History

5 releases
Index25.0 → 24.1
23 Oct 202612 Nov 2026

Questions

How good is Claude 3.5 Haiku?
Its index is 24.1 out of 100. That is place 272 of 283 ranked models.
What is Claude 3.5 Haiku best at?
Instruction: 32.6 out of 100, place 52 of 63.
What is the context window of Claude 3.5 Haiku?
200K tokens.
Is Claude 3.5 Haiku open source?
No. Its weights are not published.
When was Claude 3.5 Haiku released?
October 2024.

Badge

Claude 3.5 Haiku on Inference360: place 272 of 283, index 24.1
HTML
<a href="https://inference360.ai/models/claude-3-5-haiku"><img src="https://inference360.ai/badges/claude-3-5-haiku.svg" alt="Claude 3.5 Haiku on Inference360: place 272 of 283, index 24.1" height="20"></a>
Markdown
[![Claude 3.5 Haiku on Inference360: place 272 of 283, index 24.1](https://inference360.ai/badges/claude-3-5-haiku.svg)](https://inference360.ai/models/claude-3-5-haiku)

Compare with

Command A23.9Mistral Small 3.124.2Claude 3 Opus23.3Claude 3.5 Sonnet32.2Claude 3.7 Sonnet Thinking41.4

More from Anthropic

All models
Claude 3.7 Sonnet Thinking41.4Claude 3.7 Sonnet37.7Claude 3.5 Sonnet32.2Claude 4 Opus Thinking47.0Claude 4 Sonnet39.5