Inference
EN
EuroEval

EuroEval German

How well a model handles German, as the unweighted mean of EuroEval's primary metric over its 8 German datasets (sb10k, germeval, scala_de, germanquad, mlsum_de, include_de, multiloko_de, winogrande_de). The primary metric differs by task — MCC for classification, knowledge and common-sense reasoning, micro-F1 without MISC for entity recognition, F1 for reading comprehension, chrF++ for summarization, METEOR for simplification — and is always the first of the two the board prints. Only models measured on every dataset are included, so each mean covers the same ground. Most rows are the validation split; some older open-weight rows are the test split, which EuroEval ranks in the same table.

ModelsModels with a result.183
Top resultBest result on this benchmark.68.0%GPT-6 Astra
Top-3 spreadPoints from first to third.0.9 pts
YearYear of release.—

Result and price

38 of 191 results

Results

191 results
#ModelResultThe score from the source.
01GPT-6 Astra68.0% ±0.5
02Gemini 3.7 Flash67.3% ±0.5
03GPT-6.1 Sol67.1% ±0.5
04Gemini 3.6 Flash66.0% ±0.5
05GPT-5.6 Sol63.6% ±0.5
06GPT-6 Sol63.1% ±0.5
07Gemini 3 Flash Preview Thinkingthinking effort62.1% ±0.5
08DeepSeek Flash Lowlow effort60.7% ±0.6
09GLM-5.3-Flash59.8% ±0.6
10GPT-5.6 Terra59.7% ±0.6
11DeepSeek Flash No Thinkingno-thinking effort59.5% ±0.7
12GPT-5.4 minihigh effort57.8% ±0.5
13Qwen3.8-27B57.2% ±0.7
14Claude Sonnet 4.5 Thinkingthinking effort57.1% ±0.5
15Claude Sonnet 4.5 No Thinkingno-thinking effort56.9% ±0.5
16Qwen3.6 27B FP856.7% ±0.7
17GPT-5.4 minimedium effort56.3% ±0.5
18Gemma 4 31Bgemma-4-31B-it55.7% ±0.7
19Qwen3.5 35B A3B FP855.3% ±0.6
20Gemini 3.1 Flash Lite Preview55.0% ±0.5
21Claude Sonnet 4.654.8% ±0.5
21Grok 4.1 Fast (Reasoning)54.8% ±0.6
23GPT-5.4 minilow effort54.6% ±0.6
24Cogito V1 Preview Qwen 32B53.5% ±0.4
25Mistral Small 3.1 24B Base53.2% ±0.5
25GPT-5.4 Nonenone effort53.2% ±0.6
27GPT-5.6 Luna52.6% ±0.6
28GPT-5 mini52.5% ±0.6
29Gemini 3.5 Flash-Lite52.4% ±0.6
30Llama 3.1 70BAbove fitFar above this model's index.52.1% ±0.6
31GPT-4.151.9% ±0.6
32Mistral Small 3.1 24B51.3% ±0.5
33GPT-OSS 120B51.2% ±0.6
34Mistral Small 3.2 24B51.1% ±0.5
35Cogito V2 Preview Llama 70B50.9% ±0.4
36GPT-6 Luna50.8% ±0.5
37Gemma 3 27B Pt50.7% ±0.6
38Ministral 3 14B Base50.1% ±0.8
39Gemma 4 12B IT49.9% ±0.8
40GPT-5.249.6% ±0.5
41Gemini 2.5 Flash Lite Thinkingthinking effort49.3% ±0.6
42GPT-5.4 nanomedium effort49.2% ±0.6
43Gemma 4 26B A4Bgemma-4-26B-A4B-it49.1% ±0.6
44Qwen3 4B Thinking48.8% ±0.4
45GPT-5.4 nanohigh effort48.7% ±0.6
45Qwen3 32B No Thinkingno-thinking effort48.7% ±0.4
47Cogito V1 Preview Qwen 14B48.1% ±0.4
48Claude Haiku 4.547.8% ±0.6
49Gemma 3 27B47.2% ±0.5
50Qwen3 14B No Thinkingno-thinking effort47.1% ±0.5
51GPT-5 nano47.0% ±0.6
52NVIDIA Nemotron 3 Nano 30B A3B BF1646.7% ±0.7
52Qwen3.5 9B46.7% ±0.7
54GPT-5.4 nanolow effort46.5% ±0.6
54Ministral 3 14B Instruct46.5% ±0.6
56Ministral 3 8B Base45.9% ±0.8
57Nemotron Cascade 2 30B A3B45.8% ±0.7
57GLM-4.7 Flash NVFP445.8% ±0.7
57GPT-OSS 20Bmedium effort45.8% ±0.4
60Gemma 3 12B Pt45.3% ±0.7
61Ministral 3 14B (Reasoning)45.1% ±0.8
62Ministral 3 8B (Reasoning)44.3% ±0.6
63GPT-5.4 Mini Nonenone effort44.1% ±0.5
64EuroLLM 22B Instruct 251244.0% ±0.5
65NVIDIA Nemotron 3 Nano 30B A3B Base BF1643.4% ±0.9
66Bielik Minitron 7B V3.0 Instruct43.3%
66Ministral 3 8B Instruct43.3% ±0.6
68Gemma 3 12B42.9% ±0.5
69Bielik 11B V3.0 Instruct42.8%
70Gemma 4 31Bgemma-4-31B42.6% ±0.8
71GPT-OSS 20Blow effort42.5% ±0.6
72Apertus V1.5 70B42.4% ±0.8
73Exaone 4.0 32B41.8% ±0.6
74Gemini 2.5 Flash Lite No Thinkingno-thinking effort41.6% ±0.7
74Olmo 3.1 32B Instruct SFT41.6% ±0.5
76Gemma 4 E4B IT41.4% ±0.7
77Olmo 3.1 32B Instruct DPO41.3% ±0.5
78Nesso 4B40.9% ±0.5
79Llama 3.1 70B LatamGPT SFT-1.040.7% ±0.8
79Domyn Small V1.040.7% ±0.8
81Qwen3 8B No Thinkingno-thinking effort40.3% ±0.5
82Olmo 3.1 32B Instruct39.8% ±0.5
83Qwen3.5-4B39.7% ±0.8
84Qwen3.5 9B Base39.4% ±0.8
85Cogito V1 Preview Llama 8B38.8% ±0.6
86EuroLLM 22B 251238.7% ±0.6
86Qwen3 4B No Thinkingno-thinking effort38.7% ±0.6
88Olmo 3 32B38.5% ±0.6
89GPT-OSS 20Bhigh effort38.3% ±0.6
90Llama 3.1 Tulu 3 8B SFT38.0% ±0.6
91Gemini 3 Flash Preview No Thinkingno-thinking effort37.8% ±0.6
92Gemma 4 26B A4Bgemma-4-26B-A4B37.7% ±1.0
93Gemma 4 12B37.3% ±1.0
94Llama 3.1 8B Instruct37.0% ±0.5
95Llama 3.1 8B36.6% ±0.8
96Gemma 4 E2B IT35.8% ±0.7
97EuroLLM 9B Instruct 251235.5% ±0.6
98Aya Expanse 8B35.4% ±0.7
99DFM Mimir V1.535.2% ±0.5
100EuroLLM 9B Instruct35.1% ±0.6
100EuroLLM 9B35.1% ±0.6
102Qwen3.5 2B34.9% ±0.6
103Luciole 23B Instruct 1.134.8% ±0.7
104Ministral 3 3B (Reasoning)34.5% ±0.6
105Gemma 3 4B34.0% ±0.5
106Gemma 3 4B Pt33.9% ±0.6
106GPT-5.4 Nano Nonenone effort33.9% ±0.7
108SmolLM3 3B Base33.8% ±0.6
109Ministral 3 3B Instruct33.7% ±0.7
110Olmo 3 7B33.2% ±0.6
110AMALIA 9B DPO33.2% ±0.5
112LFM2.5 8B A1B Base33.1% ±0.7
113Qwen3.5 4B Base33.0% ±0.8
114Qwen3 1.7B No Thinkingno-thinking effort32.6% ±0.6
115Mellum2 12B A2.5B Instruct SFT31.9% ±0.7
115Mellum2-12B-A2.5B-Instruct31.9% ±0.7
117Apertus V1.1 4B31.1% ±0.8
118AMALIA 9B SFT31.0% ±0.6
118Occiglot 7B Eu531.0% ±0.7
120NVIDIA Nemotron 3 Nano 4B BF1630.9% ±0.6
121SmolLM3 3B No Thinkingno-thinking effort30.7% ±0.6
122Llama 3.2 3B Instruct30.1% ±0.6
123Apertus V1.1 4B Instruct29.7% ±0.6
124Occiglot 7B Eu5 Instruct29.4% ±0.5
124Mellum2 12B A2.5B Thinking SFT29.4% ±0.8
126DFM Mimir29.3% ±0.7
127Olmo 3 7B Instruct28.9% ±0.4
128Luciole 8B Instruct 1.128.4% ±0.7
129NVIDIA Nemotron 3 Nano 4B FP828.3% ±1.0
130Olmo 3 7B Instruct DPO27.7% ±0.6
131Mellum2-12B-A2.5B-Thinking27.3% ±0.9
132Cogito V1 Preview Llama 3B27.1% ±0.5
133Mellum2 12B A2.5B Base26.9% ±0.6
134Luciole 23B Base26.8% ±0.9
135Olmo 3 7B Instruct SFT26.5% ±0.7
136Bielik 1.5B V3.0 Instruct26.1% ±0.6
137Oellm 9B 128K Theta32m V325.8% ±0.7
138Qwen3.5 0.8B25.0% ±0.6
139Qwen3.5 2B Base23.7% ±0.7
140Teuken 7B Instruct V0.622.9% ±0.4
141Qwen3 0.6B No Thinkingno-thinking effort21.9% ±0.7
142Gemma 4 E4B21.7% ±0.9
142YuLan Mini Instruct21.7% ±0.6
144Luciole 8B Base21.2% ±0.9
144Mellum2 12B A2.5B Base Pretrain21.2% ±0.9
146Salamandra 7B Instruct21.1% ±0.4
147Qwen3.5 0.8B Base21.0% ±0.6
148Aya 23 8B20.9% ±0.7
149LFM2.5-8B-A1B20.6% ±0.9
150LFM2 1.2B20.4% ±0.5
151Gemma 3 1B IT20.1% ±0.6
152Lizzy 7B19.7% ±0.5
153Trustllm V2 8B Midtrain19.5% ±0.6
154EuroLLM 1.7B Instruct19.0% ±0.8
155Trustllm V2 8B SFT 8K17.9% ±0.5
156Trustllm V2 8B SFT17.7% ±0.4
157Apertus V1.1 1.5B17.6% ±0.7
157Apertus V1.1 0.5B Instruct17.6% ±0.6
157TildeOpen 30B17.6% ±0.8
160Trustllm V2 8B17.3% ±0.8
161NVIDIA Nemotron 3 Nano 30B A3B FP817.2% ±1.1
161Trustllm V2 8B SFT Mixb2 Exp17.2% ±0.3
163Gemma 3 1B Pt16.7% ±0.7
164Luciole 1B Instruct 1.116.5% ±0.8
165Salamandra 7B16.3% ±0.6
166Pleias 1.2B Preview15.8% ±0.6
167Apertus V1.1 1.5B Instruct15.4% ±0.7
168LFM2 350M13.9% ±0.4
169Apertus V1.1 0.5B13.5% ±0.7
170Llama 3.2 1B13.3% ±0.8
171Llama 3.2 1B Instruct13.2% ±0.5
171Luciole 1B Base13.2% ±0.8
173Gemma 4 E2B12.9% ±0.9
174LFM2 700M12.6% ±0.3
175EuroMoE 2.6B A0.6B Instruct Preview12.4% ±0.6
176Pleias 350M Preview11.8% ±0.5
177Falcon H1 Tiny Multilingual 100M Base11.7% ±0.5
178Gemma 3 270M11.3% ±0.7
179Falcon H1 Tiny Multilingual 100M Instruct9.9% ±0.5
180Gemma 3 270M IT9.8% ±0.7
180SmolLM2 360M9.8% ±0.7
182Ling Mini 2.08.1% ±1.2
183SmolLM2 360M Instruct7.8% ±0.6
184SmolLM2 135M7.3% ±0.6
185MiniCPM5-1B7.1% ±0.9
186SmolLM2 135M Instruct6.7% ±0.6
187Marco Mini Global Base4.6% ±0.4
188Baguettotron No Thinkingno-thinking effort4.2% ±0.4
189Ling Mini Base 2.03.4% ±0.8
190Monad1.2% ±0.4
191Teuken 7B Base V0.60.0%