Inference
EuroEval

EuroEval German

How well a model handles German, as the unweighted mean of EuroEval's primary metric over its 8 German datasets (sb10k, germeval, scala_de, germanquad, mlsum_de, include_de, multiloko_de, winogrande_de). The primary metric differs by task — MCC for classification, knowledge and common-sense reasoning, micro-F1 without MISC for entity recognition, F1 for reading comprehension, chrF++ for summarization, METEOR for simplification — and is always the first of the two the board prints. Only models measured on every dataset are included, so each mean covers the same ground. Most rows are the validation split; some older open-weight rows are the test split, which EuroEval ranks in the same table.

ModelsModels with a result.169
Top resultBest result on this test.68.0%GPT-6 Astra
Top-3 spreadPoints from first to third.2.1 pts
YearYear of release.N/A

Result and price

020406080100$0.03$0.1$0.3$1$3$10$30$100RESULTOUTPUT PRICE PER 1M TOKENS · LOG SCALE

Results

177 results
#ModelResultThe score from the source.
01GPT-6 Astra68.0% ±0.5
02Gemini 3.7 Flash67.3% ±0.5
03Gemini 3.6 Flash66.0% ±0.5
04GLM-5.3-Flash63.6% ±0.7
05GPT-5.6 Sol63.6% ±0.5
06Gemini 3 Flash Preview Thinkingthinking effort62.1% ±0.5
07GPT-5.6 Terra59.7% ±0.6
08GPT-5.4 minihigh effort57.8% ±0.5
09Qwen3.8-27B57.2% ±0.7
10Claude Sonnet 4.5 Thinkingthinking effort57.1% ±0.5
11Claude Sonnet 4.5 No Thinkingno-thinking effort56.9% ±0.5
12Qwen3.6 27B FP856.7% ±0.7
13GPT-5.4 minimedium effort56.3% ±0.5
14Gemma 4 31B55.7% ±0.7
15Qwen3.5 35B A3B FP855.3% ±0.6
16Gemini 3.1 Flash Lite Preview55.0% ±0.5
17Claude Sonnet 4.654.8% ±0.5
18Grok 4.1 Fast (Reasoning)54.8% ±0.6
19GPT-5.4 minilow effort54.6% ±0.6
20Cogito V1 Preview Qwen 32B53.5% ±0.4
21Mistral Small 3.1 24B Base53.2% ±0.5
22GPT-5.4 Nonenone effort53.2% ±0.6
23GPT-5.6 Luna52.6% ±0.6
24GPT-5 mini52.5% ±0.6
25Gemini 3.5 Flash-Lite52.4% ±0.6
26Llama 3.1-70B52.1% ±0.6
27GPT-4.151.9% ±0.6
28Mistral Small 3.1 24B51.3% ±0.5
29GPT-OSS 120B51.2% ±0.6
30Mistral Small 3.2 24B51.1% ±0.5
31Cogito V2 Preview Llama 70B50.9% ±0.4
32Gemma 3 27B Pt50.7% ±0.6
33Ministral 3 14B Base50.1% ±0.8
34Gemma 4 12B IT49.9% ±0.8
35GPT-5.249.6% ±0.5
36Gemini 2.5 Flash Lite Thinkingthinking effort49.3% ±0.6
37GPT-5.4 nanomedium effort49.2% ±0.6
38Gemma 4 26B A4B49.1% ±0.6
39Qwen3 4B Thinking48.8% ±0.4
40GPT-5.4 nanohigh effort48.7% ±0.6
41Qwen3 32B No Thinkingno-thinking effort48.7% ±0.4
42Cogito V1 Preview Qwen 14B48.1% ±0.4
43Claude Haiku 4.547.8% ±0.6
44Gemma 3 27B47.2% ±0.5
45Qwen3 14B No Thinkingno-thinking effort47.1% ±0.5
46GPT-5 nano47.0% ±0.6
47NVIDIA Nemotron 3 Nano 30B A3B BF1646.7% ±0.7
48Qwen3.5 9B46.7% ±0.7
49GPT-5.4 nanolow effort46.5% ±0.6
50Ministral 3 14B Instruct46.5% ±0.6
51Ministral 3 8B Base45.9% ±0.8
52Nemotron Cascade 2 30B A3B45.8% ±0.7
53GLM-4.7 Flash NVFP445.8% ±0.7
54GPT-OSS 20Bmedium effort45.8% ±0.4
55Gemma 3 12B Pt45.3% ±0.7
56Ministral 3 14B (Reasoning)45.1% ±0.8
57Ministral 3 8B (Reasoning)44.3% ±0.6
58GPT-5.4 Mini Nonenone effort44.1% ±0.5
59EuroLLM 22B Instruct44.0% ±0.5
60NVIDIA Nemotron 3 Nano 30B A3B Base BF1643.4% ±0.8
61Bielik Minitron 7B V3.0 Instruct43.3%
62Ministral 3 8B Instruct43.3% ±0.6
63Gemma 3 12B42.9% ±0.5
64Bielik 11B V3.0 Instruct42.8%
65Gemma 4 31B42.6% ±0.8
66GPT-OSS 20Blow effort42.5% ±0.6
67Apertus V1.5 70B42.4% ±0.8
68Exaone 4.0 32B41.8% ±0.6
69Gemini 2.5 Flash Lite No Thinkingno-thinking effort41.6% ±0.7
70Olmo 3.1 32B Instruct SFT41.6% ±0.5
71Gemma 4 E4B IT41.4% ±0.7
72Olmo 3.1 32B Instruct DPO41.3% ±0.5
73Nesso 4B40.9% ±0.5
74Llama 3.1 70B LatamGPT SFT-1.040.7% ±0.8
75Domyn Small V1.040.7% ±0.7
76Qwen3 8B No Thinkingno-thinking effort40.3% ±0.5
77Olmo 3.1 32B Instruct39.8% ±0.5
78Qwen3.5 4B39.7% ±0.8
79Qwen3.5 9B Base39.4% ±0.8
80Cogito V1 Preview Llama 8B38.8% ±0.6
81EuroLLM 22B38.7% ±0.6
82Qwen3 4B No Thinkingno-thinking effort38.7% ±0.6
83Olmo 3 32B38.5% ±0.6
84GPT-OSS 20Bhigh effort38.3% ±0.6
85Llama 3.1 Tulu 3 8B SFT38.0% ±0.6
86Gemini 3 Flash Preview No Thinkingno-thinking effort37.8% ±0.5
87Gemma 4 26B A4B37.7% ±1.0
88Gemma 4 12B37.3% ±1.0
89Llama 3.1 8B Instruct37.0% ±0.5
90Llama 3.1-8B36.6% ±0.8
91Gemma 4 E2B IT35.8% ±0.7
92Aya Expanse 8B35.4% ±0.7
93EuroLLM 9B Instruct35.3% ±0.6
94EuroLLM 9B35.1% ±0.6
95Qwen3.5 2B34.9% ±0.6
96Luciole 23B Instruct 1.134.8% ±0.7
97Ministral 3 3B (Reasoning)34.5% ±0.6
98Gemma 3 4B34.0% ±0.5
99Gemma 3 4B Pt33.9% ±0.6
100GPT-5.4 Nano Nonenone effort33.9% ±0.7
101SmolLM3 3B Base33.8% ±0.6
102Ministral 3 3B Instruct33.7% ±0.7
103Olmo 3 7B33.2% ±0.6
104LFM2.5 8B A1B Base33.1% ±0.7
105Qwen3.5 4B Base33.0% ±0.8
106Qwen3 1.7B No Thinkingno-thinking effort32.6% ±0.5
107Mellum2 12B A2.5B Instruct SFT31.9% ±0.6
108Mellum2-12B-A2.5B-Instruct31.9% ±0.7
109Apertus V1.1 4B31.1% ±0.6
110Occiglot 7B Eu531.0% ±0.7
111NVIDIA Nemotron 3 Nano 4B BF1630.9% ±0.6
112SmolLM3 3B No Thinkingno-thinking effort30.7% ±0.6
113Llama 3.2 3B Instruct30.1% ±0.5
114Apertus V1.1 4B Instruct29.7% ±0.6
115Occiglot 7B Eu5 Instruct29.4% ±0.5
116Mellum2 12B A2.5B Thinking SFT29.4% ±0.8
117DFM Mimir29.3% ±0.6
118Olmo 3 7B Instruct28.9% ±0.4
119Luciole 8B Instruct 1.128.4% ±0.8
120NVIDIA Nemotron 3 Nano 4B FP828.3% ±0.9
121Olmo 3 7B Instruct DPO27.7% ±0.5
122Mellum2-12B-A2.5B-Thinking27.3% ±0.9
123Cogito V1 Preview Llama 3B27.1% ±0.5
124Mellum2 12B A2.5B Base26.9% ±0.6
125Luciole 23B Base26.8% ±0.9
126Olmo 3 7B Instruct SFT26.5% ±0.6
127Bielik 1.5B V3.0 Instruct26.1% ±0.6
128Oellm 9B 128K Theta32m V325.8% ±0.7
129Qwen3.5 0.8B25.0% ±0.5
130Qwen3.5 2B Base23.7% ±0.6
131Teuken 7B Instruct V0.622.9% ±0.4
132Qwen3 0.6B No Thinkingno-thinking effort21.9% ±0.6
133Gemma 4 E4B21.7% ±0.8
134YuLan Mini Instruct21.7% ±0.5
135Luciole 8B Base21.2% ±0.9
136Mellum2 12B A2.5B Base Pretrain21.2% ±0.9
137Salamandra 7B Instruct21.1% ±0.4
138Qwen3.5 0.8B Base21.0% ±0.6
139Aya 23 8B20.9% ±0.6
140LFM2.5-8B-A1B20.6% ±0.8
141LFM2 1.2B20.4% ±0.5
142Gemma 3 1B IT20.1% ±0.6
143Lizzy 7B19.7% ±0.4
144EuroLLM 1.7B Instruct19.0% ±0.6
145Apertus V1.1 1.5B17.6% ±0.6
146Apertus V1.1 0.5B Instruct17.6% ±0.5
147TildeOpen 30B17.6% ±0.8
148NVIDIA Nemotron 3 Nano 30B A3B FP817.2% ±1.1
149Gemma 3 1B Pt16.7% ±0.8
150Luciole 1B Instruct 1.116.5% ±0.6
151Salamandra 7B16.3% ±0.6
152Pleias 1.2B Preview15.8% ±0.6
153Apertus V1.1 1.5B Instruct15.4% ±0.6
154LFM2 350M13.9% ±0.3
155Apertus V1.1 0.5B13.5% ±0.7
156Llama 3.2 1B13.3% ±0.6
157Llama 3.2 1B Instruct13.2% ±0.5
158Luciole 1B Base13.2% ±0.8
159Gemma 4 E2B12.9% ±0.9
160LFM2 700M12.6% ±0.4
161EuroMoE 2.6B A0.6B Instruct Preview12.4% ±0.5
162Pleias 350M Preview11.8% ±0.4
163Falcon H1 Tiny Multilingual 100M Base11.7% ±0.5
164Gemma 3 270M11.3% ±0.8
165Falcon H1 Tiny Multilingual 100M Instruct9.9% ±0.5
166Gemma 3 270M IT9.8% ±0.6
167SmolLM2 360M9.8% ±0.6
1686 No Thinkingno-thinking effort8.5% ±0.4
169Ling Mini 2.08.1% ±1.2
170SmolLM2 360M Instruct7.8% ±0.5
171SmolLM2 135M7.3% ±0.5
172MiniCPM5-1B7.1% ±0.7
173SmolLM2 135M Instruct6.7% ±0.7
174Marco Mini Global Base4.6% ±0.4
175Ling Mini Base 2.03.4% ±0.5
176Monad1.2% ±0.3
177Teuken 7B Base V0.60.0%