Inference
EN
EuroEval

EuroEval Polish

How well a model handles Polish, as the unweighted mean of EuroEval's primary metric over its 7 Polish datasets (polemo2, kpwr_ner, scala_pl, poquad, psc, llmzszl, winogrande_pl). The primary metric differs by task — MCC for classification, knowledge and common-sense reasoning, micro-F1 without MISC for entity recognition, F1 for reading comprehension, chrF++ for summarization, METEOR for simplification — and is always the first of the two the board prints. Only models measured on every dataset are included, so each mean covers the same ground. Most rows are the validation split; some older open-weight rows are the test split, which EuroEval ranks in the same table.

ModelsModels with a result.200
Top resultBest result on this benchmark.67.9%Gemini 3.7 Flash
Top-3 spreadPoints from first to third.1.0 pts
YearYear of release.—

Result and price

44 of 209 results

Results

209 results
#ModelResultThe score from the source.
01Gemini 3.7 Flash67.9% ±0.4
02GPT-6 Astra67.4% ±0.4
03GPT-6.1 Sol66.9% ±0.3
04Gemini 3.6 Flash65.9% ±0.4
05Qwen3.8-27B65.4% ±0.5
06Qwen3.6-27B65.3% ±0.5
07Qwen3.6 27B FP865.0% ±0.4
08Gemini 3 Pro64.8% ±0.5
09DeepSeek Flash No Thinkingno-thinking effort64.4% ±0.6
10Claude Sonnet 4.5 Thinkingthinking effort63.8% ±0.3
11GPT-5.6 Sol62.8% ±0.4
12DeepSeek Flash Lowlow effort62.7% ±0.5
13GPT-5.4 minihigh effort62.5% ±0.4
14GPT-6 Sol62.4% ±0.4
15Qwen3.5 35B A3B FP862.3% ±0.6
15Gemini 3 Flash Preview No Thinkingno-thinking effort62.3% ±0.5
17Gemma 4 31Bgemma-4-31B-it61.6% ±0.4
17Gemini 3 Flash Preview Thinkingthinking effort61.6% ±0.4
19Gemini 3.1 Flash Lite Preview61.0% ±0.4
19GLM-5.3-Flash61.0% ±0.4
21GPT-5.4 minimedium effort60.2% ±0.4
22Claude Sonnet 4.659.9% ±0.5
23GPT-5.4 Nonenone effort59.5% ±0.5
24GPT-5.6 Terra59.0% ±0.5
25GPT-5.4 minilow effort58.8% ±0.5
26Claude Sonnet 4.5 No Thinkingno-thinking effort58.7% ±0.4
27Grok 4.1 Fast (Reasoning)57.6% ±0.4
27GPT-OSS 120B57.6% ±0.5
29GPT-5 mini57.2% ±0.3
30GPT-5.257.1% ±0.4
31Qwen3.5 35B A3B Base57.0% ±0.5
32Llama 3.1 70BAbove fitFar above this model's index.56.5% ±0.5
33Gemini 2.5 Flash Lite Thinkingthinking effort56.4% ±0.5
34Mistral Small 3.1 24B Base56.3% ±0.4
34Cogito V1 Preview Qwen 32B56.3% ±0.2
36GPT-6 Luna56.0% ±0.5
37Qwen2.5 72B Instruct55.8% ±0.2
38Gemma 3 27B Pt55.7% ±0.4
39Gemma 4 26B A4Bgemma-4-26B-A4B-it55.5% ±0.7
40BgGPT Gemma 3 27B IT55.3% ±0.6
41GPT-5.4 nanohigh effort55.2% ±0.5
41Gemma 4 12B IT55.2% ±0.6
43Voxtral Small 24B 250755.1% ±0.2
44Mistral Small 3.1 24B55.0% ±0.3
44Mistral Small 3.2 24B55.0% ±0.3
44Cogito V2 Preview Llama 70B55.0% ±0.2
44Qwen3 32B No Thinkingno-thinking effort55.0% ±0.2
48Gemini 3.5 Flash-Lite54.3% ±0.5
48Llama 3.3 70B Instruct54.3% ±0.2
50GPT-5.6 Luna54.2% ±0.4
51GPT-4.154.1% ±0.5
51Ministral 3 14B Base54.1% ±0.7
53GLM-4.7-Flash53.5% ±0.6
54GPT-5.4 Mini Nonenone effort53.4% ±0.5
55NVIDIA Nemotron 3 Nano 30B A3B BF1653.3% ±0.7
56GPT-5 nano52.8% ±0.5
57Qwen3.5 9B52.7% ±0.5
58NVIDIA Nemotron 3 Nano 30B A3B FP852.5% ±0.5
59GPT-5.4 nanomedium effort52.4% ±0.6
60Bielik 11B V3.0 Instruct52.1%
61Gemma 3 27BAbove fitFar above this model's index.51.8% ±0.3
62Qwen3 4B Thinking51.4% ±0.3
63GPT-5.4 nanolow effort50.7% ±0.4
64Cogito V1 Preview Qwen 14B50.4% ±0.3
65Ministral 3 8B Base50.3% ±0.6
66GPT-OSS 20Bmedium effort50.2% ±0.3
67Qwen3 14B No Thinkingno-thinking effort50.1% ±0.3
68BgGPT Gemma 3 12B IT50.0% ±0.7
69Gemma 3 12B Pt49.7% ±0.5
70Claude Haiku 4.549.5% ±0.6
71Ministral 3 14B Instruct49.1% ±0.7
72GPT-OSS 20Blow effort48.6% ±0.3
73Olmo 3.1 32B Think48.4% ±0.6
74BgGPT Gemma 2 9B IT V1.048.0% ±0.6
74Bielik Minitron 7B V3.0 Instruct48.0%
76Gemma 3 12B47.7% ±0.3
77MamayLM Gemma 2 9B IT V0.147.6% ±0.7
78Gemma 4 31Bgemma-4-31B46.7% ±0.9
79Ministral 3 14B (Reasoning)46.3% ±0.8
79GaMS3 12B46.3% ±0.8
81Gemma 4 E4B IT46.2% ±0.7
82Ministral 3 8B Instruct46.0% ±0.6
83PLLuM 12B Nc Instruct 250745.9% ±0.6
84GaMS3 12B Instruct45.7% ±0.8
85NVIDIA Nemotron 3 Nano 30B A3B Base BF1645.4% ±0.6
86Qwen3 8B No Thinkingno-thinking effort45.3% ±0.2
86EuroLLM 22B Instruct 251245.3% ±0.4
86PLLuM 12B Instruct 241245.3% ±0.5
89Ministral 3 8B (Reasoning)44.9% ±0.3
89PLLuM 12B Nc Chat 250744.9% ±0.6
91PLLuM 8x7B Instruct 241244.6% ±0.6
92PLLuM 12B Chat 241244.4% ±0.6
93Gemma 3N E4B IT44.3% ±0.3
94Qwen3.5-4B44.2% ±0.8
95Qwen3.5 9B Base44.0% ±1.0
96PLLuM 12B Nc Instruct 241243.6% ±0.7
97Nesso 4B43.5% ±0.3
97Qwen3.5 4B Base43.5% ±0.7
97GPT-OSS 20Bhigh effort43.5% ±0.3
97PLLuM 12B Nc Chat 241243.5% ±0.7
101Gemma 4 12B43.1% ±0.8
102MamayLM Gemma 3 4B IT V1.042.3% ±0.7
103Domyn Small V1.041.7% ±0.5
104Llama PLLuM 8B Chat 241241.6% ±0.5
105Llama 3.1 8B41.4% ±0.4
106Exaone 4.0 32B41.3% ±0.6
106Qwen3 4B No Thinkingno-thinking effort41.3% ±0.2
108Olmo 3.1 32B Instruct SFT41.2% ±0.5
109Llama PLLuM 8B Instruct 241241.1% ±0.5
110Aya Expanse 8B40.8% ±0.5
110Apertus V1.5 70B40.8% ±0.6
110Llama 3.1 8B Instruct40.8% ±0.3
113EuroLLM 9B Instruct40.7% ±0.3
114BgGPT Gemma 3 4B IT40.4% ±0.6
115Olmo 3.1 32B Instruct DPO40.3% ±0.3
116Ministral 3 3B Instruct39.2% ±0.6
117EuroLLM 9B Instruct 251239.1% ±0.7
118Olmo 3.1 32B Instruct39.0% ±0.4
119Gemma 4 E2B IT38.9% ±0.8
120Ministral 3 3B (Reasoning)38.7% ±0.3
121Gemma 4 26B A4Bgemma-4-26B-A4B38.5% ±0.9
122Cogito V1 Preview Llama 8B38.4% ±0.3
123Llama PLLuM 8B Base 241238.0% ±0.8
124GPT-5.4 Nano Nonenone effort37.8% ±0.5
125Granite 4.0 Microgranite-4.0-micro37.3% ±0.3
126Gemma 3 4B Pt37.0% ±0.3
127AMALIA 9B SFT36.8% ±0.5
128EuroLLM 9B36.6% ±0.6
129AMALIA 9B DPO36.5% ±0.7
130Llama 3.1 Tulu 3 8B SFT36.3% ±0.4
131Apertus V1.1 4B35.8% ±0.9
132EuroLLM 22B 251235.0% ±0.5
133Gemma 3 4B34.8% ±0.3
134BgGPT 7B Instruct V0.234.7% ±0.8
135Qwen3.5 2B34.6% ±0.7
136Bielik 1.5B V3.0 Instruct34.1% ±0.3
136Olmo 3 32B34.1% ±0.4
138Olmo 3 7B33.4% ±0.4
139Gemini 2.5 Flash Lite No Thinkingno-thinking effort33.3% ±0.4
140LFM2.5 8B A1B Base32.9% ±0.8
141Occiglot 7B Eu5 Instruct32.6% ±0.3
142Granite 4.0 Microgranite-4.0-h-micro32.0% ±0.3
143Llama 3.2 3B Instruct31.6% ±0.3
144SmolLM3 3B Base31.4% ±0.5
144NVIDIA Nemotron 3 Nano 4B BF1631.4% ±0.4
146YugoGPT31.1% ±0.9
146Qwen3 1.7B No Thinkingno-thinking effort31.1% ±0.3
148BgGPT 7B Instruct V0.131.0% ±0.6
149Luciole 23B Instruct 1.129.8% ±0.8
150Occiglot 7B Eu529.4% ±0.5
151Oellm 9B 128K Theta32m V328.6% ±0.5
152Aya 23 8B28.4% ±0.8
153DFM Mimir V1.528.2% ±0.7
153Qwen3.5 0.8B28.2% ±0.6
155Apertus V1.1 4B Instruct28.1% ±0.4
156Cogito V1 Preview Llama 3B27.8% ±0.4
157Olmo 3 7B Instruct27.6% ±0.4
158NVIDIA Nemotron 3 Nano 4B FP827.5% ±0.8
159Salamandra 7B Instruct26.5% ±0.4
159Luciole 23B Base26.5% ±0.7
161Olmo 3 7B Instruct DPO26.3% ±0.4
162DFM Mimir25.4% ±0.6
163Gemma 4 E4B25.2% ±1.0
164Luciole 8B Instruct 1.124.3% ±0.7
165Olmo 3 7B Instruct SFT24.0% ±0.4
166Teuken 7B Instruct V0.623.9% ±0.5
167Luciole 8B Base22.5% ±1.2
168Qwen3.5 0.8B Base21.7% ±0.9
169Gemma 3 1B IT21.4% ±0.4
170Gemma 3 1B Pt20.9% ±0.5
171YuLan Mini Instruct20.3% ±0.4
171Apertus V1.1 1.5B20.3% ±0.9
173Gemma 4 E2B20.2% ±0.9
173Qwen3 0.6B No Thinkingno-thinking effort20.2% ±0.6
175Llama 3.2 1B19.9% ±0.5
176Apertus V1.1 0.5B Instruct17.6% ±1.0
177Apertus V1.1 1.5B Instruct17.4% ±0.8
178EuroLLM 1.7B Instruct16.7% ±1.2
179Salamandra 7B15.4% ±0.5
180EuroMoE 2.6B A0.6B Instruct Preview15.2% ±0.6
181Apertus V1.1 0.5B14.0% ±0.9
182Luciole 1B Instruct 1.112.5% ±0.7
183Pleias 1.2B Preview12.1% ±0.6
184Luciole 1B Base11.9% ±1.0
185SambaLingo Bulgarian Base11.7% ±0.3
186EuroMoE 2.6B A0.6B Preview11.4% ±0.6
187LFM2.5-8B-A1B10.8% ±0.8
188BgGPT Gemma 2 27B IT V1.010.7% ±1.0
189EuroLLM 1.7B10.5% ±0.3
190Pleias 3B Preview10.4% ±0.6
191Gemma 3 270M10.3% ±0.9
192LFM2 1.2B10.2% ±0.6
193Llama 3.2 1B Instruct9.7% ±0.4
194LFM2 700M8.5% ±0.4
195Gemma 3 270M IT8.1% ±0.5
196Pleias 350M Preview7.7% ±0.5
197Falcon H1 Tiny Multilingual 100M Instruct6.9% ±0.2
198SmolLM2 360M Instruct6.5% ±0.5
199LFM2 350M6.3% ±0.4
200SmolLM2 360M6.2% ±0.5
201Falcon H1 Tiny Multilingual 100M Base6.0% ±0.5
202Baguettotron No Thinkingno-thinking effort5.1% ±0.3
203SmolLM2 135M Instruct4.5% ±0.4
204SmolLM2 135M4.4% ±0.5
205Marco Mini Global Base1.9% ±0.5
206Monad1.5% ±0.3
207Muse 3B0.9% ±0.9
208Ling Mini 2.00.7% ±0.8
209Teuken 7B Base V0.60.0%