Inference
EuroEval

EuroEval Polish

How well a model handles Polish, as the unweighted mean of EuroEval's primary metric over its 7 Polish datasets (polemo2, kpwr_ner, scala_pl, poquad, psc, llmzszl, winogrande_pl). The primary metric differs by task — MCC for classification, knowledge and common-sense reasoning, micro-F1 without MISC for entity recognition, F1 for reading comprehension, chrF++ for summarization, METEOR for simplification — and is always the first of the two the board prints. Only models measured on every dataset are included, so each mean covers the same ground. Most rows are the validation split; some older open-weight rows are the test split, which EuroEval ranks in the same table.

ModelsModels with a result.189
Top resultBest result on this test.68.1%GLM-5.3-Flash
Top-3 spreadPoints from first to third.0.7 pts
YearYear of release.N/A

Result and price

020406080100$0.03$0.1$0.3$1$3$10$30$100RESULTOUTPUT PRICE PER 1M TOKENS · LOG SCALE

Results

198 results
#ModelResultThe score from the source.
01GLM-5.3-Flash68.1% ±0.4
02Gemini 3.7 Flash67.9% ±0.4
03GPT-6 Astra67.4% ±0.4
04Gemini 3.6 Flash65.9% ±0.4
05Qwen3.8-27B65.4% ±0.5
06Qwen3.6-27B65.3% ±0.5
07Qwen3.6 27B FP865.0% ±0.4
08Gemini 3 Pro64.8% ±0.5
09Claude Sonnet 4.5 Thinkingthinking effort63.8% ±0.3
10GPT-5.6 Sol62.8% ±0.4
11GPT-5.4 minihigh effort62.5% ±0.4
12Qwen3.5 35B A3B FP862.3% ±0.6
13Gemini 3 Flash Preview No Thinkingno-thinking effort62.3% ±0.5
14Gemma 4 31B61.6% ±0.4
15Gemini 3 Flash Preview Thinkingthinking effort61.6% ±0.4
16Gemini 3.1 Flash Lite Preview61.0% ±0.4
17GPT-5.4 minimedium effort60.2% ±0.4
18Claude Sonnet 4.659.9% ±0.5
19GPT-5.4 Nonenone effort59.5% ±0.5
20GPT-5.6 Terra59.0% ±0.5
21GPT-5.4 minilow effort58.8% ±0.5
22Claude Sonnet 4.5 No Thinkingno-thinking effort58.7% ±0.4
23Grok 4.1 Fast (Reasoning)57.6% ±0.4
24GPT-OSS 120B57.6% ±0.5
25GPT-5 mini57.2% ±0.3
26GPT-5.257.1% ±0.4
27Qwen3.5 35B A3B Base57.0% ±0.5
28Llama 3.1-70B56.5% ±0.5
29Gemini 2.5 Flash Lite Thinkingthinking effort56.4% ±0.5
30Mistral Small 3.1 24B Base56.3% ±0.4
31Cogito V1 Preview Qwen 32B56.3% ±0.2
32Qwen2.5 72B Instruct55.8% ±0.2
33Gemma 3 27B Pt55.7% ±0.4
34Gemma 4 26B A4B55.5% ±0.7
35BgGPT Gemma 3 27B IT55.3% ±0.6
36GPT-5.4 nanohigh effort55.2% ±0.5
37Gemma 4 12B IT55.2% ±0.6
38Voxtral Small 24B55.1% ±0.2
39Mistral Small 3.1 24B55.0% ±0.3
40Mistral Small 3.2 24B55.0% ±0.3
41Cogito V2 Preview Llama 70B55.0% ±0.2
42Qwen3 32B No Thinkingno-thinking effort55.0% ±0.2
43Gemini 3.5 Flash-Lite54.3% ±0.5
44Llama 3.3 70B Instruct54.3% ±0.2
45GPT-5.6 Luna54.2% ±0.4
46GPT-4.154.1% ±0.5
47Ministral 3 14B Base54.1% ±0.7
48GLM-4.7-Flash53.5% ±0.6
49GPT-5.4 Mini Nonenone effort53.4% ±0.5
50NVIDIA Nemotron 3 Nano 30B A3B BF1653.3% ±0.7
51GPT-5 nano52.8% ±0.5
52Qwen3.5 9B52.7% ±0.5
53NVIDIA Nemotron 3 Nano 30B A3B FP852.5% ±0.5
54GPT-5.4 nanomedium effort52.4% ±0.6
55Bielik 11B V3.0 Instruct52.1%
56Gemma 3 27B51.8% ±0.3
57Qwen3 4B Thinking51.4% ±0.3
58GPT-5.4 nanolow effort50.7% ±0.4
59Cogito V1 Preview Qwen 14B50.4% ±0.3
60Ministral 3 8B Base50.3% ±0.6
61GPT-OSS 20Bmedium effort50.2% ±0.3
62Qwen3 14B No Thinkingno-thinking effort50.1% ±0.3
63BgGPT Gemma 3 12B IT50.0% ±0.7
64Gemma 3 12B Pt49.7% ±0.5
65Claude Haiku 4.549.5% ±0.6
66Ministral 3 14B Instruct49.1% ±0.7
67GPT-OSS 20Blow effort48.6% ±0.3
68Olmo 3.1 32B Think48.4% ±0.6
69BgGPT Gemma 2 9B IT V1.048.0% ±0.6
70Bielik Minitron 7B V3.0 Instruct48.0%
71Gemma 3 12B47.7% ±0.3
72MamayLM Gemma 2 9B IT V0.147.6% ±0.7
73Gemma 4 31B46.7% ±0.9
74Ministral 3 14B (Reasoning)46.3% ±0.8
75GaMS3 12B46.3% ±0.8
76Gemma 4 E4B IT46.2% ±0.7
77Ministral 3 8B Instruct46.0% ±0.6
78GaMS3 12B Instruct45.7% ±0.8
79NVIDIA Nemotron 3 Nano 30B A3B Base BF1645.4% ±0.6
80Qwen3 8B No Thinkingno-thinking effort45.3% ±0.2
81EuroLLM 22B Instruct45.3% ±0.4
82PLLuM 12B Instruct45.3% ±0.5
83Ministral 3 8B (Reasoning)44.9% ±0.3
84PLLuM 12B Nc Instruct44.7% ±0.6
85PLLuM 8x7B Instruct44.6% ±0.6
86PLLuM 12B Chat44.4% ±0.5
87Gemma 3N E4B IT44.3% ±0.3
88Qwen3.5 4B44.2% ±0.8
89PLLuM 12B Nc Chat44.2% ±0.6
90Qwen3.5 9B Base44.0% ±0.9
91Nesso 4B43.5% ±0.3
92Qwen3.5 4B Base43.5% ±0.7
93GPT-OSS 20Bhigh effort43.5% ±0.3
94Gemma 4 12B43.1% ±0.7
95MamayLM Gemma 3 4B IT V1.042.3% ±0.7
96Domyn Small V1.041.7% ±0.5
97Llama PLLuM 8B Chat41.6% ±0.5
98Llama 3.1-8B41.4% ±0.4
99Exaone 4.0 32B41.3% ±0.6
100Qwen3 4B No Thinkingno-thinking effort41.3% ±0.2
101Olmo 3.1 32B Instruct SFT41.2% ±0.5
102Llama PLLuM 8B Instruct41.1% ±0.5
103Aya Expanse 8B40.8% ±0.5
104Apertus V1.5 70B40.8% ±0.6
105Llama 3.1 8B Instruct40.8% ±0.3
106BgGPT Gemma 3 4B IT40.4% ±0.6
107Olmo 3.1 32B Instruct DPO40.3% ±0.3
108EuroLLM 9B Instruct39.9% ±0.5
109Ministral 3 3B Instruct39.2% ±0.6
110Olmo 3.1 32B Instruct39.0% ±0.4
111Gemma 4 E2B IT38.9% ±0.7
112Ministral 3 3B (Reasoning)38.7% ±0.3
113Gemma 4 26B A4B38.5% ±0.8
114Cogito V1 Preview Llama 8B38.4% ±0.3
115Llama PLLuM 8B Base38.0% ±0.8
116GPT-5.4 Nano Nonenone effort37.8% ±0.5
117Granite 4.0 Micro37.3% ±0.3
118Gemma 3 4B Pt37.0% ±0.3
119EuroLLM 9B36.6% ±0.6
120Llama 3.1 Tulu 3 8B SFT36.3% ±0.4
121Apertus V1.1 4B35.8% ±0.8
122EuroLLM 22B35.0% ±0.5
123Gemma 3 4B34.8% ±0.3
124BgGPT 7B Instruct V0.234.7% ±0.7
125Qwen3.5 2B34.6% ±0.7
126Bielik 1.5B V3.0 Instruct34.1% ±0.3
127Olmo 3 32B34.1% ±0.4
128Olmo 3 7B33.4% ±0.4
129Gemini 2.5 Flash Lite No Thinkingno-thinking effort33.3% ±0.3
130LFM2.5 8B A1B Base32.9% ±0.8
131Occiglot 7B Eu5 Instruct32.6% ±0.3
132Granite 4.0 Micro32.0% ±0.3
133Llama 3.2 3B Instruct31.6% ±0.3
134SmolLM3 3B Base31.4% ±0.5
135NVIDIA Nemotron 3 Nano 4B BF1631.4% ±0.4
136YugoGPT31.1% ±0.7
137Qwen3 1.7B No Thinkingno-thinking effort31.1% ±0.3
138BgGPT 7B Instruct V0.131.0% ±0.5
139Luciole 23B Instruct 1.129.8% ±0.7
140Occiglot 7B Eu529.4% ±0.4
141Oellm 9B 128K Theta32m V328.6% ±0.5
142Aya 23 8B28.4% ±0.7
143Qwen3.5 0.8B28.2% ±0.5
144Apertus V1.1 4B Instruct28.1% ±0.4
145Cogito V1 Preview Llama 3B27.8% ±0.4
146Olmo 3 7B Instruct27.6% ±0.4
147NVIDIA Nemotron 3 Nano 4B FP827.5% ±0.7
148Salamandra 7B Instruct26.5% ±0.4
149Luciole 23B Base26.5% ±0.6
150Olmo 3 7B Instruct DPO26.3% ±0.4
151DFM Mimir25.4% ±0.6
152Gemma 4 E4B25.2% ±1.0
153Luciole 8B Instruct 1.124.3% ±0.7
154Olmo 3 7B Instruct SFT24.0% ±0.4
155Teuken 7B Instruct V0.623.9% ±0.5
156Luciole 8B Base22.5% ±1.2
157Qwen3.5 0.8B Base21.7% ±0.7
158Gemma 3 1B IT21.4% ±0.4
159Gemma 3 1B Pt20.9% ±0.5
160YuLan Mini Instruct20.3% ±0.4
161Apertus V1.1 1.5B20.3% ±0.8
162Gemma 4 E2B20.2% ±0.9
163Qwen3 0.6B No Thinkingno-thinking effort20.2% ±0.5
164Llama 3.2 1B19.9% ±0.5
165Apertus V1.1 0.5B Instruct17.6% ±0.9
166Apertus V1.1 1.5B Instruct17.4% ±0.7
167EuroLLM 1.7B Instruct16.7% ±1.2
168Salamandra 7B15.4% ±0.5
169EuroMoE 2.6B A0.6B Instruct Preview15.2% ±0.5
170Apertus V1.1 0.5B14.0% ±0.9
171Luciole 1B Instruct 1.112.5% ±0.6
172Pleias 1.2B Preview12.1% ±0.6
173Luciole 1B Base11.9% ±0.7
174SambaLingo Bulgarian Base11.7% ±0.3
175EuroMoE 2.6B A0.6B Preview11.4% ±0.6
176LFM2.5-8B-A1B10.8% ±0.7
177BgGPT Gemma 2 27B IT V1.010.7% ±0.9
178EuroLLM 1.7B10.5% ±0.3
179Pleias 3B Preview10.4% ±0.5
180Gemma 3 270M10.3% ±0.8
181LFM2 1.2B10.2% ±0.6
182Llama 3.2 1B Instruct9.7% ±0.3
183LFM2 700M8.5% ±0.4
184Gemma 3 270M IT8.1% ±0.5
185Pleias 350M Preview7.7% ±0.4
186Falcon H1 Tiny Multilingual 100M Instruct6.9% ±0.2
187SmolLM2 360M Instruct6.5% ±0.4
188LFM2 350M6.3% ±0.3
189SmolLM2 360M6.2% ±0.5
190Falcon H1 Tiny Multilingual 100M Base6.0% ±0.4
1916 No Thinkingno-thinking effort5.1% ±0.2
192SmolLM2 135M Instruct4.5% ±0.4
193SmolLM2 135M4.4% ±0.5
194Marco Mini Global Base1.9% ±0.5
195Monad1.5% ±0.2
196Muse 3B0.9% ±0.9
197Ling Mini 2.00.7% ±0.7
198Teuken 7B Base V0.60.0%