Inference
EuroEval

EuroEval French

How well a model handles French, as the unweighted mean of EuroEval's primary metric over its 8 French datasets (allocine, eltec, scala_fr, fquad, orange_sum, include_fr, multiloko_fr, hellaswag_fr). The primary metric differs by task — MCC for classification, knowledge and common-sense reasoning, micro-F1 without MISC for entity recognition, F1 for reading comprehension, chrF++ for summarization, METEOR for simplification — and is always the first of the two the board prints. Only models measured on every dataset are included, so each mean covers the same ground. Most rows are the validation split; some older open-weight rows are the test split, which EuroEval ranks in the same table.

ModelsModels with a result.163
Top resultBest result on this test.77.1%Gemini 3.7 Flash
Top-3 spreadPoints from first to third.1.5 pts
YearYear of release.N/A

Result and price

020406080100$0.03$0.1$0.3$1$3$10$30$100RESULTOUTPUT PRICE PER 1M TOKENS · LOG SCALE

Results

171 results
#ModelResultThe score from the source.
01Gemini 3.7 Flash77.1% ±0.3
02GPT-6 Astra76.3% ±0.4
03Gemini 3.6 Flash75.6% ±0.3
04Gemini 3 Flash Preview Thinkingthinking effort74.4% ±0.4
05GLM-5.3-Flash73.5% ±0.4
06GPT-5.6 Sol72.0% ±0.4
07GPT-5.6 Luna68.9% ±0.4
08GPT-5.6 Terra68.9% ±0.4
09Claude Sonnet 4.668.5% ±0.4
10Gemma 4 31B67.9% ±0.4
11Gemini 3.1 Flash Lite Preview67.5% ±0.4
12GPT-5.4 minihigh effort67.4% ±0.4
13GPT-5.4 minimedium effort67.3% ±0.4
14GPT-5.4 Nonenone effort67.1% ±0.5
15GPT-5.4 minilow effort67.1% ±0.4
16Qwen3.6 27B FP867.0% ±0.5
17Claude Sonnet 4.5 Thinkingthinking effort66.9% ±0.4
18Qwen3.8-27B66.8% ±0.4
19Claude Sonnet 4.5 No Thinkingno-thinking effort65.6% ±0.5
20Grok 4.1 Fast (Reasoning)65.4% ±0.4
21Gemma 4 26B A4B65.2% ±0.5
22Qwen3.5 35B A3B FP864.2% ±0.5
23Mistral Small 3.2 24B63.3% ±0.3
24Mistral Small 3.1 24B62.9% ±0.3
25GPT-4.162.3% ±0.3
26Mistral Small 3.1 24B Base62.1% ±0.4
27GPT-5.262.0% ±0.4
28Gemma 4 12B IT61.9% ±0.5
29Cogito V1 Preview Qwen 32B61.8% ±0.3
30Gemini 3.5 Flash-Lite61.6% ±0.3
31GPT-5.4 Mini Nonenone effort61.4% ±0.4
32Qwen2.5 72B Instruct61.2% ±0.3
33GPT-5 mini60.1% ±0.4
34Llama 3.3 70B Instruct60.1% ±0.3
35Cogito V2 Preview Llama 70B59.8% ±0.3
36Ministral 3 14B Base59.7% ±0.5
37Ministral 3 8B (Reasoning)59.2% ±0.4
38Cogito V1 Preview Qwen 14B58.8% ±0.3
39Ministral 3 14B Instruct58.5% ±0.5
40Llama 3.1-70B58.4% ±0.5
41Gemini 2.5 Flash Lite Thinkingthinking effort58.4% ±0.4
42Gemini 3 Flash Preview No Thinkingno-thinking effort58.4% ±0.4
43GPT-OSS 120B58.4% ±0.4
44NVIDIA Nemotron 3 Nano 30B A3B BF1658.0% ±0.5
45Gemma 3 27B Pt58.0% ±0.5
46NVIDIA Nemotron 3 Nano 30B A3B FP857.7% ±0.5
47Claude Haiku 4.557.6% ±0.4
48Gemma 3 27B57.5% ±0.3
49Ministral 3 8B Base57.3% ±0.6
50Ministral 3 14B (Reasoning)57.2% ±0.6
51Qwen3 32B No Thinkingno-thinking effort57.2% ±0.4
52GLM-4.7-Flash57.1% ±0.6
53Qwen3 14B No Thinkingno-thinking effort57.1% ±0.4
54Ministral 3 8B Instruct56.9% ±0.5
55Qwen3.5 9B55.5% ±0.6
56Gemma 3 12B55.4% ±0.3
57Qwen3 4B Thinking55.3% ±0.3
58Bielik 11B V3.0 Instruct55.0%
59Nemotron Cascade 2 30B A3B54.9% ±0.5
60GPT-5.4 nanomedium effort54.9% ±0.5
61GPT-5.4 nanohigh effort54.8% ±0.5
62Apertus V1.5 70B54.7% ±0.5
63Gemma 3 12B Pt54.5% ±0.4
64EuroLLM 22B Instruct54.0% ±0.5
65NVIDIA Nemotron 3 Nano 30B A3B Base BF1653.9% ±0.7
66Exaone 4.0 32B53.9% ±0.6
67GPT-5 nano53.9% ±0.5
68Gemma 4 E4B IT53.7% ±0.5
69Gemini 2.5 Flash Lite No Thinkingno-thinking effort53.1% ±0.5
70GPT-5.4 nanolow effort53.0% ±0.5
71GPT-OSS 20Bmedium effort52.9% ±0.3
72Bielik Minitron 7B V3.0 Instruct52.6%
73Qwen3 8B No Thinkingno-thinking effort52.5% ±0.4
74Llama 3.1 70B LatamGPT SFT-1.052.2% ±0.6
75Nesso 4B52.1% ±0.4
76GPT-OSS 20Blow effort51.7% ±0.3
77Qwen3.5 4B51.5% ±0.5
78Gemma 4 31B50.0% ±0.8
79Luciole 23B Instruct 1.150.0% ±0.6
80Qwen3 4B No Thinkingno-thinking effort49.4% ±0.3
81Ministral 3 3B (Reasoning)48.7% ±0.4
82Ministral 3 3B Instruct48.5% ±0.4
83Gemma 4 E2B IT47.5% ±0.6
84Aya Expanse 8B47.4% ±0.6
85Cogito V1 Preview Llama 8B47.1% ±0.6
86EuroLLM 9B Instruct47.0% ±0.5
87Qwen3.5 9B Base46.7% ±1.0
88Llama 3.1 8B Instruct46.7% ±0.4
89Qwen3.5 2B46.1% ±0.6
90Llama 3.1-8B45.9% ±0.5
91Llama 3.1 Tulu 3 8B SFT45.7% ±0.4
92GPT-5.4 Nano Nonenone effort45.5% ±0.5
93GPT-OSS 20Bhigh effort44.8% ±0.6
94Gemma 4 26B A4B44.4% ±1.0
95Gemma 3 4B44.4% ±0.6
96EuroLLM 22B43.9% ±0.4
97EuroLLM 9B43.7% ±0.4
98SmolLM3 3B Base43.2% ±0.5
99Gemma 4 12B42.8% ±0.9
100Qwen3 1.7B No Thinkingno-thinking effort42.6% ±0.3
101Luciole 8B Instruct 1.142.6% ±0.5
102AMALIA 9B DPO42.5% ±0.5
103LFM2.5 8B A1B Base42.1% ±0.5
104Gemma 3 4B Pt41.9% ±0.5
105AMALIA 9B SFT40.7% ±0.5
106DFM Mimir40.2% ±0.4
107NVIDIA Nemotron 3 Nano 4B BF1639.4% ±0.4
108NVIDIA Nemotron 3 Nano 4B FP839.3% ±0.6
109Llama 3.2 3B Instruct39.2% ±0.4
110Apertus V1.1 4B38.7% ±0.7
111SmolLM3 3B No Thinkingno-thinking effort38.0% ±0.5
112Occiglot 7B Eu5 Instruct37.7% ±0.3
113Occiglot 7B Eu537.4% ±0.4
114Lucie 7B Instruct V1.137.0% ±0.6
115Qwen3.5 4B Base37.0% ±0.8
116Luciole 23B Base37.0% ±0.7
117Bielik 1.5B V3.0 Instruct36.0% ±0.3
118Domyn Small V1.035.9% ±1.2
119Apertus V1.1 4B Instruct35.6% ±0.5
120Cogito V1 Preview Llama 3B34.1% ±0.5
121Oellm 9B 128K Theta32m V333.7% ±0.7
122Aya 23 8B33.0% ±0.7
123Luciole 8B Base32.5% ±0.6
124Qwen3.5 0.8B31.7% ±0.5
125YuLan Mini Instruct31.6% ±0.3
126Lucie 7B Instruct Human Data30.5% ±0.7
127Apertus V1.1 1.5B30.4% ±0.6
128Gemma 4 E4B30.1% ±0.9
129LFM2.5-8B-A1B29.6% ±0.8
130Qwen3.5 2B Base29.3% ±0.8
131Apertus V1.1 1.5B Instruct27.0% ±0.6
132Teuken 7B Instruct V0.626.1% ±0.7
133Luciole 1B Instruct 1.125.8% ±0.5
134Salamandra 7B25.2% ±0.4
135LFM2 1.2B24.5% ±0.3
136TildeOpen 30B24.4% ±0.5
137Gemma 4 E2B24.3% ±0.8
138Gemma 3 1B Pt24.3% ±0.4
139Llama 3.2 1B24.2% ±0.3
140Gemma 3 1B IT24.0% ±0.4
141Qwen3.5 0.8B Base23.8% ±0.8
142Salamandra 7B Instruct23.7% ±1.0
143Qwen3 0.6B No Thinkingno-thinking effort23.6% ±1.0
144LFM2 700M23.2% ±0.3
145Pleias 3B Preview22.9% ±0.5
146Apertus V1.1 0.5B22.0% ±0.6
147Pleias 1.2B Preview21.6% ±0.6
148EuroMoE 2.6B A0.6B Instruct Preview20.9% ±0.7
149Luciole 1B Base20.8% ±0.8
150SmolLM2 360M17.5% ±0.6
151Apertus V1.1 0.5B Instruct17.2% ±0.5
152Gemma 3 270M16.4% ±0.8
153Llama 3.2 1B Instruct15.7% ±0.6
154Nesso 0.4B Agentic15.6% ±0.9
155Falcon H1 Tiny Multilingual 100M Instruct14.5% ±0.5
156LFM2 350M14.1% ±0.6
157Falcon H1 Tiny Multilingual 100M Base13.8% ±0.6
158Pleias 350M Preview13.7% ±0.4
159SmolLM2 360M Instruct12.6% ±0.6
160MiniCPM5-1B11.7% ±0.9
161Gemma 3 270M IT10.9% ±0.7
162SmolLM2 135M9.8% ±0.5
163SmolLM2 135M Instruct8.5% ±0.5
164Zagreus 0.4B Ita8.2% ±0.7
165Ling Mini 2.07.6% ±0.8
166Ling Mini Base 2.05.8% ±0.9
1676 No Thinkingno-thinking effort4.7% ±0.3
168Marco Mini Global Base3.6% ±0.4
169Monad1.8% ±0.2
170Muse 3B0.6% ±0.6
171Teuken 7B Base V0.60.0%