Multilingual Grade School Math

A multilingual benchmark that translates 250 grade school math problems from GSM8K into 10 typologically diverse languages: Bengali, German, Spanish, French, Japanese, Russian, Swahili, Telugu, Thai, and Chinese.

ModelsModels with a result.13
Top resultBest result on this benchmark.93.7%o4-mini
Top-3 spreadPoints from first to third.3.8 pts
YearYear of release.2022

Results

13 results
#ModelResultThe score from the source.
01o4-mini93.7%
03o1-mini89.9%
04GPT-4.586.9%
07Phi-480.6%