Inference

Scientific Code Benchmark

SciCode evaluates language models on generating code for realistic scientific research problems across 16 subfields of physics, math, chemistry, biology, and material science. Problems decompose into 338 subproblems requiring domain knowledge recall, scientific reasoning, and precise code synthesis. Based on real scripts from published research.

  • In index
  • Coding
ModelsModels with a result.27
Top resultBest result on this test.60.1%Sakana Fugu
Top-3 spreadPoints from first to third.1.4 pts
YearYear of release.2024

Result and price

020406080100$0.03$0.1$0.3$1$3$10$30RESULTOUTPUT PRICE PER 1M TOKENS · LOG SCALE

Results

27 results
#ModelResultThe score from the source.
01Sakana Fugu60.1%
02Step 5 Preview58.9%
03Sakana Fugu-Ultra58.7%
04Qwen3.7 Max53.5%
05Gemini 3.5 Flash53.1%
06Kimi K2.652.2%
07Qwen3.7 Plus51.3%
08Inkling-Small48.7%
08Kimi K2.548.7%
10Grok 4.347.3%
11Qwen 3.6 Max (preview)47.0%
12Nemotron 3 Ultra44.6%
13Muse Glimmer 30B43.6%
14Ling 3.0 Flash41.2%
15Hy3 Preview41.2%
16A.X K241.0%
17Ling 3.0 Flash FP840.4%
18Granite 4.2 30B38.8%
19Mercury 2.538.0%
20K-EXAONE 2.037.4%
21Granite 4.2 8B36.1%
22Nemotron 3 Nano Omni 30B A3B32.0%
23Nemotron 3.5 Lightning 30B A3B NVFP431.4%
24Agents-A1-4B29.6%
25Ling 2.6 Flash27.0%
26MiniCPM5-2B26.3%
27Granite 4.2 3B24.1%