Inference
EN
Xiang Deng et al.

SWE-bench Pro

A long-horizon repository benchmark built to test realistic software engineering work. Its scores need a task-quality and setup check before they support a coding-agent decision.

ModelsModels with a result.83
Top resultBest result on this benchmark.93.0%Claude Fable 5.2
Top-3 spreadPoints from first to third.11.7 pts
YearYear of release.2025

Result and price

55 of 83 results

Results

83 results
#ModelResultThe score from the source.
01Claude Fable 5.293.0%
02Claude Opus 5.589.9%
03Claude Sonnet 5.581.3%
04Claude Fable 5.181.2%
05Claude Mythos 580.3%
06Claude Fable 580.0%
07Claude Opus 579.2%
08Sakana Fugu-Ultra73.7%
09Claude Opus 4.869.2%
10Ling 3.1 Flash68.7%
11Qwen3.8 Max67.7%
12Beam 1.0 501B-A23B66.1%
13Beam 1.0 501B-A23B FP865.9%
14Hy4 preview65.7%
15Beam 1.0 501B-A23B NVFP465.6%
16Ornith-1.5-397B65.1%
17Grok 4.564.7%
18GPT-5.6 Sol64.6%
19Claude Opus 4.7 (Adaptive)64.3%
20GPT-5.6 Terra63.4%
21Qwen3.8-Omni-Flash63.3%
22Claude Sonnet 563.2%
23GPT-5.6 Luna62.7%
24Qwen3.8-Flash-Next62.5%
25Ornith-1.0-397B62.2%
26GLM-5.262.1%
27Qwen3.8-27B61.7%
28Muse Spark 1.161.5%
29Ripple-65B Preview61.4%
30Sage-A1.561.2%
31dots3-note Preview61.0%
32Qwen3.7 Max60.6%
33Atria Dawn Preview59.6%
33Ornith-1.5-35B-A3B59.6%
35Laguna S 2.159.4%
36Sakana Fugu59.0%
36MiniMax M359.0%
38GPT-5.558.6%
38Kimi K2.658.6%
40GLM-5.158.4%
41GPT-5.457.7%
42Qwen3.7 Plus57.6%
43Qwen 3.6 Max (preview)57.3%
44MiMo-V2.5-Pro57.2%
45Claude Opus 4.557.1%
46GPT-5.3 Codex56.8%
47Qwen3.6 Plus56.6%
47Ling 3.0 Flash56.6%
49Step 3.7 Flash56.3%
50MiniMax M2.756.2%
51MiMo-V2.556.1%
52Inkling-Small55.9%
53GPT-5.255.6%
54DeepSeek V4 Pro 081355.4%
55Gemini 3.5 Flash55.1%
55GLM-555.1%
57Inkling54.3%
57Spark-X2.6-4B54.3%
59Gemini 3.5 Flash-Lite54.2%
60Qwen3.6-27B53.5%
61Claude Opus 4.653.4%
62MAI-Thinking-152.8%
63DeepSeek V4 Flash 073152.6%
64Muse Spark52.4%
65Grok 4.2051.8%
65Nemotron 3.5 Super VL 120B A12B51.8%
67Nemotron 3.5 Super VL 120B A12B NVFP451.3%
68Muse Glimmer 30B51.2%
69Qwen3.5 397B50.9%
70Kimi K2.550.7%
71Ornith-1.0-35B50.4%
72Qwen3.6-35B-A3B49.5%
73Laguna M.149.2%
74Laguna XS 2.147.6%
75Ornith-1.5-9B47.5%
76Laguna XS.246.3%
77Ornith-1.0-9B42.9%
78LongCat-Flash-Lite-Sparse40.6%
79Granite 4.2 30B33.3%
80LLaDA2.2-flash30.1%
81Mellum2.1-12B-A2.5B-Thinking28.0%
82Granite 4.2 8B19.1%
83MiniCPM5-2B14.4%