Xiang Deng et al.

SWE-bench Pro

A long-horizon repository benchmark built to test realistic software engineering work. Its scores need a task-quality and setup check before they support a coding-agent decision.

In index
Coding
Source ↗

Models71

Top result89.9%Claude Opus 5.5

Top-3 spread9.6 pts

Year2025

Result and price

Results

71 results

#ModelResult

Claude Opus 5.589.9%

Claude Fable 5.181.2%

Claude Mythos 580.3%

Claude Fable 580.0%

Claude Opus 579.2%

06SASakana Fugu-Ultra73.7%

Claude Opus 4.869.2%

Qwen3.8 Max67.7%

09TEHy4 preview65.7%

10OAOrnith-1.5-397B65.1%

Grok 4.564.7%

GPT-5.6 Sol64.6%

Claude Opus 4.7 (Adaptive)64.3%

GPT-5.6 Terra63.4%

Qwen3.8-Omni-Flash63.3%

Claude Sonnet 563.2%

GPT-5.6 Luna62.7%

Qwen3.8-Flash-Next62.5%

19DAOrnith-1.0-397B62.2%

GLM-5.262.1%

Qwen3.8-27B61.7%

Muse Spark 1.161.5%

23DSdots3-note Preview61.0%

Qwen3.7 Max60.6%

25SAAtria Dawn Preview59.6%

25OAOrnith-1.5-35B-A3B59.6%

27POLaguna S 2.159.4%

MiniMax M359.0%

28SASakana Fugu59.0%

GPT-5.558.6%

30MAKimi K2.658.6%

GLM-5.158.4%

GPT-5.457.7%

Qwen3.7 Plus57.6%

Qwen 3.6 Max (preview)57.3%

MiMo-V2.5-Pro57.2%

Claude Opus 4.557.1%

GPT-5.3 Codex56.8%

Qwen3.6 Plus56.6%

39INLing 3.0 Flash56.6%

41STStep 3.7 Flash56.3%

MiniMax M2.756.2%

MiMo-V2.556.1%

44TMInkling-Small55.9%

GPT-5.255.6%

DeepSeek V4 Pro 081355.4%

Gemini 3.5 Flash55.1%

GLM-555.1%

49TMInkling54.3%

Gemini 3.5 Flash-Lite54.2%

Qwen3.6-27B53.5%

Claude Opus 4.653.4%

MAI-Thinking-152.8%

DeepSeek V4 Flash 073152.6%

Muse Spark52.4%

Grok 4.2051.8%

Muse Glimmer 30B51.2%

Qwen3.5 397B50.9%

59MAKimi K2.550.7%

60DAOrnith-1.0-35B50.4%

Qwen3.6-35B-A3B49.5%

62POLaguna M.149.2%

63POLaguna XS 2.147.6%

64OAOrnith-1.5-9B47.5%

65POLaguna XS.246.3%

66DAOrnith-1.0-9B42.9%

67MELongCat-Flash-Lite-Sparse40.6%

68IBGranite 4.2 30B33.3%

69INLLaDA2.2-flash30.1%

70IBGranite 4.2 8B19.1%