Inference
EN
Vals AI

SWE-bench

Solving production software engineering tasks

ModelsModels with a result.87
Top resultBest result on this benchmark.97.0%Claude Opus 5
Top-3 spreadPoints from first to third.0.8 pts
UpdatedDate the source changed the results.1 Sep 2026

Result and cost

86 of 87 results

Results

87 results
#ModelResultThe score from the source.CostUS dollars to run one task.TimeTime to run one task.
01Claude Opus 597.0% ±0.8$1.2910 min
02DeepSeek V4 Pro 081396.4% ±0.8$0.1034 min
03GPT-5.6 Sol96.2% ±0.9$1.153 min
04Grok 4.695.6% ±0.9$0.78510 min
05GPT-5.6 Terra95.4% ±0.9$0.4013 min
05GLM-5.395.4% ±0.9$0.33814 min
07Claude Fable 595.0% ±1.0$2.056 min
08Kimi K393.4% ±1.1$0.7610 min
09GPT-5.6 Luna93.0% ±1.1$0.0433 min
10GLM-5.3-Flash92.0% ±1.2$0.01864 min
11DeepSeek V4 Flash 073188.8% ±1.4$0.00993 min
12Claude Opus 4.888.6% ±1.4$1.929 min
13Muse Spark 1.286.6% ±1.5$0.5548 min
13Grok 4.586.6% ±1.5$0.543 min
15Qwen3.8-27B86.0% ±1.6$1.0810 min
16Claude Opus 4.8 Claude CodeClaude Code85.8% ±1.6$0.6722 min
17Qwen3.8 Max85.6% ±1.6$1.1342 min
18GLM-5.282.8% ±1.7$0.7111 min
19GPT-5.582.6% ±1.7$1.367 min
20Inkling-Small82.2% ±1.7$0.1344 min
21Muse Spark 1.182.0% ±1.7$0.3523 min
21Claude Opus 4.782.0% ±1.7$2.427 min
23Gemini 3.7 Flash80.8% ±1.8$1.446 min
24Gemini 3.8 Flash80.0% ±1.8$2.1911 min
25Claude Sonnet 579.6% ±1.8$1.4916 min
25Gemini 3.6 Flash79.6% ±1.8$1.1913 min
25Composer 2.5Cursor CLI79.6% ±1.8—0.0 s
28Gemini 3.5 Flash78.8% ±1.8$0.9544 min
28Gemini 3.1 Pro Preview78.8% ±1.8$0.7835 min
30GPT-5.478.2% ±1.8$0.8015 min
30Claude Opus 4.6 (Adaptive)78.2% ±1.8$1.226 min
30Kimi K2.7 Code78.2% ±1.8$0.26515 min
33GPT-5.3 Codex78.0% ±1.9$0.4554 min
34Inkling77.6% ±1.9$0.5867 min
35DeepSeek V4 Pro 042377.4% ±1.9$0.43711 min
35Claude Sonnet 4.677.4% ±1.9$1.309 min
37Gemini 3 Pro76.4% ±1.9$0.7727 min
37Claude Opus 4.5 Thinking76.4% ±1.9$1.055 min
37GLM-5.176.4% ±1.9$0.4579 min
37GPT-5.5 CodexCodex76.4% ±1.9$0.6522 min
41Kimi K2.676.2% ±1.9$0.2759 min
41GPT-5.5 FactoryFactory76.2% ±1.9$1.137 min
43GPT-5.275.8% ±1.9$1.4511 min
44Gemini 3 Flash Preview75.0% ±1.9$0.285 min
44MiniMax M375.0% ±1.9$0.41612 min
44Gemini 3.5 Flash-Lite75.0% ±1.9$0.3097 min
47MiniMax M2.174.8% ±1.9$0.4396 min
48Muse Spark74.4% ±2.0$0.4448 min
49MiniMax M2.574.2% ±2.0$0.4577 min
50MiMo-V2.5-Pro74.0% ±2.0$0.0238 min
51MiniMax M2.773.8% ±2.0$0.46615 min
52Qwen3.6 Plus73.4% ±2.0$0.8697 min
53GPT-5.4 mini73.0% ±2.0$0.5115 min
54Qwen 3.6 Max (preview)72.8% ±2.0$1.5718 min
55GPT-5.2-Codex72.4% ±2.0$0.3123 min
56Grok 4.20 (Reasoning)72.2% ±2.0$0.5162 min
57GLM-5 Thinking71.4% ±2.0$0.4499 min
57Grok 4.371.4% ±2.0$0.2232 min
59Qwen3.5 Plus Thinking71.2% ±2.0$0.9648 min
60MiMo-V2.571.0% ±2.0$0.014 min
61Claude Sonnet 4.5 Thinking70.0% ±2.1$0.8076 min
61Kimi K2.5 Thinking70.0% ±2.1$0.1686 min
61Qwen3.6-27B70.0% ±2.1$0.91417 min
64GPT-5.169.8% ±2.1$0.7138 min
64GPT-5.4 nano69.8% ±2.1$0.0984 min
66GLM-4.769.4% ±2.1$0.2046 min
67GPT-569.0% ±2.1$0.274 min
68Qwen3.7 Max68.8% ±2.1$3.2316 min
69DeepSeek V3.2 (Thinking)67.6% ±2.1$0.6739 min
70Claude Haiku 4.5 Thinking66.6% ±2.1$0.3664 min
71Mistral Medium 3.566.4% ±2.1$4.0019 min
72Ling 3.0 Flash65.2% ±2.1$0.0323 min
73Qwen3.5 Flash64.4% ±2.1$0.2636 min
74Gemini 3.1 Flash Lite Preview62.8% ±2.2$0.1052 min
74Devstral 2 251262.8% ±2.2$0.7474 min
76GPT-5 mini60.8% ±2.2$0.053 min
77Kimi K2 Thinking60.2% ±2.2$0.36627 min
78Grok 457.8% ±2.2$1.4210 min
79Laguna M.157.6% ±2.2$0.5215 min
80Laguna XS.255.2% ±2.2$0.175 min
81Gemini 2.5 Pro54.4% ±2.2$0.3523 min
82Nemotron Lightning 3p5 30B A3B52.8% ±2.2$0.03215 min
83Grok 4 Fast (Reasoning)45.4% ±2.2$0.0512 min
84Grok 4.1 Fast (Reasoning)41.4% ±2.2$0.0282 min
84Mistral Large 341.4% ±2.2$0.3223 min
86GPT-OSS 120B33.6% ±2.1$0.1724 min
87Command A7.8% ±1.2$0.214 min