SWE-bench team

SWE-bench Lite · swebench.com

Resolved-issue rate on real GitHub issues, as published by the SWE-bench team.

In index
Coding
Source ↗

Models15

Top result56.7%Claude 4 Sonnet

Top-3 spread15.0 pts

Updated11 Sept 2025

Result and price

Results

15 results

#ModelResult

01

Claude 4 SonnetSWE-agent56.7%

02

Claude 3.7 SonnetSWE-agent48.0%

03UNCodeAct v2.1 (claude-3-5-sonnet-20241022)OpenHands41.7%

04

o3-miniAgentless Lite32.3%

05

DeepSeek V3Moatless Tools30.7%

06

Claude 3.5 SonnetKortix AI30.0%

07UNCodeAct v1.8OpenHands26.7%

08

Qwen2.5 (7B + 72B)SWE-Fixer24.7%

09

GPT-4oAppMap Navie21.7%

10

GPT-4 (1106)SWE-agent10.3%

11

Claude 3 OpusSWE-agent8.0%

12

Claude 2RAG baseline3.0%

13UNSWE-Llama 7BRAG baseline1.3%

14UNSWE-Llama 13BRAG baseline1.0%

15

GPT-3.5RAG baseline0.3%