Inference
SWE-bench team

SWE-bench Lite · SWE-bench/experiments

Resolved-issue rate on the 300-issue Lite split, from the submission records behind the leaderboard.

ModelsModels with a result.13
Top resultBest result on this test.57.5%Claude 4 Sonnet
Top-3 spreadPoints from first to third.10.2 pts
UpdatedDate the source changed the results.11 Sept 2025

Result and price

020406080100$0.1$0.3$1$3$10$30$100RESULTOUTPUT PRICE PER 1M TOKENS · LOG SCALE

Results

15 results
#ModelResultThe score from the source.
01Claude 4 SonnetSWE-agent57.5%
02Claude 3.7 SonnetSWE-agent48.0%
03Qwen3 Coder 30B A3B InstructEntroPO + R2E47.3%
04Claude 3.5 SonnetAgentless-1.540.8%
05DeepSeek V3KGCompass36.7%
06o3-miniAegis31.3%
07DeepSeek V3Moatless Tools30.7%
08GPT-4oSIMA27.8%
09Qwen 2.5SWE-Fixer24.0%
10GPT-4OpenCSG StarShip CodeGenAgent23.7%
11GPT-4AutoCodeRover19.0%
12MCTS Refine 7BMCTS-Refine-7B16.3%
13Claude 3 OpusRAG baseline4.3%
14Claude 2RAG baseline3.0%
15SWE LlamaRAG baseline1.3%