SWE-bench team

SWE-bench Verified · SWE-bench/experiments

Resolved-issue rate on the 500 human-validated SWE-bench issues, from the submission records behind the leaderboard.

In index
Coding
Best in coding
Source ↗

Models65

Top result76.8%Claude Opus 4.5

Top-3 spread1.0 pts

Updated1 Sep 2026

Result and cost

30 of 65 results

Results

65 results

#ModelResultCost

01Claude Opus 4.5medium effort · live-SWE-agent76.8%—

01Claude 4.5 Opushigh effort · mini-SWE-agent76.8%$0.754

03Gemini 3 Flash Previewhigh effort · mini-SWE-agent75.8%$0.356

03MiniMax M2.5high effort · mini-SWE-agent75.8%$0.073

05Claude Opus 4.6mini-SWE-agent75.6%$0.552

06Claude Sonnet 4Harness AI74.8%—

07GPT-5.2-Codexmini-SWE-agent72.8%$0.449

07GPT-5.2high effort · mini-SWE-agent72.8%$0.474

07GLM-5high effort · mini-SWE-agent72.8%$0.534

10Gemini 3.5 Flashmedium effort · mini-SWE-agent71.8%—

11Claude Sonnet 4.5high effort · mini-SWE-agent71.4%$0.658

12Kimi K2 0905Lingxi v1.571.2%—

13Kimi K2.5high effort · mini-SWE-agent70.8%$0.147

13Claude 4 SonnetMoatless Tools70.8%—

15Claude 4 OpusTools70.4%—

16DeepSeek V3.2high effort · mini-SWE-agent70.0%$0.448

17Gemini 3 Prohigh effort · mini-SWE-agent69.6%$0.96

18Claude Haiku 4.5high effort · mini-SWE-agent66.6%$0.331

19GPT-5.1medium effort · mini-SWE-agent66.0%$0.306

19GPT-5.1-Codexmedium effort · mini-SWE-agent66.0%$0.589

21Kimi K2OpenHands65.4%—

22GPT-5medium effort · mini-SWE-agent65.0%$0.28

23o4-miniPatchPilot-v1.164.6%—

23o1-previewW&B Programmer O1 crosscheck564.6%—

25Kimi K2 Thinkingmini-SWE-agent63.4%$0.438

26Qwen3-Coder 480B/A35B InstructOpenHands62.5%—

27Claude 3.7 SonnetSWE-agent62.4%—

28GLM-4.6mini-SWE-agent61.8%$0.097

29MiniMax M2mini-SWE-agent61.0%$0.428

30DeepSeek V3.2 Reasonermini-SWE-agent60.0%$0.028

31GPT-5 minimedium effort · mini-SWE-agent59.8%$0.035

32GLM-4.5Undisclosed59.2%—

33o3mini-SWE-agent58.4%$0.334

34Devstral Small (2512)mini-SWE-agent56.4%—

35Devstral 2 2512mini-SWE-agent53.8%—

36Gemini 2.5 Promini-SWE-agent53.6%$0.288

36FrogBoss 32BFrogBoss-32B-251053.6%—

38Claude 3.5 SonnetEngine Labs53.0%—

39Qwen3 Coder 30B A3B InstructEntroPO + R2E52.2%—

40DeepSWE PreviewDeepSWE-Preview50.5%—

41Kimi K2 InstructCodeSweep - SWE-agent48.6%—

42DevStral Small 2505OpenHands46.8%—

43FrogMini 14BFrogMini-14B-251045.0%—

44Gemini 2.0 Flash ExpCodeShellAgent44.2%—

45o3-miniAgentless Lite42.4%—

45Amazon.nova Premier V1:0Amazon Nova Premier 1.042.4%—

47DeepSeek V3 0324SWE-Exp42.0%—

48Llama3 SWE RL 70BAgentless Mini41.2%—

49Claude 3 HaikuTools40.6%—

50GPT-4.1mini-SWE-agent39.6%$0.146

51DevStral Small 2507SWE-agent38.0%—

51Qwen2.5 Coder 32B InstructSkywork-SWE-32B38.0%—

53GPT-5 nanomedium effort · mini-SWE-agent34.8%$0.038

54Qwen 2.5SWE-Fixer32.8%—

55Gemini 2.5 Flashmini-SWE-agent28.7%$0.134

56GPT-OSS 120Bmini-SWE-agent26.0%$0.057

57GPT-4.1 minimini-SWE-agent23.9%$0.435

58MCTS Refine 7BMCTS-Refine-7B23.2%—

59GPT-4omini-SWE-agent21.6%$1.53

60Llama 4 Maverick Instructmini-SWE-agent21.0%$0.314

61Gemini 2.0 Flashmini-SWE-agent13.5%$0

62Claude 3 OpusRAG baseline11.4%—

63Llama 4 Scout Instructmini-SWE-agent9.1%$0.116

64Claude 2RAG baseline4.4%—

65SWE LlamaRAG baseline1.4%—