SWE-bench team

SWE-bench Verified · SWE-bench/experiments

Resolved-issue rate on the 500 human-validated SWE-bench issues, from the submission records behind the leaderboard.

In index
Coding
Source ↗

Models62

Top result76.8%Claude Opus 4.5

Top-3 spread1.0 pts

Updated1 Sept 2026

Result and cost

Results

62 results

#ModelResultCost

01

Claude Opus 4.5medium effort · live-SWE-agent76.8%—

01

Claude 4.5 Opushigh effort · mini-SWE-agent76.8%$0.754

03

Gemini 3 Flash Previewhigh effort · mini-SWE-agent75.8%$0.356

03

MiniMax M2.5high effort · mini-SWE-agent75.8%$0.073

05

Claude Opus 4.6mini-SWE-agent75.6%$0.552

06

Claude Sonnet 4Harness AI74.8%—

07

GPT-5.2-Codexmini-SWE-agent72.8%$0.449

07

GLM-5high effort · mini-SWE-agent72.8%$0.534

09

GPT-5.2high effort · mini-SWE-agent72.3%$0.52

10

Gemini 3.5 Flashmedium effort · mini-SWE-agent71.8%—

11

Claude Sonnet 4.5high effort · mini-SWE-agent71.4%$0.658

12MAKimi K2.5high effort · mini-SWE-agent70.8%$0.147

12

Claude 4 SonnetMoatless Tools70.8%—

14

Claude 4 OpusTools70.4%—

15

DeepSeek V3.2high effort · mini-SWE-agent70.0%$0.448

16

Gemini 3 Prohigh effort · mini-SWE-agent69.6%$0.96

17

Claude Haiku 4.5high effort · mini-SWE-agent66.6%$0.331

18

GPT-5.1medium effort · mini-SWE-agent66.0%$0.306

18

GPT-5.1-Codexmedium effort · mini-SWE-agent66.0%$0.589

20MAKimi K2OpenHands65.4%—

21

GPT-5medium effort · mini-SWE-agent65.0%$0.28

22

o1-previewW&B Programmer O1 crosscheck564.6%—

23MAKimi K2 Thinkingmini-SWE-agent63.4%$0.438

24

Qwen3 Coder 480B A35B InstructOpenHands62.5%—

25

Claude 3.7 SonnetSWE-agent62.4%—

26

GLM-4.6mini-SWE-agent61.8%$0.097

27

MiniMax M2mini-SWE-agent61.0%$0.428

28

DeepSeek V3.2 Reasonermini-SWE-agent60.0%$0.028

29

GPT-5 minimedium effort · mini-SWE-agent59.8%$0.035

30

GLM-4.5Undisclosed59.2%—

31

o3mini-SWE-agent58.4%$0.334

32

o4-mini (high)PatchPilot-v1.154.8%—

33

Devstralmini-SWE-agent53.8%—

34

Gemini 2.5 Promini-SWE-agent53.6%$0.288

34UNFrogBoss 32BFrogBoss-32B-251053.6%—

36

Claude 3.5 SonnetEngine Labs53.0%—

37

Qwen3 Coder 30B A3B InstructEntroPO + R2E52.2%—

38UNDeepSWE PreviewDeepSWE-Preview50.5%—

39MAKimi K2 InstructCodeSweep - SWE-agent48.6%—

40

Devstral SmallOpenHands46.8%—

41UNFrogMini 14BFrogMini-14B-251045.0%—

42

Gemini 2.0 Flash ExpCodeShellAgent44.2%—

43

o3-miniAgentless Lite42.4%—

43UNAmazon.nova Premier V1:0Amazon Nova Premier 1.042.4%—

45

DeepSeek V3SWE-Exp42.0%—

46

Llama3 SWE RL 70BAgentless Mini41.2%—

47

Claude 3 HaikuTools40.6%—

48

GPT-4.1mini-SWE-agent39.6%$0.146

49

Qwen2.5 Coder 32B InstructSkywork-SWE-32B38.0%—

50

GPT-5 nanomedium effort · mini-SWE-agent34.8%$0.038

51

Qwen 2.5SWE-Fixer32.8%—

52

Gemini 2.5 Flashmini-SWE-agent28.7%$0.134

53

GPT-4oAppMap Navie26.6%—

54

GPT-OSS 120Bmini-SWE-agent26.0%$0.057

55

GPT-4.1 minimini-SWE-agent23.9%$0.435

56UNMCTS Refine 7BMCTS-Refine-7B23.2%—

57

Llama 4 Maverick Instructmini-SWE-agent21.0%$0.314

58

Gemini 2.0 Flashmini-SWE-agent13.5%$0

59

Claude 3 OpusRAG baseline11.4%—

60

Llama 4 Scout Instructmini-SWE-agent9.1%$0.116

61

Claude 2RAG baseline4.4%—

62UNSWE LlamaRAG baseline1.4%—