NL2Repo

A repository-understanding benchmark that measures whether models can map natural-language requests onto the right code locations and system changes.

In index
Coding

Models29

Top result65.4%DeepSeek V4.1 Flash

Top-3 spread5.9 pts

YearN/A

Result and price

Results

29 results

#ModelResult

01

DeepSeek V4.1 Flash65.4%

02

DeepSeek V4 Pro 081361.5%

03OAOrnith-1.5-397B59.5%

04TEHy4 preview58.9%

05

GLM-5.358.0%

06

GLM-5.3-Flash56.3%

07

Qwen3.8 Max55.9%

08

DeepSeek V4 Flash 073154.2%

09DSdots3-note Preview49.8%

10

Qwen3.8-Omni-Flash48.9%

10

GLM-5.248.9%

12DAOrnith-1.0-397B48.2%

13

Qwen3.8-Flash-Next48.1%

14

Qwen3.7 Max47.2%

15

Seed 2.1 Pro47.0%

16OAOrnith-1.5-35B-A3B46.2%

17

Seed 2.1 Turbo43.7%

18

Claude Opus 4.543.2%

19

Qwen 3.6 Max (preview)42.9%

20

GLM-5.142.7%

21

Qwen3.8-27B42.3%

22

MiniMax M342.1%

23

Qwen3.7 Plus41.1%

24

MiniMax M2.739.8%

25

Qwen3.6-27B36.2%

26DAOrnith-1.0-35B34.6%

27OAOrnith-1.5-9B32.4%

28

Qwen3.6-35B-A3B29.4%

29DAOrnith-1.0-9B27.2%