Inference
Vals AI

SRE Bench

Can AI agents reverse engineer real-world, contamination-free binaries?

ModelsModels with a result.10
Top resultBest result on this test.56.9%GPT-6 Astra
Top-3 spreadPoints from first to third.34.0 pts
UpdatedDate the source changed the results.16 Sept 2026

Result and cost

020406080100$0.3$1$3$10$30$100RESULTUS DOLLARS PER TASK · LOG SCALE

Results

10 results
#ModelResultThe score from the source.CostUS dollars to run one task.TimeTime to run one task.
01GPT-6 Astra56.9%$13.518 min
02GPT-5.6 Sol30.5%$42.5288 min
03Claude Fable 5.122.9%$32.232.0 h
04Claude Opus 512.2%$23.6381 min
05Gemini 3.7 Flash4.6%$21.331.8 h
06GPT-5.53.8%$17.7746 min
07Hy4 preview2.3%$4.91.7 h
08DeepSeek V4.1 Flash0.8%$0.54961 min
08Grok 4.50.8%$13.4666 min
10GLM-5.20.0%$26.582.5 h