Artificial Analysis CyberGym-E2E-AA

Artificial Analysis' implementation of Berkeley RDI's CyberGym-E2E: agents find a real memory-safety vulnerability in a C/C++ open-source project, write a proof-of-concept input that crashes it, and patch it without breaking the project's tests.

ModelsModels with a result.19
Top resultBest result on this benchmark.94.7%Kimi K3.1 Pro
Top-3 spreadPoints from first to third.16.1 pts
YearYear of release.2026

Result and price

17 of 19 results

Results

19 results
#ModelResultThe score from the source.
01Kimi K3.1 ProAbove fitFar above this model's index.94.7%
02Mistral Large 4Above fitFar above this model's index.79.4%
03MiMo-V2.6-ProAbove fitFar above this model's index.78.6%
04GPT-6 LunaAbove fitFar above this model's index.77.9%
05Step 5 PreviewAbove fitFar above this model's index.74.8%
06Grok 4.7Above fitFar above this model's index.74.0%
06GLM-5.3-FlashAbove fitFar above this model's index.74.0%
08Kimi K3Above fitFar above this model's index.58.0%
09Beam 1.0 501B-A23BAbove fitFar above this model's index.45.0%
10GLM-5.329.0%
11MiniMax M324.4%
17GPT-6 Sol0.0%