David Rein et al.

Graduate-Level Google-Proof Q&A

A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Designed to be difficult even for skilled non-experts with access to Google.

In index
Knowledge
Source ↗

Models84

Top result96.0%GPT-6 Astra

Top-3 spread0.5 pts

Year2023

Result and price

Results

84 results

#ModelResult

GPT-6 Astra96.0%

02SASakana Fugu95.5%

02SASakana Fugu-Ultra95.5%

GPT-5.6 Sol94.6%

Claude Opus 4.7 (Adaptive)94.2%

Claude Mythos 594.1%

GPT-5.593.6%

Claude Opus 4.893.6%

09MAKimi K393.5%

GPT-5.6 Terra92.9%

GPT-5.492.8%

11OAOrnith-1.5-397B92.8%

Qwen3.8 Max92.6%

Qwen3.7 Max92.4%

GPT-5.292.4%

16TEHy4 preview92.3%

GPT-5.6 Luna92.3%

Gemini 3.5 Flash92.2%

Qwen3.8-Flash-Next91.7%

Claude Opus 4.691.3%

GLM-5.291.2%

Qwen3.8-Omni-Flash91.0%

DeepSeek V4.1 Flash90.9%

24MAKimi K2.690.5%

Qwen3.6 Plus90.4%

Qwen3.7 Plus90.3%

DeepSeek V4 Pro 081390.1%

Grok 4.390.1%

Claude Sonnet 4.689.9%

29INInterfaze Beta89.9%

31TMInkling-Small89.5%

Qwen3.8-27B89.2%

32OAOrnith-1.5-35B-A3B89.2%

Qwen3.5 397B88.4%

DeepSeek V4 Flash 073188.1%

GPT-5.4 mini88.0%

37TMInkling87.9%

Qwen3.6-27B87.8%

39MAKimi K2.5 (Reasoning)87.6%

39MAKimi K2.587.6%

41TEHy3 Preview87.2%

Nemotron 3 Ultra87.0%

Claude Opus 4.587.0%

Qwen3.5-122B-A10B86.6%

45OAOrnith-1.5-9B86.4%

GLM-586.0%

Qwen3.6-35B-A3B86.0%

48PMTernary Bonsai 2 27B85.8%

GLM-4.785.7%

Qwen3.5-27B85.5%

51INLing 3.0 Flash85.0%

Gemma 4 31B84.3%

MAI-Thinking-184.2%

Qwen3.5-35B-A3B84.2%

55INLing 3.0 Flash FP884.0%

MiMo-V2-Flash83.7%

Claude Sonnet 4.583.4%

Gemini 2.5 Pro83.0%

GPT-5.4 nano82.8%

o1-pro79.0%

Gemma 4 12B78.8%

Qwen3 235B 250777.5%

o3-mini77.2%

o175.7%

Nemotron 3.5 Lightning 30B A3B NVFP475.6%

Nemotron 3 Nano Omni 30B A3B72.2%

67ZYZAYA1-8B71.0%

68IBGranite 4.2 30B66.4%

GPT-4.166.3%

GPT-4.1 mini64.2%

71IBGranite 4.2 8B64.1%

Claude 3.5 Sonnet59.4%

DeepSeek V359.1%

74INLing 2.6 Flash59.0%

Gemma 4 E4B58.6%

76JEMellum2-12B-A2.5B-Thinking57.6%

77ZYZAYA1-74B-Preview57.3%

78IBGranite 4.2 3B54.8%

GPT-4.1 nano50.3%

Gemma 4 E2B43.4%

80SPSoofi S 30B-A3B43.4%

82JEMellum2-12B-A2.5B-Instruct40.9%

83LILFM2.5-VL-450M25.7%

84LILFM2.5-230M25.4%