Vals AI

CaseLaw v2

Private question-answer benchmark over Canadian court-cases.

Not in index
Legal
Source ↗

Models53

Top result79.3%Grok 4.3

Top-3 spread9.4 pts

Updated4 May 2026

Result and cost

52 of 53 results

Results

53 results

#ModelResultCostTime

01Grok 4.379.3% ±1.4$0.0443 s

02GPT-5.173.4% ±0.8$0.04324 s

03GPT-4.169.9% ±0.3$0.0476.7 s

04GPT-5 mini68.5% ±1.1$0.01143 s

05Claude Opus 4.768.4% ±0.4$0.16218 s

06GPT-566.5% ±0.7$0.0672 min

07GPT-5.566.2% ±0.3$0.15525 s

08GPT-5.266.0% ±0.6$0.10486 s

09Grok 465.8% ±0.2$0.15932 s

10Kimi K2 Thinking65.7% ±1.8$0.0152 min

10Grok 4 Fast (Reasoning)65.7% ±0.4$0.007324 s

12Gemini 3.1 Pro Preview64.8% ±1.2$0.13657 s

13Command A64.5% ±1.1$0.09842 s

14Claude Sonnet 4.664.0% ±0.6$0.14536 s

15Gemini 2.5 Pro63.9% ±0.6$0.25744 s

16GPT-5.463.8% ±2.2$0.10157 s

17Muse Spark63.1% ±0.8$0.006562 s

18Claude Opus 4.5 Thinking62.6% ±0.1$0.22917 s

19Claude Sonnet 4.5 Thinking62.2% ±1.5$0.25625 s

20Claude Opus 4.6 (Adaptive)62.1% ±0.4$0.23132 s

21Mistral Large 361.4% ±1.0$0.01941 s

22Kimi K2.661.2% ±2.1$0.02646 s

23MiniMax M2.760.9% ±0.8$0.003122 s

24Grok 4.1 Fast (Reasoning)60.5% ±0.8$0.01119 s

25Qwen3.5 Plus Thinking59.7% ±1.1$0.032 min

25GPT-4o59.7% ±0.4$0.06741 s

27DeepSeek V4 Pro 042359.4% ±1.5$0.01348 s

28Kimi K2.5 Thinking58.7% ±0.7$0.0254 min

29Trinity-Large-Thinking57.9% ±1.9$0.008915 s

30Claude Haiku 4.5 Thinking56.5% ±0.9$0.0533 s

31Qwen3.5 Flash55.9% ±1.2$0.004944 s

32Gemini 3 Flash Preview55.8% ±0.1$0.0254 min

32MiniMax M2.155.8% ±0.5$0.00837 s

34DeepSeek V3.2 (Thinking)55.4% ±0.5$0.01866 s

35Gemini 3.1 Flash Lite Preview55.0% ±0.7$0.00934.6 s

36GLM-4.754.9% ±0.9$0.02160 s

37Grok 4.20 (Reasoning)54.4% ±0.3$0.06811 s

38DeepSeek V3.153.9% ±0.6$0.01741 s

39MiniMax M2.553.5% ±0.5$0.01231 s

40Qwen3.6-27B53.2% ±2.2$0.02442 s

41Gemini 3 Pro53.1% ±0.2$0.13832 s

42Gemma 4 31B52.6% ±1.3$029 s

42GPT-5 nano52.6% ±1.0$0.004256 s

44GLM-5 Thinking52.5% ±1.7$0.0342 min

45GPT-5.4 nano51.9% ±1.1$0.00117.6 s

46GPT-5.4 mini51.7% ±0.9$0.04158 s

47GLM-5.151.6% ±1.1$0.03449 s

48Qwen3.6 Plus51.4% ±1.2$0.04744 s

49GPT-OSS 120B48.8% ±0.9$0.004368 s

50Qwen 3.6 Max (preview)47.9% ±1.1$0.05243 s

51Qwen3 Max47.5% ±0.5$0.05185 s

52Mistral Medium 3.544.2% ±1.1$0.08565 s

53GPT-OSS 20B43.8% ±0.9$0.002286 s