Sierra Research

τ²-bench Retail

Share of tasks an agent completes on one attempt, against a simulated user who holds information it must ask for.

In index
Agentic
Best in agentic
Source ↗

Models23

Top result85.3%Gemini 3.0 Pro

Top-3 spread1.1 pts

Updated30 Apr 2026

Result and price

18 of 23 results

Results

23 results

#ModelResultCost

01Gemini 3.0 ProSierra85.3%—

02Qwen3.5 397B A17Benabled effort · Sierra84.4%—

03Nemotron Orchestrator 8BNVIDIA84.2%—

04RAFT 30B A3BNortheastern University HAI Lab82.5%$0

05Claude Opus 4.1Anthropic82.4%—

06GPT-5.2high effort · Sierra81.6%$0.072

06GPT-5Sierra81.6%$0.106

08Claude Opus 4Anthropic81.4%—

09DeepSeek V3.2DeepSeek81.1%—

10Claude Sonnet 4Anthropic80.5%—

11Claude Opus 4.5high effort · Sierra79.6%$0.387

12Qwen3 Max ThinkingQwen79.4%—

13Claude Sonnet 4.5enabled effort · Sierra79.3%$0.255

14Gemini 3 Flashhigh effort · Sierra76.8%—

15Gemini 3 Prohigh effort · Sierra75.9%—

16GPT-4.1Sierra74.0%—

17o3OpenAI73.9%—

18GLM-5enabled effort · Sierra73.7%—

19Qwen3 MaxAlibaba72.2%—

20Claude 3.7 SonnetSierra72.1%—

21Kimi K2Moonshot AI70.6%—

22o4-miniSierra68.3%—

23GPT-4.1 miniSierra61.4%—