Sierra Research

τ²-bench Telecom

Share of tasks an agent completes on one attempt, against a simulated user who holds information it must ask for.

In index
Agentic
Best in agentic
Source ↗

Models18

Top result98.2%Qwen3 Max Thinking

Top-3 spread0.4 pts

Updated2 Mar 2026

Result and price

15 of 18 results

Results

18 results

#ModelResultCost

01Qwen3 Max ThinkingQwen98.2%—

02Gemini 3.0 ProSierra98.0%—

03Qwen3.5 397B A17Benabled effort · Sierra97.8%—

04DeepSeek V3.2DeepSeekAbove fit96.2%—

05GPT-5Sierra95.8%$0.165

06Claude Opus 4.5high effort · Sierra92.3%$0.72

07Claude Sonnet 4.5enabled effort · Sierra91.4%$0.488

08Gemini 3 Flashhigh effort · Sierra91.2%—

09Gemini 3 Prohigh effort · Sierra91.0%—

10GPT-5.2high effort · Sierra89.7%$0.093

11Nemotron Orchestrator 8BNVIDIA88.6%—

12GLM-5enabled effort · Sierra86.8%—

13Qwen3 MaxAlibaba84.2%—

14Kimi K2Moonshot AI65.8%—

15o4-miniSierra50.2%—

16Claude 3.7 SonnetSierra49.0%—

17GPT-4.1 miniSierra48.9%—

18GPT-4.1Sierra34.0%—