Sierra Research

τ²-bench Telecom

Share of tasks an agent completes on one attempt, against a simulated user who holds information it must ask for.

In index
Agentic
Source ↗

Models18

Top result98.2%Qwen3 Max Thinking

Top-3 spread0.4 pts

Updated2 Mar 2026

Result and price

Results

18 results

#ModelResultCost

01

Qwen3 Max ThinkingQwen98.2%—

02

Gemini 3.0 ProSierra98.0%—

03

Qwen3.5 397B A17Benabled effort · Sierra97.8%—

04

DeepSeek V3.2DeepSeek96.2%—

05

GPT-5Sierra95.8%$0.165

06

Claude Opus 4.5high effort · Sierra92.3%$0.72

07

Claude Sonnet 4.5enabled effort · Sierra91.4%$0.488

08

Gemini 3 Flashhigh effort · Sierra91.2%—

09

Gemini 3 Prohigh effort · Sierra91.0%—

10

GPT-5.2high effort · Sierra89.7%$0.093

11

Nemotron Orchestrator 8BNVIDIA88.6%—

12

GLM-5enabled effort · Sierra86.8%—

13

Qwen3 MaxAlibaba84.2%—

14MAKimi K2Moonshot AI65.8%—

15

o4-mini (high)Sierra50.2%—

16

Claude 3.7 SonnetSierra49.0%—

17

GPT-4.1 miniSierra48.9%—

18

GPT-4.1Sierra34.0%—