Sierra Research

τ²-bench Airline

Share of tasks an agent completes on one attempt, against a simulated user who holds information it must ask for.

In index
Agentic
Source ↗

Models22

Top result84.0%Claude Opus 4.5

Top-3 spread1.5 pts

Updated2 Mar 2026

Result and price

Results

22 results

#ModelResultCost

01

Claude Opus 4.5high effort · Sierra84.0%$0.399

02

GPT-5.2high effort · Sierra83.0%$0.114

03

Gemini 3 Flashhigh effort · Sierra82.5%—

03

GLM-5enabled effort · Sierra82.5%—

05

Qwen3.5 397B A17Benabled effort · Sierra81.5%—

06

Gemini 3 Prohigh effort · Sierra80.5%—

07

Gemini 3.0 ProSierra73.0%—

08

Claude Sonnet 4.5enabled effort · Sierra71.0%$0.296

09

Qwen3 Max ThinkingQwen69.0%—

10

Claude 3.7 SonnetSierra64.2%—

11

DeepSeek V3.2DeepSeek63.8%—

12

GPT-5Sierra62.5%$0.134

13

Claude Sonnet 4Anthropic60.0%—

14

Claude Opus 4Anthropic59.6%—

15

Qwen3 MaxAlibaba59.5%—

16MAKimi K2Moonshot AI56.5%—

17

Claude Opus 4.1Anthropic56.0%—

17

GPT-4.1Sierra56.0%—

17

Nemotron Orchestrator 8BNVIDIA56.0%—

20

o4-mini (high)Sierra52.1%—

21

o3OpenAI52.0%—

22

GPT-4.1 miniSierra48.7%—