Sierra Research

τ²-bench Banking

Share of tasks an agent completes on one attempt, against a simulated user who holds information it must ask for.

In index
Agentic
Source ↗

Models26

Top result55.2%Qwen3.8 Max

Top-3 spread7.2 pts

Updated4 Aug 2026

Result and price

Results

26 results

#ModelResultCost

01

Qwen3.8 Maxxhigh effort · Sierra55.2%—

02

Claude Opus 5max effort · Sierra48.7%—

03

Grok 4.5high effort · Sierra47.9%—

04

GPT-5.6 Solxhigh effort · Sierra46.9%—

05

GPT-5.5xhigh effort · Sierra44.6%—

06

Muse Spark 1.1xhigh effort · Sierra40.5%—

07

Claude Opus 4.7max effort · Sierra40.2%—

08

Claude Fable 5max effort · Sierra39.7%—

08

Claude Opus 4.8max effort · Sierra39.7%—

10

GPT-5.4xhigh effort · Sierra39.4%$1.84

11

GLM-5.2xhigh effort · Sierra37.1%—

12

GPT-5.2high effort · Sierra32.2%—

13UNDistyl ButtonAgenthigh effort · Distyl AI31.2%—

14

Claude Opus 4.6max effort · Sierra27.3%—

14

Gemini 3 Flashhigh effort · Sierra27.3%—

16

Gemini 3.1 Pro Previewhigh effort · Sierra26.0%—

17

Claude Sonnet 4.5enabled effort · Sierra25.3%$4.05

18TMInklingmax effort · Sierra25.0%—

19

Claude Opus 4.5high effort · Sierra24.7%—

20

Gemini 3 Prohigh effort · Sierra18.0%—

20

Grok 4.2high effort · Sierra18.0%—

22

Grok 4 fasthigh effort · Sierra15.7%—

23

Gemini 2.5 Prohigh effort · Sierra13.7%—

24

Grok 4.1 Fasthigh effort · Sierra13.1%—

25

GLM-5enabled effort · Sierra9.8%—

25

Qwen3.5 397B A17Benabled effort · Sierra9.8%—