Sierra Research

τ²-bench Banking

Share of tasks an agent completes on one attempt, against a simulated user who holds information it must ask for.

In index
Agentic
Best in agentic
Source ↗

Models28

Top result73.2%PrincipledMachines Cortex 0.1

Top-3 spread24.5 pts

Updated20 Oct 2026

Result and price

23 of 28 results

Results

28 results

#ModelResultCost

01PrincipledMachines Cortex 0.1medium effort · Principled Machines73.2%$0.596

02Qwen3.8 Maxxhigh effort · Sierra55.2%—

03Claude Opus 5max effort · Sierra48.7%—

04Grok 4.5high effort · Sierra47.9%—

05GPT-5.6 Solxhigh effort · Sierra46.9%—

06GPT-5.5xhigh effort · Sierra44.6%—

07Muse Spark 1.1xhigh effort · Sierra40.5%—

08Claude Opus 4.7max effort · Sierra40.2%—

09Claude Fable 5max effort · Sierra39.7%—

09Claude Opus 4.8max effort · Sierra39.7%—

11GPT-5.4xhigh effort · Sierra39.4%$1.84

12Kimi K3max effort · Sierra37.1%—

12GLM-5.2xhigh effort · Sierra37.1%—

14GPT-5.2high effort · Sierra32.2%—

15Distyl ButtonAgenthigh effort · Distyl AI31.2%—

16Claude Opus 4.6max effort · Sierra27.3%—

16Gemini 3 Flashhigh effort · Sierra27.3%—

18Gemini 3.1 Pro Previewhigh effort · Sierra26.0%—

19Claude Sonnet 4.5enabled effort · Sierra25.3%$4.05

20Inklingmax effort · Sierra25.0%—

21Claude Opus 4.5high effort · Sierra24.7%—

22Gemini 3 Prohigh effort · Sierra18.0%—

22Grok 4.2high effort · Sierra18.0%—

24Grok 4 fasthigh effort · Sierra15.7%—

25Gemini 2.5 Prohigh effort · Sierra13.7%—

26Grok 4.1 Fasthigh effort · Sierra13.1%—

27GLM-5enabled effort · Sierra9.8%—

27Qwen3.5 397B A17Benabled effort · Sierra9.8%—