---
title: "tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains"
description: "tau-bench evaluates tool-using agents inside multi-turn retail and airline conversations with domain policies and mutable databases. Read it to see how final-state grading separates fluent talk from completed work, why pass^k measures repeated reliability, and why the original scores are now historical rather than a current leaderboard."
canonical_url: "https://fanout.sh/daily/2026-10-13-tau-bench-tool-agent-user-interaction"
md_url: "https://fanout.sh/daily/2026-10-13-tau-bench-tool-agent-user-interaction.md"
last_updated: "2026-10-13"
access: "public"
---

# tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

tau-bench evaluates tool-using agents inside multi-turn retail and airline conversations with domain policies and mutable databases. Read it to see how final-state grading separates fluent talk from completed work, why pass^k measures repeated reliability, and why the original scores are now historical rather than a current leaderboard.

## Paper details

- Authors: Shunyu Yao et al.

- Venue: ICLR 2025

- Track: AI Research

- Difficulty: Medium

- Reading time: 34 min

- Original paper: https://arxiv.org/abs/2406.12045v1

## What you will learn

- How a simulated user, policy document, API tools, and hidden target state form one stateful agent task.

- Why the reward checks both the final database and required information in the agent's reply.

- How pass^k exposes inconsistent agents and why corrected task releases matter for present-day comparisons.

## Continue exploring

[Compare ReAct](https://fanout.sh/daily/2026-08-10-react-reasoning-acting): See how interleaved reasoning and tool actions work before inspecting how tau-bench evaluates the resulting conversation and state changes.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md