SKU RLE-1206 · Sold by External
tau-bench (τ-bench)
Product specifications
| SKU | RLE-1206 |
|---|---|
| Data type | RL environments |
| Volume | 165 tasks (retail 115 + airline 50) |
| Size on disk | Not published by source |
| Format | Python package + JSON domain data |
| Access model | PUBLIC LICENSE |
| Pricing | Free · open-source license |
| Quality score | — |
| License | MIT |
τ-bench evaluates language agents in dynamic customer-service conversations across two domains — retail (115 tasks) and airline (50 tasks) — where the agent must follow a domain policy document while calling domain-specific API tools and responding to an LLM-simulated user. Agents perform read/write database actions (e.g., modify_reservation, cancel) and are scored by comparing the final database/output state, reported with pass@k (pass^1..pass^k) reliability metrics. Distributed as a pip-installable Python package (MIT-licensed) by Sierra Research.