PolyWorkBench benchmark: 67 multilingual long-horizon tasks

PolyWorkBench introduces a benchmark for evaluating large language model agents on multilingual long-horizon workplace workflows, presenting 67 tasks across five domains and a hybrid evaluation framework. The paper, submitted on 7 Jul 2026 by Hongliang Li and seven co-authors, finds state-of-the-art LLM agents suffer significant performance degradation in multilingual settings compared to monolingual runs.

What is PolyWorkBench?

PolyWorkBench is a benchmark suite made up of 67 tasks across five domains: commerce, knowledge work, legal analysis, localization, and manufacturing. The tasks require agents to process heterogeneous multilingual inputs, perform iterative reasoning, call external tools, and produce structured outputs within extended workflows.

The authors framed the suite specifically to probe multilinguality inside agentic execution rather than only at input or output stages. The paper describes these as "multilingual long-horizon workplace workflows," where language variation interleaves with planning, tool use, and stepwise decision-making. The submission is 15 pages long and contains 6 figures illustrating the benchmark and experimental results.

How are agents evaluated on PolyWorkBench?

The benchmark uses a hybrid framework that combines structural grading, executable verification, and LLM-based semantic assessment to measure both functional correctness and linguistic consistency. Structural grading checks expected output formats, executable verification validates actions or tool calls where possible, and LLM-based semantic assessment judges content-level alignment when exact execution checks are insufficient.

Tasks demand iterative reasoning and external tool invocation, so the evaluation captures errors that arise across multiple procedural steps. Empirical results in the paper show state-of-the-art LLM agents experience "significant performance degradation in multilingual workflow settings compared to monolingual counterparts," and the authors argue multilinguality produces compounding effects across reasoning and execution steps. The benchmark therefore stresses both procedural decision-making and cross-lingual fidelity rather than isolated translation or classification accuracy.

Why does this matter?

Multilingual inputs are common in real workplace workflows, and PolyWorkBench exposes that current agentic systems do not simply lose a bit of accuracy — their errors compound across planning, tool use, and output synthesis. If multilinguality degrades both reasoning steps and execution, agents that pass monolingual benchmarks may still fail in real multilingual pipelines.

That gap matters for developers who deploy agents in internationalized settings: evaluation must capture linguistic consistency and functional correctness across multiple interdependent steps. PolyWorkBench offers concrete task diversity and an executable verification component that forces researchers to confront errors that cascade beyond single-turn language understanding.

What did the authors conclude and what to watch next?

The authors conclude that jointly modeling language variation and procedural decision-making is important for agent evaluation and future design. Watch for follow-up work that adapts planning or tool-invocation policies to multilingual contexts, and for models or benchmarks that publish multilingual, stepwise execution metrics rather than single-shot scores.

Concrete signals to track: whether future agent benchmarks adopt executable verification elements like PolyWorkBench, and whether papers report cross-lingual degradation versus monolingual baselines across multi-step tasks. The paper itself, submitted on 7 Jul 2026, supplies the task set and evaluation framework for others to reproduce and extend.

Paper and availability

The dataset and benchmark design are presented in a 15-page paper with 6 figures by Hongliang Li et al., submitted to arXiv on 7 Jul 2026 under the title "PolyWorkBench: Benchmarking Multilingual Long-Horizon LLM Agents."