WALLA: Decentralized LLM aggregation via wagering mechanisms

WALLA, a family of wagering mechanisms for aggregating large language model predictions, was introduced in an arXiv paper submitted on 5 Jul 2026 (arXiv:2607.04389) by Yuhong Luo, David M. Pennock and Xintong Wang. The 32-page manuscript proposes that each model reports a prediction and a learned wager, and that predictions be aggregated using wagers as weights, while preserving strategic robustness and decentralization.

What is WALLA?

WALLA is a family of advantage-aligned wagering mechanisms in which each model supplies a prediction plus a learned wager, and the mechanism uses wagers as aggregation weights. The paper frames WALLA to achieve three explicit properties: (1) dominant-strategy incentive compatibility of prediction under arbitrary belief structure, (2) advantage-wager alignment where the optimal wager is proportional to the model's expected score advantage, and (3) prediction-agnostic wager optimization that enables decentralized learning of wager policies.

The authors present WALLA as a decentralized alternative to centralized aggregation, intended for settings where models have private tools or data and cannot reveal internal information, yet aggregation must remain robust to strategic reporting.

How does WALLA work in practice?

WALLA collects from each participating model a pair: a prediction and a learned wager, aggregates predictions using wagers as weights, and computes net payouts that include a leave-one-out baseline. The leave-one-out baseline is embedded into the net payout function; according to the paper, this design yields the three desirable properties listed above and makes the wager an advantage-aligned signal.

The paper also instantiates two mechanism variants that trade off normality and no-arbitrage while maintaining a bounded worst-case deficit for the mechanism. Those variants are presented as practical choices for implementers who must balance financial or accounting constraints (normality) against arbitrage prevention (no-arbitrage). The design explicitly enables decentralized learning because wager optimization is prediction-agnostic, so models can learn wager policies without requiring globally optimal predictions.

What evidence do the authors provide?

The authors report experiments on question-answering and forecasting benchmarks across heterogeneous models and private-information settings, finding that WALLA "matches centralized aggregation methods in predictive performance," while simultaneously delivering decentralized learning, advantage-aligned aggregation weights, uncertainty awareness, and incentive-compatible prediction. The paper provides these empirical claims alongside the theoretical properties and the two practical mechanism variants.

Concrete bibliographic facts: the paper is arXiv:2607.04389 (submitted 5 Jul 2026), runs 32 pages, and is available with DOI https://doi.org/10.48550/arXiv.2607.04389.

Why it matters

WALLA addresses a recurring tension: aggregators want the performance gains of ensembling diverse models, but cannot always access those models' private data or internals. By turning wagers into aggregation weights and aligning wagers with expected advantage, WALLA gives models an incentive-compatible channel to reveal relative confidence without exposing private information. That preserves the predictive benefits of aggregation while enabling decentralized learning of wager policies, a capability the authors argue is crucial in federated or multi-party settings.

What to watch

Look for follow-up work applying WALLA to real multi-vendor stacks and for open-source implementations of the two mechanism variants that trade normality and no-arbitrage. A clear next milestone would be independent replication of the reported experiments on question-answering and forecasting benchmarks under realistic payment or deficit constraints.