Conformal Thinking: Risk Control for LLM Reasoning (ICML 2026)

Conformal Thinking: Risk Control for Reasoning on a Compute Budget, a paper published July 2026, reframes the practical problem of setting token budgets for reasoning as a risk-control task. The authors introduce an upper stopping threshold and a novel parametric lower threshold, and use distribution-free risk control with a validation set to specify these stops so users can limit the error rate while minimizing compute.

How does the method decide when to stop reasoning?

The paper sets two concrete stopping mechanisms: an upper threshold that stops reasoning when the model is confident, and a parametric lower threshold that preemptively stops when an instance appears unsolvable. The upper threshold carries the risk of producing an incorrect output by halting early; the lower threshold carries the opposite risk, prematurely stopping progress on solvable problems. Given a user-specified target risk and a validation set, the framework applies distribution-free risk control to optimally choose both thresholds so the error rate is bounded while computation is reduced.

What evidence do the authors provide that this works?

Empirical results across diverse reasoning tasks and models show that the approach meets the specified risk targets and yields computational savings. The paper reports computational efficiency gains from the parametric lower threshold and from ensemble stopping mechanisms, while still adhering to the user-specified risk target. The work is authored by Xi Wang, Anushri Suresh, Alvin Zhang, Rishi More, William Jurayj, Benjamin Van Durme, Mehrdad Farajtabar, Daniel Khashabi, and Eric Nalisnick, with Xi Wang, Anushri Suresh, Alvin Zhang, and Rishi More marked as equal contributors, and † denoting Johns Hopkins University. Code accompanies the paper at https://github.com/xidulu/reasoning_risk_control/.

Why does reframing token budgets as risk control matter?

Framing budget-setting as risk control replaces heuristic tuning with statistically principled constraints: a target risk and a validation set determine stopping rules that balance error and compute. This directly addresses two deployment problems the paper highlights: wasting compute on hopeless instances, and stopping too early on solvable ones. The addition of a parametric lower threshold specifically targets unsolvable cases to save tokens, and ensemble stopping mechanisms further improve compute efficiency while preserving the user-specified error cap.

What to watch

Watch the authors' GitHub repository for code and experiments at https://github.com/xidulu/reasoning_risk_control/, and for follow-up studies that apply distribution-free risk control to more reasoning benchmarks or different model families. A concrete signal of broader impact will be other teams adopting the dual-threshold scheme or reporting comparable efficiency gains under explicit risk budgets.