Fixed-Point Reasoners: FPRM looped Transformers paper (2026)

Fixed-Point Reasoners (FPRM) is a Transformer-based model that uses fixed-point convergence as an end-to-end halting mechanism in a looped architecture, the authors say in a paper submitted to arXiv on 16 Jun 2026 (arXiv:2606.18206). The paper, authored by Sajad Movahedi, Vera Milovanović, Shlomo Libo Feigin, Alexander Theus, Thomas Hofmann, Valentina Boeva, T. Konstantin Rusch and Antonio Orvieto, describes architectural fixes that address signal propagation problems in deep looped models and reports effectiveness on reasoning benchmarks including Sudoku, Maze, state-tracking and ARC-AGI.

What did the authors build and publish?

FPRM is a Transformer-based Fixed-Point Reasoning Model that combines looped architecture with fixed-point convergence to decide halting, and the paper documenting it was submitted on 16 Jun 2026 as arXiv:2606.18206. The authors present two architectural modifications, pre-norm layers and residual scaling, aimed at fixing the signal propagation issue that arises when halting decisions are postponed in deep looped models.

The paper frames looped architectures as an inductive bias for step-by-step compositional reasoning, where the number of effective layers reached by looping determines solution quality. It states that like deep architectures, looped architectures suffer from a signal propagation problem induced by depth. To address that, the authors apply pre-norm layers and residual scaling, and then introduce fixed-point halting so the loop can converge and stop end-to-end. The submission includes code availability information.

How does fixed-point halting work in FPRM and why are the architectural changes needed?

Fixed-point halting means the model uses fixed-point convergence as an end-to-end mechanism to decide when to stop looping, which lets FPRM adapt its compute to task difficulty. In practice, the paper positions fixed-point convergence as the halting decision inside a looped Transformer; convergence indicates the loop can stop and produce an output.

The authors identify a signal propagation problem that appears when depth increases because the halting decision is postponed. They counter this by using pre-norm layers and residual scaling, architectural modifications that improve stability in deep looped settings. Those changes are presented as the foundation that enables reliable fixed-point halting in an end-to-end trained looped model.

What benchmarks did the paper test, and what claims are made about results?

The paper reports FPRM is effective on a set of common reasoning benchmarks: Sudoku, Maze, state-tracking, and ARC-AGI. The text states fixed-point halting allows FPRM to adapt its compute to task difficulty and that the model is effective on those benchmarks. The paper does not provide numeric metrics in the arXiv abstract text, but it explicitly lists these four evaluation domains as targets for the method.

Why it matters

Looped architectures aim to learn iterative, stepwise procedures that match how many reasoning tasks are naturally solved. The combination of stability fixes and an internal halting mechanism changes two things at once: it mitigates depth-induced signal degradation, and it gives the model a built-in, adaptive stopping rule. That could reduce the need to pick a fixed number of unrolled steps and let compute scale with problem difficulty, affecting efficiency and flexibility when applying Transformers to procedural reasoning tasks.

What to watch

Check the paper and the linked code for experiment details, training setups and numeric results for Sudoku, Maze, state-tracking and ARC-AGI. The arXiv identifier is arXiv:2606.18206 and the submission date is 16 Jun 2026; the authors list includes Sajad Movahedi and seven coauthors. The next concrete signals will be the code release details and any follow-up papers or replication studies that publish per-benchmark metrics and ablations for pre-norm and residual scaling.