STAR: SpatioTemporal RL post-training for Stable Diffusion

STAR is a SpatioTemporal Adaptive Reward Allocation method for RL post-training of text-to-image diffusion and flow models, presented by Jinjie Shen and coauthors. The paper, submitted 16 Jun 2026 and revised 18 Jun 2026, introduces spatially resolved policy updates that target the parts of an image and the denoising steps most responsible for reward.

What is STAR and how does it work?

STAR constructs spatial allocation maps from text-image attention inside the generative model and applies group-relative advantages to the latent regions that matter most, dynamically varying those maps across denoising steps and rollouts. The method starts from the core content named in the prompt, allocates the same group-relative advantage to more relevant latent regions with almost no additional computational overhead, and then applies a spatially resolved policy objective so updates are stronger where they will affect the external reward.

The paper frames the problem as a granularity mismatch: conventional RL post-training compresses the final-image reward into a single scalar advantage and applies it uniformly across the generative trajectory. STAR addresses temporal structure by recognizing that different denoising steps correspond to different generation stages, and it addresses spatial structure by locating the content that determines text alignment only in parts of the image.

How was STAR evaluated and what were the results?

The authors used Stable Diffusion 3.5 Medium as the base model and tested STAR on three tasks: GenEval, OCR text rendering, and PickScore, reporting specific numeric improvements. STAR achieved 0.9759 on GenEval, 0.9757 on OCR, and 23.60 on PickScore, all without changing the external reward source.

Evaluation focused on compositional semantic alignment, text rendering quality, and preference optimization. The paper emphasizes that the improvements come from reallocating the same final-image reward into spatially and temporally targeted signals during policy updates, rather than altering the reward function itself. The reported scores give concrete, source-attributed benchmarks for STAR’s performance on those three tasks.

Why does it matter?

Text-to-image generation naturally decomposes across time and space: denoising steps map to stages of synthesis, and the prompt-relevant content often occupies only part of the canvas. By aligning policy updates with that structure, STAR lets training signal concentrate where it can change outputs meaningfully. That matters because it addresses an optimization mismatch that can otherwise blunt RL post-training, improving alignment and text rendering while keeping the same external reward metrics and nearly the same computational cost.

Putting the reward strength where it will change pixels reduces wasted gradient signal and makes post-training updates more efficient. For practitioners tuning diffusion or flow generators, the method promises a way to boost task-specific metrics while retaining existing reward sources.

What to watch

Check whether STAR’s gains at the reported 0.9759 GenEval, 0.9757 OCR and 23.60 PickScore hold across other base models and datasets, and whether the claim of almost no additional computational overhead scales as the allocation maps are applied in larger or more complex rollouts. The paper’s next confirmations will be reproductions and broader tests that keep the external reward source unchanged.

Paper metadata: arXiv identifier arXiv:2606.17979, version history shows initial submission on 16 Jun 2026 and a revision on 18 Jun 2026. The method and numbers above are drawn directly from the authors’ abstract and evaluation summary.