Rubric-Conditioned Self-Distillation: arXiv paper beats GRPO

Rubric-Conditioned Self-Distillation, submitted to arXiv on 17 Jun 2026 by Siyi Gu, Jialin Chen, Sophia Zhou, Arman Cohan and Rex Ying, proposes a new post-training framework that conditions teacher models on rubric criteria and uses those teachers to provide token-level guidance to students. The paper evaluates the approach on a suite of science reasoning benchmarks and reports that the method surpasses GRPO by 1.0 points and OPSD by 0.9 points on average.

What is rubric-conditioned self-distillation?

Rubric-Conditioned Self-Distillation is a framework that uses criterion-level rubrics as structured, fine-grained feedback and conditions the teacher model on those rubrics to generate token-level guidance for the student. The method avoids using a single reference rationale as the sole supervision target, instead turning rubric-level criteria into guidance over sampled student trajectories so credit assignment is more fine-grained than scalar reward optimization.

The paper contrasts this approach with two common post-training signals. Distillation typically depends on chain-of-thought annotations, which the authors note are expensive and can be noisy, incomplete or partially incorrect. Reinforcement learning with verified rewards compresses evaluative feedback into a scalar signal, which the authors say obscures which aspects of a response should be improved.

How does the method work in practice?

The authors instantiate the framework with a two-stage pipeline: first learn to generate task-specific rubrics, then train a rubric-guided reasoner that receives token-level guidance from a teacher conditioned on those rubrics. In short: generate rubrics, condition a teacher on rubric criteria, sample student trajectories, and use teacher outputs to supervise token predictions rather than relying on a single gold rationale or a scalar reward.

That two-stage pipeline is central to the design the authors describe. By conditioning the teacher on criterion-level rubrics, the paper claims the teacher can assign credit at a finer granularity across the reasoning process, enabling more targeted corrections than scalar reward signals provide.

How was it evaluated and what were the results?

The authors evaluated the approach on a diverse suite of science reasoning benchmarks and report average gains over two baselines: the rubric-conditioned method surpasses GRPO by 1.0 points and OPSD by 0.9 points on average. Those numeric improvements are the concrete performance claims provided in the paper's abstract.

The evaluation framing emphasizes converting rubric-level criteria into token-level guidance over the reasoning process, positioning the reported 1.0 and 0.9 point margins as evidence that rubric-conditioned supervision yields measurable gains over existing approaches like GRPO and OPSD.

Why it matters

The paper addresses two persistent training frictions: reliance on single annotated rationales and the loss of structure when feedback is reduced to a scalar reward. If rubric-conditioned supervision consistently turns rubric criteria into effective token-level corrections, models can learn which individual reasoning steps to change rather than only whether a final answer is right or wrong. That could lower dependence on expensive chain-of-thought annotations and make on-policy distillation signals more informative.

What to watch

Look for independent reproductions and broader evaluations beyond the paper's science reasoning suite: confirmation would be seeing the rubric-generation and rubric-guided reasoner pipeline reproduced and the reported average gains (1.0 point vs GRPO, 0.9 point vs OPSD) replicated across other tasks. The authors' two-stage design also invites scrutiny of how robust rubric generation is across domains and whether token-level guidance scales to larger or more open-ended reasoning tasks.

References: paper "Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation," arXiv:2606.19327, submitted 17 Jun 2026, authors Siyi Gu; Jialin Chen; Sophia Zhou; Arman Cohan; Rex Ying.