GUI Grounding: Quality-Aware Self-Distillation (arXiv 2026)

Trust the Right Teacher: Quality-Aware Self-Distillation for GUI Grounding, by Jingyuan Huang, Zuming Huang, Yucheng Shi, Tianze Yang, Xiaoming Zhai, Wei Chu and Ninghao Liu, was posted to arXiv as arXiv:2606.18101 (v1 submitted 16 Jun 2026, revised v2 17 Jun 2026). The paper targets GUI grounding, the coordinate-sensitive task where vision-language models must identify small target elements in high-resolution screenshots and predict precise screen coordinates.

How does quality-aware self-distillation work?

The method improves on-policy self-distillation, or OPSD, by filtering and calibrating teacher token signals so they remain reliable when the student has deviated from the ground truth. OPSD supplies dense token-level teacher signals beyond hard coordinate labels, but the paper argues naive OPSD can produce unreliable coordinate-token supervision once a student-generated prefix has strayed from the target coordinate. The authors propose two complementary mechanisms: a soft correctness-aware gate that down-weights teacher signals when the teacher's current coordinate-token prediction cannot be completed into the ground-truth box under the student prefix, and teacher-probability scaling that uses the teacher's confidence to calibrate the remaining gated supervision.

What are the core components and why are both needed?

Both components together improve signal quality because they address different failure modes: correctness-aware gating suppresses outright unreliable coordinate-token supervision, while teacher-probability scaling adjusts the strength of the remaining signals using teacher confidence. The paper reports a key empirical finding that neither component alone improves overall performance, whereas combining them consistently improves the base model and outperforms strong baselines. The description of the gate and scaling is the central technical contribution: the gate checks whether a teacher prediction can still be completed into the ground-truth box under a student-generated prefix, and the scaling multiplies the gated signal by the teacher's probability as a lightweight calibration factor.

How well does it perform on benchmarks?

The method was evaluated across six GUI grounding benchmarks, and the authors state it consistently improves the base model and outperforms strong baselines. The paper frames GUI grounding as a coordinate-sensitive task where dense token-level signals can help, and shows that quality-aware OPSD yields systematic gains across the evaluated datasets. The submission lists the experimental claim at scale: experiments across six GUI grounding benchmarks demonstrate the consistent improvement. The paper also links to its arXiv record via DOI https://doi.org/10.48550/arXiv.2606.18101.

Why it matters

GUI grounding requires precise coordinate prediction on cluttered, high-resolution screenshots, a setting where sparse, hard labels provide limited learning signal. Improving the reliability of teacher supervision inside OPSD addresses a practical failure mode of sequence-style coordinate generation: once a student prefix diverges, naive teacher feedback can be misleading. By gating out unrecoverable teacher tokens and scaling the remaining signals by teacher confidence, the approach raises the signal-to-noise ratio of post-training supervision, which should make VLMs more robust to early decoding errors in coordinate-sensitive tasks.

What to watch

Look for code, model checkpoints, or detailed per-benchmark numbers from the authors; the arXiv entry provides links to PDF, TeX source and ancillary materials but the public record here summarizes the method and high-level experimental claim. Also watch whether follow-up work separates the individual contributions of different gating criteria and confidence calibrations across more varied coordinate-generation tasks beyond the six GUI grounding benchmarks the paper evaluates.