DRIFT: On-Policy Data Attribution for SFT, boosts 7B models

DRIFT, a method described in an arXiv paper submitted 16 Jun 2026, refines supervised fine-tuning data by attributing training instances using on-policy Influence Functions. The paper, by Zefan Wang, Lincheng Li, Tianyu Yu and Yuan Yao (arXiv:2606.18307), argues that replacing off-policy references with model rollouts reduces attribution error and lifts the performance ceiling for SFT.

What problems does DRIFT address?

DRIFT targets two structural failures the authors identify in standard Influence Function based attribution: a proximity gap from off-policy validation targets, and a strong bias toward gradient norm. Standard formulations use external validation targets that sit off the model's parameter manifold, creating a proximity mismatch; they also overweight examples with large gradient norms, which the authors call a gradient-hacking bias. DRIFT reframes attribution to eliminate those two failure modes.

How does DRIFT work?

DRIFT uses the model's on-policy rollouts as the validation targets, then computes influence scores that are signed by trajectory correctness and debiased against gradient-norm bias. Using on-policy rollouts empirically minimizes the parameter proximity gap and better meets the local neighborhood assumption behind Influence Functions. The method applies signed weighting based on whether a rollout's trajectory is correct, and it explicitly debiases influence scores so a small set of validation queries can reliably anchor attribution across the full dataset.

How was DRIFT evaluated and what were the results?

The paper reports experiments on 7B-parameter instruction and reasoning models, and finds that DRIFT consistently raises the performance ceiling compared with existing data curation baselines. The authors state that DRIFT outperforms prior data curation baselines across those 7B models, showing that refining the data distribution toward instances most likely to improve the final model can increase ultimate capability beyond mere training-speed or budget-preservation gains.

Why it matters

DRIFT shifts focus from selecting smaller subsets that preserve performance to actively refining the data distribution to improve final-model capability. Using on-policy rollouts as validation targets aligns attribution with the model's current behavior, which should make per-example signals more meaningful for tuning instruction-following and reasoning abilities. If the approach generalizes, teams fine-tuning LLMs could obtain higher ceilings without larger architectures, simply by changing how they attribute and weight training instances.

What to watch

Look for independent reproductions on other model scales and for open-source implementations linked to the arXiv entry. The next clear milestone will be evaluations of DRIFT on non-7B checkpoints or public instruction-tuning corpora that confirm whether on-policy attribution and the debiasing steps generalize across architectures and data sources.

Paper details: DRIFT: Refining Instruction Data via On-Policy Data Attribution, authors Zefan Wang, Lincheng Li, Tianyu Yu, Yuan Yao; arXiv:2606.18307, submitted 16 Jun 2026. DOI: 10.48550/arXiv.2606.18307.