Task Insensitivity in Language Agents, Task-Perturbed NLL

Jingyu Liu, Xiaopeng Wu, Kehan Chen, Chuan Yu and Yong Liu identify task insensitivity as a key source of out-of-distribution failure in language agents and propose a targeted training fix. The paper "Diagnosing Task Insensitivity in Language Agents" was submitted to arXiv on 25 Jun 2026 (arXiv:2606.26918, v1) and the submission includes a 3,055 KB PDF.

What did the authors find?

The authors find that language models acting as long-horizon agents often apply learned patterns from training instead of following new task instructions, producing what they call task insensitivity: models continue with actions aligned with the original task even when the instruction is semantically corrupted. They report a consistent training-time attention drift away from task tokens and toward local observations, which the paper interprets as an optimization bias toward shortcuts.

The paper frames two concrete failure modes. First, when an instruction is semantically corrupted and cannot be directly answered, models may nevertheless persist with actions appropriate to the original task. Second, replacing the task description in a trained prompt with a similar but distinct task can still elicit the same action sequence, showing weak dependence of action on the instruction text.

How does Task-Perturbed NLL Optimization work?

Task-Perturbed NLL Optimization is presented as a lightweight contrastive regularizer that explicitly encourages action dependence on the task instruction. In practice the method perturbs task descriptions during negative-log-likelihood (NLL) training so the model must discriminate correct task-conditioned actions from those tied to spurious training patterns.

The paper states the intervention as "Task-Perturbed NLL Optimization, a lightweight contrastive regularizer," and positions it as an augmentation to existing NLL objectives rather than a full architectural change. The authors argue this shifts attention back toward task tokens and reduces the optimization bias that favors local observation cues over instructions.

What evidence do they provide?

The authors describe extensive evaluations showing the intervention improves task sensitivity and out-of-distribution generalization while preserving more stable attention to task tokens. The arXiv entry links the full PDF and auxiliary materials; the submission record for version one appears with the timestamp and file size noted above (Submitted on 25 Jun 2026, [v1] Thu, 25 Jun 2026 11:53:41 UTC, 3,055 KB). The abstract summarizes qualitative and training-time attention findings as the core empirical observations.

Why it matters

Task insensitivity exposes a specific optimization failure: models learn shortcuts tied to training artifacts rather than forming robust instruction-conditioned policies. That matters for any deployment relying on language models to follow changing instructions across long-horizon tasks, because the failure mode can persist even when the instruction is explicitly changed or corrupted. A lightweight, training-time regularizer that increases instruction dependence addresses the problem without requiring model redesign, which could make it attractive to researchers and engineers updating agent training pipelines.

What to watch

Look for the paper's experimental details, accompanying code, and independent replications that quantify how much Task-Perturbed NLL reduces attention drift and improves OOD task success. The arXiv entry lists links and toggles for code and data resources associated with the article; those resources will be the next concrete signals of adoption and reproducibility.

References and identifiers: the paper is available as arXiv:2606.26918 [cs.AI], submitted 25 Jun 2026 by Jingyu Liu and coauthors. The authors summarize their contribution as an explicit regularizer to encourage action dependence on task instructions and report improvements to task sensitivity and OOD generalization in their evaluations.