OpenAgent: Agents' fragility under open-world tool shifts

OpenAgent formalizes the problem of agents operating under open-world shifts and shows such agents break when their training is static. The paper, "Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use," was submitted to arXiv on 1 Jul 2026 (arXiv:2607.01084) by Song-Lin Lv and four coauthors and accepted by ICML 2026.

What did the authors define and test?

OpenAgent is a problem setting that models distributional shifts across queries, actions, observations, and domains; the authors implement it by building a controlled sandbox that defines fine-grained environmental shifts across a four-tier hierarchy: Perception, Interaction, Reasoning, and Internalization. The sandbox lets the paper isolate how each kind of shift affects an agent trained in static conditions. The paper links its experiments, code and resources at https://github.com/LAMDA-NeSy/OpenAgent.

The paper compares agents trained by Supervised Fine-Tuning (SFT) and by Reinforcement Learning (RL), examining how both training regimes respond when the environment departs from their training distributions. The experiments are framed to diagnose which dimensions of mismatch most undermine tool use and overall agent utility.

How did agents behave under open-world shifts?

SFT and RL agents both experienced performance degradation when exposed to open environmental shifts, though the authors report varying degrees of fragility depending on the shift type. The analysis breaks down the failure modes by the four-tier hierarchy so readers can see whether perception changes, interaction changes, reasoning changes, or deeper internalization gaps drive the drop in performance.

Beyond diagnosing failures, the authors propose an intervention called Perturbation-Augmented Fine-Tuning, a disturbance-based strategy applied to SFT aimed at improving robustness and utility in realistic environments. The paper positions that intervention as a foundation for hardening SFT-trained agents against the kinds of distributional shifts defined in the OpenAgent setting.

Why it matters

OpenAgent reframes a frequent mismatch: benchmarks used in static testing do not capture the multi-dimensional shifts agents will face in deployment. The paper supplies a concrete taxonomy of those shifts and a sandbox to measure them, which clarifies where current SFT and RL pipelines fall short. For practitioners, that means systems validated on static benchmarks may underperform when tool sets, user queries, or observation patterns change. The proposed perturbation-based fine-tuning is a direct, testable step toward closing that gap.

What the paper contributes to the field

The paper contributes three practical elements: the OpenAgent problem formalism, a controlled experimental sandbox with a four-tier shift hierarchy (Perception, Interaction, Reasoning, Internalization), and a candidate robustness method, Perturbation-Augmented Fine-Tuning. The work appears aimed at both researchers who need a standardized way to measure open-world generalization and engineers seeking interventions that can be applied to SFT pipelines.

What to watch

Watch for the GitHub release the authors cite: https://github.com/LAMDA-NeSy/OpenAgent, which should provide the sandbox and code used in the paper. Also watch for follow-up evaluations at ICML 2026 where the community will see peer feedback and replication of the perturbation-augmented strategy across different agent architectures.

Details grounded in the paper: the submission appears on arXiv as arXiv:2607.01084, it lists Song-Lin Lv and four coauthors, and the authors state the work was accepted by ICML 2026. The paper frames OpenAgent as a formal problem of tool-use agents in open-world conditions and reports that both SFT and RL training regimes suffer performance degradation under those shifts.