ASK+: uncertainty-gated LLM help raises DoorKey success to 93%

ASK+ converts small language models into occasional correctors for reinforcement learning agents operating under partial observability, the authors show in a paper submitted on 2 Jul 2026 by Juarez Monteiro and five coauthors. Across standard benchmarks, ASK+ supplies a stateful, trajectory-aware prompt and structured chain-of-thought to the SLM and raises DoorKey success to 93%, up from 89% for vanilla ASK, and pushes FourRooms success from 53% to 70%. HigherLower accuracy reaches 73.7%, matching the SLM-only upper bound.

How does ASK+ work?

ASK+ supplies the SLM with trajectory-aware context (a partially revealed map, visited positions, and action history) plus a structured chain-of-thought, turning the model from a passive redundancy check into an active consultant that occasionally issues independent actions. The paper contrasts this with the "bare egocentric prompt," which it identifies as insufficient context for genuine reasoning, and reports that vanilla uncertainty-gated approaches suffer an overwrite rate "at or near zero." ASK+ instead uses a stateful prompt so the SLM can use trajectory information to form action proposals that sometimes override the policy.

The authors also treat the gating signal carefully: they show the predictive entropy measure used for selective querying reflects action uncertainty rather than state uncertainty and remains informative inside POMDPs. That preserves the core idea of uncertainty-gated assistance beyond fully observable settings while changing what is fed to the SLM and how its output is judged.

How does ASK+ perform across environments and models?

ASK+ drives measurable gains on three reported benchmarks. On DoorKey, vanilla ASK equals PPO at 89% success while ASK+ reaches 93% success. On FourRooms, success improves from 53% to 70% with ASK+. On HigherLower, ASK+ achieves 73.7% accuracy, which the paper notes matches the SLM-only upper bound. Across all tested environments the paper finds Qwen3.5-2B matches or exceeds Qwen3.5-4B, indicating that prompt design and selective gating have a larger impact than model scale in these experiments.

Those numbers expose two concrete effects. First, richer context and chain-of-thought convert the SLM from rarely acting to occasionally correcting the agent. Second, smaller SLMs can match or beat larger ones when given better prompts and gating, reducing the apparent premium on model size for this assistance role.

Why it matters

ASK+ shows that the failure of prior uncertainty-gated schemes was a context problem rather than an LLM capacity problem. Providing a partially revealed map, visitation history, and action trace gives small LLMs the situational grounding they need to offer useful, sometimes corrective, guidance. The finding that Qwen3.5-2B matches or exceeds Qwen3.5-4B reinforces that careful prompt engineering and gating can outweigh raw model scale for in-loop assistance under partial observability.

That matters for teams wanting to add LLM guidance inside POMDPs without shipping large models, and for research on human-in-the-loop or mixed-autonomy systems where occasional model intervention must be both accurate and selective.

What to watch

Check the paper's arXiv page for the linked code and data to reproduce these results, and watch for follow-up evaluations that test the predictive-entropy gating signal and stateful prompts on larger or more complex POMDPs. Also look for comparisons that measure the frequency and conditions under which the SLM issues independent actions versus serving as a redundancy check.