Agent Step Value ASV: 1,100 Transitions, Bayesian Surprise

Andrew Zhang and Chengzhan Li introduced Agent Step Value (ASV) in an arXiv preprint submitted 5 Jul 2026 and revised 7 Jul 2026. ASV is a replay framework that scores before/after states with a stateless LLM evaluator over a fixed candidate set and quantifies how individual transitions move belief and final outcomes.

What is Agent Step Value (ASV)?

ASV is a replay-based scorer that measures how single agent transitions change downstream evaluations: it reports entropy movement, a Bayesian surprise measure of belief change, and an offline gold-margin gain toward a reviewed target. The framework runs a stateless LLM evaluator over fixed candidate sets and replays frozen transitions under changed projection, rationale, prompt, or scoring rules to quantify evaluator-channel sensitivity.

ASV therefore separates step-level signal from final-answer scores. The paper frames this as a way to expose which transitions helped or harmed a trace and to turn prompt sensitivity into a measured channel effect that can be localized across components.

How was ASV tested and what did it find?

The authors ran ASV on a 100-question open-QA study with live PubMed retrieval and DeepSeek log-probability scoring, evaluating 1,100 transitions in total. The replayed traces produced a measured entropy movement of 0.000 and a mean Bayesian surprise of 2.693, indicating near-one-hot belief pivots in the evaluated traces.

ASV also compared different evaluator channels and rationale protocols. Under a 128-token rationale-conditioned protocol, ASV measured a mean gold-margin gain of -2.335 with a 95% confidence interval of [-3.395, -1.272]. Re-scoring the same frozen transitions with direct one-token scoring instead gave a mean gold-margin gain of +4.033. A focused 100-transition component audit attributed this reversal to short generated rationales replacing full states as the scored channel.

Those raw numbers come directly from the paper's open-QA experiments and the authors emphasize that replaying identical transitions under changed scoring rules is central to revealing these channel effects.

Why it matters

The ASV results show that evaluator design and the form of the scored channel can flip a trace-level assessment from a net loss to a net gain, as shown by the switch from -2.335 (rationale-conditioned) to +4.033 (one-token scoring). That magnitude and the near-zero entropy movement alongside a mean Bayesian surprise of 2.693 suggest transitions often produce sharp shifts in evaluator belief rather than gradual uncertainty change. For teams auditing model traces or building evaluator chains, ASV gives a concrete mechanism to localize where generated rationales, projection choices, or prompt differences cause evaluator instability.

By measuring evaluator-channel sensitivity directly, ASV moves beyond reporting final-answer accuracy and toward attributing which steps in a trace genuinely contributed to outcomes. The paper presents both the metrics and an audit method that linked the observed reversal to short generated rationales, showing how component-level analysis can explain puzzling aggregate results.

What to watch

Look for ASV applied to other domains and larger trace sets to see whether the pattern—near-zero entropy movement with large Bayesian surprise and large protocol-dependent swings in gold-margin gain—holds beyond PubMed open-QA traces. Also watch for follow-up audits that vary rationale length, projection, and prompt systematically to map which channel changes produce the largest evaluation reversals.

References and data points drawn from the arXiv preprint "Agent Step Value: Probing the Observer Effect in Black-Box Traces" by Andrew Zhang and Chengzhan Li (submitted 5 Jul 2026, revised 7 Jul 2026). Specific experimental figures: 100-question open-QA study, 1,100 evaluated transitions, entropy movement 0.000, mean Bayesian surprise 2.693, mean gold-margin gain -2.335 (95% CI [-3.395, -1.272]) under a 128-token rationale-conditioned protocol, and +4.033 under direct one-token scoring. A 100-transition component audit traced the reversal to short generated rationales.