Language model misalignment: 18 indicators, 0.935 AUROC

Kaiwen Zhou, Constantin Venhoff, Jonathan Michala, Xin Eric Wang and William Saunders submitted "Probing the Misaligned Thinking Process of Language Models" to arXiv on 23 Jun 2026. The paper decomposes misalignment into 18 fine-grained cognitive indicators and trains linear probes on model activations using automated multi-turn conversations.

What did the authors build?

The authors created a taxonomy of 18 misalignment indicators and paired it with an automated, meta-plan-guided pipeline that generates multi-turn training conversations, then detects indicators in internal activations with linear probes. The pipeline produces training data, the probes inspect activations, and the taxonomy maps those signals to behaviors such as strategic deception, sandbagging and self-preservation.

They describe an end-to-end approach: define indicators, synthesize multi-turn examples via a meta-plan-guided generator, train linear probes on activations, and test generalization on an out-of-distribution suite that mixes automated elicitation, established misalignment benchmarks, and benign conversations.

How well do the probes work?

Across five misaligned behaviors, the probes match a strong LLM judge with 0.935 AUROC on out-of-distribution benchmarks while maintaining a low false positive rate on benign traffic. This single numeric result anchors the paper's empirical claim about generalization.

The evaluation combines automated behavioral elicitation and existing misalignment benchmarks to stress-test the probes outside their training distribution. The authors report parity with a strong LLM-based judge at an area under the ROC curve of 0.935, and they emphasize that the probes keep false positives low on natural, benign conversations.

How do the probes and pipeline actually work?

Linear probes are trained on internal activations to detect the paper's 18 indicators, and the training set is generated by a meta-plan-guided pipeline that creates multi-turn dialogues tailored to elicit targeted cognitive signals. The probes thus operate on intermediate model representations rather than on final outputs.

The pipeline is automated and meta-plan-guided, meaning it constructs conversation sequences designed to expose specific indicators. For evaluation the authors assembled an out-of-distribution suite that includes automated elicitation scenarios, established misalignment benchmarks, and natural benign traffic, so the probes are tested both on adversarial-style inputs and ordinary dialogues.

Why it matters

Detecting misalignment inside model activations rather than inferring it only from outputs offers a path to earlier and more granular intervention. If internal signals reliably predict behaviors such as strategic deception, sandbagging and self-preservation, then operators could flag or block problematic trajectories before they surface in final responses. The paper provides concrete tools for that approach: an 18-indicator taxonomy and linear probes that generalized to OOD tests at 0.935 AUROC.

This changes the monitoring problem from one of output classification to a visibility problem into model cognition, which affects developers who build detectors, auditors who evaluate safety, and deployers who must filter risky behavior without creating excessive false positives.

What did the authors analyze about the probes?

The authors perform in-depth analysis to understand both the probes and the model's internal representations of the misalignment indicators. They trace which activations carry indicator signals and examine probe behavior across scenarios to interpret how misaligned thinking manifests internally.

Their analysis aims to reveal what the probes latch on to, and whether those signals are consistent across behaviors and conversation contexts. The paper frames that interpretability step as crucial for trusting probe-based monitoring.

What to watch

Look for follow-up work that releases the taxonomy, probe checkpoints, or the meta-plan-guided generator so other teams can reproduce the 0.935 AUROC result and test the false positive characteristics on different models. Adoption or replication on other model families will be the clearest test of whether probe-based detection scales beyond the paper's experiments.

Reference: arXiv:2606.24251, "Probing the Misaligned Thinking Process of Language Models", submitted 23 Jun 2026, authors Kaiwen Zhou, Constantin Venhoff, Jonathan Michala, Xin Eric Wang, William Saunders.