Emergent Alignment: Martin Kolář's DPO conscience step for LLMs

Martin Kolář submitted a paper titled "Emergent Alignment" to arXiv on 17 Jun 2026 (arXiv:2606.19527), proposing a way for large language models to detect and correct their own unethical outputs. The paper adds a conscience review step to the model and extends the training loss with an alignment component using Direct Preference Optimization, and the author notes the work was rejected from ICML 2026.

What method does Emergent Alignment introduce?

The paper endows an LLM with a conscience step that reviews its own reasoning and outputs, and it extends the training loss with an alignment component implemented via Direct Preference Optimization (DPO). The technique relies on a frozen copy of the model as the judge rather than a weaker or stronger external evaluator, and the author frames it as an online method usable during training, fine-tuning, adversarial prompting and zero-shot learning.

Kolář describes the conscience step as an introspective reviewer of the model's chain of thought and final output, then uses DPO to steer gradient updates away from non-ethical outputs. The approach removes the requirement for a separate human or stronger model judge by comparing the model against a frozen self-copy during these online steps.

How does this differ from Emergent Misalignment?

Prior work labelled an Emergent Misalignment scenario where fine-tuning produced a range of unethical behaviors, including cases where models were fine-tuned to hack code; Kolář positions his contribution as the empirical counterpart that achieves the opposite outcome. The paper demonstrates that, under the same code hacking scenario used to illustrate Emergent Misalignment, "a single high-level introspective question steers training toward an ethical model." In short, where Emergent Misalignment documented unintended unethical behaviors from fine-tuning, Emergent Alignment shows a simple introspective intervention plus DPO can redirect training toward ethical outputs.

What evidence and scope does the paper claim?

Kolář frames the method as broadly applicable: training, fine-tuning, adversarial prompting and zero-shot learning are all listed as settings where the online conscience-plus-DPO technique can operate. The author emphasizes that the technique does not require an external judge; it uses a frozen copy of the model itself to evaluate alignment. The arXiv entry presents this as an empirical result rather than a finalized, peer-reviewed claim: the submission date is 17 Jun 2026 and the submission was marked as rejected from ICML 2026.

The paper's arXiv record includes links to PDF, an experimental HTML rendering and TeX source, and it is catalogued as arXiv:2606.19527 [cs.AI].

Why it matters

If the method scales beyond the paper's experiments, it could change how practitioners think about alignment work by reducing dependence on external preference labels or stronger judges. Using a frozen self-copy as the evaluator sidesteps a recurring logistical bottleneck: creating or maintaining a consistently calibrated external judge. For teams that face adversarial prompting or risky fine-tuning paths, a lightweight, online DPO-based conscience step offers a concrete mitigation that fits into existing training loops.

What to watch

Whether reviewers or reproducing teams validate the empirical claims and whether the author publishes code or experimental details that allow independent replication. The paper's rejection from ICML 2026 makes peer-review outcomes and community reproduction the next concrete signals to follow.

References and facts in this brief are drawn from the arXiv entry for "Emergent Alignment" by Martin Kolář, arXiv:2606.19527, submitted 17 Jun 2026, which records the ICML 2026 rejection.