Personalized Multimodal Generation: NaviGen method and experiments

NaviGen converts a user’s interaction history into executable instructions for image and video synthesis, according to an arXiv paper (arXiv:2606.24196) submitted 23 Jun 2026 and revised 24 Jun 2026. The method encodes behavior as paired identifiers and trains a two-stage supervised fine-tuning plus reinforcement learning pipeline to distill instruction-writing skill and align generation with user intent.

What is NaviGen and how does it represent behavior?

NaviGen represents each item with a dual identifier that couples a collaborative code and a textual code, using that pair as a behavioral substrate and a semantic bridge in one token stream. The dual identifier design is the core representation choice: the collaborative code captures behavioral signals while the textual code provides language-grounded semantics suitable for downstream instruction writing.

The paper frames this representation as necessary because modern AIGC pipelines assume a well-formed creation instruction while real end users rarely articulate visual details. By turning interaction histories into a token stream of dual identifiers, NaviGen makes behavior legible to language reasoning and to models that need to write instructions for multimodal synthesis.

How does the SFT+RL pipeline train instruction-writing and align generation?

NaviGen uses a two-stage SFT+RL pipeline: stage one distills preference reasoning and instruction writing from evolutionarily searched supervision, and stage two aligns generation with user intent through hierarchical and self-consistent rewards. The pipeline first applies supervised fine-tuning to learn instruction-writing behavior, then uses reinforcement learning with specifically designed reward structure to push outputs toward user-aligned, visually generatable instructions.

Key training components are named in the paper: evolutionarily searched supervision for the initial distillation, and hierarchical plus self-consistent rewards for the RL alignment stage. The authors position these elements to address two obstacles they identify: encoding behavior in a language-legible form, and teaching instruction-writing skills that neither pretraining nor raw behavior data provide.

How was NaviGen evaluated and what did the experiments show?

The authors evaluated NaviGen across product, game, and short-video domains and report that it improves personalized image and video generation, strengthens next-item prediction, and yields more specific, relevant, and visually generatable instructions. The experiments span three application areas to test cross-domain efficacy rather than a single vertical.

The paper includes 16 pages of content, supported by 15 figures and 5 tables, and the authors provide code at the listed URL. Those artifacts accompany the experimental claims and the described training pipeline, and the paper is available under arXiv identifier arXiv:2606.24196 (DOI https://doi.org/10.48550/arXiv.2606.24196).

Why it matters

NaviGen tackles a practical gap between how users behave and how generative models expect prompts: users supply interaction history, not polished creation instructions. Converting behavior into language-ready instructions could let existing multimodal generators produce outputs that better match individual preferences without requiring users to craft detailed prompts. If the dual-identifier representation and the two-stage SFT+RL recipe generalize, they provide a repeatable path from recommendation-style signals to actionable synthesis instructions.

What to watch

Look for the released code and accompanying replication materials at the paper’s provided URL and for follow-up results that quantify gains on specific generation metrics. The next milestones to check are public benchmarks or community replications that measure how much instruction specificity and downstream visual quality improve in each domain (product, game, short-video).

Paper metadata: authors Hengji Zhou, Yufeng Liu, Ye Liu, Yong Xu, Lianghao Xia, Liqiang Nie; first submitted 23 Jun 2026, revised 24 Jun 2026; 16 pages, 15 figures, 5 tables; arXiv:2606.24196, DOI https://doi.org/10.48550/arXiv.2606.24196. Code is released at the URL given in the paper.