LongWebBench benchmark: Evaluating long-horizon webpage generation

LongWebBench, submitted to arXiv on 16 Jun 2026 by Yi Zhao and co-authors, is a new benchmark that evaluates webpage generation across long-horizon settings both structurally and functionally. The suite includes 490 real-world long webpages for structural evaluation and 507 goal-oriented interaction tasks distributed over 129 webpages for functional testing.

What is LongWebBench?

LongWebBench is a benchmark and evaluation pipeline for long-horizon webpage generation that separates structural fidelity from executable function. It contains 490 real-world long webpages for structural fidelity assessment and 507 goal-oriented interaction tasks across 129 webpages for functional evaluation, and the authors provide both code and data at a public URL.

The benchmark is motivated by limitations in prior evaluations that focused on short, single-screen, mostly static webpages. LongWebBench targets longer, multi-screen pages and multi-step interactions to better reflect real-world webpage complexity.

How does LongWebBench measure structure and function?

LongWebBench uses two complementary protocols: a multi-dimensional VLM-based metric for assessing long-range structural coherence, and a DOM-augmented agent-based pipeline for end-to-end functional verification. The VLM-based metric scores structural fidelity across long webpages, while the DOM-augmented agent pipeline attempts to execute multi-step, goal-oriented interactions on generated pages.

The paper evaluates state-of-the-art open-source and proprietary vision-language models under both single-image and multi-image input settings. Experiments show structural fidelity degrades as webpage length increases, and visually plausible generations frequently cannot support executable multi-step interactions.

What did the experiments find?

Evaluations with existing VLMs reveal two core gaps: structural degradation on longer pages and failures in interaction execution despite reasonable visuals. The authors report that while generated pages can appear visually plausible, they often fail to support executable multi-step interactions when tested by the DOM-augmented agent pipeline.

The benchmark setup therefore stresses that visual similarity alone is insufficient: functional verification via interaction is needed to judge whether generated pages truly replicate usable webpages. The study spans both single-image and multi-image generation settings to probe how input scope affects outcomes.

Why it matters

Benchmarks shape research priorities. By providing 490 long webpages and 507 interaction tasks, LongWebBench shifts evaluation from surface-level visual fidelity toward practical, multi-step functionality. That shift exposes limits in current VLMs: models may hallucinate visually convincing layouts that break under interaction, which matters for any application that must generate usable, multi-screen web content.

Researchers and system builders aiming to deploy webpage-generation models will need metrics that capture executable behavior, not only image similarity. LongWebBench supplies both the datasets and an agent-based verification pipeline to make that possible.

What to watch

Track the public code and data the authors link in the paper for reproducible runs and community benchmarks. Also watch whether subsequent work reports improvements on the DOM-augmented agent-based functional tests or reduces structural degradation on the 490 long webpages in LongWebBench.

References and notes

• Paper title: LongWebBench: Evaluating Structural and Functional Webpage Generation in Long-Horizon Settings. Submission date: 16 Jun 2026. Authors: Yi Zhao, Zhen Yang, Mengpan Chen, Mingde Xu, Shanghui Gong, Xijun Liu, Jibing Gong, Jie Tang. The paper is 49 pages with 38 figures and is available on arXiv with DOI https://doi.org/10.48550/arXiv.2606.17727.