BIM-Edit: Benchmarking LLMs for IFC-based BIM Editing

BIM-Edit, a benchmark posted to arXiv on 18 June 2026 by Bharathi Kannan Nithyanantham and colleagues, tests large language models on natural-language edits of Building Information Models stored in the Industry Foundation Classes format. The benchmark contains 324 editing tasks spanning 11 realistic building models and 36 synthetic scenes; the best-performing model achieves a 49.5% average across three metrics, and no model fully solves more than 3.4% of tasks.

What is BIM-Edit?

BIM-Edit is a task suite that measures LLMs on editing existing BIM files, not just generating new geometry: it focuses on IFC-based models and covers semantic and relational structure as well as geometry. The benchmark contains 324 editing tasks expressed across three instruction categories — direct, spatial, and topological — and uses 11 realistic building models plus 36 synthetic scenes to cover diverse scenarios.

BIM-Edit frames edits as natural-language instructions that require scene understanding, correct modification, and preservation of semantic relations. The authors describe the domain as challenging because building models encode geometry together with semantic and relational structure, so success requires more than geometric correctness.

How well did current LLMs perform on the benchmark?

Across the evaluated LLMs, the top model posts a 49.5% average score when judged on three evaluation dimensions: geometric accuracy, semantic validity, and topological consistency, and no model fully solves more than 3.4% of the 324 tasks. These numbers come directly from the benchmark results reported in the paper.

The evaluation breaks outputs into three dimensions to reflect engineering requirements: geometric accuracy checks shape and placement, semantic validity checks that entities and attributes remain correct, and topological consistency checks relations and connectivity. The authors present the aggregate figures above to show there is a substantial gap between current LLM capabilities and the expectations of structured CAD and BIM workflows.

Why does this benchmark matter?

BIM-Edit shifts evaluation from generation-only CAD benchmarks to editing existing, semantically rich models, and that change exposes shortcomings current LLMs have with preserving relationships and semantics. The benchmark’s combination of realistic models and synthetic scenes, plus its three-category instruction set, specifically stresses scene-grounded edits where geometry, semantics, and topology interact.

For practitioners, the results signal that relying on LLMs for IFC edits will likely require additional safeguards or hybrid workflows: the best average score reported is 49.5%, and complete success on tasks is rare, with a maximum of 3.4% fully solved. For researchers, BIM-Edit supplies a concrete, reproducible testbed aimed at closing that gap.

What to watch

Watch for future model evaluations that raise the reported 49.5% average and push the fully-solved rate above 3.4%. Progress on any of the three reported metrics — geometric accuracy, semantic validity, or topological consistency — would be a clear signal that LLMs are becoming more suitable for IFC-based BIM editing.

The dataset and task breakdown in BIM-Edit give researchers a specific baseline: 324 tasks, three instruction categories (direct, spatial, topological), 11 realistic building models, and 36 synthetic scenes. Improvements that are measured against those exact figures will provide a direct comparison to the paper’s findings.

Authors and submission details: the paper, titled "BIM-Edit: Benchmarking Large Language Models for IFC-Based Building Information Modeling," lists Bharathi Kannan Nithyanantham, Clemens Kujat, Tobias Sesterhenn, Stefan Telgmann, Jörn Plönnigs, Stefan Lüdtke, and Christian Bartelt, and was submitted to arXiv on 18 June 2026.