Foundation Models for Automatic CAD: 97-problem benchmark

J de Curtò, Victoria Guillén and I. de Zarzà published "Foundation Models for Automatic CAD Generation" on arXiv on 6 Jul 2026, presenting LLMForge, a multi-model text-to-CAD framework and a curated benchmark of 97 engineering design problems. The paper evaluates seven foundation models across two critique regimes and four canonical geometry families.

What did the authors build and test?

They built LLMForge, a text-to-CAD pipeline that integrates JSON-schema validation, analytic feature scoring, mesh synthesis and multi-round iterative refinement, and tested it on 97 engineering design problems spanning four geometry families. LLMForge runs under two critique regimes: IterTracer, which uses a Phong-shaded ray-trace renderer plus analytic visual metrics, and IterVision, which replaces the analytic scorer with a vision-language model critic (Qwen2.5-VL-72B) that applies chain-of-thought visual reasoning.

The analytic metrics used with IterTracer include silhouette IoU, hole visibility, edge clearance and aspect-ratio conformance. IterTracer provides lightweight geometry-aware feedback across refinement rounds. IterVision evaluates rendered views semantically, assessing spatial coherence and design intent rather than strictly geometric measures.

How did the models perform on the benchmark?

Seven foundation models were evaluated: DeepSeek-V3.2, Qwen3-235B-A22B, Llama-3.3-70B, Gemma-3-27B, GLM-4.5, MiniMax-M2.1 and INTELLECT; the paper reports a tight cluster of top performance under IterTracer and very high mesh success overall. Under the IterTracer regime the four highest-ranked models form a tight cluster with an overall mean in [0.885, 0.890], and the pipeline achieved 98.97% mesh success across the benchmark. When the authors applied VLM-based critique in IterVision, the leading model produced 100% watertight meshes while revealing systematic difficulty on rotationally symmetric geometries such as flanged cylinders, where visual and semantic scoring diverged most.

The benchmark covers four canonical geometry families described in the paper: plates with holes and bolt circles, multi-feature boxes, flanged cylinders and L-brackets. The study compares foundation models of differing scale and tuning, and finds that compact instruction-tuned models can match substantially larger systems under geometry-aware analytic critique.

Why it matters

The paper shows that compact instruction-tuned models can achieve near-parity with much larger systems on automated CAD generation when paired with geometry-aware evaluation. That result, together with a 98.97% mesh success rate and a 100% watertight outcome for the leading model under VLM critique, implies automated pipelines can produce industrially usable geometry more reliably than naive text-to-3D attempts. The authors also discuss failure modes, CAD-oriented prompting and implications for industrial workflows and scalable automated mechanical design, signaling practical relevance beyond academic benchmarks.

What to watch

Watch for follow-up work that addresses the paper's reported weakness on rotationally symmetric geometries, especially flanged cylinders where visual and semantic critics diverge. Also expect the authenticated Springer chapter in the series Learning and Analytics in Intelligent Systems, where the accepted version will appear in Advances in Global Applied Artificial Intelligence, to give the final curated results and additional context.

Details and paper metadata

• Submission: arXiv:2607.05573, submitted 6 Jul 2026.
• Authors: J de Curtò, Victoria Guillén, I. de Zarzà.
• Framework: LLMForge, with IterTracer (analytic critic) and IterVision (VLM critic using Qwen2.5-VL-72B).
• Benchmark: 97 engineering design problems across plates with holes and bolt circles, multi-feature boxes, flanged cylinders and L-brackets.
• Models evaluated: DeepSeek-V3.2, Qwen3-235B-A22B, Llama-3.3-70B, Gemma-3-27B, GLM-4.5, MiniMax-M2.1, INTELLECT.
• Key metrics reported: IterTracer top-cluster overall mean in [0.885, 0.890], 98.97% mesh success, IterVision produced 100% watertight meshes for the leading model.

The paper is accepted as a book chapter in "Advances in Global Applied Artificial Intelligence" and will appear in Springer series: Learning and Analytics in Intelligent Systems; an authenticated version will be published by Springer.