LLMs as Optimizers: Direct vs Tool-Augmented vs Tool-Creating

Roko Peran, Luka Hobor, Mihael Kovac and Mario Brcic submitted a survey titled "Large Language Models as Optimizers" to arXiv on 9 Apr 2026 that maps three LLM-as-optimizer paradigms and their performance frontiers. The paper runs six pages, includes one figure and two tables, and was accepted at the 49th ICT and Electronics Convention, MIPRO (Paper ID: #23463).

What are the three LLM-as-optimizer approaches?

Direct optimization, tool-augmented optimization and tool-creating optimization are the three paradigms the paper defines, each handling search and solution refinement differently. Direct optimization uses iterative prompting and heuristic generation to navigate solution spaces; tool-augmented optimization translates natural language problems into formal specifications and orchestrates external solvers; tool-creating optimization has LLMs discover reusable algorithms or heuristics that can be deployed at zero marginal LLM cost.

The authors present these paradigms as distinct strategies for turning real-world search problems into better solutions. Direct optimization keeps the loop inside prompting and heuristic tweaks, tool-augmented hands tasks off to formal solvers, and tool-creating extracts reusable code or heuristics so repeated families of problems can run without repeated LLM invocation.

How do their performance frontiers compare and what gaps remain?

The paper describes current performance frontiers based on benchmarks from the literature, and identifies a critical reasoning gap in current architectures that limits direct approaches while favouring tool-based strategies for auditability. Direct optimization pushes models to search and refine solutions directly but is constrained by reasoning limitations; tool-augmented approaches gain formal rigor through external solvers and better audit trails; tool-creating shifts cost and effort into reusable artifacts, reducing per-instance LLM expense.

Peran et al. do not present a single numeric benchmark in the abstract, but they state the survey synthesizes existing benchmark results to place these paradigms on their performance frontiers. The authors argue there is a trade-off between the future potential of direct optimization and the auditability of tool-augmented optimization, and they suggest even more powerful future models may favour tool-creating because it improves operational efficiency for repetitive problem families.

Why it matters

The taxonomy reframes how LLMs are evaluated on optimization tasks: not just by raw model capability but by how they are used to search, formalize and automate solutions. That shifts evaluation from model-only metrics to system design choices: whether to iterate in prompts, orchestrate classical solvers, or invest once in reusable heuristics. The paper highlights that a persistent "critical reasoning gap" could steer developers toward hybrid pipelines that trade some model autonomy for verifiability and lower marginal cost.

What to watch

Watch whether follow-up benchmarks explicitly separate model reasoning performance from system-level gains produced by tool orchestration or tool creation. The paper's arXiv entry (arXiv:2606.15577) and its acceptance at the 49th ICT and Electronics Convention, MIPRO provide concrete touchpoints for forthcoming presentations and comparisons.

References and publication details: submitted 9 Apr 2026 to arXiv as arXiv:2606.15577, six pages long with one figure and two tables, accepted at the 49th ICT and Electronics Convention, MIPRO, Paper ID: #23463.