MIRA-Math benchmark: Minimal Information Requesting in Math

MIRA-Math, submitted to arXiv on 8 Jul 2026 by Charbel Al Bateh and Samer Saab Jr, is a new benchmark that isolates the capability to request minimal information during mathematical reasoning. The dataset contains 2,310 generated instances spanning 22 typed mathematical families and ships with generators, verifiers, prompts, run metadata, and documentation.

What is MIRA-Math?

MIRA-Math is a diagnostic benchmark for solving mathematical problems whose full latent state has a unique answer, but whose solver-facing view is missing exactly one necessary atomic fact. The solver must request that missing fact in natural language under a strict budget and then integrate the returned fact into an exact final answer.

MIRA-Math narrows the scope compared with broader interactive benchmarks that mix reasoning with tools, retrieval, and long-horizon dialogue. The paper frames the task as a test of minimal information requesting: identifying the single atomic fact needed, phrasing a request that matches the dataset-provided hint, and using the returned fact to compute a deterministic final answer.

How does MIRA-Math evaluate requests and answers?

A fixed constrained LLM responder sees only the dataset-provided atomic fact and must either offer the quoted fact when the request matches it, or decline otherwise; instance generation, typed hint specifications, validation, and final-answer verification are deterministic. Request metrics are therefore measured under a fixed LLM-mediated responder channel.

The benchmark includes 2,310 instances drawn from 22 typed mathematical families, including algebra, probability, linear systems, discrete structures, signal processing, Markov chains, circuits, interpolation, and numerical boundary-value problems. The authors ran experiments "across frontier and small models" and found that request success and final-answer accuracy are separable: models may ask for the right fact yet fail the downstream computation, or fail before obtaining the canonical hint.

MIRA-Math also provides the full supporting artifacts the authors say are necessary for reproducible evaluation: generators, verifiers, prompts, run metadata, and dataset documentation are released alongside the instances.

Why it matters

MIRA-Math isolates a compact, measurable skill: recognizing and requesting exactly one missing atomic fact and combining it with correct computation. That separation exposes distinct failure modes, where a model's ability to ask the right question does not guarantee a correct final answer. By making request behavior interact with a fixed responder, the benchmark creates deterministic verification of both the request and the subsequent computation.

This structure helps researchers evaluate whether models fail because they cannot identify missing information, because they cannot phrase a matching request, or because they flub the downstream arithmetic or symbolic work. The dataset's breadth across 22 mathematical families gives those diagnostics coverage from algebra and probability to numerical boundary-value problems.

What to watch

Watch for community uptake of the released generators, verifiers, prompts, and run metadata and for follow-up experiments that report separate request-success and final-answer accuracy numbers. Also look for whether adopters reproduce the paper's finding that request success and final-answer accuracy can diverge when evaluating both frontier and small models.

Details: the paper appears on arXiv as arXiv:2607.07391 (submitted 8 Jul 2026) and is authored by Charbel Al Bateh and Samer Saab Jr. The repository and documentation accompanying the release aim to support reproducible evaluation of minimal information requesting in mathematical reasoning.