SageMath-augmented LLM agents: +9.7pp gains; GPT-5.5 75.2%

Pavel Snopov and German Magai submitted an arXiv paper on 7 July 2026 that evaluates a ReAct-style agent which combines LLM reasoning with verifiable SageMath feedback and Context7 documentation. The paper reports that access to SageMath improves model performance on the RealMath benchmark by +9.7 percentage points on average, with gains ranging from 1.5 pp to 27.8 pp.

How did the SageMath-augmented agent work?

The agent is a ReAct-style setup that pairs an LLM with a Computer Algebra System, using SageMath for verifiable computational feedback and Context7 for up-to-date documentation. The authors evaluated the agent in a setting that emulates a computational-mathematics research loop, and they proposed a refinement to the RealMath benchmark consisting of a multi-step post-processing procedure plus a multi-stage validation pipeline to improve problem quality and reliability.

The pipeline combines free-form LLM reasoning with programmatic checks from SageMath, feeding results back into the model for iterative problem solving. The paper details experiments across multiple frontier models and measures both solve rates and token usage under tool-enabled configurations.

What were the key results?

SageMath access delivered substantial performance gains across all evaluated models, raising average solve rates by +9.7 percentage points and producing model-specific gains that span from 1.5 pp to 27.8 pp. The authors report that Qwen 3.7-Max benefits the most from SageMath. GPT-5.5 achieved the highest solve rate among tool-enabled configurations, reaching 75.2 percent, and the paper states GPT-5.5 also had the lowest token usage in those configurations.

The authors also note that providing CAS access narrows the gap between open-weight and closed models on their tasks. The experiments were run on a refined RealMath problem set, generated with the paper's multi-step post-processing and validated through a multi-stage pipeline, both intended to improve the extracted dataset's quality.

Why does this matter?

CAS-augmented agents blend symbolic, verifiable computation with generative reasoning, giving models an empirical handle on mathematical computations rather than relying solely on fluency. The paper frames this hybrid as a promising direction for assisting mathematicians in computational exploration and positions the work as a step toward automated conjecture discovery. Narrowing the performance gap between open-weight and closed models means broader access to effective research tools could become feasible if CAS integrations scale and generalize.

What to watch

The paper was accepted to the 3rd AI for Math Workshop at ICML 2026 and the project repository is available online, making code and experimental details public for replication. Watch for community uptake of the authors' multi-step post-processing procedure and multi-stage validation pipeline for RealMath, and for follow-up experiments that apply SageMath-augmented agents to other computational-mathematics tasks.

Additional specifics from the paper: it is 37 pages long with 16 figures, and the arXiv submission identifier is arXiv:2607.06820. The experiments explicitly measure average and per-model gains from SageMath access, the reported average improvement is +9.7 percentage points, gains range from 1.5 pp to 27.8 pp, Qwen 3.7-Max is the single model with the largest benefit, and GPT-5.5 records a 75.2% solve rate and the lowest token usage among tool-enabled configurations.