Databricks sets GLM 5.2 as default coding engine after cost test

Databricks found that the Chinese open-source model GLM 5.2 matched Anthropic’s Opus 4.8 on an internal benchmark of its multi-million-line codebase and will make GLM 5.2 its day-to-day coding model. GLM 5.2 hit the top performance cluster at $1.28 per task versus $1.94 for Opus, and Databricks says developer feedback from pilots supported the move.

What did Databricks test and how?

Databricks ran an internal benchmark built from real pull requests against a multi-million-line codebase spanning more than ten languages, including Python, Go, TypeScript, Scala, and Rust. The team required tasks to be recent and human-written, paired tasks with high-quality tests reviewed by hand, rewrote tests to allow alternative implementations, and scored only on passing tests rather than an LLM judge; they also truncated Git history to stop models from searching for answers.

The company says public datasets often do not represent its stack and can leak solutions into training data, so it created its own harness and measurements. Unity AI Gateway analysis of those internal tasks found 61 percent were medium complexity, 19 percent low, and 12 percent high, which informed Databricks’ plan to route work by task complexity.

How did GLM 5.2 compare on performance and cost?

GLM 5.2 landed in the top performance cluster and was statistically tied with Opus 4.8, with the top group hitting pass rates from 82 to 90 percent; GPT 5.5 in certain configurations also appeared in that cluster. Databricks reports GLM 5.2 cost $1.28 per task versus $1.94 per task for Opus in their test.

Databricks grouped tested models into three tiers: a top group at 82 to 90 percent pass rates (Opus 4.8, GLM 5.2, GPT 5.5 in some configs), a middle group at 71 to 82 percent (Sonnet 4.6, Sonnet 5, GPT 5.4 among others), and a bottom tier at 51 to 60 percent (GPT 5.4-mini and Haiku 4.5). The company also emphasized that token price and task cost diverge because token efficiency depends on the software environment. For example, Databricks’ Pi harness sent much less context than some native coding environments: for Opus 4.8 at "high effort," Pi was 2.08x cheaper at comparable quality (85 versus 87 percent). For GPT 5.5, Codex used 1,235,000 tokens versus 665,000 for Pi in Databricks’ tests.

Databricks says the Pareto frontier of quality-to-cost is shaped by models from OpenAI, Anthropic, and open source, and that only a mix of providers delivers frontier-level efficiency. The company plans to route more work to cheaper tiers based on task complexity and says it is already working on "running GLM at peak performance." The benchmark authors, including Databricks co-founder Matei Zaharia, concluded, "The evidence shows it's time to start deploying these as daily drivers for coding."

Databricks is not alone in testing Chinese and open-source models: the post notes Coinbase moved to Chinese models including GLM-5.2 and Kimi 2.7 and cut AI spending in half, Lindy replaced Claude with Deepseek v4 and saved millions, and Snowflake found GLM-5.2 nearly tied with Opus 4.7 at a fraction of the cost. On OpenRouter, Chinese models reached 30 percent of weekly traffic since February 2026, up from 11 percent the prior year, at 60 to 90 percent lower cost than Western alternatives.

Why it matters

A model that matches top-tier quality while lowering per-task cost shifts the default economics of developer tooling. Databricks’ test shows token efficiency and harness design can change which models are cost-effective for routine coding work. If teams adopt cheaper but comparable models as daily drivers, that could reduce cloud AI spend and change procurement choices across engineering organizations.

What to watch

Watch for Databricks’ rollout to developers and whether the company successfully runs GLM "at peak performance" and routes work to cheaper tiers as planned. Also track independent comparisons from other large codebases, such as the Snowflake test cited, to see if GLM 5.2’s cost-performance parity holds across different stacks.