Estonian Language benchmark: Claude tops 60-model propaganda test

The Institute of the Estonian Language released a benchmark on June 16, 2026 measuring how susceptible AI language models are to Russian propaganda. Sixty models were tested with 75 questions in three languages covering 14 propaganda narratives, and each answer was scored on a scale of 1 to 5 where 1 means the model repeats Russian talking points.

What the benchmark tested

Questions were phrased in three styles: neutral, biased, and manipulative. The evaluation used a calibrated Claude Opus 4.5 as the scoring model, and that evaluation setup was validated by disinformation experts at the organization Propastop. Test candidates had no access to web search or other external tools during the runs, so the benchmark measures only the language models themselves, not tool-enabled retrieval or fact-checking layers.

The dataset targeted 14 specific propaganda narratives and was delivered in three languages. The primary threat vector named in the materials is organized disinformation: the benchmark notes that Russian networks such as "Pravda" deliberately feed AI systems millions of disinformation articles, and it cites a recent episode in which OpenAI shut down a Russian campaign that used ChatGPT to spread propaganda ahead of Germany's federal election.

How models performed

Anthropic's Claude models occupied the top positions in the ranking. Claude Fable 5 led the benchmark with a score of 95.2; the report lists Claude Opus 4.7 directly behind it. Nvidia's Nemotron 3 and Alibaba's Qwen 3.6 Plus appear after Anthropic in the upper tier of results. By contrast, Mistral's models, including Medium 3.5, landed in the bottom third of the ranking.

The benchmark's findings echo a separate Newsguard study cited in the report that recorded a steady misinformation rate of 36.67 percent for Mistral. The report also notes business context: Mistral is negotiating a 3 billion euro funding round at a 20 billion euro valuation.

Why it matters

The benchmark isolates the model's internal resistance to propaganda-style prompts, so high scores indicate a model's native ability to identify and reject manipulated framing without external tools. That matters for deployments where models cannot or do not use search, tool chains, or curated safety intermediaries. Models that struggle on this test are more likely to echo adversarial talking points when given manipulative prompts.

The results carry reputational and commercial weight. Anthropic's dominance on this particular benchmark strengthens its safety positioning, while the consistency between the benchmark and the Newsguard finding places additional scrutiny on Mistral, which markets itself as a European alternative even as it negotiates large fundraising. The broader operational risk is concrete: the report cites organized campaigns that feed models disinformation at scale and a known case in which ChatGPT was used as part of a Russian campaign ahead of an election.

What to watch

Whether Claude Fable 5 remains unavailable outside the U.S. is a near-term factor for how widely its leading score will influence procurement choices. Watch for follow-up releases from the Institute of the Estonian Language for full score tables and methodology details, and for any public responses from Mistral, Anthropic, Nvidia, or Alibaba about model changes or mitigations. Also track the outcome of Mistral's funding negotiations and whether that capital shift coincides with product or safety updates.