SpeechDx multi-task benchmark: 12 datasets, 27 clinical tasks

SpeechDx, a new multi-task benchmark for clinical speech AI, was submitted to arXiv on 15 Jun 2026 by Sejal Bhalla, Larry Kieu, Aina Merchant, Eyal de Lara and Alex Mariakakis (arXiv:2606.17339). The benchmark aggregates 12 datasets and 27 tasks, and evaluates 12 state-of-the-art audio encoders across those tasks and in zero-shot cross-condition transfer.

What is included in SpeechDx?

SpeechDx bundles 12 datasets and 27 tasks spanning diverse health conditions, and it organizes those tasks according to which stage of speech production they disrupt: conceptualization, formulation, and articulation. The benchmark intentionally includes tasks with limited labeled data and evaluates the same health condition across multiple datasets to separate clinically meaningful patterns from dataset artefacts.

The paper frames the motivation succinctly: "Speech offers a uniquely informative window into health," and uses that premise to justify a shared evaluation framework that spans multiple clinical mechanisms and datasets rather than isolated condition-specific studies.

How were models evaluated and what were the results?

The authors systematically evaluated 12 state-of-the-art audio encoders across all 27 tasks and under zero-shot cross-condition transfer. Across the benchmark the paper finds that large-scale speech models provide the strongest overall baselines, domain-specific models only improve performance on closely matched tasks, and no current representation generalizes reliably across the clinical speech landscape.

Those conclusions come from the cross-task and cross-dataset structure of SpeechDx, which stresses both within-condition performance and generalization to underrepresented tasks and datasets. The benchmark specifically measures how representations trained at scale compare to domain-specific encoders when faced with the breadth of clinical speech problems the authors assembled.

Why did the authors structure tasks by speech-production stage?

Structuring tasks by conceptualization, formulation, and articulation lets SpeechDx group problems by the clinical mechanisms they perturb rather than by diagnostic label or single-dataset convenience. This design choice aims to reveal when models latch onto clinically meaningful signals and when they exploit dataset-specific artefacts. By testing the same condition across multiple datasets, SpeechDx helps distinguish consistent clinical patterns from spurious correlations tied to individual collections.

That organization underpins the benchmark’s central claim: evaluating across shared clinical mechanisms gives a clearer picture of representation generality and clinical relevance than hunting for high scores on narrow, isolated tasks.

Why it matters

SpeechDx provides a practical standard researchers can use to compare models on how well they capture clinically relevant speech phenomena and how well they generalize. The paper’s finding that even large-scale models do not generalize reliably across the assembled clinical landscape signals that progress toward general-purpose clinical speech representations remains incomplete. For clinicians and researchers, that means promising single-task results should be treated cautiously until they scale across the benchmark’s multi-dataset, multi-task tests.

What to watch

Watch for public releases of SpeechDx code, data splits, and benchmark leaderboards tied to arXiv:2606.17339; those will let external teams reproduce the evaluations the paper reports and measure whether newer models close the generalization gap identified by the authors. Also track follow-up work that reports improvements on the benchmark’s zero-shot cross-condition transfer evaluations.

References and provenance: the benchmark and results described above come from the arXiv submission "SpeechDx: A Multi-Task Benchmark for Clinical Speech AI" (arXiv:2606.17339), submitted 15 Jun 2026 by Sejal Bhalla, Larry Kieu, Aina Merchant, Eyal de Lara and Alex Mariakakis.