Patient-centred AI: communication style alters triage in arXiv

João Matos and co-authors submitted an arXiv paper on 9 Jul 2026 that analysed 2,053 real patient-chatbot conversations and found communication style can change urgency assessment by large language models. The paper, "The complexities of patient-centred conversational artificial intelligence," tested simulated patients and evaluated LLM triage across clinician-graded cases.

What did the researchers analyse and how large was the dataset?

The team analysed 2,053 real patient-chatbot conversations to characterise variability in how people communicate symptoms and emotions. From that base, they built a patient simulator that separates clinical content, emotional state, conversational strategy, and communication style so evaluations do not rely on cooperative, articulate, simulated patients.

The researchers then used five distinct patient personae across 1,164 clinician-graded cases to evaluate performance, and they tested four separate large language models in urgency assessment tasks. Those concrete counts anchor the paper's claim that real-world communication diversity matters for consumer-facing health chatbots.

How realistic were the simulated patients?

Simulated conversations were nearly indistinguishable from real ones in a Turing-inspired realism test, with 15 human graders achieving an accuracy of 55 percent in distinguishing simulated from real interactions. The simulator explicitly models clinical content, emotional state, conversational strategy, and communication style to capture the variability observed in the 2,053 conversation sample.

That 55 percent grader accuracy is the paper's primary realism metric. The authors report the simulation succeeded at reproducing the wide variation in expression and emotion found in actual users, producing dialogues that human graders struggled to separate from the originals.

How did communication style affect LLM urgency assessments?

Communication style materially shifted triage outcomes when the authors evaluated four LLMs across clinician-graded cases. Using five patient personae to vary how the same clinical content was presented, the study found that different expression patterns and conversational strategies produced different urgency assessments from the models.

The paper does not present a single aggregate accuracy number for the LLMs in the abstract, but it emphasises that systems trained and validated on idealised, cooperative simulated patients risk underperforming when confronted by the communication diversity observed in practice.

Why it matters

Models that interpret patient language will encounter wide variation in how people describe symptoms and emotions. If conversational AI is validated only on articulate, cooperative, or homogeneous simulations, it may deliver inconsistent triage decisions across different communication styles. The authors warn that such mismatches risk amplifying health disparities when deployed in the real world.

This shifts part of the evaluation burden from synthetic benchmarks to realism-focused testing: training pipelines and safety evaluations should measure performance across diverse personae and clinician-graded outcomes, not only on polished, simulated prompts.

What to watch

Look for follow-up evaluations that publish cross-style urgency-assessment benchmarks using real conversations and clinician grading. A concrete signal will be whether researchers or LLM developers adopt multi-persona simulators or release results that report model variability across the five personae and the 1,164 clinician-graded cases the paper used.

The paper and its supplementary materials run 36 pages of main text and 129 pages of supplementary material, and it is available on arXiv as arXiv:2607.08625 for readers who want the full experimental details.