LLM-based Models for Detecting Emerging Service Feedback Topics

Mahsa Tavakoli, Ruth Bankey and Cristián Bravo submitted a paper on 25 Jun 2026 to arXiv (arXiv:2606.26595) that presents a methodology combining large language models, statistical techniques and human-AI collaboration to detect emerging topics in multilingual service feedback. The authors frame the work around public sector needs, especially tax administrations, where rising feedback volumes make scalable, context-aware analysis necessary.

What did the paper propose?

The paper proposes a hybrid framework that pairs fine-tuned, quantized LLMs with statistical techniques and human expert oversight to detect emerging service-quality topics and potential inequities. The authors describe the architecture as one that produces "accurate, computationally efficient, and context-aware analyses," explicitly targeting multilingual customer feedback and the needs of public sector organizations.

The approach emphasizes human-AI collaboration to reduce LLM fabrication and to improve the relevance of insights. The methodology aims to be practical for organizations such as tax administrations, where fairness and service quality influence trust and compliance.

How was the approach evaluated?

The authors evaluated the framework using similarity analysis and assessments from experienced tax officers, and they report stronger alignment with expert judgments than baseline models. The paper states that combining fine-tuned, quantized LLMs with expert oversight led to results that matched tax officers' assessments more closely than baselines the authors compared against.

The evaluation highlights two concrete measures used in the study: similarity analysis as an automated check and direct human assessment by experienced tax officers. The human-in-the-loop element also served to reduce instances of LLM fabrication while improving the reliability and relevance of generated insights.

Why it matters

Public sector bodies receive multilingual feedback at scale and need methods that surface emerging problems and potential disparities without relying solely on static, expert-defined indicators. A system that aligns better with experienced officers and reduces fabricated outputs can help agencies act faster on service issues and on inequities that would otherwise go unnoticed. That shifts scarce human attention from routine sorting toward verification and intervention.

What to watch

Look for the paper's formal record: the arXiv entry notes an arXiv-issued DOI via DataCite is pending registration. Also watch for follow-up work or operational trials in tax administrations that would test whether the reported alignment with expert judgments translates into timely, evidence-based decisions in live service settings.

Bibliographic note: the paper appears on arXiv as arXiv:2606.26595, submitted 25 Jun 2026, authored by Mahsa Tavakoli, Ruth Bankey and Cristián Bravo.