Supplementary file 1_Public-facing health information on achalasia generated by large language models: a multidimensional comparative study.docx

We therefore compared six LLMs in answering public questions about achalasia, an uncommon esophageal motility disorder, focusing on safety, accuracy, empathy, information quality and reliability, and readability.</p>Methods<p>In this cross-sectional comparative study, 40 patient-oriented questions about achalasia were submitted once to each of six LLMs (ChatGPT 5.5, Claude Opus 4.7, DeepSeek V4 Pro, Gemini 3.1 Pro Thinking, Grok-4.3, and Qwen3-Max), between May 7 and May 12, 2026.

Three gastroenterologists independently assessed 240 responses for safety, accuracy, empathy, information quality, and readability using DISCERN, EQIP, the JAMA benchmark criteria assessing authorship, attribution, disclosure, and currency, and the Global Quality Score (GQS). Readability was evaluated using six established readability indices.</p>Results<p>Overall, 27 of 240 responses (11.3%) were classified as potentially unsafe, with proportions ranging from 7.5 to 15.0% across models; the overall between-model difference in safety was not statistically significant.