Supplementary file 1_From risk classification to clinical action: a prespecified paired pilot benchmark of public large language model interfaces for diabetes-related foot ulcer prevention.docx

Additional outcomes included screening frequency, need for referral, specialty and urgency of referrals, completeness, interreviewer reliability, repeated-generation stability, and safety.</p>Results<p>Each interface classified 20/20 cases in concordance with the frozen IWGDF reference (100%; Wilson 95% CI 83.9%-100.0%). The 16.1-percentage-point interval below the observed ceiling indicates limited precision and remains compatible with clinically meaningful error in new cases.

Of 100 primary outputs, agreement was observed in 89/100 (89.0%) for referral need, 68/75 (90.7%) for referral specialty, and 92/100 (92.0%) for urgency. Median information-item and mandatory-measure coverage was 100.0% for both measures; however, completeness scoring had limited inter-reviewer reliability and should be interpreted cautiously. In an exploratory, hypothesis-generating four-case repeated-generation substudy, all-three-generation consensus concordance was observed in 16/20 interface-case combinations for referral need, 14/15 eligible combinations for specialty, and 19/20 combinations for urgency.

These descriptive counts are not estimates of failure probability or tail behavior. Under the prespecified curated no-search benchmark conditions, no major safety errors were observed among 140 outputs; this absence of observed events does not establish safety in routine clinical use.</p>Conclusions<p>Performance was highest for structured guideline mapping, though reliability diminished in referral and individualized management across repeated generations.

These findings highlight the necessity for auditable, clinician-supervised decision support instead of autonomous deployment.</p>