SciGraphChat: Retrieval-Augmented Generation over Scientific Knowledge Graphs

This paper presents the design and deployment of such a scholarly knowledge discovery system, along with practical design recommendations for integrating Semantic Web technologies and generative AI within research infrastructures. We evaluate the system with NFDIxCS stakeholders and external users via the System Usability Scale (SUS) and the User Experience Questionnaire–Short (UEQ-S), complemented by qualitative feedback.

Results show a mean SUS score of 69.4, positive mean pragmatic quality (UEQ-S mean 0.93), and neutral hedonic quality (UEQ-S mean 0.70). Qualitative feedback indicates that users value the combination of conversational access and graph-based inspection, while also requesting tighter integration with scholarly workflows such as reference export, corpus scoping, and adaptive result presentation. Our findings suggest that combining conversational retrieval with transparent graph-based evidence inspection can improve trust, explainability, and usability in knowledge discovery.</p>