Springe direkt zu Inhalt

Evaluation of a retrieval-augmented generation-powered chatbot for preCT informed consent: a prospective comparative study

Tim Landgraf, Felix Busch, Lukas Kaibel, Hai Nguyen, Tristan Lemke, Sebastian Ziegelmayer, Markus Graf, Alexander W marka, Lukas Endrös, Philipp Prucker, Daniel Spitzl, Markus Mergen, Marcus R Makowski, Keno Bressem, Sebastian Petzoldt, Lisa C. Adams – 2025

This study aims to investigate the feasibility, usability, and effectiveness of a Retrieval-Augmented Generation (RAG)-powered Patient Information Assistant (PIA) chatbot for pre-CT information counseling compared to the standard physician consultation and informed consent process. This prospective comparative study included 86 patients scheduled for CT imaging between November and December 2024. Patients were randomly assigned to either the PIA group (n = 43), who received pre-CT information via the PIA chat app, or the control group (n = 43), with standard doctor-led consultation. Patient satisfaction, information clarity and comprehension, and concerns were assessed using six ten-point Likert-scale questions after information counseling with PIA or the doctor’s consultation. Additionally, consultation duration was measured, and PIA group patients were asked about their preference for pre-CT consultation, while two radiologists rated each PIA chat in five categories. Both groups reported similarly high ratings for information clarity (PIA: 8.64 ± 1.69; control: 8.86 ± 1.28; p = 0.82) and overall comprehension (PIA: 8.81 ± 1.40; control: 8.93 ± 1.61; p = 0.35). However, the doctor consultation group showed greater effectiveness in alleviating patient concerns (8.30 ± 2.63 versus 6.46 ± 3.29; p = 0.003). The PIA group demonstrated significantly shorter subsequent consultation times (median: 120 s [interquartile range (IQR): 100–140] versus 195 s [IQR: 170–220]; p = 0.04). Both radiologists rated overall quality, scientific and clinical evidence, clinical usefulness and relevance, consistency, and up-to-dateness of PIA high. The RAG-powered PIA effectively provided pre-CT information while significantly reducing physician consultation time. While both methods achieved comparable patient satisfaction and comprehension, physicians were more effective at addressing worries or concerns regarding the examination.

Titel
Evaluation of a retrieval-augmented generation-powered chatbot for preCT informed consent: a prospective comparative study
Verfasser
Tim Landgraf, Felix Busch, Lukas Kaibel, Hai Nguyen, Tristan Lemke, Sebastian Ziegelmayer, Markus Graf, Alexander W marka, Lukas Endrös, Philipp Prucker, Daniel Spitzl, Markus Mergen, Marcus R Makowski, Keno Bressem, Sebastian Petzoldt, Lisa C. Adams
Verlag
Springer International Publishing
Schlagwörter
Artificial Intelligence, Medical Informations, Natural language processing, Radiology, Large language models
Datum
2025-03-21
Kennung
https://doi.org/10.1007/s10278-025-01483-w
Erschienen in
Journal of Imaging Informatics in Medicine 2025.
Zitierweise
Busch, F., Kaibel, L., Nguyen, H. et al. Evaluation of a Retrieval-Augmented Generation-Powered Chatbot for Pre-CT Informed Consent: a Prospective Comparative Study. J Digit Imaging. Inform. med. (2025). https://doi.org/10.1007/s10278-025-01483-w
Sprache
eng
Art
Text
Größe oder Länge
pp 1-12
Rechte
Open Access