Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students
Standard
Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students. / Knitza, Johannes; Tascilar, Koray; Gruber, Eva; Kaletta, Hannah; Hagen, Melanie; Liphardt, Anna-Maria; Schenker, Hannah; Krusche, Martin; Wacker, Jochen; Kleyer, Arnd; Simon, David; Vuillerme, Nicolas; Schett, Georg; Hueber, Axel J.
in: ARTHRITIS RES THER, Jahrgang 23, Nr. 1, 06.09.2021, S. 233.Publikationen: SCORING: Beitrag in Fachzeitschrift/Zeitung › SCORING: Zeitschriftenaufsatz › Forschung › Begutachtung
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students
AU - Knitza, Johannes
AU - Tascilar, Koray
AU - Gruber, Eva
AU - Kaletta, Hannah
AU - Hagen, Melanie
AU - Liphardt, Anna-Maria
AU - Schenker, Hannah
AU - Krusche, Martin
AU - Wacker, Jochen
AU - Kleyer, Arnd
AU - Simon, David
AU - Vuillerme, Nicolas
AU - Schett, Georg
AU - Hueber, Axel J
N1 - © 2021. The Author(s).
PY - 2021/9/6
Y1 - 2021/9/6
N2 - BACKGROUND: An increasing number of diagnostic decision support systems (DDSS) exist to support patients and physicians in establishing the correct diagnosis as early as possible. However, little evidence exists that supports the effectiveness of these DDSS. The objectives were to compare the diagnostic accuracy of medical students, with and without the use of a DDSS, and the diagnostic accuracy of the DDSS system itself, regarding the typical rheumatic diseases and to analyze the user experience.METHODS: A total of 102 medical students were openly recruited from a university hospital and randomized (unblinded) to a control group (CG) and an intervention group (IG) that used a DDSS (Ada - Your Health Guide) to create an ordered diagnostic hypotheses list for three rheumatic case vignettes. Diagnostic accuracy, measured as the presence of the correct diagnosis first or at all on the hypothesis list, was the main outcome measure and evaluated for CG, IG, and DDSS.RESULTS: The correct diagnosis was ranked first (or was present at all) in CG, IG, and DDSS in 37% (40%), 47% (55%), and 29% (43%) for the first case; 87% (94%), 84% (100%), and 51% (98%) in the second case; and 35% (59%), 20% (51%), and 4% (51%) in the third case, respectively. No significant benefit of using the DDDS could be observed. In a substantial number of situations, the mean probabilities reported by the DDSS for incorrect diagnoses were actually higher than for correct diagnoses, and students accepted false DDSS diagnostic suggestions. DDSS symptom entry greatly varied and was often incomplete or false. No significant correlation between the number of symptoms extracted and diagnostic accuracy was seen. It took on average 7 min longer to solve a case using the DDSS. In IG, 61% of students compared to 90% in CG stated that they could imagine using the DDSS in their future clinical work life.CONCLUSIONS: The diagnostic accuracy of medical students was superior to the DDSS, and its usage did not significantly improve students' diagnostic accuracy. DDSS usage was time-consuming and may be misleading due to prompting wrong diagnoses and probabilities.TRIAL REGISTRATION: DRKS.de, DRKS00024433 . Retrospectively registered on February 5, 2021.
AB - BACKGROUND: An increasing number of diagnostic decision support systems (DDSS) exist to support patients and physicians in establishing the correct diagnosis as early as possible. However, little evidence exists that supports the effectiveness of these DDSS. The objectives were to compare the diagnostic accuracy of medical students, with and without the use of a DDSS, and the diagnostic accuracy of the DDSS system itself, regarding the typical rheumatic diseases and to analyze the user experience.METHODS: A total of 102 medical students were openly recruited from a university hospital and randomized (unblinded) to a control group (CG) and an intervention group (IG) that used a DDSS (Ada - Your Health Guide) to create an ordered diagnostic hypotheses list for three rheumatic case vignettes. Diagnostic accuracy, measured as the presence of the correct diagnosis first or at all on the hypothesis list, was the main outcome measure and evaluated for CG, IG, and DDSS.RESULTS: The correct diagnosis was ranked first (or was present at all) in CG, IG, and DDSS in 37% (40%), 47% (55%), and 29% (43%) for the first case; 87% (94%), 84% (100%), and 51% (98%) in the second case; and 35% (59%), 20% (51%), and 4% (51%) in the third case, respectively. No significant benefit of using the DDDS could be observed. In a substantial number of situations, the mean probabilities reported by the DDSS for incorrect diagnoses were actually higher than for correct diagnoses, and students accepted false DDSS diagnostic suggestions. DDSS symptom entry greatly varied and was often incomplete or false. No significant correlation between the number of symptoms extracted and diagnostic accuracy was seen. It took on average 7 min longer to solve a case using the DDSS. In IG, 61% of students compared to 90% in CG stated that they could imagine using the DDSS in their future clinical work life.CONCLUSIONS: The diagnostic accuracy of medical students was superior to the DDSS, and its usage did not significantly improve students' diagnostic accuracy. DDSS usage was time-consuming and may be misleading due to prompting wrong diagnoses and probabilities.TRIAL REGISTRATION: DRKS.de, DRKS00024433 . Retrospectively registered on February 5, 2021.
KW - Humans
KW - Physicians
KW - Rheumatic Diseases/diagnosis
KW - Students, Medical
U2 - 10.1186/s13075-021-02616-6
DO - 10.1186/s13075-021-02616-6
M3 - SCORING: Journal article
C2 - 34488887
VL - 23
SP - 233
JO - ARTHRITIS RES THER
JF - ARTHRITIS RES THER
SN - 1478-6354
IS - 1
ER -