Implementing and evaluating a fully functional AI-enabled model for chronic eye disease screening in a real clinical environment
Standard
Implementing and evaluating a fully functional AI-enabled model for chronic eye disease screening in a real clinical environment. / Skevas, Christos; de Olaguer, Nicolás Pérez; Lleó, Albert; Thiwa, David; Schroeter, Ulrike; Lopes, Inês Valente; Mautone, Luca; Linke, Stephan J; Spitzer, Martin Stephan; Yap, Daniel; Xiao, Di.
in: BMC OPHTHALMOL, Jahrgang 24, Nr. 1, 01.02.2024, S. 51.Publikationen: SCORING: Beitrag in Fachzeitschrift/Zeitung › SCORING: Zeitschriftenaufsatz › Forschung › Begutachtung
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Implementing and evaluating a fully functional AI-enabled model for chronic eye disease screening in a real clinical environment
AU - Skevas, Christos
AU - de Olaguer, Nicolás Pérez
AU - Lleó, Albert
AU - Thiwa, David
AU - Schroeter, Ulrike
AU - Lopes, Inês Valente
AU - Mautone, Luca
AU - Linke, Stephan J
AU - Spitzer, Martin Stephan
AU - Yap, Daniel
AU - Xiao, Di
N1 - © 2024. The Author(s).
PY - 2024/2/1
Y1 - 2024/2/1
N2 - BACKGROUND: Artificial intelligence (AI) has the potential to increase the affordability and accessibility of eye disease screening, especially with the recent approval of AI-based diabetic retinopathy (DR) screening programs in several countries.METHODS: This study investigated the performance, feasibility, and user experience of a seamless hardware and software solution for screening chronic eye diseases in a real-world clinical environment in Germany. The solution integrated AI grading for DR, age-related macular degeneration (AMD), and glaucoma, along with specialist auditing and patient referral decision. The study comprised several components: (1) evaluating the entire system solution from recruitment to eye image capture and AI grading for DR, AMD, and glaucoma; (2) comparing specialist's grading results with AI grading results; (3) gathering user feedback on the solution.RESULTS: A total of 231 patients were recruited, and their consent forms were obtained. The sensitivity, specificity, and area under the curve for DR grading were 100.00%, 80.10%, and 90.00%, respectively. For AMD grading, the values were 90.91%, 78.79%, and 85.00%, and for glaucoma grading, the values were 93.26%, 76.76%, and 85.00%. The analysis of all false positive cases across the three diseases and their comparison with the final referral decisions revealed that only 17 patients were falsely referred among the 231 patients. The efficacy analysis of the system demonstrated the effectiveness of the AI grading process in the study's testing environment. Clinical staff involved in using the system provided positive feedback on the disease screening process, particularly praising the seamless workflow from patient registration to image transmission and obtaining the final result. Results from a questionnaire completed by 12 participants indicated that most found the system easy, quick, and highly satisfactory. The study also revealed room for improvement in the AMD model, suggesting the need to enhance its training data. Furthermore, the performance of the glaucoma model grading could be improved by incorporating additional measures such as intraocular pressure.CONCLUSIONS: The implementation of the AI-based approach for screening three chronic eye diseases proved effective in real-world settings, earning positive feedback on the usability of the integrated platform from both the screening staff and auditors. The auditing function has proven valuable for obtaining efficient second opinions from experts, pointing to its potential for enhancing remote screening capabilities.TRIAL REGISTRATION: Institutional Review Board of the Hamburg Medical Chamber (Ethik-Kommission der Ärztekammer Hamburg): 2021-10574-BO-ff.
AB - BACKGROUND: Artificial intelligence (AI) has the potential to increase the affordability and accessibility of eye disease screening, especially with the recent approval of AI-based diabetic retinopathy (DR) screening programs in several countries.METHODS: This study investigated the performance, feasibility, and user experience of a seamless hardware and software solution for screening chronic eye diseases in a real-world clinical environment in Germany. The solution integrated AI grading for DR, age-related macular degeneration (AMD), and glaucoma, along with specialist auditing and patient referral decision. The study comprised several components: (1) evaluating the entire system solution from recruitment to eye image capture and AI grading for DR, AMD, and glaucoma; (2) comparing specialist's grading results with AI grading results; (3) gathering user feedback on the solution.RESULTS: A total of 231 patients were recruited, and their consent forms were obtained. The sensitivity, specificity, and area under the curve for DR grading were 100.00%, 80.10%, and 90.00%, respectively. For AMD grading, the values were 90.91%, 78.79%, and 85.00%, and for glaucoma grading, the values were 93.26%, 76.76%, and 85.00%. The analysis of all false positive cases across the three diseases and their comparison with the final referral decisions revealed that only 17 patients were falsely referred among the 231 patients. The efficacy analysis of the system demonstrated the effectiveness of the AI grading process in the study's testing environment. Clinical staff involved in using the system provided positive feedback on the disease screening process, particularly praising the seamless workflow from patient registration to image transmission and obtaining the final result. Results from a questionnaire completed by 12 participants indicated that most found the system easy, quick, and highly satisfactory. The study also revealed room for improvement in the AMD model, suggesting the need to enhance its training data. Furthermore, the performance of the glaucoma model grading could be improved by incorporating additional measures such as intraocular pressure.CONCLUSIONS: The implementation of the AI-based approach for screening three chronic eye diseases proved effective in real-world settings, earning positive feedback on the usability of the integrated platform from both the screening staff and auditors. The auditing function has proven valuable for obtaining efficient second opinions from experts, pointing to its potential for enhancing remote screening capabilities.TRIAL REGISTRATION: Institutional Review Board of the Hamburg Medical Chamber (Ethik-Kommission der Ärztekammer Hamburg): 2021-10574-BO-ff.
KW - Humans
KW - Artificial Intelligence
KW - Diabetic Retinopathy/diagnosis
KW - Mass Screening/methods
KW - Software
KW - Macular Degeneration/diagnosis
KW - Glaucoma/diagnosis
U2 - 10.1186/s12886-024-03306-y
DO - 10.1186/s12886-024-03306-y
M3 - SCORING: Journal article
C2 - 38302908
VL - 24
SP - 51
JO - BMC OPHTHALMOL
JF - BMC OPHTHALMOL
SN - 1471-2415
IS - 1
ER -