Empowering Early Career Neurosurgeons in the Critical Appraisal of Artificial Intelligence and Machine Learning: The Design and Evaluation of a Pilot Course

Abstract

Artificial intelligence (AI) is expected to play a greater role in neurosurgery. There is a need for neurosurgeons capable of critically appraising AI literature to evaluate its implementation or communicate information to patients. However, there are a lack of courses delivered at a level appropriate for individuals to develop such skills. We assessed the impact of a 2-day (non-credit bearing) online digital literacy course on the ability of individuals to critically appraise AI literature in neurosurgery. We performed a prospective, quasi-experimental non-randomized, controlled study with an intervention arm comprising individuals enrolled in our 2-day digital health literacy course and a waiting-list control arm used for comparison. We assessed participants' pre- and post-course knowledge, confidence, and course acceptability using Qualtrics surveys designed for the purpose of this study. A total of 62 individuals (33 participants, 29 waitlist controls), including neurosurgical trainees and both undergraduate and post-graduate students, attended the course and completed the pre-course survey. The 2 groups did not vary significantly in terms of age or demographics. Following the course, participants significantly improved in their knowledge of AI (mean difference=3.86, 95% CI=2.97-4.75, P-value < 0.0001) and confidence in critically appraising literature using AI (P-value=0.002). Similar differences in knowledge (mean difference=3.15, 95% CI=1.82-4.47, P-value < 0.0001) and confidence (P-value < 0.0001) were found when compared to the control group. Bespoke courses delivered at an appropriate level can improve clinicians' understanding of the application of AI in neurosurgery, without the need for in-depth technical knowledge or programming skills.