RADHawk-an AI-based knowledge recommender to support precision education, improve reporting productivity, and reduce cognitive load.

Abstract

Using artificial intelligence (AI) to augment knowledge is key to establishing precision education in modern radiology training. Our department has developed a novel AI-derived knowledge recommender, the first reported precision education program in radiology, RADHawk (RH), that augments the training of radiology residents and fellows by pushing personalized and relevant educational content in real-time and in context with the case being interpreted. To assess the impact on trainees of an AI-based knowledge recommender compared to traditional knowledge sourcing for radiology reporting through reporting time, quality, cognitive load, and learning experiences. A mixed methods prospective study allocated trainees to intervention and control groups, working with and without access to RH, respectively. Validated questionnaires and observed and graded simulated picture archiving and communication system (PACS)-based reporting at the start and end of a month's rotation assessed technology acceptance, case report quality, case report time and sourcing time, cognitive load, and attitudes toward modified learning strategies. Non-parametric regression analyses and Mann-Whitney tests were used to compare outcomes between groups, with significance set at P<0.05. The intervention group (n=28) demonstrated astatistically significant reduction in the case report time by -162s per case (95%CI -275.76s to -52.40s) (P-value = 0.002) and an increase of 14% (95%CI 8.1-19.8%) (P-value <0.001) in accuracy scores compared to the control group (n=29) at the end of the rotation. The intervention group also showed lower levels of mental demand (P=0.030) and experienced less effort (P=0.030) and frustration (P=0.030) while reporting. Additionally, >78% of the intervention group gave positive ratings on RH's effectiveness, increase in productivity, job usefulness, and ease of use. Eighty-nine percent of participants in the intervention group requested access to RH for their next rotation. This study demonstrates that RH, as the first reported AI-derived knowledge recommender for radiology education, significantly reduces reporting time and improves reporting accuracy while reducing overall workload and mental demand for radiology trainees. The high acceptance among trainees suggests its potential for supporting self-directed learning. Further testing of a larger external cohort will support more widespread implementation of RH for precision education.