The variation of radiologists' performance over the course of a reading session

Abstract

The radiologist’s task of reviewing many cases successively is highly repetitive and requires a high level of concentration. Fatigue effects have, for example, been shown in studies comparing performance at different times of day. However, little is known about changes in performance during an individual reading session. During a session reading an enriched case set, performance may be affected by both fatigue (i.e. decreasing performance) and training (i.e. increasing performance) effects. In this paper, we reanalyze 3 datasets from 4 studies for changes in radiologist performance during a reading session. Studies feature 8-20 radiologists reading and assessing 27-60 cases in single, uninterrupted sessions. As the studies were not designed for this analysis, study setups range from bone fractures to mammograms and randomization varies between studies. Thus, they are analyzed separately using mixed-effects models. There is some indication that, as time goes on, specificity increases (shown with p<0.05 for 2 out of 3 datasets, no significant difference for the other) while sensitivity may also increase (p<0.05 for 1 out of 3 datasets). The difficulty of ‘normal’ (healthy / non-malignant) and ‘abnormal’ (unhealthy / malignant) cases differs (p<0.05 for 3 out of 3 datasets) and the reader’s experience may also be relevant (p<0.05 for 1 out of 3 datasets). These results suggest that careful planning of breaks and session length may help optimize reader performance. Note that the overall results are still inconclusive and a targeted study to investigate fatigue and training effects within a reading session is recommended.