Expertise development in volumetric image interpretation of radiology residents: what do longitudinal scroll data reveal?

Dorien Van Montfort,Koen Vincken,Cécile Ravesloot,Dirk Rutgers,Marieke Van Der Schaaf,Ellen Kok,Anouk Van Der Gijp

doi:10.1007/s10459-020-09995-6

Dorien Van Montfort, Koen Vincken + Show 5 more

Open Access

https://doi.org/10.1007/s10459-020-09995-6

Copy DOI

Abstract

The current study used theories on expertise development (the holistic model of image perception and the information reduction hypothesis) as a starting point to identify and explore potentially relevant process measures to monitor and evaluate expertise development in radiology residency training. It is the first to examine expertise development in volumetric image interpretation (i.e., CT scans) within radiology residents using scroll data collected longitudinally over five years of residency training. Consistent with the holistic model of image perception, the percentage of time spent on full runs, i.e. scrolling through more than 50% of the CT-scan slices (global search), decreased within residents over residency training years. Furthermore, the percentage of time spent on question-relevant areas in the CT scans increased within residents over residency training years, consistent with the information reduction hypothesis. Second, we examined if scroll patterns can predict diagnostic accuracy. The percentage of time spent on full runs and the percentage of time spent on question-relevant areas did not predict diagnostic accuracy. Thus, although scroll patterns over training years are consistent with visual expertise theories, they could not be used as predictors of diagnostic accuracy in the current study. Therefore, the relation between scroll patterns and performance needs to be further examined, before process measures can be used to monitor and evaluate expertise development in radiology residency training.

Highlights

The interpretation of medical images is central in radiology
Regarding H1a, a significant negative linear relation between training time (TrTime) and PercTimeFullRunsAvg was found (b = − 0.94), indicating that moving one year forward in the radiology training program led to a decrease of the percentage of time spent on full runs per DRPT of 0.94 percent point
Regarding H2a and H2b, the odds ratios of the fixed effects for PercTimeFullRunsAvg and PercTimeFullRunsAvg were not significantly different from 1, indicating that both the percentage of time spent on full runs and the percentage of time spent on relevant area did not significantly predict the number of questions correct per number of questions in a DRPT

Summary

Introduction

The interpretation of medical images is central in radiology. Image interpretation is considered to be a highly complex task (Drew et al 2013; Krupinski 2011; Van der Gijp et al 2014). Process measures are increasingly valued as additional sources of information about residents’ competence (Kok 2019), for example in the context of (formative) assessment and monitoring Process measures such as computer-log data (time-stamped information on interactions with the computer, for example, scrolling, panning, windowing and zooming) and eye tracking data (time-stamped information on where a person looks, how long and in what order; Kok and Jarodzka 2016) provide information that goes beyond outcome variables, for example, about the efficiency of visual search and strategies use (Drew et al 2013; Manning et al 2006; Venjakob et al 2012; Van der Gijp et al 2017). This requires that we know which process measures reflect developing expertise and predict performance

Objectives

Methods

Results

Conclusion