Experiences with a self-test for Dutch breast screening radiologists: lessons learnt

Abstract

To evaluate a self-test for Dutch breast screening radiologists introduced as part of the national quality assurance programme. A total of 144 radiologists were invited to complete a test-set of 60 screening mammograms (20 malignancies). Participants assigned findings such as location, lesion type and BI-RADS. We determined areas under the receiver operating characteristics (ROC) curves (AUC), case and lesion sensitivity and specificity, agreement (kappa) and correlation between reader characteristics and case sensitivity (Spearman correlation coefficients). A total of 110 radiologists completed the test (76%). Participants read a median number of 10,000 screening mammograms/year. Median AUC value was 0.93, case and lesion sensitivity was 91% and case specificity 94%. We found substantial agreement for recall (κ = 0.77) and laterality (κ = 0.80), moderate agreement for lesion type (κ = 0.57) and BI-RADS (κ = 0.45) and no correlation between case sensitivity and reader characteristics. Areas under the ROC curve, case sensitivity and lesion sensitivity were satisfactory and recall agreement was substantial. However, agreement in lesion type and BI-RADS could be improved; further education might be aimed at reducing interobserver variation in interpretation and description of abnormalities. We offered individual feedback on interpretive performance and overall feedback at group level. Future research will determine whether performance has improved. • We introduced and evaluated a self-test for Dutch breast screening radiologists. • ROC curves, case and lesion sensitivity and recall agreement were all satisfactory. • Agreement in BI-RADS interpretation and description of abnormalities could be improved. • These are areas that should be targeted with further education and training. • We offered individual feedback on interpretative performance and overall group feedback.