Do proficiency test results correlate with the work performance of screeners who screen Papanicolaou smears?

Abstract

We rescreened Papanicolaou smear slides from 40,245 women, which had been examined by 81 cytology screeners, scored the screeners' work performance, and compared these scores with the results of the screeners' performance on glass slide and computer-based proficiency tests. All diagnoses (i.e., from the proficiency tests, the original slides, and the rescreened slides) were classified in the 4 diagnostic categories specified in the Clinical Laboratory Improvement Amendments. The rescreening scores were standardized to account for different distributions of abnormalities in the proficiency tests and rescreened slides. We compared a standardized score with the proficiency test scores. Of the cases, 91% were categorized as normal, benign, or reactive changes when rescreened, and 98% of these agreed with the original diagnosis. Sixteen percent of low-grade and 15% of high-grade intraepithelial lesions were classified as normal. The rank correlation between the rescreening scores and both proficiency tests was 0.24 using a scoring scheme for cytotechnologists. The correlation between the rescreening and proficiency testing scores indicates that performance on a 10-slide test gives some indication of the true performance of screeners. The computer-based test shows promise as an alternative to the glass slide test but needs further development and validation.