System operating characteristic and cost analysis in automated cytodiagnosis of gynecologic specimens.

Abstract

Automated cytodiagnosis of gynecologic specimens is analogous to the engineering problem of signal detection. Misclassification gives either false positive or false negative errors. The system operating characteristic (SOC) describes system performance in terms of these two types of error. Weights are assigned to the two types of error and take into account the incidence of disease in the population and the monetary and societal costs of misdiagnosis. The overall loss due to misclassification is calculated by cost analysis and is combined with the SOC to arrive at the optimal operating point for a system. The techniques of SOC and cost analysis are illustrated by application to a preliminary study in which 54 gynecologic specimens are analyzed by flow cytometry for cellular DNA content and size. Classification is based on the number of signals corresponding to sextuploid or greater DNA content. Eleven of the specimens have abnormal but nonneoplastic cytologies and are excluded from the analysis. When the analysis is optimized to an equal weight cost structure, the false negative error rate is 2/36, the false positive error rate is 2/7, and the overall loss is 0.34 relative to the cost of rescreening a woman who has been misdiagnosed as positive. The overall loss is a benchmark ofsystem performance. It is used to compare results obtained by different systems for automated cytodiagnosis. Objective measures of classification capability are good descriptors of system performance, but other considerations, such as specimen adequacy, effect of normal cells causing false alarms, and standardization of cytodiagnostic criteria, are important in assessing the clinical acceptability of a system for automated cytodiagnosis.