Concordance of Self-reported Data and Medical Record Audit for Six Cancer Screening Procedures

Abstract

Self-reported data about the interval since last cancer screening are often used to determine whether individuals are due for periodic screening and to monitor adherence to guidelines for early cancer detection. In a study conducted within the Kaiser Permanente Medical Care Program, we examined the concordance of self-reported information and medical record documentation about recency of and reasons for six procedures for early cancer detection. We also assessed the concordance of population-level estimates of screening rates based on these two sources. Data were obtained from a mailed questionnaire or telephone interview completed by 779 men and women. The data from these randomly selected study participants (431 women and 348 men), who had been members of the health plan for the previous 5 years, were compared with information obtained from their medical charts. Intersource agreement about whether each procedure was done within the last 2 years was evaluated, with the medical record used as the gold standard. To assess the accuracy of patient self-reporting, we also calculated sensitivity, false-positive and false-negative results, and Kappa statistics. Concordance between self-reported data and medical record documentation was greater for procedures that generated a test report (mammogram, Pap smear, fecal occult blood test, and sigmoidoscopy) than for those generating a physician's note (clinical breast examination and digital rectal examination). Kappa statistics showed a similar pattern. Sensitivity of self-reported data was more than 90% for mammogram, clinical breast examination, Pap smear, and fecal occult blood test and nearly 80% for sigmoidoscopy and digital rectal examination. However, false-positive results were above 40%, except for fecal occult blood test and sigmoidoscopy. For all six procedures, estimated population-level rates of screening within the past 2 years would have been significantly higher (P < .0001) if self-reported data were used instead of medical record audit data. Self-reported data may overestimate the percentage of the population that has been screened and underestimate the interval since the last cancer detection procedures. Such data should be used cautiously for clinical decision making, research, and surveillance activities at both individual and population levels. Also, comparability of data should be considered when population screening rates are evaluated on the basis of different data sources.