Agreement Between Self-Reports and Medical Records Was Only Fair in a Cross-Sectional Study of Performance of Annual Eye Examinations Among Adults With Diabetes in Managed Care

Abstract

Despite consensus about the importance of measuring quality of diabetes care and the widespread use of self-reports and medical records to assess quality, little is known about the degree of agreement between these data sources. To evaluate agreement between self-reported and medical record data on annual eye examinations and to identify factors associated with agreement. Data from interviews and medical records were available for 8409 adults with diabetes who participated in the baseline round of the Translating Research Into Action for Diabetes (TRIAD) Study. Agreement between self-reports and medical records was evaluated as concordance and Cohen's kappa coefficient. Self-reports indicated a higher performance of annual dilated eye examinations than did medical records (75.9% vs. 38.8%). Concordance between the data sources was 57.9%. Agreement was only fair (kappa coefficient = 0.25; 95% confidence interval, 0.23-0.26). Nearly two-thirds (64.6%) of discordance was due to lack of evidence in the medical record to support self-reported performance of the procedure. After adjustment, agreement was most strongly related to health plan (chi = 977.9, df = 9; P < 0.0001), and remained significantly better for 3 of the 10 health plans (P < 0.00001) and for persons younger than 45 years of age (P = 0.00002). The low level of agreement between self-report and medical records suggests that many providers of diabetes care do not have easily available accurate information on the eye examination status of their patients.