Does self-report data on HIV primary care utilization agree with medical record data for socially marginalized populations in the United States?

Abstract

To test whether self-report data agree with medical record data in marginalized, HIV-infected populations, we collected information about HIV primary care visits over a 6-month period from both sources. Patients were drawn from a large study of engagement and retention in care conducted between 2003 and 2005. Self-report data were collected in face-to-face interviews and medical records were extracted using a rigorous, standardized protocol with multiple quality checks. We found poor overall agreement (weighted kappa = 0.36, 95% confidence interval = 0.28, 0.43). Factors associated with disagreement included younger age (adjusted odds ratio for 20 versus 40 years = 1.25, 95% confidence interval = 0.98, 1.60), non-Hispanic black race/ethnicity (adjusted odds ratio for non-Hispanic blacks versus non-Hispanic whites = 1.48, 95% confidence interval = 1.03, 2.13), lower education (adjusted odds ratio for high school education, GED, or less versus some college or college graduate = 1.43, 95% confidence interval = 0.96, 2.13), and substance use (adjusted odds ratio for any illicit drug/heavy alcohol use in the past 6 months versus no use = 1.39, 95% confidence interval = 1.02, 1.90). These findings do not support a conclusion that unconfirmed self-report data of HIV primary care visits are a sufficient substitute for rigorously collected medical record data in studies focusing on marginalized populations. Use of other data sources (e.g., administrative data), use of other self-reported outcome measures that have better concordance with medical records/administrative data (e.g., CD4 counts), or incorporation of rigorous measures to increase reliability of self-report data may be needed. Limitations of this study include the lack of a true gold standard with which to compare self-report data.