The Effects of Survey Enhancements on the Quality of Reporting in the Medical Expenditure Panel Survey, 2008–2015

Abstract

Abstract It is well established that survey respondents imperfectly recall health care use in surveys. However, careful attention to both survey design and fielding procedures can enhance recall. We examine the effects of a comprehensive, multi-pronged approach to changing field procedures in the Medical Expenditure Panel Survey (MEPS) to improve quality of health care use reporting. Conducted annually since 1996, the MEPS is the leading large-scale nationally representative health survey with detailed individual and household information on health care use and expenditures. These survey enhancements were undertaken in 2013–2014 because of concerns over a drop in the quality of reporting in 2010 that persisted into 2011–2012. The approach combined focused retraining of field supervisors and interviewers, developing quality metrics and reports for ongoing monitoring of interviewers, and revising advanced letters and materials sent to respondents. We seek to determine the extent to which changes in field procedures and trainings improved interviewer and respondent behaviors associated with better reporting, and more importantly, improved reporting accuracy. We use longitudinal MEPS data from 2008 through 2015, combining household reported use with sociodemographic and health status characteristics, and paradata on the characteristics of the interviews and interviewers. We exploit the longitudinal data and timings of major trainings and changes in field procedures in regression models, separating out the effects of the trainings and other fielding changes to the extent possible. We find that the 2013–2014 data quality improvement activities substantially improved reporting quality. Positive interviewer behaviors increased substantially to above pre-2010 levels, and utilization reporting has recovered to above pre-2010 levels, returning MEPS to trend. Importantly, these substantial gains occurred in 2013, prior to extensive in-person training for most of the field force. We examine the lessons learned from this data quality initiative both for the MEPS program and for other large household surveys.