Modeling the Under Reporting Bias in Panel Survey Data

Abstract

Panel survey data have been gaining importance in marketing. However, one challenge of estimating econometric models based on panel survey data is how to account for under reporting, that is, respondents do not report behavioral incidences which actually occur. Under reporting is especially likely to occur in a panel survey because the data recording mechanism is often tedious, complex and effortful. The probability of under reporting is likely to vary across respondents and also over the duration of the survey period. In this paper, we propose a model to simultaneously study reported behavioral incidences and partially observed actual behavioral incidences. We propose a Bayesian approach for estimating the proposed model. We treat those unobserved actual behavioral incidences as latent variables, and the Gibbs Sampler makes it convenient to impute the non-reported consumption incidences along with making inferences on other model parameters. Our proposed method has two advantages. First, it offers a model-based approach to remove the under reporting bias in panel survey data, and therefore allows marketing researchers to make accurate inferences about consumers’ actual behavior. Second, the method also offers a natural way to study factors that influence respondents’ propensity to under report. Since we treat those under reported behavioral incidences as non-missing-at-random (NMAR), this under reporting propensity varies across respondents and over time. This understanding can help marketing researchers design the right strategy to intervene and incentivize respondents to authentically report and hence improve the quality of survey data. The proposed model and estimation approach are tested on both synthetic data and an actual panel survey data on consumer reported beverage-drinking behavior. Our analysis suggests that under reporting can significantly mask respondents’ true behavior.