A Realistic Evaluation of Methods for Handling Missing Data When There is a Mixture of MCAR, MAR, and MNAR Mechanisms in the Same Dataset

Abstract

The impact of missing data on statistical inference varies depending on several factors such as the proportion of missingness, missing-data mechanism, and method employed to handle missing values. While these topics have been extensively studied, most recommendations have been made assuming that all missing values are from the same missing-data mechanism. In reality, it is very likely that a mixture of missing-data mechanisms is responsible for missing values in a dataset and even within the same pattern of missingness. Although a mixture of missing-data mechanisms and causes within a dataset is a likely scenario, the performance of popular missing-data methods under these circumstances is unknown. This study provides a realistic evaluation of methods for handling missing data in this setting using Monte Carlo simulation in the context of regression. This study also seeks to identify acceptable proportions of missing values that violate the missing-data mechanism assumed by the method used to handle missing values. Results indicate that multiple imputation (MI) performs better than other principled or ad-hoc methods. Different missing-data methods are also compared via the analysis of a real dataset in which mixtures of missingness mechanisms are created. Recommendations are provided for the use of different methods in practice.