Missing data imputation on biomedical data using deeply learned clustering and L2 regularized regression based on symmetric uncertainty

Abstract

Big data era in healthcare led to the generation of high dimensional datasets like genomic datasets, electronic health records etc. One among the critical issues to be addressed in such datasets is handling incomplete data that may yield misleading results if not handled properly. Imputation is considered to be an effective way when the missing data rate is high. While imputation accuracy and classification accuracy are the two important metrics generally considered by most of the imputation techniques, high dimensional datasets such as genomic datasets motivated the need for imputation techniques that are also computationally efficient and preserves the structure of the dataset. This paper proposes a novel approach to missing data imputation in biomedical datasets using an ensemble of deeply learned clustering and L2 regularized regression based on symmetric uncertainty. The experiments are conducted with different proportion of missing data on both genomic and non-genomic biomedical datasets for different types of missingness pattern. Our proposed approach is compared with seven proven baseline imputation methods and two recently proposed imputation approaches. The results show that the proposed approach outperforms the other approaches considered in our experimentation in terms of imputation accuracy and computational efficiency despite preserving the structure of the dataset. Thus, the overall classification accuracy of the biomedical classification tasks is also improved when our proposed missing data imputation technique is used.