Autoencoder-based multi-task learning for imputation and classification of incomplete data

Abstract

The existence of missing values in real-world datasets increases the difficulty of data analysis. In this paper, we propose an autoencoder (AE)-based multi-task learning (MTL) model and optimize missing values dynamically to classify incomplete datasets having interdependencies among attributes. Specifically, we first design the input structure of hidden neurons in a dynamic way to enhance the imputation performance of AE, and then reorganize the output layer and construct an MTL model to achieve imputation and classification simultaneously. During network training and prediction, missing values are treated as variables and optimized dynamically accompanying with network parameters under the consideration of the incomplete model input. The optimization of missing values promotes the MTL model to match the regression and classification structures implied in incomplete data, thus reducing the impact of the perturbation caused by missing values effectively. The experiments on several datasets validate the effectiveness of the proposed method.