Linear guided autoencoder: Representation learning with linearity

Abstract

In materials science, good representations of materials are important for use with prediction models in order to ensure accurate prediction of the properties of the output. In this paper, in order to address this issue, we use a learning system, linear guided autoencoder (LGAE) we call, which consists of an autoencoder and a linear predictor. For the autoencoder, we adopt a variant of the denoising autoencoder. In the LGAE, the learning addresses the unsupervised and supervised tasks simultaneously. Thus, the LGAE can be regarded as a form of nonlinear partial least squares (PLS) regression. Previous studies have not found the optimal solution for the encoder for an objective that contains both tasks. Our main contributions are a first-order approximation of the optimal solution and determination of the condition for linear solution that applies to the LGAE after training, in order to acquire knowledge from the nonlinear model (i.e., the LGAE). The main drawback of nonlinear PLS regression is that it is difficult to interpret the latent representation. Therefore, we propose a technical method for interpreting the latent representation. Experiments on benchmark datasets are conducted in order to compare the LGAE with kernel PLS regression, which is a powerful nonlinear PLS regression method. We also applied the LGAE to a dataset of methane storage materials in order to interpret the methane uptake based on the input variables and obtained reasonable results.