NeuralCP: Bayesian Multiway Data Analysis with Neural Tensor Decomposition

Abstract

Multiway data are widely observed in neuroscience, health informatics, food science, etc. Tensor decomposition is an important technique for capturing high-order interactions among such multiway data. Classical tensor decomposition methods, such as the Tucker decomposition and the CANDECOMP/PARAFAC (CP), assume that the complex interactions among objects are multi-linear and thus insufficient to represent nonlinear relationships in data. To effectively model the complex nonlinear relationships of a tensor, we design a neural model joining neural networks with the Bayesian tensor decomposition, in which the high-order interactions are captured by neural networks. By taking advantages of the nonlinear modeling provided by the neural networks and the uncertainty modeling provided by Bayesian models, we replace the multi-linear product in traditional Bayesian tensor decomposition with a more flexible neural function (i.e., a multi-layer perceptron) whose parameters can be learned from data. Our model can be efficiently optimized with stochastic gradient descent. Accordingly, it is scalable to large real-world tensor. We conducted experiments on both synthetic data and real-world chemometrics tensor data. Experimental results have demonstrated that the proposed model can achieve significantly higher prediction performance than the state-of-the-art tensor decomposition approaches. The proposed nonlinear tensor decomposition method, i.e., NeuralCP, has been demonstrated to obtain promising prediction results on many multi-way data.