A distributed proximal gradient descent method for tensor completion

Abstract

In this paper, we propose a novel proximal method for CANDECOMP/PARAFAC (CP) decomposition to deal with the tensor completion problem. This approach is based on solving local optimization problems, rather than confronting the entire optimization problem at once. In addition, we propose two distributed algorithms for dealing with data of high dimensionality that scale up to tensors of dimension 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">8</sup> ×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">8</sup> ×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">8</sup> . We show that our proximal method outperforms the Stochastic Gradient Descent (SGD) for CP decomposition in terms of convergence accuracy by a factor of 2.8 and it can efficiently and effectively reconstruct a color image by observing only 10% of its entries. Experimental results show that our distributed methods are scalable in terms of dimensionality, factorization rank, number of machines and perform efficiently in both dense and sparse settings. The proposed distributed proximal approach outperforms existing distributed methods in terms of speed of convergence by a factor of two. Moreover, it can successfully recover a hyperspectral image, by observing 10% of its values.