Inertial accelerated SGD algorithms for solving large-scale lower-rank tensor CP decomposition problems

Abstract

The stochastic gradient descent (SGD) method has been applied to the tensor CANDECOMP/PARAFAC (CP) decomposition problem to reduce the computational cost. However, SGD usually takes numerous iterations to converge. In this paper, we propose inertial accelerated SGD methods for the subproblems of CP decomposition concerning one block factor. We consider two methods for the selection of a block index in the subproblem: the stochastic manner with replacement and the randomly permuted order without replacement. First, we propose an inertial accelerated block-randomized SGD algorithm for CP decomposition (iBrasCPD) and show each cluster point is a stochastic stationary point. Then we introduce a randomized permutation inertial stochastic algorithm (RP-iSPALM) for randomized order without replacement. Under some mild conditions, we show the global convergence and convergence rate of RP-iSPALM under the Kurdyka-Łojasiewicz (KL) property. Numerical experiments on six real datasets demonstrate that our proposed algorithms are efficient and can achieve better performance than the existing state-of-the-art methods.