A Lagrange–Newton algorithm for tensor sparse principal component analysis

Abstract

This paper is concerned with the tensor sparse principal component analysis (TSPCA) by obtaining principal components which are linear combinations of a small subset of the original features for tensorial data. The core mathematical model can be formulated as a nonsmooth nonconvex optimization problem with a polynomial objective function, and with the sparsity constraint and the unit Euclidean spherical constraint. By employing the tools in tensor analysis, along with the variational properties for the involved -norm, the optimality condition of TSPCA is analysed in terms of stationary points. To well resolve the problem, we reformulate the stationary conditions into the Lagrange stationary equation system via the property of the projection operator onto the sparsity constraint set. With special emphasis on the Jacobian nonsingularity of the corresponding nonlinear system, we propose the Lagrange–Newton algorithm for pursuing the stationary point, which serves as a promising approximation of the optimal solution to TSPCA. The locally quadratic convergence rate is also established under mild conditions. Numerical experiments illustrate the effectiveness of our proposed TSPCA approach in terms of the solution accuracy as well as the computation time.