A New Look at the Power Method for Fast Subspace Tracking

Abstract

A class of fast subspace tracking methods such as the Oja method, the projection approximation subspace tracking (PAST) method, and the novel information criterion (NIC) method can be viewed as power-based methods. Unlike many non-power-based methods such as the Given's rotation based URV updating method and the operator restriction algorithm, the power-based methods with arbitrary initial conditions are convergent to the principal subspace of a vector sequence under a mild assumption. This paper elaborates on a natural version of the power method. The natural power method is shown to have the fastest convergence rate among the power-based methods. Three types of implementations of the natural power method are presented in detail, which require respectively O(n2p), O(np2), and O(np) flops of computation at each iteration (update), where n is the dimension of the vector sequence and p is the dimension of the principal subspace. All of the three implementations are shown to be globally convergent under a mild assumption. The O(np) implementation of the natural power method is shown to be superior to the O(np) equivalent of the Oja, PAST, and NIC methods. Like all power-based methods, the natural power method can be easily modified via subspace deflation to track the principal components and, hence, the rank of the principal subspace.