A Dual Purpose Principal and Minor Subspace Gradient Flow

Abstract

The dual purpose principal and minor subspace gradient flow can be used to track principal subspace (PS) and if altered simply by the sign, it can also serve as a minor subspace (MS) trackor. This is of practical significance in the implementations of algorithms. In this paper, a unified information criterion is proposed and a dual purpose principal and minor subspace gradient flow is derived based on the information criterion. In this dual purpose gradient flow, the weight matrix length is self-stabilizing, i.e., moving towards unit length in each learning step. The energy function associated with the dual purpose gradient flow for tracking PS and MS is given, and it exhibits a unique global minimum attained if and only if its state matrices span the PS or MS of the autocorrelation matrix of a vector data stream. The other stationary points of its energy function are (unstable) saddle points. The proposed dual purpose gradient flow can efficiently track an orthonormal basis of the PS or MS, which is illustrated through simulation experiments.