Minor Subspace Analysis Research Articles

Fast Dual Purpose Algorithm Based on Novel Unified Cost Function

Principal subspace analysis (PSA) and minor subspace analysis (MSA) are considered two robust instruments in many fields. The dual purpose algorithm is capable of solving the PSA and MSA by simply switching the sign of one algorithm. Until today, there have been few dual purpose algorithms that are able to find their corresponding cost functions. In this paper, a novel unified cost function (NUCF) is proposed that possesses a global maximum, which is achieved only in the case where the weight matrix encompasses the desired principal or minor subspace. With the use of the gradient ascent method in the NUCF, we propose a novel dual purpose algorithm, which possesses lower computational complexity when compared with some existing algorithms. Numerical simulations and real applications illustrate that the proposed dual purpose algorithm is capable of tracking the desired subspace, and it converges faster than some similar types of algorithms.

Dual subspace learning via geodesic search on Stiefel manifold

Oja’s principal subspace algorithm is a well-known and powerful technique for learning and tracking principal information of time series. However, Oja’s algorithm is divergent when performing the task of minor subspace analysis. In the present paper, we transform Oja’s algorithm into a dual learning algorithm in the sense of fulfilling principal subspace analysis as well as minor subspace analysis via geodesic search on Stiefel manifold. Also inherent stability is guaranteed for the proposed geodesic based algorithm due to the fact the weight matrix rigourously evolves on the compact Stiefel manifold. The effectiveness of the proposed algorithm is further verified in the section of numerical simulation.

A self-stabilizing MSA algorithm in high-dimension data stream

Minor subspace analysis (MSA) is a statistical method for extracting the subspace spanned by all the eigenvectors associated with the minor eigenvalues of the autocorrelation matrix of a high-dimension vector sequence. In this paper, we propose a self-stabilizing neural network learning algorithm for tracking minor subspace in high-dimension data stream. Dynamics of the proposed algorithm are analyzed via a corresponding deterministic continuous time (DCT) system and stochastic discrete time (SDT) system methods. The proposed algorithm provides an efficient online learning for tracking the MS and can track an orthonormal basis of the MS. Computer simulations are carried out to confirm the theoretical results.

A new minor component analysis method based on Douglas-Kung-Amari minor subspace analysis method

This letter presents a new minor component analysis algorithm that is based on the transformation of the known minor subspace analysis algorithm. The minor subspace analysis method that is adopted is the one proposed by Douglas, Kung, and Amari. The proposed minor component analysis algorithm extracts N minor components of the K-dimensional vector stationary random process N<K. Since the chosen minor subspace analysis method is fully homogeneous from the point of view of the individual neuron, the proposed minor component analysis method is also fully homogeneous from the point of view of the individual neuron.

Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold

In this paper we extend the natural gradient method for neural networks to the case where the weight vectors are constrained to the Stiefel manifold. The proposed methods involve numerical integration techniques of the gradient flow without violating the manifold constraints. The extensions are based on geodesics. We rigorously formulate the previously proposed natural gradient and geodesics on the manifold exploiting the fact that the Stiefel manifold is a homogeneous space having a transitive action by the orthogonal group. Based on this fact, we further develop a simpler updating rule and one parameter family of its generalizations. The effectiveness of the proposed methods is validated by experiments in minor subspace analysis and independent component analysis.

Adaptive minor component extraction with modular structure

An information criterion for adaptively estimating multiple minor eigencomponents of a covariance matrix is proposed. It is proved that the proposed criterion has a unique global minimum at the minor subspace and that all other equilibrium points are saddle points. Based on the gradient search approach of the proposed information criterion, an adaptive algorithm called adaptive minor component extraction (AMEX) is developed. The proposed algorithm automatically performs the multiple minor component extraction in parallel without the inflation procedure. Similar to the adaptive lattice filter structure, the AMEX algorithm also has the flexibility wherein increasing the number of the desired minor component does not affect the previously extracted minor components. The AMEX algorithm has a highly modular structure and the various modules operate completely in parallel without any delay. Simulation results are given to demonstrate the effectiveness of the AMEX algorithm for both the minor component analysis (MCA) and the minor subspace analysis (MSA).

Self-stabilized gradient algorithms for blind source separation with orthogonality constraints

Recently, developments in self-stabilized algorithms for gradient adaptation of orthonormal matrices have resulted in simple but powerful principal and minor subspace analysis methods. In this paper, we extend these ideas to develop algorithms for instantaneous prewhitened blind separation of homogeneous signal mixtures. Our algorithms are proven to be self-stabilizing to the Stiefel manifold of orthonormal matrices, such that the rows of the adaptive demixing matrix do not need to be periodically reorthonormalized. Several algorithm forms are developed, including those that are equivariant with respect to the prewhitened mixing matrix. Simulations verify the excellent numerical properties of the proposed methods for the blind source separation task.

A minor subspace analysis algorithm

This paper proposes a learning algorithm which extracts adaptively the minor subspace spanned by the eigenvectors corresponding to the smallest eigenvalues of the autocorrelation matrix of an input signal. We show both analytically and by simulation results that the weight vectors provided by the proposed algorithm are guaranteed to converge to the minor subspace of the input signal. For wider applications, we also present the complex valued version of the proposed minor subspace analysis algorithm.