True Eigenvectors Research Articles

Rate-optimal posterior contraction for sparse PCA

Principal component analysis (PCA) is possibly one of the most widely used statistical tools to recover a low-rank structure of the data. In the high-dimensional settings, the leading eigenvector of the sample covariance can be nearly orthogonal to the true eigenvector. A sparse structure is then commonly assumed along with a low rank structure. Recently, minimax estimation rates of sparse PCA were established under various interesting settings. On the other side, Bayesian methods are becoming more and more popular in high-dimensional estimation, but there is little work to connect frequentist properties and Bayesian methodologies for high-dimensional data analysis. In this paper, we propose a prior for the sparse PCA problem and analyze its theoretical properties. The prior adapts to both sparsity and rank. The posterior distribution is shown to contract to the truth at optimal minimax rates. In addition, a computationally efficient strategy for the rank-one case is discussed.

Second order accurate distributed eigenvector computation for extremely large matrices

We propose a second-order accurate method to estimate the eigenvectors of extremely large matrices thereby addressing a problem of relevance to statisticians working in the analysis of very large datasets. More specifically, we show that averaging eigenvectors of randomly subsampled matrices efficiently approximates the true eigenvectors of the original matrix under certain conditions on the incoherence of the spectral decomposition. This incoherence assumption is typically milder than those made in matrix completion and allows eigenvectors to be sparse. We discuss applications to spectral methods in dimensionality reduction and information retrieval.

Principal components, minor components, and linear neural networks

Many neural network realizations have been recently proposed for the statistical technique of Principal Component Analysis (PCA). Explicit connections between numerical constrained adaptive algorithms and neural networks with constrained Hebbian learning rules are reviewed. The Stochastic Gradient Ascent (SGA) neural network is proposed and shown to be closely related to the Generalized Hebbian Algorithm (GHA). The SGA behaves better for extracting the less dominant eigenvectors. The SGA algorithm is further extended to the case of learning minor components. The symmetrical Subspace Network is known to give a rotated basis of the dominant eigenvector subspace, but usually not the true eigenvectors themselves. Two extensions are proposed: in the first one, each neuron has a scalar parameter which breaks the symmetry. True eigenvectors are obtained in a local and fully parallel learning rule. In the second one, the case of an arbitrary number of parallel neurons is considered, not necessarily less than the input vector dimension.

Lanczos algorithm with selective reorthogonalization for eigenvalue extraction in structural dynamic and stability analysis

The Lanczos algorithm has been gaining increasing popularity as the eigensolver for structural analysis in recent years. The notorious numerical instability problem associated with the algorithm as originally proposed was overcome by performing full reorthogonalization among the Lanczos vectors serving as the bases of the solution subspace. Recent studies have shown that reorthogonalization is not required unless convergence to a true eigenvector from the Lanczos subspace is imminent. Hence, the reorthogonalization process performed among the Lanczos vectors may be significantly reduced, and the efficiency of the Lanczos algorithm may be further improved. The current article is the result from the implementation of this “selective reorthogonalization” principle. In this paper, the basics of the Lanczos algorithm and the full reorthogonalization procedure are first reviewed, and then the selective reorthogonalization algorithm is outlined. Next the implementation aspects are discussed, and finally verification as well as comparison examples are given. It is demonstrated that in most cases the selective reorthogonalization approach to the partial solution of an eigensystem is not only numerically stable but also is more efficient in computer time than that of the full reorthogonalization, and only comparable memory space is required.

Error bounds for computed eigenvalues and eigenvectors

On the basis of an existence theorem for solutions of nonlinear systems, a method is given for finding rigorous error bounds for computed eigenvalues and eigenvectors of real matrices. It does not require the usual assumption that the true eigenvectors span the whole space. Further, a priori error estimates for eigenpairs corrected by an iterative method are given. Finally the results are illustrated with numerical examples.