Computation Of Subspaces Research Articles

Lanczos, Householder transformations, and implicit deflation for fast and reliable dominant singular subspace computation

AbstractMany applications, such as subspace‐based models in information retrieval and signal processing, require the computation of singular subspaces associated with the k dominant, or largest, singular values of an m×n data matrix A, where k≪min(m,n). Frequently, A is sparse or structured, which usually means matrix–vector multiplications involving A and its transpose can be done with much less than 𝒪(mn) flops, and A and its transpose can be stored with much less than 𝒪(mn) storage locations. Many Lanczos‐based algorithms have been proposed through the years because the underlying Lanczos method only accesses A and its transpose through matrix–vector multiplications. We implement a new algorithm, called KSVD, in the Matlab environment for computing approximations to the singular subspaces associated with the k dominant singular values of a real or complex matrix A. KSVD is based upon the Lanczos tridiagonalization method, the WY representation for storing products of Householder transformations, implicit deflation, and the QR factorization. Our Matlab simulations suggest it is a fast and reliable strategy for handling troublesome singular‐value spectra. Copyright © 2001 John Wiley & Sons, Ltd.

Optimal reduced-rank estimation and filtering

This paper provides a unified view of, and a further insight into, a class of optimal reduced-rank estimators and filters. An alternating power (AP) method for computing the optimal reduced-rank estimators and filters is derived and analyzed. The AP method is a generalization of the conventional power method for subspace computation, which is shown to be globally and exponentially convergent under weak conditions. When the rank reduction is relatively large, the AP method is computationally more efficient than the conventional methods. The AP method is useful for adaptive computation of the canonical components of a desired reduced-rank estimate, which in turn facilitates the detection of a time-varying rank. The study shown in this paper is particularly useful for applications that involve a large number of sources and a large number of receivers, where rank reduction is either inherent in the multivariate system or required to reduce the model complexity and/or the computational load.

Smoothing techniques and computation of invariant subspaces of elliptic operators

We propose a generalized Krylov subspace method for computing the invariant subspace associated with the smallest eigenvalues of an elliptic operator. The success of the method relies upon a smoothing operator whose main purposes is to enhance the components of the smallest eigenvalues while dumping those of the largest ones. We show numerically that multigrid techniques can be used to construct an efficient smoothing operator.

A novel computational method for solving finite qbd processes

We present a novel numerical method that exploits invariant subspace computations for finding the stationary probability distribution of a finite QBD process. Assuming that the QBD state space is defined in two dimensions with m phases and K+1 levels, the solution vector π k for level, is known to be expressible in the mixed matrix-geometric form , where R 1and R 2are certain solutions to two quadratic matrix equations, and v 1and v 2are vectors to be determined using the boundary conditions. We show that the matrix-geometric factors R 1and R 2can be simultaneously obtained irrespective of K via finding arbitrary bases for the left-and right-invariant subspaces of a certain real matrix of size 2m To find these bases, we employ either Schur decomposition or a matrix-sign function iteration with quadratic convergence rate. The vectors v 1and v 2are obtained by solving a linear matrix equation, which is constructed with a time complexity of. Therefore, the effect of the number of levels on the overall complexity is minimal. Besides its numerical efficiency, the proposed method is numerically stable, and in the limiting case of , it is shown to yield the well-known matrix-geometric solution for infinite QBD processes. We also extend the method to the cases of level-dependent and non-canonical transitions, and provide a numerical example to demonstrate its computational features in comparison with other well-known methods

A note on finite-element spectral computations for integro-differential operators

This work deals with the numerical computation of eigenvalues and invariant subspaces of an integro-differential operator by finite element methods.

Componentwise perturbation theory for linear systems with multiple right-hand sides

Existing definitions of componentwise backward error and componentwise condition number for linear systems are extended to systems with multiple right-hand sides and to a general class of componentwise measure of perturbations involving Hölder p-norms. It is shown that for a system of order n with r right-hand sides, the componentwise backward error can be computed by finding the minimum p-norm solutions to n underdetermined linear systems, and an explicit expression is obtained in the case r = 1. A perturbation bound is derived, and from this the componentwise condition number is obtained to within a multiplicative constant. Applications of the results are discussed to invariant subspace computations, quasi-Newton methods based on multiple secant equations, and an inverse ODE problem.

Componentwise perturbation theory for linear systems with multiple right-hand sides

Existing definitions of componentwise backward error and componentwise condition number for linear systems are extended to systems with multiple right-hand sides and to a general class of componentwise measure of perturbations involving Hölder p -norms. It is shown that for a system of order n with r right-hand sides, the componentwise backward error can be computed by finding the minimum p -norm solutions to n underdetermined linear systems, and an explicit expression is obtained in the case r = 1. A perturbation bound is derived, and from this the componentwise condition number is obtained to within a multiplicative constant. Applications of the results are discussed to invariant subspace computations, quasi-Newton methods based on multiple secant equations, and an inverse ODE problem.

On Hamiltonian and symplectic Hessenberg forms

Characterizations are given for the Hamiltonian matrices that can be reduced to Hamiltonian Hessenberg form and the symplectic matrices that can be reduced to symplectic Hessenberg form by orthogonal symplectic similarity transformations. The reduction to these special Hessenberg forms is the missing link in the solution of the open problem of constructing a stable structure-preserving QR-like method of complexity O( n 3) for the computation of invariant subspaces of Hamiltonian and symplectic matrices. Our considerations lead us to propose an approach to the computation of Lagrangian invariant subspaces of a Hamiltonian or symplectic matrix.

Inner and outer geometry for singular systems with computation of subspaces

Abstract Output-nulling and unknown-input subspaces are defined for singular systems, with recursions given for their computation. Inner (i.e. domain) subspaces are distinguished from outer (i.e. codomain) subspaces. A general duality theory is presented, and the composite image, composite pre-image, and composite subspaces are introduced. It is shown how output-nulling and unknown-input subspaces may be computed via the singular system structure algorithm and a new square-root algorithm. The latter algorithm is shown to be related to the solution of a Riccati equation for singular systems. A simplified Riccati solution procedure is introduced that only requires the solution of a (non-square) Lyapunov equation and also applies in the state-variable case.

An Algorithm for Subspace Computation, with Applications in Signal Processing

An algorithm for computing the eigenvectors corresponding to the m algebraically smallest or largest eigenvalues of an $n \times n$ symmetric matrix ${\bf A}$ is described. The algorithm consists of repeated applications of the Rayleigh-Ritz procedure to a sequence of subspaces of dimension $m + 1$ which converges to the desired subspace. The method is closely related to the Lanczos method, but requires a constant amount of computation at each iteration. Applications of the algorithm include the adaptive covariance eigenstructure computation, in which the matrix ${\bf A}$ can change while the algorithm is in progress.

Generalized output nulling subspaces: Riccati equation computation and applications

Four generalized output-nulling subspaces (ONS) are defined and it is shown that many subspaces important in the analysis of linear systems can be expressed as special cases of the ONS. On the other hand, it is shown that the ONS can be computed from the solutions to square-root Riccati equations. Applications to computation of subspaces contained in the state space and applications to system inversion are demonstrated. The properties of the ONS considered as time-dependent subspace sequences are investigated.

On the computation of controllability subspaces

A procedure to determine parametric expressions for the basis matrices of all possible controllability subspaces (c.s.)  of a pair (A, B) is presented. Based on this procedure the paramotrization of the basis matrices corresponding to the family of c.s. of (A, B) that are contained in the kernel of the output map c becomes possible. This result leads to an alternative method of computing the maximal c.s.  of )A, B) contained in the kernel of c, which avoids the intermediate computation of the maximal (A, B)-invarianfc subspaco contained in ker C.

Design of high-gain state-feedback controllers for linear multivariable systems

It is shown that recent results in the theory of entire eigen-structure assignment greatly facilitate the design of high-gain state-feedback controllers. The design of such controllers is reduced to the separate computation of well-defined subspaces associated with the `slow´ and `fast´ modes of closed-loop systems incorporating high-gain controllers.