Dominant Subspace Research Articles

A fast algorithm for the recursive calculation of dominant singular subspaces

In many engineering applications it is required to compute the dominant subspace of a matrix A of dimension m × n , with m ⪢ n . Often the matrix A is produced incrementally, so all the columns are not available simultaneously. This problem arises, e.g., in image processing, where each column of the matrix A represents an image of a given sequence leading to a singular value decomposition-based compression [S. Chandrasekaran, B.S. Manjunath, Y.F. Wang, J. Winkeler, H. Zhang, An eigenspace update algorithm for image analysis, Graphical Models and Image Process. 59 (5) (1997) 321–332]. Furthermore, the so-called proper orthogonal decomposition approximation uses the left dominant subspace of a matrix A where a column consists of a time instance of the solution of an evolution equation, e.g., the flow field from a fluid dynamics simulation. Since these flow fields tend to be very large, only a small number can be stored efficiently during the simulation, and therefore an incremental approach is useful [P. Van Dooren, Gramian based model reduction of large-scale dynamical systems, in: Numerical Analysis 1999, Chapman & Hall, CRC Press, London, Boca Raton, FL, 2000, pp. 231–247]. In this paper an algorithm for computing an approximation of the left dominant subspace of size k of A ∈ R m × n , with k ⪡ m , n , is proposed requiring at each iteration O ( mk + k 2 ) floating point operations. Moreover, the proposed algorithm exhibits a lot of parallelism that can be exploited for a suitable implementation on a parallel computer.

Entropic sampling via Wang-Landau random walks in dominant energy subspaces

Dominant energy subspaces of statistical systems are defined with the help of restrictive conditions on various characteristics of the energy distribution, such as the probability density and the fourth order Binder's cumulant. Our analysis generalizes the ideas of the critical minimum energy subspace (CRMES) technique, applied previously to study the specific heat's finite-size scaling. Here, we illustrate alternatives that are useful for the analysis of further finite-size anomalies and the behavior of the corresponding dominant subspaces is presented for the two-dimensional (2D) Baxter-Wu and the 2D and 3D Ising models. In order to show that a CRMES technique is adequate for the study of magnetic anomalies, we study and test simple methods which provide the means for an accurate determination of the energy-order-parameter (E,M) histograms via Wang-Landau random walks. The 2D Ising model is used as a test case and it is shown that high-level Wang-Landau sampling schemes yield excellent estimates for all magnetic properties. Our estimates compare very well with those of the traditional Metropolis method. The relevant dominant energy subspaces and dominant magnetization subspaces scale as expected with exponents alpha/nu and gamma/nu, respectively. Using the Metropolis method we examine the time evolution of the corresponding dominant magnetization subspaces and we uncover the reasons behind the inadequacy of the Metropolis method to produce a reliable estimation scheme for the tail regime of the order-parameter distribution.

Model-order reduction by dominant subspace projection: error bound, subspace computation, and circuit applications

Balanced truncation is a well-known technique for model-order reduction with a known uniform reduction error bound. However, its practical application to large-scale problems is hampered by its cubic computational complexity. While model-order reduction by projection to approximate dominant subspaces without balancing has produced encouraging experimental results, the approximation error bound has not been fully analyzed. In this paper, a square-integral reduction error bound is derived for unbalanced dominant subspace projection by using a frequency-domain solution of the Lyapunov equation. Such an error bound is valid in both the frequency and time domains. Then, a dominant subspace computation scheme together with three Krylov subspace options is introduced. It is analytically justified that the Krylov subspace for moment matching at low frequencies is able to provide a better dominant subspace approximation than the Krylov subspace at high frequencies, while a rational Krylov subspace with a proper real shift parameter is capable of achieving superior approximation than the Krylov subspace at low frequency. A heuristic method of choosing a real shift parameter is also introduced based on its new connection to the discretization of a continuous-time model. The computation algorithm and theoretical analysis are then examined by several numerical examples to demonstrate the effectiveness. Finally, the dominant subspace computation scheme is applied to the model-order reduction of two large-scale interconnect circuit examples.

Smart antennas for UTRA TDD

AbstractDue to the increasing number of users in mobile communications and the demand for mobile multimedia services with high data rates, third generation mobile radio systems are currently one of the key communication technologies in research, development, and international standardization bodies. Smart antennas can be used to increase the capacity of wireless communication systems even further. TDD systems are particularly attractive for the employment of smart antennas, since the channel information (e. g., in terms of spatial covariance matrices) estimated on the uplink can be used directly for downlink beamforming. In this paper, we discuss efficient uplink and downlink processing techniques with smart antennas for UTRA TDD, the TDD mode of third generation mobile radio systems that is based on TD—CDMA. On the uplink, joint space—time processing is used to eliminate co—channel interference created by other users that transmit on the same frequency and in the same time slot. These joint detection techniques are efficiently implemented in the frequency domain. On the downlink, the data is only transmitted in the direction of the desired mobile user while interference in the direction of co—channel users is avoided. Moreover, short—term spatial channel estimates can be improved by projecting them into the dominant subspace of the long—term spatial covariance matrix. System level simulations have been performed to evaluate the gain in spectral efficiency (available bit rate per bandwidth per cell) if simple downlink beamforming techniques are used at the base stations. There is a two— to three—fold spectral efficiency gain if antenna arrays of M = 8 elements are deployed in a sectorized macro—cellular environment.

Fast orthonormal PAST algorithm

Subspace decomposition has proven to be an important tool in adaptive signal processing. A number of algorithms have been proposed for tracking the dominant subspace. Among the most robust and most efficient methods is the projection approximation and subspace tracking (PAST) method. This paper elaborates on an orthonormal version of the PAST algorithm for fast estimation and tracking of the principal subspace or/and principal components of a vector sequence. The orthonormal PAST (OPAST) algorithm guarantees the orthonormality of the weight matrix at each iteration. Moreover, it has a linear complexity like the PAST algorithm and a global convergence property like the natural power (NP) method.

A numerical algorithm for the diffusion equation using 3D FEM and the Arnoldi method

In this paper we introduce a new computational method for solving the diffusion equation. In particular, we construct a "generalized" state-space system and compute the impulse response of an equivalent truncated state-space system. In this effort, we use a 3D finite element method (FEM) to obtain the state-space system. We then use the Arnoldi iteration to approximate the state impulse response by projecting on the dominant controllable subspace. The idea exploited here is the approximation of the impulse response of the linear system. We study the homogeneous and heterogeneous cases and discuss the approximation error. Finally, we compare our computational results to our experimental setup.

A new time-domain narrowband velocity estimation technique for Doppler ultrasound flow imaging. I. Theory

A significant improvement in blood velocity estimation accuracy can be achieved by simultaneously processing both temporal and spatial information obtained from a sample volume. Use of the spatial information becomes especially important when the temporal resolution is limited. By using a two-dimensional sequence of spatially sampled Doppler signal "snapshots" an improved estimate of the Doppler correlation matrix can be formed. Processing Doppler data in this fashion addresses the range-velocity spread nature of the distributed red blood cell target, leading to a significant reduction in spectral speckle. Principal component spectral analysis of the "snapshot" correlation matrix is shown to lead to a new and robust Doppler mode frequency estimator. By processing only the dominant subspace of the Doppler correlation matrix, the Cramer-Rao bounds on the estimation error of target velocity is significantly reduced in comparison to traditional narrowband blood velocity estimation methods and achieves almost the same local accuracy as a wideband estimator. A time-domain solution is given for the velocity estimate using the root-MUSIC algorithm, which makes the new estimator attractive for real-time implementation.

Reduced models in the medium frequency range for general dissipative structural-dynamics systems

This paper presents a theoretical approach for constructing a reduced model, in the medium frequency range, in the area of structural dynamics for a general three-dimensional anisotropic and inhomogeneous viscoelastic bounded medium. All the results presented can be used for beams, plates and shells. The boundary value problem in the frequency domain and its variational formulation are presented. For a given medium frequency band, an energy operator which is intrinsic to the dynamic system is introduced and mathematically studied. This energy operator depends on the dissipative part of the dynamical system. It is proved that this operator is a positive-definite symmetric trace operator in a Hilbert space and that its dominant eigensubspace allows a reduced model to be constructed using the Ritz-Galerkin method. A finite dimension approximation of the continuous case is presented (for instance using the finite element method). An effective construction of the dominant subspace using the subspace iteration method is developed. Finally, an example is given to validate the concepts and the algorithms.

Reduced models in the medium-frequency range for general external structural-acoustic systems

This paper presents a theoretical approach for constructing a reduced model in the medium-frequency range in the area of structural acoustics for a general three-dimensional dissipative structure made of an anisotropic, inhomogeneous, viscoelastic bounded medium coupled with an external acoustic fluid. All the results presented can be used if the structure is made of beams, plates, and shells. The boundary value problem in the frequency domain and its variational formulation are presented. For a fixed medium-frequency band, an energy operator related to the structural-acoustic system is introduced. This operator is symmetric positive definite and has a countable set of positive eigenvalues. Its dominant eigensubspace allows a reduced model to be constructed using the Ritz–Galerkin method. A finite dimension approximation of the three-dimensional continuous case is presented and an effective construction of the dominant subspace using the subspace iteration method is developed. As an example, the reduced model is used for constructing the time-stationary random response of the structural-acoustic system submitted to a random wall pressure field. Finally, the theory is validated for a finite length circular cylindrical shell coupled with several dashpots and springs and immersed in a gas (air) and in a liquid (water).

ULV and generalized ULV subspace tracking adaptive algorithms

Traditional adaptive filters assume that the effective rank of the input signal is the same as the input covariance matrix or the filter length N. Therefore, if the input signal lives in a subspace of dimension less than N, these filters fail to perform satisfactorily. In this paper, we present two new algorithms for adapting only in the dominant signal subspace. The first of these is a low-rank recursive-least-squares (RLS) algorithm that uses a ULV decomposition (Stewart 1992) to track and adapt in the signal subspace. The second adaptive algorithm is a subspace tracking least-mean-squares (LMS) algorithm that uses a generalized ULV (GULV) decomposition, developed in this paper, to track and adapt in subspaces corresponding to several well-conditioned singular value clusters. The algorithm also has an improved convergence speed compared with that of the LMS algorithm. Bounds on the quality of subspaces isolated using the GULV decomposition are derived, and the performance of the adaptive algorithms are analyzed.

Performance analysis of an adaptive algorithm for tracking dominant subspaces

This paper provides a performance analysis of a least mean square (LMS) dominant invariant subspace algorithm. Based on an unconstrained minimization problem, this algorithm is a stochastic gradient algorithm driving the columns of a matrix W to an orthonormal basis of a dominant invariant subspace of a correlation matrix. We consider the stochastic algorithm governing the evolution of WW/sup H/ to the projection matrix onto this dominant invariant subspace and study its asymptotic distribution. A closed-form expression of its asymptotic covariance is given in the case of independent observations and is further analyzed to provide some insights into the behavior of this LMS type algorithm. In particular, it is shown that even though the algorithm does not constrain W to have orthonormal columns, there is deviation from orthonormality of the first order. We also give a closed-form expression of the asymptotic covariance of the DOAs estimated by the MUSIC algorithm applied to the adaptive estimate of the projector. It is found that the asymptotic distributions have a structure that is very similar to those describing batch estimation techniques because both algorithms are obtained from the minimization of the same criterion. Finally, the accuracy of the asymptotic analysis is checked by numerical simulations and is found to be valid not only for a small step size but in a very large domain.

Factored spherical subspace tracking

Tracking the dominant subspace of a data matrix is an essential part of many signal processing algorithms. We present a modification to the so-called spherical subspace tracking algorithm. This algorithm has a low computational complexity, but suffers from accumulation of numerical errors. We show that this numerical instability can be circumvented by using a minimal orthogonal parameterization of the subspace basis matrix. The resulting factored spherical SVD updating algorithm then consists exclusively of rotation operations. In view of implementing the algorithm in real-time applications with high data rates, a linear systolic architecture is derived.

SPHERICAL SUBSPACE TRACKING ON SYSTOLIC ARRAYS

The spherical subspace tracker is a low complexity adaptive algorithm that converges to the dominant (or subdominant) subspace. Spherical subspace updating is a fast way to monitor slowly time-varying subspaces, which arise in high resolution direction of arrival tracking algorithms. In real time sonar and radar array processing applications, it becomes necessary to find parallel algorithms and pipelined architectures that provide high speed solutions to these tracking problems. In this paper the spherical subspace tracker is parallelized and mapped onto systolic array architectures. The arrays are ideally suited for implementation with custom VLSI or networks of available parallel numeric processing chips such as iWarps and TMS320C40s. C-language like cell programs specify the functionality and timing of the systolic array in concrete terms.

A subspace fitting method for identification of linear state-space models

A new method is presented for the identification of systems parameterized by linear state-space models. The method relies on the concept of subspace fitting, wherein an input/output data model parameterized by the state matrices is found that best fits, in the least-squares sense, the dominant subspace of the measured data. Some empirical results are included to illustrate the performance advantage of the algorithm compared to standard techniques.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Subspace invariance: the RO-FST and TQR-SVD adaptive subspace tracking algorithms

Subspace decomposition and tracking are quintessential ingredients in high-resolution adaptive array processing. MUSIC, minimum norm, and eigenbeamforming (projection nulling) are examples. Unfortunately, high computational complexity limits the use of subspace tracking in real-time systems. Adaptive algorithms with lower complexities have been proposed to address this limitation. The authors compare two such algorithms: TQR-SVD and fast subspace tracking (FST). Both have lower complexity than traditional approaches, with FST's complexity being lower than TQR-SVD's by a factor of r (the dimension of the dominant subspace). The authors show that a simplified version of FST (called RO-FST-refinement only-FST) produces the same subspace estimates as the TQR-SVD algorithm. >

Analysis of the Enright-Kamel Partitioning Method for Stiff Ordinary Differential Equations

The use of implicit formulae in the solution of stiff ODEs gives rise to systems of nonlinear equations which are usually solved iteratively by a modified Newton scheme. The linear algebra costs associated with such schemes may form a substantial part of the overall cost of the solution. The work of W. H. Enright and M. S. Kamel attempts to reduce the cost of the iteration by automatically transforming and partitioning the system. We provide new theoretical justification for this method in the case where the stiff eigenvalues of the Jacobian matrix used in the modified Newton iteration are small in number and well separated from the other eigenvalues. The theory of Y. Saad is introduced and adapted to show that the method uses the projection of the Jacobian onto a Krylov subspace which virtually contains the dominant subspace. This is shown to have favourable consequences. Numerical evidence is provided to support the theory.

Multigrid method for stability problems

The problem of calculating the stability of steady state solutions of differential equations is treated. Leading eigenvalues (i.e., having maximal real part) of large matrices that arise from discretization are to be calculated. An efficient multigrid method for solving these problems is presented. The method begins by obtaining an initial approximation for the dominant subspace on a coarse level using a damped Jacobi relaxation. This proceeds until enough accuracy for the dominant subspace has been obtained. The resulting grid functions are then used as an initial approximation for appropriate eigenvalue problems. These problems are solved first on coarse levels, followed by refinement until a desired accuracy for the eigenvalues has been achieved. The method employs local relaxation on all levels together with a global change on the coarsest level only, which is designed to separate the different eigenfunctions as well as to update their corresponding eigenvalues. Coarsening is done using the FAS formulation in a nonstandard way in which the right-hand side of the coarse grid equations involves unknown parameters to be solved for on the coarse grid. This in particular leads to a new multigrid method for calculating the eigenvalues of symmetric problems. Numerical experiments with a model problem that are presented demonstrate the effectiveness of the method proposed. Using an FMG algorithm a solution to the level of discretization errors is obtained in just a few work units (less than 10), where a work unit is the work involved in one Jacobi relaxation on the finest level.