Orthogonal Similarity Transformations Research Articles

On the convergence properties of the orthogonal similarity transformations to tridiagonal and semiseparable (plus diagonal) form

In this paper, we will compare the convergence properties of three basic reduction methods, by placing them in a general framework. It covers the reduction to tridiagonal, semiseparable and semiseparable plus diagonal form. These reductions are often used as the first step in the computation of the eigenvalues and/or eigenvectors of arbitrary matrices. In this way, the calculation of the eigenvalues using, for example, the QR-algorithm reduces in complexity. First we will investigate the convergence properties of these three reduction algorithms. It will be shown that for the partially reduced matrices at step k of any of these reduction algorithms, the lower right k × k (already reduced) sub-block will have the Lanczos–Ritz values, w.r.t. a certain starting vector. It will also be shown that the reductions to semiseparable and to semiseparable plus diagonal form have an extra convergence behavior a special type of subspace iteration is performed on the lower right k × k submatrix, which contains these Ritz-values. Secondly we look in more detail at the behavior of the involved subspace iteration. It will be shown that the reduction method can be interpreted as a nested type of multi-shift iteration. Theoretical results will be presented, making it possible to predict the convergence behavior of these reduction algorithms. Also a theoretical bound on the convergence rate is presented. Finally we illustrate by means of numerical examples, how it is possible to tune the convergence behavior such that it can become a powerful tool for certain applications.

Orthogonal similarity transformation of a symmetric matrix into a diagonal-plus-semiseparable one with free choice of the diagonal

In this paper we describe an orthogonal similarity transformation for transforming arbitrary symmetric matrices into a diagonal-plus-semiseparable matrix, where we can freely choose the diagonal. Very recently an algorithm was proposed for transforming arbitrary symmetric matrices into similar semiseparable ones. This reduction is strongly connected to the reduction to tridiagonal form. The class of semiseparable matrices can be considered as a subclass of the diagonalplus- semiseparable matrices. Therefore we can interpret the proposed algorithm here as an extension of the reduction to semiseparable form. A numerical experiment is performed comparing thereby the accuracy of this reduction algorithm with respect to the accuracy of the traditional reduction to tridiagonal form, and the reduction to semiseparable form. The experiment indicates that all three reduction algorithms are equally accurate. Moreover it is shown in the experiments that asymptotically all the three approaches have the same complexity, i.e. that they have the same factor preceding the n 3 term in the computational complexity. Finally we illustrate that special choices of the diagonal create a specific convergence behavior.

Necessary and sufficient conditions for orthogonal similarity transformations to obtain the Arnoli(Lanczos)–Ritz values

It is a well-known fact that while reducing a symmetric matrix into a similar tridiagonal one, the already tridiagonal matrix in the partially reduced matrix has as eigenvalues the Lanczos–Ritz values. This behavior is also shared by the reduction algorithm which transforms symmetric matrices via orthogonal similarity transformations to semiseparable form. Moreover also the orthogonal reduction to Hessenberg form has a similar behavior with respect to the Arnoldi–Ritz values.In this paper we investigate the orthogonal similarity transformations creating this behavior. Two easy conditions are derived, which provide necessary and sufficient conditions, such that the partially reduced matrices have the desired convergence behavior. The conditions are easy to check as they demand that in every step of the reduction algorithm two particular matrices need to have a zero block.

An Implicit Q Theorem for Hessenberg-like Matrices

The implicit Q theorem for Hessenberg matrices is a widespread and powerful theorem. It is used in the development of, for example, implicit QR algorithms to compute the eigendecomposition of Hessenberg matrices. Moreover it can also be used to prove the essential uniqueness of orthogonal similarity transformations of matrices to Hessenberg form. The theorem is also valid for symmetric tridiagonal matrices, proving thereby also in the symmetric case its power. Currently there is a growing interest to so-called semiseparable matrices. These matrices can be considered as the inverses of tridiagonal matrices. In a similar way, one can consider Hessenberg-like matrices as the inverses of Hessenberg matrices. In this paper, we formulate and prove an implicit Q theorem for the class of Hessenberg-like matrices. We introduce the notion of strongly unreduced Hessenberg-like matrices and also a method for transforming matrices via orthogonal transformations to this form is proposed. Moreover, as the theorem is valid for Hessenberg-like matrices it is also valid for symmetric semiseparable matrices.

An Orthogonal Similarity Reduction of a Matrix into Semiseparable Form

An algorithm to reduce a symmetric matrix to a similar semiseparable one of semiseparability rank 1, using orthogonal similarity transformations, is proposed in this paper. It is shown that, while running to completion, the proposed algorithm gives information on the spectrum of the similar initial matrix. In fact, the proposed algorithm shares the same properties of the Lanczos method and the Householder reduction to tridiagonal form. Furthermore, at each iteration, the proposed algorithm performs a step of the QR method without shift to a principal submatrix to retrieve the semiseparable structure. The latter step can be considered a kind of subspace-like iteration method, where the size of the subspace increases by one dimension at each step of the algorithm. Hence, when during the execution of the algorithm the Ritz values approximate the dominant eigenvalues closely enough, diagonal blocks will appear in the semiseparable part where the norm of the corresponding subdiagonal blocks goes to zero in the subsequent iteration steps, depending on the corresponding gap between the eigenvalues. A numerical experiment is included, illustrating the properties of the new algorithm.

Orthogonal similarity transformation into block-semiseparable matrices of semiseparability rankk

Very recently, an algorithm, which reduces any symmetric matrix into a semiseparable one of semi- separability rank 1 by similar orthogonality transformations, has been proposed by Vandebril, Van Barel and Mastronardi. Partial execution of this algorithm computes a semiseparable matrix whose eigenvalues are the Ritz-values obtained by the Lanczos' process applied to the original matrix. Also a kind of nested subspace iteration is performed at each step. In this paper, we generalize the above results and propose an algorithm to reduce any symmetric matrix into a similar block-semiseparable one of semiseparability rank k, with k ∈ ℕ, by orthogonal similarity transformations. Also in this case partial execution of the algorithm computes a block-semiseparable matrix whose eigenvalues are the Ritz-values obtained by the block-Lanczos' process with k starting vectors, applied to the original matrix. Subspace iteration is performed at each step as well. Copyright © 2005 John Wiley & Sons, Ltd.

QRT: A QR-Based Tridiagonalization Algorithm for Nonsymmetric Matrices

The stable similarity reduction of a nonsymmetric square matrix to tridiagonal form has been a long-standing problem in numerical linear algebra. The biorthogonal Lanczos process is in principle a candidate method for this task, but in practice it is confined to sparse matrices and is restarted periodically because roundoff errors affect its three-term recurrence scheme and degrade the biorthogonality after a few steps. This adds to its vulnerability to serious breakdowns or near-breakdowns, the handling of which involves recovery strategies such as the look-ahead technique, which needs a careful implementation to produce a block-tridiagonal form with unpredictable block sizes. Other candidate methods, geared generally towards full matrices, rely on elementary similarity transformations that are prone to numerical instabilities. Such concomitant difficulties have hampered finding a satisfactory solution to the problem for either sparse or full matrices. This study focuses primarily on full matrices. After outlining earlier tridiagonalization algorithms from within a general framework, we present a new elimination technique combining orthogonal similarity transformations that are stable. We also discuss heuristics to circumvent breakdowns. Applications of this study include eigenvalue calculation and the approximation of matrix functions.

Computation of Kalman Decompositions of Periodic Systems

We consider the numerically reliable computation of reachability and observability Kalman decompositions of a periodic system with time-varying dimensions. These decompositions generalize the controllability/observability Kalman decompositions for standard state space systems and have immediate applications in the structural analysis of periodic systems. We propose a structure exploiting numerical algorithm to compute the periodic controllability form by employing exclusively orthogonal state-space similarity transformations. The new algorithm is computationally efficient and backward stable, thus fulfils all requirements for a satisfactory algorithm for periodic systems.

An efficient method for computing eigenvalues of a real normal matrix

Jacobi-based algorithms have attracted attention as they have a high degree of potential parallelism and may be more accurate than QR-based algorithms. In this paper we discuss how to design efficient Jacobi-like algorithms for eigenvalue decomposition of a real normal matrix. We introduce a block Jacobi-like method. This method uses only real arithmetic and orthogonal similarity transformations and achieves ultimate quadratic convergence. A theoretical analysis is conducted and some experimental results are presented.

Right coprime factorisations using system upper Hessenberg forms—the multi-input system case

Based on a method for right coprime factorisations of linear systems using matrix elementary transformations, it is shown that a very simple iteration formula exists for right coprime factorisations of multi-input linear systems in the system upper Hessenberg forms. This formula gives directly the coefficient matrices of the pair of solutions to the right coprime factorisation of the system Hessenberg form, and involves only inverses of a few triangular matrices and some matrix products and summations. Based on this formula, a simple, efficient procedure for determining a right coprime factorisation of a multi-input linear system is proposed, which first converts a given linear system into its system Hessenberg form using some orthogonal similarity transformations and then applies the iteration formula to the converted system Hessenberg form. An example demonstrates the use of the approach.

IMPROVED DETECTION OF BIFURCATIONS IN LARGE NONLINEAR SYSTEMS VIA THE CONTINUATION OF INVARIANT SUBSPACES ALGORITHM

The Continuation of Invariant Subspaces (CIS) algorithm [Demmel et al., 2001; Dieci & Friedman, 2001] produces a smooth orthogonal similarity transformation to block triangular form of a parameter dependent matrix A(s). The CIS algorithm is an adaptation of an iterative refinement technique for improving the accuracy of computed invariant subspaces. On the basis of the CIS algorithm, new test functions are defined for an improved, more reliable detection of bifurcations in problems with simple bifurcations near multiple bifurcations, problems with multiple bifurcations, and problems with symmetries. Illustrative examples are given.

A framework for symmetric band reduction

We develop an algorithmic framework for reducing the bandwidth of symmetric matrices via orthogonal similarity transformations. This framework includes the reduction of full matrices to banded or tridiagonal form and the reduction of banded matrices to narrower banded or tridiagonal form, possibly in multiple steps. Our framework leads to algorithms that require fewer floating-point operations than do standard algorithms, if only the eigenvalues are required. In addition, it allows for space-time tradeoffs and enables or increases the use of blocked transformations.

Fast and stable eigendecomposition of symmetric banded plus semi-separable matrices

A new fast and stable algorithm to reduce a symmetric banded plus semi-separable matrix to tridiagonal form via orthogonal similarity transformations is presented.

Computing the logarithm of a symmetric positive definite matrix

A numerical method for computing the logarithm of a symmetric positive definite matrix is developed in this paper. It is based on reducing the original matrix to a tridiagonal matrix by orthogonal similarity transformations and applying Padé approximations to the logarithm of the tridiagonal matrix. Theoretical studies and numerical experiments indicate that the method is quite efficient when the matrix is not very ill-conditioned.

A general framework for state feedback pole assignment of singular systems

In this paper, the state feedback pole assignment problem of singular systems is investigated. It is well known that the pole assignment of singular systems is equivalent to the one for normal systems. The above theory is re-developed through the application of the orthogonal similarity transformation, which is numerically stable. The merit of this approach is that almost all existing methods for assigning poles in normal systems can be generalized to solve the problem in a reliable way. Parallel algorithms for this generalization procedure are also discussed.

A PARALLEL TRIANGULAR SYLVESTER EQUATION SOLVER BASED ON THE HESSENBERG-SCHUR METHOD∗

The Hessenberg-Schur method is one of the most efficient and stable algorithms for solving linear matrix equations. When it is applied to the Sylvester equation, AX + X B = C, matrix A is reduced to the Hessenberg form and matrix B to the real Schur for using orthogonal similarity transformations. In this paper we present a parallel algorithm for solving the Sylvester matrix equation when A and B are in the above-mentioned forms on a Distributed Memory Multiprocessor. Both a complexity analysis and an experimental study of the new algorithm are also presented.

A Hybrid Tridiagonalization Algorithm for Symmetric Sparse Matrices

This paper considers sequential direct methods for the reduction of a sparse symmetric matrix to tridiagonal form using sequences of orthogonal similarity transformations. Currently, the best published approach combines a bandwidth-reducing preordering algorithm, perhaps Gibbs–Poole–Stockmeyer (GPS), with a band-preserving tridiagonalization such as the EISPACK BANDR routine. This paper introduces a new hybrid tridiagonalization algorithm, BANDHYB, based upon band-preserving reduction techniques that rearrange the elimination sequence of nonzero entries to take better advantage of the sparsity within the band of the permuted matrix. For many practical sparse problems a GPS-BANDHYB approach substantially reduces the CPU requirements of tridiagonalization, compared to a GPS-BANDR scheme, without increasing storage requirements. Over a wide range of sparse problems the new algorithm reduced CPU time by an average of 31%, with reductions of more than 60% observed for some problems.

Fast Plane Rotations with Dynamic Scaling

This paper presents fast plane rotations for orthogonal similarity and orthogonal one-sided transformations. Fast rotations have the advantage that they reduce the number of square roots and multiplications. The authors' new rotations have further advantages over the existing fast rotations: they obviate the rescaling that has been necessary to guard against underflow or overflow and they give higher efficiency, especially on vector processors. An error analysis, in the case of the $QR$ decomposition, and computational results that illustrate the effects of the dynamic scaling are presented.

On swapping diagonal blocks in real Schur form

We discuss a new version of an existing algorithm for reordering the eigenvalues on the diagonal of a matrix in real Schur form by performing an orthogonal similarity transformation. A detailed error analysis and software description are presented. Numerical examples show the superiority of our algorithm over previous algorithms.

Computation of Zeros of Generalized State-Space Systems

Abstract The paper presents an efficient and numerically stable algorithm for the computation of finite zeros of generalized state-space systems. The proposed algorithm reduces the system matrix pencil to a regular pencil whose finite generalized eigenvalues are the finite zeros of the system. The reduction is performed using unitary (in complex case) or orthogonal (in real case) pencil similarity transformations. The method is general, it can handle both square and non-square systems, degene-rate systems as well as singular systems. It exploits efficiently the particular structure of the system matrix pencil. The proposed algorithm can be viewed as an extension of a similar algorithm for stan-dard state-space systems. Numerical properties and algorithmic details are discussed. Numerical examples are given for illustrating the performances of the algorithm.