Displacement Rank Research Articles

Linear Operators on Matrices: Preserving Spectrum and Displacement Structure

In this paper we characterize those linear operators on general matrices that preserve singular values and displacement rank. We also characterize those linear operators on Hermitian matrices that preserve eigenvalues and displacement inertia.

A Look-Ahead Block Schur Algorithm for Toeplitz-Like Matrices

We derive a look-ahead recursive algorithm for the block triangular factorization of Toeplitz-like matrices. The derivation is based on combining the block Schur/Gauss reduction procedure with displacement structure and leads to an efficient block-Schur complementation algorithm. For an $n \times n$ Toeplitz-like matrix, the overall computational complexity of the algorithm is $O( rn^2 + \frac{n^3 }{t} )$ operations, where r is the matrix displacement rank and t is the number of diagonal blocks. These blocks can be of any desirable size. They may, for example, correspond to the smallest nonsingular leading submatrices or, alternatively, to numerically well-conditioned blocks.

Displacement Structures of Covariance Matrices, Lossless Systems, and Numerical Algorithm Design

Low displacement rank theory underlies many fast algorithms designed for structured covariance matrices. Some of these have gained notoriety for their numerical instability problems, particularly fast least-squares algorithms. Recent studies have shown that instability is not inherent to fast algorithms, but rather comes from the violation of backward consistency constraints. This paper thus details the connection between covariance matrices of a given displacement inertia and lossless rational matrices, as well as the role of this connection in numerically consistent algorithms. This basic connection allows displacement structures to be parametrized via a sequence of rotation angles obtained from a lossless system. The utility of this approach is that, irrespective of errors in the rotation parameter set, they remain consistent with a positive definite matrix of a prescribed displacement inertia. This property in turn may be rephrased as meaningful forms of backward consistency in numerical algorithms. The rotation parameters then take the form of Givens or Jacobi angles applied to data, in contrast to classical approaches which directly manipulate dyadic decompositions of the displacement structure. The concepts are illustrated in popular signal processing applications. In particular, these connections lend clear insight into the stable computation of reflection coefficients of Toeplitz systems, and also serve to resolve the numerical instability problem of fast least-squares algorithms.

On the Stability of the Bareiss and Related Toeplitz Factorization Algorithms

This report contains a numerical stability analysis of factorization algorithms for computing the Cholesky decomposition of symmetric positive definite matrices of displacement rank 2. The algorithms in the class can be expressed as sequences of elementary downdating steps. The stability of the factorization algorithms follows directly from the numerical properties of algorithms for realizing elementary downdating operations. It is shown that the Bareiss algorithm for factorizing a symmetric positive definite Toeplitz matrix is in the class and hence the Bareiss algorithm is stable. Some numerical experiments that compare behavior of the Bareiss algorithm and the Levinson algorithm are presented. These experiments indicate that in general (when the reflection coefficients are not all positive) the Levinson algorithm is not stable; certainly it can give much larger residuals than the Bareiss algorithm.

Displacement structure of generalized inverse matrices

It is shown that matrices with a UV-displacement structure possess generalized inverses with a VU-displacement structure. Estimations for the displacement rank of the generalized inverses are presented. The results apply to matrices of Toeplitz, Vandermonde, and Cauchy type and provide formulas for generalized inverses which are important for fast algorithms.

The fast subsampled-updating recursive least-squares (FSU RLS) algorithm for adaptive filtering based on displacement structure and the FFT

In this paper, we derive a new fast algorithm for Recursive Least-Squares (RLS) adaptive filtering. This algorithm is especially suited for adapting very long filters such as in the acoustic echo cancelation problem. The starting point is to introduce subsampled updating (SU) in the RLS algorithm. In the SU RLS algorithm, the Kalman gain and the likelihood variable are matrices. Due to the shift invariance of the adaptive FIR filtering problem, these matrices exhibit a low displacement rank. This leads to a representation of these quantities in terms of sums of products of triangular Toeplitz matrices. Finally, the product of these Toeplitz matrices with a vector can be computed efficiently by using the Fast Fourier Transform (FFT).

A new algorithm to test the stability of 2-D digital recursive filters

Abstract Any algorithm for testing the stability of 2-D digital recursive filters has to process a 2-D polynomial of degree n and m with respect to each variable. The number of computations needed by the algorithm can be expressed as a polynomial of the variables n and m . The total degree of this polynomial defines the complexity order of the algorithm. In this paper we establish that the matrix associated with Bezout's resultant appearing in the stability test of causal or semicausal recursive filters has 2 as its displacement rank. This permits us to apply the generalized Levinson-Szego algorithm to derive a new algorithm for testing the 2-D digital filters' stability. This algorithm is proposed for quarter-plane or nonsymmetric half-plane recursive filters with real coefficients and without nonessential singularities of the second kind. Its complexity order is equal to 4. Note that the complexity order of the fastest existing algorithms is equal to 5.

Displacement structure of pseudoinverses

A matrix or a linear operator A is said to possess an UV-displacement structure if rank( AU − VA) is small compared with the rank of A. Estimates for the rank of A † V − UA † and more general displacements of A † are presented, where A † is the pseudoinverse of A. The general results are applied to close-to-Toeplitz, close-to-Vandermonde, and generalized Cauchy matrices, Bezoutians, Toeplitz and Hankel Operators, singular integral operators, and integral operators with displacement kernel. This leads to formulas for A † which can be used for the fast computation of pseudosolutions. For Vandermonde matrices the exact displacement rank of A † is evaluated. It turns out that this rank is not always small.

Displacement Rank of Generalized Inverses of Persymmetric Matrices

Toeplitz matrices are persymmetric matrices belonging to the large class of so-called structured matrices, characterized by their displacement rank. This characterization was introduced 12 years ago by Kailath and others. In this framework, properties of singular structured persymmetric matrices are investigated with the goal of proving the possible existence of fast algorithms for computing their pseudo-inverses. Loosely speaking, it is proved that the pseudo-inverses of some structured matrices with displacement rank r have a displacement rank bounded by $2r$.

Concurrent iterative algorithm for Toeplitz-like linear systems

A nonsingular n*n matrix A is given with its short displacement generator. It has small displacement rank bounded by a fixed constant. The class of such matrices generalizes Toeplitz matrices. A good initial approximation to a short displacement generator for A/sup -1/ is readily available. Ways to refine this approximation and numerically compute a displacement generator of A/sup -1/ and the solution vector x=A/sup -1/b to a linear system Ax=b by using O(log/sup 2/n) parallel arithmetic steps and n processors are presented. These results are extended to some other important classes of dense structure matrices.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Decreasing the Displacement Rank of a Matrix

This paper approximates a matrix B having a displacement rank r by a matrix $B_d $ having a lower displacement rank d (for a fixed $d < r $), so as to minimize the approximation error norm. $\| B_d - C \|_2 $ for any matrix C in terms of $\| B - C \|_2 $ is estimated.

Finite-precision analysis of a covariance algorithm for least squares FIR filtering and AR modeling

A numerically stable, fast, order-recursive algorithm for solving the covariance problem in signal modeling is described. The propagation of finite arithmetic errors as well as data acquisition errors is studied in detail. First, linearization of the main algorithmic recursions is carried out. Then, a suitable transformation converts the resulting state equations of the accumulated errors into their residual form. Subsequently, bounds for the residuals are computed. The derivation of these bounds depends heavily on the Levinson type structure of the algorithm and the low displacement rank of the problem. The main result is that the algorithm is weakly numerically stable. The proposed order-recursive algorithm is subsequently utilized as a block adaptive method. Its performance is also demonstrated by long run simulations. >

On a new class of reproducing kernel spaces and a new generalization of the Iohvidov laws

A new class of finite dimensional reproducing kernel spaces of m × 1 vector valued analytic functions on a fairly general domain Ω + is introduced. The reproducing kernels of these spaces have a special form which is based on an m × m matrix valued function Θ, which is J unitary on the boundary of Ω +. Every invertible Hermitian matrix can be interpreted as the Gram matrix of a suitably chosen basis in such a space, where m is equal to the appropriately defined displacement rank of the given matrix. Orthogonal decompositions of the space are developed in terms of Schur complements of the matrix and factorizations of the Θ, much as in the classical cases wherein Ω + is either the unit disc or the half plane. Finally, a new generalization of the Iohvidov laws (which extends earlier generalizations by the same authors) is deduced as an application of the theory.

Finite Precision Analysis of a Covariance Algorithm for LS Fir Filtering and AR Modeling

In this paper a numerical stable fast order recursive algorithm for the solution of the соvariance approach in signal modeling is described. The propagation of finite arithmetic errors as well as data acquisition errors are studied in detail. The proposed algorithm builds from the scheme developed in Kalouptsidis et al (1985), by adding some extra computational redundancy to achieve error cancellation and to simplify the numerical stability analysis. The error analysis method used is similar to the method employed by Cybenko (1980) in his analysis of Durbin's algorithm for linear prediction. First, linearization of the main algorithmic recursions is carried out. Then, a suitable transformation converts the resulting state equations of the accumulated errors into their residual form. Subsequently, bounds for the residuals are computed. The derivation of these bounds depends heavily on the Levinson type structure of the algorithm and the low displacement rank of the problem.

Parallel solution of toeplitzlike linear systems

The known algorithms invert an n × n Toeplitz matrix in sequential arithmetic time O( n log 2 n), but their parallel time is not better than linear in n. We present a weakly numerically stable algorithm that numerically inverts an n × n well-conditioned Toeplitz matrix A in sequential time O( n log 4 n log log n) or in parallel time of O(log 3 n) using O( n log n log log n) processors. The algorithm keeps all its advantages being applied to the inversion of Toeplitzlike matrices, having smaller displacement ranks (which makes it applicable to pseudo-inversion of Toeplitzlike matrices of full rank).

Regularized fast recursive least squares algorithms for adaptive filtering

Fast recursive least squares (FRLS) algorithms are developed by using factorization techniques which represent an alternative way to the geometrical projections approach or the matrix-partitioning-based derivations. The estimation problem is formulated within a regularization approach, and priors are used to achieve a regularized solution which presents better numerical stability properties than the conventional least squares one. The numerical complexity of the presented algorithms is explicitly related to the displacement rank of the a priori covariance matrix of the solution. It then varies between O(5m) and that of the slow RLS algorithms to update the Kalman gain vector, m being the order of the solution. An important advantage of the algorithms is that they admit a unified formulation such that the same equations may equally treat the prewindowed and the covariance cases independently from the used priors. The difference lies only in the involved numerical complexity, which is modified through a change of the dimensions of the intervening variables. Simulation results are given to illustrate the performances of these algorithms.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Divide-and-Conquer Solutions of Least-Squares Problems for Matrices with Displacement Structure

A divide-and-conquer implementation of a generalized Schur algorithm enables (exact and) least-squares solutions of various block-Toeplitz or Toeplitz-block systems of equations with $O ( \alpha ^3 n\log ^2 n )$ operations to be obtained, where the displacement rank $\alpha $ is a small constant (typically between two to four for scalar near-Toeplitz matrices) independent of the size of the matrices.

Quick solution of least square equations and inversion of block matrices of low displacement rank

A general algorithm for the inversion of block matrices of low displacement rank is proposed. It is particularly adapted to the solution of the least squares linear prediction problem. Some explicit formulas for the inverses of such matrices and a characterization of them are also provided. The method is based on the representation of the shift difference of the considered matrix as the difference between two low-rank positive matrices. >

Hyperbolic Householder Algorithms for Factoring Structured Matrices

Efficient algorithms for computing triangular decompositions of Hermitian matrices with small displacement rank using hyperbolic Householder matrices are derived. These algorithms can be both vectorized and parallelized. Implementations along with performance results on an Alliant FX/80, Cray X-MP/48, and Cray-2 are discussed. The use of Householder-type transformations is shown to improve performance for problems with nontrivial displacement ranks. In special cases, the general algorithm reduces to the well-known Schur algorithm for factoring Toeplitz matrices and Elden’s algorithm for solvig structured regularization problems. It gives a Householder formulation to the class of algorithms based on hyperbolic rotations studied by Kailath, Lev-Ari, Chun, and their colleagues for Hermitian matrices with small displacement structure. In addition, an extension to the efficient factorization of indefinite systems is described.

Structured invariant spaces of vector valued functions, sesquilinear forms, and a generalization of the lohvidov laws

Vector spaces of pairs of rational vector valued functions, which are (1) invariant under the generalized backward shift and (2) endowed with a sesquilinear form which is subject to a structural identity, are studied. It is shown that any matrix can be viewed as the “Gram” matrix of a suitably defined basis for such a space. This identification is used to show that a rule due to Iohvidov for evaluating the rank of certain subblocks of a Toeplitz (or Hankel) matrix is applicable to a wider class of matrices with (appropriately defined) displacement rank equal to two. Enroute, a theory of reproducing kernel spaces is developed for nondegenerate spaces of the type mentioned above.