Positive Definite Toeplitz Matrices Research Articles

Entries of the inverses of large positive definite Toeplitz matrices

This is an expository paper embarking on the asymptotic behaviorof the entries of the inverses of positive definite symmetric Toeplitzmatrices as the matrix dimension goes to infinity. We consider the behaviorof the entries in neighborhoods of the four corners as well as thedensity of the distribution of the entries over all of the inverse matrix.

Empirical Bayes identification of stationary processes and approximation of Toeplitz spectra

We study Statistical Consistency of an approximate subspace identification procedure for the infinite dimensional a posteriori model of a frequency estimation problem in an Empirical Bayesian framework. By first imposing a natural uniform prior probability density on the unknown frequencies, the estimation of the hyperparameters of the a priori distribution can be accomplished by a sequence of subspace identification techniques. These techniques exploit the special structure of the covariance matrix of the a posteriori process which is discovered by making a connection with classical results on energy concentration in deterministic signal processing. The convergence of the spectrum of the subspace estimates to the (nonrational) spectral density of the a posteriori process has an analytic counterpart in the approximation of symmetric positive definite Toeplitz matrices by submatrices of finite rank. This is proven by a weak-sense convergence theorem for Toeplitz spectra.

Entries of the inverses of large positive definite Toeplitz matrices

AbstractThis is an expository paper embarking on the asymptotic behavior of the entries of the inverses of positive definite symmetric Toeplitz matrices as the matrix dimension goes to infinity. We consider the behavior of the entries in neighborhoods of the four corners as well as the density of the distribution of the entries over all of the inverse matrix.

On the inverse problem of fractional Brownian motion and the inverse of infinite Toeplitz matrices

The inverse problem of fractional Brownian motion and other Gaussian processes with stationary increments involves inverting an infinite hermitian positively definite Toeplitz matrix (a matrix that has equal elements along its diagonals). The problem of inverting Toeplitz matrices is interesting on its own and has various applications in physics, signal processing, statistics, etc. A large body of literature has emerged to study this question since the seminal work of Szegö on Toeplitz forms in 1920's. In this paper we obtain, for the first time, an explicit general formula for the inverse of infinite hermitian positive definite Toeplitz matrices. Our formula is explicitly given in terms of the Szegö function associated to the spectral density of the matrix. These results are applied to the fractional Brownian motion and to m-diagonal Toeplitz matrices and we provide explicit examples.

The Generalized Schur Algorithm and Some Applications

The generalized Schur algorithm is a powerful tool allowing to compute classical decompositions of matrices, such as the Q R and L U factorizations. When applied to matrices with particular structures, the generalized Schur algorithm computes these factorizations with a complexity of one order of magnitude less than that of classical algorithms based on Householder or elementary transformations. In this manuscript, we describe the main features of the generalized Schur algorithm. We show that it helps to prove some theoretical properties of the R factor of the Q R factorization of some structured matrices, such as symmetric positive definite Toeplitz and Sylvester matrices, that can hardly be proven using classical linear algebra tools. Moreover, we propose a fast implementation of the generalized Schur algorithm for computing the rank of Sylvester matrices, arising in a number of applications. Finally, we propose a generalized Schur based algorithm for computing the null-space of polynomial matrices.

Norms of positive definite Toeplitz matrices and detection of almost periodic components in random signals

For positive definite Toeplitz matrices QN =( b(j −k)) N−1 j,k=0 generated by trigonomet- ric moments b(j) of a non-negative measure dσ(θ),θ ∈ (−π,π), we note that the Hilbert- Schmidt normQN � 2 and the maximal eigenvalue λm(N) satisfy the following relations

A Divide and Conquer Algorithm for the Superfast Solution of Toeplitz-like Systems

In this paper a new $O(N \,\log^3N)$ solver for $N\times N$ Toeplitz-like systems, based on a divide and conquer technique, is presented. Similarly to the superfast algorithm MBA for the inversion of a Toeplitz-like matrix [R. R. Bitmead and B. D. O. Anderson, Linear Algebra Appl., 34 (1980), pp. 103--116; M. Morf, Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 1980, pp. 954--959], it exploits the displacement properties. In order to avoid the well-known numerical instability of the explicit inversion, the new algorithm relies on the triangular factorization and back-substitution formula for the system seen as a $2\times 2$ block system with blocks of half size. This idea is the one used in [M. Stewart, SIAM J. Matrix Anal. Appl., 25 (2003), pp. 669--693] to improve the numerical stability of superfast methods based on the generalized Schur algorithm for positive definite Toeplitz matrices, but the algorithm we propose can be applied also to nonsymmetric Toeplitz-like systems. The stability of the algorithm is examined through numerical experiments.

Pivot tightening for direct methods for solving symmetric positive definite systems of linear interval equations

The paper considers systems of linear interval equations, i.e., linear systems where the coefficients of the matrix and the right hand side vary between given bounds. We focus on symmetric matrices and consider direct methods for the enclosure of the solution set of such a system. One of these methods is the interval Cholesky method, which is obtained from the ordinary Cholesky decomposition by replacing the real numbers by the related intervals and the real operations by the respective interval operations. We present a method by which the diagonal entries of the interval Cholesky factor can be tightened for positive definite interval matrices, such that a breakdown of the algorithm can be prevented. In the case of positive definite symmetric Toeplitz matrices, a further tightening of the diagonal entries and also of other entries of the Cholesky factor is possible. Finally, we numerically compare the interval Cholesky method with interval variants of two methods which exploit the Toeplitz structure with respect to the computing time and the quality of the enclosure of the solution set.

A modified T. Chan’s preconditioner for Toeplitz systems

We present a modified T. Chan’s preconditioner for solving Toeplitz linear systems by the preconditioned conjugate gradient (PCG) method in this paper. Especially, we give some results when the matrices are Hermitian positive definite Toeplitz matrices. The operation and convergence of the PCG method are discussed. Numerical examples presented illustrate the effectiveness of the preconditioner obtained.

A Schur-based algorithm for computing bounds to the smallest eigenvalue of a symmetric positive definite Toeplitz matrix

Recent progress in signal processing and estimation has generated c onsiderable interest in the problem of computing the smallest eigenvalue of symmetric positive definite Toeplitz matrices. Several algorithms have been proposed in the literature. They compute the smallest eigenvalue in an iterative fashion, many of them relying on the Levinson–Durbin solution of sequences of Yule–Walker systems. Exploiting the properties of two algorithms recently developed for estimating a lower and an upper bound of the smallest singular value of upper triangular matrices, respectively, an algorithm for computing bounds to the smallest eigenvalue of a symmetric positive definite Toeplitz matrix is derived. The algorithm relies on the computation of the R factor of the QR-factorization of the Toeplitz matrix and the inverse of R. The simultaneous computation of R and R −1 is efficiently accomplished by the generalized Schur algorithm.

Computation of symmetric positive definite Toeplitz matrices by the hybrid steepest descent method

This paper studies the problem of finding the nearest symmetric positive definite Toeplitz matrix to a given symmetric one. Additional design constraints, which are also formed as closed convex sets in the real Hilbert space of all symmetric matrices, are imposed on the desired matrix. An algorithmic solution to the problem given by the hybrid steepest descent method is established also in the case of inconsistent design constraints.

A polynomial fit preconditioner for band Toeplitz matrices in image reconstruction

The preconditioned conjugate gradient (CG) is often applied in image reconstruction as a regularizing method. When the blurring matrix has Toeplitz structure, the modified circulant preconditioner and the inverse Toeplitz preconditioner have been shown to be effective. We introduce here a preconditioner for symmetric positive definite Toeplitz matrices based on a trigonometric polynomial fit which has the same effectiveness of the previous ones but has a lower cost when applied to band matrices. The case of band block Toeplitz matrices with band Toeplitz blocks (BTTB) corresponding to separable point spread functions (PSFs) is also considered.

Hartley Transform Representations of Symmetric Toeplitz Matrix Inverses with Application to Fast Matrix-Vector Multiplication

Representations for inverses of real symmetric Toeplitz matrices involving discrete Hartley transformations are presented which can be used for fast matrix-vector multiplication. In this way, multiplication of a column vector by an inverse real symmetric Toeplitz matrix can be carried out with the help of six Hartley transformations plus two for preprocessing. Besides complexity, stability issues will also be discussed. In particular, it is shown that, for positive definite Toeplitz matrices, the relative error of the computed vector can be estimated by a multiple of the condition number of the matrix.

Preconditioners for ill-conditioned Toeplitz matrices

This paper is concerned with the solution of systems of linear equations A N x = b, where $$\{ A_N \} _{N \in \mathbb{N}}$$ denotes a sequence of positive definite Hermitian ill-conditioned Toeplitz matrices arising from a (real-valued) nonnegative generating function f ∈ C 2π with zeros. We construct positive definite Hermitian preconditioners M N such that the eigenvalues of M N −1 A N are clustered at 1 and the corresponding PCG-method requires only O(N log N) arithmetical operations to achieve a prescribed precision. We sketch how our preconditioning technique can be extended to symmetric Toeplitz systems, doubly symmetric block Toeplitz systems with Toeplitz blocks and non-Hermitian Toeplitz systems. Numerical tests confirm the theoretical expectations.

Symmetric Gaussian elimination for cauchy-type matrices with application to positive definite Toeplitz matrices

The problem of solving linear equations with a Toeplitz matrix \(T\) appears in many applications. Often \(T\) is positive definite but ill-conditioned with many small eigenvalues. In this case fast and superfast algorithms may show a very poor behavior or even break down. In recent papers the transformation of a Toeplitz matrix into a Cauchy-type matrix is proposed. The resulting new linear equations can be solved in \(O(n^2)\) operations using standard pivoting strategies which leads to very stable fast methods also for ill-conditioned systems. The basic tool is the formulation of Gaussian elimination for matrices with low displacement rank. In this paper, we will transform a Hermitian Toeplitz matrix \(T\) into a Cauchy-type matrix \(B=FTF^{\rm H}\) by applying the Fourier transform. We will prove some useful properties of \(B\) and formulate a symmetric Gaussian elimination algorithm for positive definite \(B\). Using the symmetry and persymmetry of \(T\) we can reduce the total costs of this algorithm compared with unsymmetric Gaussian elimination. For complex Hermitian \(T\), the complexity of the new algorithm is then nearly the same as for the Schur algorithm. Furthermore, it is possible to include some strategies for ill-conditioned positive definite matrices that are well-known in optimization. Numerical examples show that this new algorithm is fast and reliable.

Qualitative Analysis of One-Step Algebraic Models with Tridiagonal Toeplitz Matrices

One-step algebraic models with symmetric and positive definite (SPD) tridiagonal Toeplitz matrices are introduced and their qualitative properties are studied. Theoretical results are applied to the numerical solution of parabolic differential equations, with illustrations by numerical examples. Possible extensions as well as arising open problems are discussed in concluding remarks.

Classical foundations of algorithms for solving positive definite Toeplitz equations

The ongoing development and analysis of efficient algorithms for solving positive definite Toeplitz equations is motivated to a large extent by the importance of these equations in signal processing applications. The role of positive definite Toeplitz matrices in this and other areas of mathematics and engineering stems from Schur's study of bounded analytic functions on the unit disk, and Szegő's theory of polynomials orthogonal on the unit circle. These ideas underlie several Toeplitz solvers, and provide a useful framework for understanding the relationships among these algorithms. In this paper we give an overview of several direct algorithms for solving positive definite Toeplitz systems of linear equations from this classical viewpoint.

Application of ADI Iterative Methods to the Restoration of Noisy Images

The restoration of two-dimensional images in the presence of noise by Wiener's minimum mean square error filter requires the solution of large linear systems of equations. When the noise is white and Gaussian, and under suitable assumptions on the image, these equations can be written as a Sylvester's equation \[ T_1^{-1}\hat{F}+\hat{F}T_2=C \] for the matrix $\hat{F}$ representing the restored image. The matrices $T_1$ and $T_2$ are symmetric positive definite Toeplitz matrices. We show that the ADI iterative method is well suited for the solution of these Sylvester's equations, and illustrate this with computed examples for the case when the image is described by a separable first-order Markov process. We also consider generalizations of the ADI iterative method, propose new algorithms for the generation of iteration parameters, and illustrate the competitiveness of these schemes.

Fast Gaussian Elimination with Partial Pivoting for Matrices with Displacement Structure

Fast O(n 2 ) implementation of Gaussian elimination with partial pivoting is designed for matrices possessing Cauchy-like displacement structure. We show how Toeplitz-like, Toeplitz-plus-Hankel-like and Vandermonde-like matrices can be transformed into Cauchy-like matrices by using Discrete Fourier, Cosine or Sine Transform matrices. In particular this allows us to propose a new fast O(n 2 ) Toeplitz solver GKO, which incorporates partial pivoting. A large set of numerical examples showed that GKO demonstrated stable numerical behavior and can be recommended for solving linear systems, especially with nonsymmetric, indefinite and ill-conditioned positive definite Toeplitz matrices. It is also useful for block Toeplitz and mosaic Toeplitz ( Toeplitz-block ) matrices. The algorithms proposed in this paper suggest an attractive alternative to look-ahead approaches, where one has to jump over ill-conditioned leading submatrices, which in the worst case requires O(n 3 ) operations.

Fast Gaussian elimination with partial pivoting for matrices with displacement structure

Fast O ( n 2 ) O({n^2}) implementation of Gaussian elimination with partial pivoting is designed for matrices possessing Cauchy-like displacement structure. We show how Toeplitz-like, Toeplitz-plus-Hankel-like and Vandermonde-like matrices can be transformed into Cauchy-like matrices by using Discrete Fourier, Cosine or Sine Transform matrices. In particular this allows us to propose a new fast O ( n 2 ) O({n^2}) Toeplitz solver GKO, which incorporates partial pivoting. A large set of numerical examples showed that GKO demonstrated stable numerical behavior and can be recommended for solving linear systems, especially with nonsymmetric, indefinite and ill-conditioned positive definite Toeplitz matrices. It is also useful for block Toeplitz and mosaic Toeplitz (Toeplitz-block) matrices. The algorithms proposed in this paper suggest an attractive alternative to look-ahead approaches, where one has to jump over ill-conditioned leading submatrices, which in the worst case requires O ( n 3 ) O({n^3}) operations.