Toeplitz Preconditioners Research Articles

Fast solution methods for Riesz space fractional diffusion equations with non-separable coefficients

In this paper, efficient preconditioned iterative methods for solving the Riesz space fractional diffusion equations with variable diffusion coefficients are considered. Crank–Nicolson and weighted and shifted Grünwald difference scheme are applied to discretize the problem. For one-dimensional problems, we transform the asymmetric discretized linear system into a symmetric one and solve the resulted linear system by preconditioned conjugate gradient method. The preconditioned conjugate gradient method with a symmetric Toeplitz preconditioner and a sine transform based preconditioner are employed to solve the resulting linear system. Theoretically, we respectively prove that the condition numbers of preconditioned matrices have uniform upper bounds, which is independent of discretization step-sizes and fractional order. For two-dimensional problems, we propose an optional and symmetric splitting iteration method, which can be embedded by conjugate gradient method although the diffusion coefficients are non-separable. We prove that the optional and symmetric splitting iteration method is unconditionally convergent. Numerical results are reported to illustrate the efficiency of the proposed methods.

A block Toeplitz preconditioner for all-at-once systems from linear wave equations

In this work, we propose a novel parallel-in-time preconditioner for an all-at-once system, arising from the numerical solution of linear wave equations. Namely, our main result concerns a block tridiagonal Toeplitz preconditioner that can be diagonalized via fast sine transforms, whose effectiveness is theoretically shown for the nonsymmetric block Toeplitz system resulting from discretizing the concerned wave equation. Our approach is to first transform the original linear system into a symmetric one and subsequently develop the desired preconditioning strategy based on the spectral symbol of the modified matrix. Various Krylov subspace methods are considered. That is, we show that the minimal polynomial of the preconditioned matrix is of low degree, which leads to fast convergence when the generalized minimal residual method is used. To fully utilize the symmetry of the modified matrix, we additionally construct an absolute-value preconditioner which is symmetric positive definite. Then, we show that the eigenvalues of the preconditioned matrix are clustered around $\pm 1$, which gives a convergence guarantee when the minimal residual method is employed. Numerical examples are given to support the effectiveness of our preconditioner. Our block Toeplitz preconditioner provides an alternative to the existing block circulant preconditioner proposed by McDonald, Pestana, and Wathenin [SIAM J. Sci. Comput., 40 (2018), pp. A1012–A1033],advancing the symmetrization preconditioning theory that originated from the same work.

An Efficient Second-Order Convergent Scheme for One-Side Space Fractional Diffusion Equations with Variable Coefficients

In this paper, a second-order finite-difference scheme is investigated for time-dependent space fractional diffusion equations with variable coefficients. In the presented scheme, the Crank–Nicolson temporal discretization and a second-order weighted-and-shifted Grünwald–Letnikov spatial discretization are employed. Theoretically, the unconditional stability and the second-order convergence in time and space of the proposed scheme are established under some conditions on the variable coefficients. Moreover, a Toeplitz preconditioner is proposed for linear systems arising from the proposed scheme. The condition number of the preconditioned matrix is proven to be bounded by a constant independent of the discretization step-sizes, so that the Krylov subspace solver for the preconditioned linear systems converges linearly. Numerical results are reported to show the convergence rate and the efficiency of the proposed scheme.

A limited-memory block bi-diagonal Toeplitz preconditioner for block lower triangular Toeplitz system from time–space fractional diffusion equation

A block lower triangular Toeplitz system arising from time-space fractional diffusion equation is discussed. For efficient solutions of such the linear system, the preconditioned biconjugate gradient stabilized method and flexible general minimal residual method are exploited. The main contribution of this paper has two aspects: (i) A block bi-diagonal Toeplitz preconditioner is developed for the block lower triangular Toeplitz system, whose storage is of $\mathcal{O}(N)$ with $N$ being the spatial grid number; (ii) A new skew-circulant preconditioner is designed to fast calculate the inverse of the block bi-diagonal Toeplitz preconditioner multiplying a vector. Numerical experiments are given to demonstrate the efficiency of our preconditioners.

Band Toeplitz preconditioners for non-symmetric real Toeplitz systems by preconditioned GMRES method

In this paper we study n×n real, non-symmetric, ill conditioned Toeplitz systems of the form Tn(f)x=b. The corresponding generating function is a complex valued one of the form f=f1+if2, where i=−1, is known a priori and has roots. We note that f1 is a 2π-periodic even function while f2 is a 2π-periodic odd one. In order to solve the above system efficiently, we use the Preconditioned Generalized Minimal Residual (PGMRES) method and the Preconditioned Conjugate Gradient method applied to the Normal Equations (PCGN). We present a specific preconditioning technique that combines elimination of the roots of f and best uniform approximation or interpolation by trigonometric polynomials. The proposed preconditioner is a band Toeplitz matrix Tn(p) generated by the trigonometric polynomial p=gq. The trigonometric polynomial g is an appropriate complex polynomial having the same zeros as f while q is the trigonometric polynomial derived by the uniform approximation or interpolation of the function fg. Using Tn(p) as a preconditioner, we achieve a good clustering of the singular values of the preconditioned matrix in a small interval around 1, as well as a good clustering of the eigenvalues in a small domain in the right half-plane of the complex plane, far from zero. Moreover, we describe on how the presented preconditioning technique can be extended to the two level Toeplitz systems. Finally, we show, by various numerical experiments, that the proposed preconditioning technique can solve a Toeplitz system of the above form in a small number of iterations.

Multigrid with FFT smoother for a simplified 2D frictional contact problem

SUMMARYThis paper aims to develop a fast multigrid (MG) solver for a Fredholm integral equation of the first kind, arising from the 2D elastic frictional contact problem. After discretization on a rectangular contact area, the integral equation gives rise to a linear system with the coefficient matrix being dense, symmetric positive definite and Toeplitz. A so‐called fast Fourier transform (FFT) smoother is proposed. This is based on a preconditioner M that approximates the inverse of the original coefficient matrix, and that is determined using the FFT technique. The iterates are then updated by Richardson iteration: adding the current residuals preconditioned with the Toeplitz preconditioner M. The FFT smoother significantly reduces most components of the error but enlarges several smooth components. This causes divergence of the MG method. Two approaches are studied to remedy this feature: subdomain deflation (SD) and row sum modification (RSM). MG with the FFT + RSM smoother appears to be more efficient than using the FFT + SD smoother. Moreover, the FFT + RSM smoother can be applied as an efficient iterative solver itself. The two methods related to RSM also show rapid convergence in a test with a wavy surface, where the Toeplitz structure is lost. Copyright © 2014 John Wiley & Sons, Ltd.

Banded Toeplitz Preconditioners for Toeplitz Matrices from Sinc Methods

We give general expressions, analyze algebraic properties and derive eigenvalue bounds for a sequence of Toeplitz matrices associated with the sinc discretizations of various orders of differential operators. We demonstrate that these Toeplitz matrices can be satisfactorily preconditioned by certain banded Toeplitz matrices through showing that the spectra of the preconditioned matrices are uniformly bounded. In particular, we also derive eigenvalue bounds for the banded Toeplitz preconditioners. These results are elementary in constructing high-quality structured preconditioners for the systems of linear equations arising from the sinc discretizations of ordinary and partial differential equations, and are useful in analyzing algebraic properties and deriving eigenvalue bounds for the corresponding preconditioned matrices. Numerical examples are given to show effectiveness of the banded Toeplitz preconditioners.

Inverse product Toeplitz preconditioners for non-Hermitian Toeplitz systems

In this paper, we first propose product Toeplitz preconditioners (in an inverse form) for non-Hermitian Toeplitz matrices generated by functions with zeros. Our inverse product-type preconditioner is of the form \(T_F T_L^{-1} T_U^{-1}\) where T F , T L , and T U are full, band lower triangular, and band upper triangular Toeplitz matrices, respectively. Our basic idea is to decompose the generating function properly such that all factors T F , T L , and T U of the preconditioner are as well-conditioned as possible. We prove that under certain conditions, the preconditioned matrix has eigenvalues and singular values clustered around 1. Then we use a similar idea to modify the preconditioner proposed in Ku and Kuo (SIAM J Sci Stat Comput 13:1470–1487, 1992) to handle the zeros in rational generating functions. Numerical results, including applications to the computation of the stationary probability distribution of Markovian queuing models with batch arrival, are given to illustrate the good performance of the proposed preconditioners.

An explicit formula for the inverse of band triangular Toeplitz matrix

In order to estimate the condition number of the preconditioned matrix proposed in [F.R. Lin, W.K. Ching, Inverse Toeplitz preconditioners for Hermitian Toeplitz systems, Numer. Linear Algebra Appl. 12 (2005) 221–229], we study the inverse of band triangular Toeplitz matrix. We derive an explicit formula for the entries of the inverse of band lower triangular Toeplitz matrix by means of divided difference and use the formula to estimate the condition number of the preconditioned matrices. In particular, we prove that the minimal eigenvalue of preconditioned matrix is well separated from the origin.

Regularizing Inverse Preconditioners for Symmetric Band Toeplitz Matrices

Image restoration is a widely studied discrete ill-posed problem. Among the many regularization methods used for treating the problem, iterative methods have been shown to be effective. In this paper, we consider the case of a blurring function defined by space invariant and band-limited PSF, modeled by a linear system that has a band block Toeplitz structure with band Toeplitz blocks. In order to reduce the number of iterations required to obtain acceptable reconstructions, in 13 an inverse Toeplitz preconditioner for problems with a Toeplitz structure was proposed. The cost per iteration is of operations, where is the pixel number of the 2D image. In this paper, we propose inverse preconditioners with a band Toeplitz structure, which lower the cost to and in experiments showed the same speed of convergence and reconstruction efficiency as the inverse Toeplitz preconditioner.

Block band Toeplitz preconditioners derived from generating function approximations: analysis and applications

We are concerned with the study and the design of optimal preconditioners for ill-conditioned Toeplitz systems that arise from a priori known real-valued nonnegative generating functions f(x,y) having roots of even multiplicities. Our preconditioned matrix is constructed by using a trigonometric polynomial θ(x,y) obtained from Fourier/kernel approximations or from the use of a proper interpolation scheme. Both of the above techniques produce a trigonometric polynomial θ(x,y) which approximates the generating function f(x,y), and hence the preconditioned matrix is forced to have clustered spectrum. As θ(x,y) is chosen to be a trigonometric polynomial, the preconditioner is a block band Toeplitz matrix with Toeplitz blocks, and therefore its inversion does not increase the total complexity of the PCG method. Preconditioning by block Toeplitz matrices has been treated in the literature in several papers. We compare our method with their results and we show the efficiency of our proposal through various numerical experiments.

The Spectra of Preconditioned Toeplitz Matrix Sequences Can Have Gaps

Different than for the case of Toeplitz matrix sequences $\{T_n(f)\}$, $f\in L^1$, we can prove that the closure of the union of all the spectra of preconditioned matrix sequences of the form $\{T_n^{-1}(g)T_n(f)\}$, $f,g\in L^1$, $g\ge 0$, can have gaps if the essential range of f/g is not connected. The result has important consequences on the practical use of band Toeplitz preconditioners widely used in the literature both for (multilevel) ill-conditioned positive definite and (multilevel) indefinite Toeplitz linear systems.

Practical Band Toeplitz Preconditioning and Boundary Layer Effects

In the last decade many efficient iterative solvers for n×n Hermitian positive definite Toeplitz systems have been devised. Many of them are based on band Toeplitz preconditioners: they are optimal but require the knowledge of the zeros of the underlying generating function. In some cases this information is available and in some cases is not. In [27] an economic numerical procedure for finding these zeros within a given precision has been devised. Here we provide conditions on the approximation error of these zeros in order to maintain the optimality that is a convergence rate independent of the dimension n of the considered linear systems.

A polynomial fit preconditioner for band Toeplitz matrices in image reconstruction

The preconditioned conjugate gradient (CG) is often applied in image reconstruction as a regularizing method. When the blurring matrix has Toeplitz structure, the modified circulant preconditioner and the inverse Toeplitz preconditioner have been shown to be effective. We introduce here a preconditioner for symmetric positive definite Toeplitz matrices based on a trigonometric polynomial fit which has the same effectiveness of the previous ones but has a lower cost when applied to band matrices. The case of band block Toeplitz matrices with band Toeplitz blocks (BTTB) corresponding to separable point spread functions (PSFs) is also considered.

Spectral and structural analysis of high precision finite difference matrices for elliptic operators

In this paper we study the structural properties of matrices coming from high-precision Finite Difference (FD) formulae, when discretizing elliptic (or semielliptic) differential operators L(a,u) of the form (−) k d k dx k a(x) d k dx k u(x) . Strong relationships with Toeplitz structures and Linear Positive Operators (LPO) are highlighted. These results allow one to give a detailed analysis of the eigenvalues localisation/distribution of the arising matrices. The obtained spectral analysis is then used to define optimal Toeplitz preconditioners in a very compact and natural way and, in addition, to prove Szegö-like and Widom-like ergodic theorems for the spectra of the related preconditioned matrices. A wide numerical experimentation, confirming the theoretical results, is also reported.

Inverse Toeplitz preconditioners for ill-posed problems

It has been shown recently that iterative regularization using conjugate gradient type methods for image restoration problems can be effectively preconditioned with circulant approximations. Here it is shown that the theoretical properties of this approach are not restricted to circulant matrices. Specifically, a Toeplitz approximate inverse preconditioning scheme for discrete ill-posed problems is considered. It is proved that the preconditioned system approximates the prolate matrix, and that this property implies that fast convergence of conjugate gradient type methods can be expected. In addition, it is shown that these results can be generalized to two-dimensional problems. An image restoration application is used to demonstrate the properties of the preconditioner.

Block adaptive filtering algorithm based on the preconditioned conjugate gradient method

As a new block adaptive algorithm based on the preconditioned conjugate gradient method, the TBCG is formulated by employing a Toeplitz preconditioner assumed to be symmetric Toeplitz. By computer simulations, it is shown that the convergence rate of the proposed algorithm is superior to those of the BCG and NLMS, and comparable to the RLS. Moreover, like the RLS, the convergence property of the proposed algorithm is insensitive to the eigenvalue spread of input signals.

Properties of iteration of toeplitz operators with toeplitz preconditioners

We consider the problems of preconditioning and iterative inversion of Toeplitz operators on sequences of complex numbers. We divide the preconditioned operator into two parts, of which one is compact and the other is regarded as a small perturbartion. It will be shown that the Krylov subspace methods (such as GMRES) will perform initially at superlinear speed when applied to such a preconditioned system. However, with large iteration numbers, the speed will settle down to linear order. Most of our results are stated in terms of the symbol of the Toeplitz operator in question.

Toeplitz Preconditioners Constructed from Linear Approximation Processes

Preconditioned conjugate gradients (PCG) are widely and successfully used methods to solve Toeplitz linear systems An(f)x=b. Here we consider preconditioners belonging to trigonometric matrix algebras and to the band Toeplitz class and we analyze them from the viewpoint of the function theory in the case where f is supposed continuous and strictly positive. First we prove that the necessary (and sufficient) condition, in order to devise a superlinear PCG method, is that the spectrum of the preconditioners is described by a sequence of approximation operators "converging" to f . The other important information we deduce is that while the matrix algebra approach is substantially not sensitive to the approximation features of the underlying approximation operators, the band Toeplitz approach is. Therefore, the only class of methods for which we may obtain impressive evidence of superlinear convergence behavior is the one [S. Serra, Math. Comp., 66 (1997), pp. 651--665] based on band Toeplitz matrices with weakly increasing bandwidth.

On the asymptotic spectrum of Hermitian block Toeplitz matrices with Toeplitz blocks

We study the asymptotic behaviour of the eigenvalues of Hermitian n × n n\times n block Toeplitz matrices A n , m A_{n,m} , with m × m m\times m Toeplitz blocks. Such matrices are generated by the Fourier coefficients of an integrable bivariate function f f , and we study their eigenvalues for large n n and m m , relating their behaviour to some properties of f f as a function; in particular we show that, for any fixed k k , the first k k eigenvalues of A n , m A_{n,m} tend to inf f \inf f , while the last k k tend to sup f \sup f , so extending to the block case a well-known result due to Szegö. In the case the A n , m A_{n,m} ’s are positive-definite, we study the asymptotic spectrum of P n , m − 1 A n , m P_{n,m}^{-1}A_{n,m} , where P n , m P_{n,m} is a block Toeplitz preconditioner for the conjugate gradient method, applied to solve the system A n , m x = b A_{n,m}x=b , obtaining strict estimates, when n n and m m are fixed, and exact limit values, when n n and m m tend to infinity, for both the condition number and the conjugate gradient convergence factor of the previous matrices. Extensions to the case of a deeper nesting level of the block structure are also discussed.