Circulant Preconditioners Research Articles

Circulant preconditioners for ill-conditioned boundary integral equations from potential equations

In this paper, we consider solving potential equations by the boundary integral equation approach. The equations so derived are Fredholm integral equations of the first kind and are known to be ill-conditioned. Their discretized matrices are dense and have condition numbers growing like O(n) where n is the matrix size. We propose to solve the equations by the preconditioned conjugate gradient method with circulant integral operators as preconditioners. These are convolution operators with periodic kernels and hence can be inverted efficiently by using fast Fourier transforms. We prove that the preconditioned systems are well conditioned, and hence the convergence rate of the method is linear. Numerical results for two types of regions are given to illustrate the fast convergence. © 1998 John Wiley & Sons, Ltd.

Circulant preconditioners for ill‐conditioned boundary integral equations from potential equations

In this paper, we consider solving potential equations by the boundary integral equation approach. The equations so derived are Fredholm integral equations of the first kind and are known to be ill-conditioned. Their discretized matrices are dense and have condition numbers growing like O(n) where n is the matrix size. We propose to solve the equations by the preconditioned conjugate gradient method with circulant integral operators as preconditioners. These are convolution operators with periodic kernels and hence can be inverted efficiently by using fast Fourier transforms. We prove that the preconditioned systems are well conditioned, and hence the convergence rate of the method is linear. Numerical results for two types of regions are given to illustrate the fast convergence. © 1998 John Wiley & Sons, Ltd.

Addendum to “a note on construction of circulant preconditioners from kernels”

A unified treatment of construction of block circulant preconditioners for block Toeplitz systems from the viewpoint of kernels has been given [Appl. Math. Comput. 83 (1997) 3]. It was shown that some well-known block circulant preconditioners can be derived from convoluting the generating functions of systems with some famous kernels. A convergence analysis was also given there. For solving a large class of block Toeplitz systems, a linear convergence rate is obtained by using the preconditioned conjugate gradient (PCG) method with block circulant preconditioner. In this addendum, by using a given convergence result [Linear Algebra Appl. 232 (1996) 1], a superlinear convergence rate is obtained. Numerical results are given to illustrate the rate of convergence.

Circulant preconditioners for failure prone manufacturing systems

This paper studies the application of preconditioned conjugate-gradient methods in solving for the steady-state probability distribution of manufacturing systems. We consider the optimal hedging policy for a failure prone one-machine system. The machine produces one type of product, and its demand has finite batch arrival. The machine states and the inventory levels are modeled as Markovian processes. We construct the generator matrix for the machine-inventory system. The preconditioner is constructed by taking the circulant approximation of the near-Toeplitz structure of the generator matrix. We prove that the preconditioned linear system has singular values clustered around one when the number of inventory levels tends to infinity. Hence conjugate-gradient methods will converge very fast when applied to solving the preconditioned linear system. Numerical examples are given to verify our claim. The average running cost for the system can be written in terms of the steady state probability distribution. The optimal hedging point can then be obtained by varying different values of the hedging point.

Circulant block-factorization preconditioning of anisotropic elliptic problems

The recently introduced circulant block-factorization preconditioners are studied in this paper. The general approach is first formulated for the case of block-tridiagonal sparse matrices. Then an estimate of the condition number of the preconditioned matrix for a model anisotropic Dirichlet boundary value problem is derived in the formκ<√2e(n+1)+2, whereN=n 2 is the size of the discrete problem, ande stands for the ratio of the anisotropy. Various numerical tests demonstrating the behavior of the circulant block-factorization preconditioners for anisotropic problems are presented.

Sine transform based preconditioners for elliptic problems

We consider applying the preconditioned conjugate gradient (PCG) method to solving linear systems Ax = b where the matrix A comes from the discretization of second-order elliptic operators with Dirichlet boundary conditions. Let (L + Σ)Σ−1(Lt + Σ) denote the block Cholesky factorization of A with lower block triangular matrix L and diagonal block matrix Σ. We propose a preconditioner M = (Lˆ + Σˆ)Σˆ−1(Lˆt + Σˆ) with block diagonal matrix Σˆ and lower block triangular matrix Lˆ. The diagonal blocks of Σˆ and the subdiagonal blocks of Lˆ are respectively the optimal sine transform approximations to the diagonal blocks of Σ and the subdiagonal blocks of L. We show that for two-dimensional domains, the construction cost of M and the cost for each iteration of the PCG algorithm are of order O(n2 log n). Furthermore, for rectangular regions, we show that the condition number of the preconditioned system M−1A is of order O(1). In contrast, the system preconditioned by the MILU and MINV methods are of order O(n). We will also show that M can be obtained from A by taking the optimal sine transform approximations of each sub-block of A. Thus, the construction of M is similar to that of Level-1 circulant preconditioners. Our numerical results on two-dimensional square and L-shaped domains show that our method converges faster than the MILU and MINV methods. Extension to higher-dimensional domains will also be discussed. © 1997 John Wiley & Sons, Ltd.

A note on construction of circulant preconditioners from kernels

Circulant preconditioners for Hermitian Toeplitz systems are studied by R. Chan and Yeung from the viewpoint of kernels. They show that some well-known circulant preconditioners can be derived from convoluting the generating functions of systems with some famous kernels. A convergence analysis is also given there. In this paper, we generalize the results of R. Chan and Yeung from the point matrix case to the block matrix case.

Circulant Preconditioners for Markov-Modulated Poisson Processes and Their Applications to Manufacturing Systems

The Markov-modulated Poisson process (MMPP) is a generalization of the Poisson process and is commonly used in modeling the input process of communication systems such as data traffic systems and ATM networks. In this paper, we give fast algorithms for solving queueing systems and manufacturing systems with MMPP inputs. We consider queueing systems where the input of the queues is a superposition of the MMPP which is still an MMPP. The generator matrices of these processes are tridiagonal block matrices with each diagonal block being a sum of tensor products of matrices. We are interested in finding the steady state probability distributions of these processes which are the normalized null vectors of their generator matrices. Classical iterative methods, such as the block Gauss--Seidel method, are usually employed to solve for the steady state probability distributions. They are easy to implement, but their convergence rates are slow in general. The number of iterations required for convergence increases like $O(m)$, where $m$ is the size of the waiting spaces in the queues. Here, we propose to use the preconditioned conjugate gradient method. We construct our preconditioners by taking circulant approximations of the tensor blocks of the generator matrices. We show that the number of iterations required for convergence increases at most like $O(\log_2 m)$ for large $m$. Numerical results are given to illustrate the fast convergence. As an application, we apply the MMPP to model unreliable manufacturing systems. The production process consists of multiple parallel machines which produce one type of product. Each machine has exponentially distributed up time, down time, and processing time for one unit of product. The interarrival of a demand is exponentially distributed and finite backlog is allowed. We consider hedging point policy as the production control. The average running cost of the system can be written in terms of the steady state probability distribution. Our numerical algorithm developed for the queueing systems can be applied to obtain the steady state distribution for the system and hence the optimal hedging point. Furthermore, our method can be generalized to handle the case when the machines have a more general type of repairing process distribution such as the Erlangian distribution.

Optimum block adaptive filtering algorithms using the preconditioning technique

We propose three block adaptive algorithms using the preconditioning technique. The Toeplitz-preconditioned optimum block adaptive (TOBA) algorithm employs a preconditioner assumed to be Toeplitz, the symmetric successive overrelaxation (SSOR)-preconditioned optimum block adaptive (SOBA) algorithm uses a product of triangular matrices as a preconditioner, and the circulant-preconditioned OBA (COBA) algorithm is based on a circulant preconditioner. It is also shown that their tracking properties and convergence rates are superior to those of the OBA algorithm, the self-orthogonizing block adaptive filter (SOBAF), and the normalized frequency-domain OBA (NFOBA) algorithm.

Preconditioning of Block Toeplitz Matrices by Sine Transforms

The iterative solution of a block Toeplitz linear system by the conjugate gradient method is analyzed, the preconditioning step being solved by means of a discrete sine transform. Convergence properties are established and compared to the behavior of the block circulant preconditioner recently proposed in the literature. As in the scalar case, the new approach takes advantage if the system is ill conditioned.

Optimal, quasi-optimal and superlinear band-Toeplitz preconditioners for asymptotically ill-conditioned positive definite Toeplitz systems

In this paper we are concerned with the solution of n × n n\times n Hermitian Toeplitz systems with nonnegative generating functions f f . The preconditioned conjugate gradient (PCG) method with the well–known circulant preconditioners fails in the case where f f has zeros. In this paper we consider as preconditioners band–Toeplitz matrices generated by trigonometric polynomials g g of fixed degree l l . We use different strategies of approximation of f f to devise a polynomial g g which has some analytical properties of f f , is easily computable and is such that the corresponding preconditioned system has a condition number bounded by a constant independent of n n . For each strategy we analyze the cost per iteration and the number of iterations required for the convergence within a preassigned accuracy. We obtain different estimates of l l for which the total cost of the proposed PCG methods is optimal and the related rates of convergence are superlinear. Finally, for the most economical strategy, we perform various numerical experiments which fully confirm the effectiveness of approximation theory tools in the solution of this kind of linear algebra problems.

Fast Recursive Least Squares Adaptive Filtering by Fast Fourier Transform-Based Conjugate Gradient Iterations

Recursive least squares (RLS) estimations are used extensively in many signal processing and control applications. In this paper we consider RLS with sliding data windows involving multiple (rank k) updating and downdating computations. The least squares estimator can be found by solving a near-Toeplitz matrix system at each step. Our approach is to employ the preconditioned conjugate gradient method with circulant preconditioners to solve such systems. Here we iterate in the time domain (using Toeplitz matrix-vector multiplications) and precondition in the Fourier domain, so that the fast Fourier transform (FFT) is used throughout the computations. The circulant preconditioners are derived from the spectral properties of the given input stochastic process. When the input stochastic process is stationary, we prove that with probability 1, the spectrum of the preconditioned system is clustered around 1 and the method converges superlinearly provided that a sufficient number of data samples are taken, i.e.,...

Toeplitz-Circulant Preconditioners for Toeplitz Systems and their Applications to Queueing Networks with Batch Arrivals

The preconditioned conjugate gradient method is employed to solve Toeplitz systems $T_n {\bf x} = {\bf b}$ where the generating functions of the n-by-n Toeplitz matrices $T_n $ are functions with zeros. In this case, circulant preconditioners are known to give poor convergence, whereas band-Toeplitz preconditioners offer only linear convergence and can handle only real-valued functions with zeros of even orders. We propose here preconditioners hich are products of band-Toeplitz matrices and circulant matrices. The band-Toeplitz matrices are used to cope with the zeros of the given generating function and the circulant matrices are used to speed up the convergence rate of the algorithm. Our preconditioner can handle complex-valued functions with zeros of arbitrary orders. We prove that the preconditioned Toeplitz matrices have singular values clustered around 1 for large n. We apply our preconditioners to solve the stationary probability distribution vectors of Markovian queueing models with batch arrivals. We show that if the number of servers is fixed independent of the queue size n, then the preconditioners are invertible and the preconditioned matrices have singular values clustered around 1 for large n. Numerical results are given to illustrate the fast convergence of our methods.

Generalization of Strang's Preconditioner with Applications to Toeplitz Least Squares Problems

In this paper, we propose a method to generalize Strang's circulant preconditioner for arbitrary n-by-n matrices An. The th column of our circulant preconditioner Sn is equal to the th column of the given matrix An. Thus if An is a square Toeplitz matrix, then Sn is just the Strang circulant preconditioner. When Sn is not Hermitian, our circulant preconditioner can be defined as . This construction is similar to the forward-backward projection method used in constructing preconditioners for tomographic inversion problems in medical imaging. We show that if the matrix An has decaying coefficients away from the main diagonal, then is a good preconditioner for An. Comparisons of our preconditioner with other circulant-based preconditioners are carried out for some 1-D Toeplitz least squares problems: min ∥ b - Ax∥2. Preliminary numerical results show that our preconditioner performs quite well, in comparison to other circulant preconditioners. Promising test results are also reported for a 2-D deconvolution problem arising in ground-based atmospheric imaging.

Fast iterative solvers for symmetric Toeplitz systems — A survey and an extension

Fast iterative Toeplitz solvers based on the preconditioned conjugate gradient (PCG) methods with circulant preconditioners were proposed in 1985. Since then, Sine-transform preconditioner and Hartley-transform preconditioner were proposed in 1990 and 1993, respectively. For solving a large family of Toeplitz systems T n x = b, it requires only O( n log n) operations by using these preconditioners. In this paper, we give a brief survey and unify the analysis of all these preconditioners. An extension to the block Toeplitz systems is also given here.

Which circulant preconditioner is better?

The eigenvalue clustering of matrices S n − 1 A n S_n^{-1}A_n and C n − 1 A n C_n^{-1}A_n is experimentally studied, where A n A_n , S n S_n and C n C_n respectively are Toeplitz matrices, Strang, and optimal circulant preconditioners generated by the Fourier expansion of a function f ( x ) f(x) . Some illustrations are given to show how the clustering depends on the smoothness of f ( x ) f(x) and which preconditioner is preferable. An original technique for experimental exploration of the clustering rate is presented. This technique is based on the bisection idea and on the Toeplitz decomposition of a three-matrix product C A C CAC , where A A is a Toeplitz matrix and C C is a circulant. In particular, it is proved that the Toeplitz (displacement) rank of C A C CAC is not greater than 4, provided that C C and A A are symmetric.

Sine transform based preconditioners for symmetric Toeplitz systems

The optimal circulant preconditioner for a given matrix A is defined to be the minimizer of ∥ C − A∥ F over the set of all circulant matrices C. Here ∥·∥ F is the Frobenius norm. Optimal circulant preconditioners have been proved to be good preconditioners in solving Toeplitz systems with the preconditioned conjugate gradient method. In this paper, we construct an optimal sine transform based preconditioner which is defined to be the minimizer of ∥ B − A∥ F over the set of matrices B that can be diagonalized by sine transforms. We will prove that for general n-by- n matrices A, these optimal preconditioners can be constructed in O( n 2) real operations and in O( n) real operations if A is Toeplitz. We will also show that the convergence properties of these optimal sine transform preconditioners are the same as that of the optimal circulant ones when they are employed to solve Toeplitz systems. Numerical examples are given to support our convergence analysis.

A Note on Preconditioned Block Toeplitz Matrices

In [SIAM J. Sci. Statist. Comput., 13 (1992), pp. 948–966], Ku and Kuo proposed and analysed a block circulant preconditioner $R_{mn} $ for solving a family of block Toeplitz systems $T_{mn} v = b$. For a special class of block matrices called the quadrantally symmetric Toeplitz matrices, they proved that the eigenvalues of $R_{mn}^{ - 1} T_{mn} $ are clustered around one except at most $O(m + n)$ outliers with $T_{mn} $ generated by a two-dimensional rational function. The superior convergence rate of the preconditioned conjugate gradient (PCG) method is explained by the clustering property of the spectrum of $R_{mn}^{ - 1} T_{mn} $. However, in their analysis, there is no discussion on the positive definiteness of the matrix $T_{mn} $, and the preconditioner $R_{mn} $ is assumed to be invertible. In this paper, we give some results on these two aspects. Under the assumptions in the above-referenced paper, we prove that if the generating function $f(x,y)$ of $T_{mn} $ is positive, then $T_{mn} $ is positive definite. Moreover, we show that $R_{mn} $ is uniformly invertible when m and n are sufficiently large.

Fast Transform Based Preconditioners for Toeplitz Equations

We present a new preconditioner for $n \times n$ symmetric, positive definite Toeplitz systems. This preconditioner is an element of the n-dimensional vector space of matrices that are diagonalized by the discrete sine transform. Conditions are given for which the preconditioner is positive definite and for which the preconditioned system has asymptotically clustered eigenvalues. The diagonal form of the preconditioner can be calculated in $O( n \log ( n ) )$ operations if $n = 2^k - 1$. Thus only n additional parameters need be stored. Moreover, complex arithmetic is not needed. To use the preconditioner effectively, we develop a new technique for computing a fast convolution using the discrete sine transform (also requiring only real arithmetic). The results of numerical experimentation with this preconditioner are presented. Our preconditioner is comparable, and in some cases superior, to the standard circulant preconditioner of Tony Chan. Possible generalizations for other fast transforms are also indicated.

Best-conditioned circulant preconditioners

We discuss the solutions to a class of Hermitian positive definite systems Ax = b by the preconditioned conjugate gradient method with circulant preconditioner C. In general, the smaller the condition number κ( C −1/2 AC −1/2) is, the faster the convergence of the method will be. The circulant matrix C b that minimizes κ( C −1/2 AC −1/2) is called the best-conditioned circulant preconditioner for the matrix A. We prove that if F AF∗ has Property A, where F is the Fourier matrix, then C b minimizes ∥ C − A∥ F over all circulant matrices C. Here ∥ · ∥ F denotes the Frobenius norm. We also show that there exists a noncirculant Toeplitz matrix A such that F AF∗ has Property A.