Circulant Preconditioners Research Articles

A block α-circulant based preconditioned MINRES method for wave equations

In this work, we propose an absolute value block α-circulant preconditioner for the minimal residual (MINRES) method to solve an all-at-once system arising from the discretization of wave equations. Motivated by the absolute value block circulant preconditioner proposed in [E. McDonald, J. Pestana, and A. Wathen. SIAM J. Sci. Comput., 40(2):A1012-A1033, 2018], we propose an absolute value version of the block α-circulant preconditioner. Since the original block α-circulant preconditioner is non-Hermitian in general, it cannot be directly used as a preconditioner for MINRES. Our proposed preconditioner is the first Hermitian positive definite variant of the block α-circulant preconditioner for the concerned wave equations, which fills the gap between block α-circulant preconditioning and the field of preconditioned MINRES solver. The matrix-vector multiplication of the preconditioner can be fast implemented via fast Fourier transforms. Theoretically, we show that for a properly chosen α the MINRES solver with the proposed preconditioner achieves a linear convergence rate independent of the matrix size. To the best of our knowledge, this is the first attempt to generalize the original absolute value block circulant preconditioner in the aspects of both theory and performance the concerned problem. Numerical experiments are given to support the effectiveness of our preconditioner, showing that the expected optimal convergence can be achieved.

An efficient fourth-order accurate conservative scheme for Riesz space fractional Schrödinger equation with wave operator

In this study, we investigate a high-order accurate conservative finite difference scheme by utilizing a fourth-order fractional central finite difference method for the two-dimensional Riesz space-fractional nonlinear Schrödinger wave equation. The conservation laws of the discrete difference scheme are shown. Meanwhile, the exactness, uniqueness, and prior estimate of the numerical solution are rigorously established. Then, it is proved that the proposed scheme is unconditionally convergent in the discrete L2 and Hγ/2 norm, where γ is a fractional order. Furthermore, we demonstrate that when the fractional order γ and the spatial grid number J increase, the block-Toeplitz coefficient matrix generated by the spatial discretization becomes ill-conditioned. As a result, we adopt an effective linearized iteration method for the nonlinear system, allowing it to be solved efficiently by the Krylov subspace solver with an appropriate circulant preconditioner, in which the fast Fourier transform is applied to speed up the computational cost at each iterative step. Finally, numerical experiments are presented to validate the theoretical findings and the efficiency of the fast algorithm.

A block Toeplitz preconditioner for all-at-once systems from linear wave equations

In this work, we propose a novel parallel-in-time preconditioner for an all-at-once system, arising from the numerical solution of linear wave equations. Namely, our main result concerns a block tridiagonal Toeplitz preconditioner that can be diagonalized via fast sine transforms, whose effectiveness is theoretically shown for the nonsymmetric block Toeplitz system resulting from discretizing the concerned wave equation. Our approach is to first transform the original linear system into a symmetric one and subsequently develop the desired preconditioning strategy based on the spectral symbol of the modified matrix. Various Krylov subspace methods are considered. That is, we show that the minimal polynomial of the preconditioned matrix is of low degree, which leads to fast convergence when the generalized minimal residual method is used. To fully utilize the symmetry of the modified matrix, we additionally construct an absolute-value preconditioner which is symmetric positive definite. Then, we show that the eigenvalues of the preconditioned matrix are clustered around $\pm 1$, which gives a convergence guarantee when the minimal residual method is employed. Numerical examples are given to support the effectiveness of our preconditioner. Our block Toeplitz preconditioner provides an alternative to the existing block circulant preconditioner proposed by McDonald, Pestana, and Wathenin [SIAM J. Sci. Comput., 40 (2018), pp. A1012–A1033],advancing the symmetrization preconditioning theory that originated from the same work.

A Mixed-Type Circulant Preconditioner for a Nonlocal Elastic Model

A Mixed-Type Circulant Preconditioner for a Nonlocal Elastic Model

Kansa–RBF algorithms for elliptic BVPs in annular domains with mixed boundary conditions

We employ a Kansa–radial basis function (RBF) method for the numerical solution of elliptic boundary value problems in annular domains with mixed Dirichlet/Neumann boundary conditions. By exploiting the circular boundaries and the properties of circulant matrices we employ, in an efficient way, the pre-conditioned Krylov subspace iterative solvers GMRES and BiCGSTAB for the solution of the resulting linear systems. In particular, we employ block circulant pre-conditioners which allow for the efficient solution of the relevant systems in the iterative solution. Moreover, by exploiting the properties of circulant matrices we perform the matrix–vector multiplications involved in the iterative solvers efficiently. The feasibility of the proposed techniques is illustrated by several numerical examples.

Tensorized low-rank circulant preconditioners for multilevel Toeplitz linear systems from high-dimensional fractional Riesz equations

The solution of d-dimensional fractional partial differential equations (FPDEs) is challenging in numerical computation. This paper proposes a low-rank preconditioned conjugate gradient (PCG) method for discretizations with a low-rank right-hand side stemming from the d-dimensional fractional Riesz equation. At each iteration step, a combination of low-rank in Tensor Train (TT) format technique and Fast Fourier Transform (FFT) is used to accelerate multilevel Toeplitz matrix-vector multiplications. Furthermore, a low-rank circulant preconditioner is constructed based on the spectrum of the coefficient matrix. The reported numerical results demonstrate the competitiveness of the new method compared with a state-of-the-art matrix-free approach based on FFT.

Optimal block circulant preconditioners for block Toeplitz systems with application to evolutionary PDEs

In this work, we propose a preconditioned minimal residual (MINRES) method for a class of non-Hermitian block Toeplitz systems. Namely, considering an mn-by-mn non-Hermitian block Toeplitz matrix T(n,m) with m-by-m commuting Hermitian blocks, we first premultiply it by a simple permutation matrix to obtain a Hermitian matrix and then construct a Hermitian positive definite block circulant preconditioner for the modified matrix. Under certain conditions, we show that the eigenvalues of the preconditioned matrix are clustered around ±1 when n is sufficiently large. Due to the Hermitian nature of the modified matrix, MINRES with our proposed preconditioner can achieve theoretically guaranteed superlinear convergence under suitable conditions. In addition, we provide several useful properties of block circulant matrices with commuting Hermitian blocks, including diagonalizability and symmetrization. A generalization of our result to the multilevel block case is also provided. We in particular indicate that our work can be applied to the all-at-once systems arising from solving evolutionary partial differential equations. Numerical examples are given to illustrate the effectiveness of our preconditioning strategy.

Learning a preconditioner to accelerate compressed sensing reconstructions in MRI.

PurposeTo learn a preconditioner that accelerates parallel imaging (PI) and compressed sensing (CS) reconstructions.MethodsA convolutional neural network (CNN) with residual connections was used to train a preconditioning operator. Training and validation data were simulated using 50% brain images and 50% white Gaussian noise images. Each multichannel training example contains a simulated sampling mask, complex coil sensitivity maps, and two regularization parameter maps. The trained model was integrated in the preconditioned conjugate gradient (PCG) method as part of the split Bregman CS method. The acceleration performance was compared with that of a circulant PI‐CS preconditioner for varying undersampling factors, number of coil elements and anatomies.ResultsThe learned preconditioner reduces the number of PCG iterations by a factor of 4, yielding a similar acceleration as an efficient circulant preconditioner. The method generalizes well to different sampling schemes, coil configurations and anatomies.ConclusionIt is possible to learn adaptable preconditioners for PI and CS reconstructions that meet the performance of state‐of‐the‐art preconditioners. Further acceleration could be achieved by optimizing the network architecture and the training set. Such a preconditioner could also be integrated in fully learned reconstruction methods to accelerate the training process of unrolled networks.

Band-Toeplitz preconditioners for ill-conditioned Toeplitz systems

Preconditioning for Toeplitz systems has been an active research area over the past few decades. Along this line of research, circulant preconditioners have been recently proposed for the Toeplitz-like system arising from discretizing fractional diffusion equations. A common approach is to combine a circulant preconditioner with the preconditioned conjugate gradient normal residual (PCGRN) method for the coefficient system. In this work, instead of using PCGRN for the normal equation system, we propose a simple yet effective preconditioning approach for solving the original system using the preconditioned minimal residual (PMINRES) method that can achieve convergence guarantees depending only on eigenvalues. Namely, for a large class of ill-conditioned Toeplitz systems, we propose a number of preconditioners that attain the overall \({\mathcal {O}}(n\log {n})\) complexity. We first symmetrize the given Toeplitz system by using a permutation matrix and construct a band-Toeplitz plus circulant preconditioner for the modified system. Then, under certain assumptions, we show that the eigenvalues of the preconditioned system are clustered around \(\pm 1\) except a number of outliers and hence superlinear convergence rate of PMINRES can be achieved. Particularly, we indicate that our solver can be applied to solve certain fractional diffusion equations. An extension of this work to the block Toeplitz case is also included. Numerical examples are provided to demonstrate the effectiveness of our proposed method.

Grid independent convergence using multilevel circulant preconditioning: Poisson\u2019s equation

We consider the iterative solution of the discrete Poisson’s equation with Dirichlet boundary conditions. The discrete domain is embedded into an extended domain and the resulting system of linear equations is solved using a fixed point iteration combined with a multilevel circulant preconditioner. Our numerical results show that the rate of convergence is independent of the grid’s step sizes and of the number of spatial dimensions, despite the fact that the iteration operator is not bounded as the grid is refined. The embedding technique and the preconditioner is derived with inspiration from theory of boundary integral equations. The same theory is used to explain the behaviour of the preconditioned iterative method.

A Note on Parallel Preconditioning for the All-at-Once Solution of Riesz Fractional Diffusion Equations

The $p$-step backwards difference formula (BDF) for solving the system of ODEs can result in a kind of all-at-once linear systems, which are solved via the parallel-in-time preconditioned Krylov subspace solvers (see McDonald, Pestana, and Wathen [SIAM J. Sci. Comput., 40(2) (2018): A1012-A1033] and Lin and Ng [arXiv:2002.01108, 17 pages]. However, these studies ignored that the $p$-step BDF ($p\geq 2$) is not selfstarting, when they are exploited to solve time-dependent PDEs. In this note, we focus on the 2-step BDF which is often superior to the trapezoidal rule for solving the Riesz fractional diffusion equations, but its resultant all-at-once discretized system is a block triangular Toeplitz system with a low-rank perturbation. Meanwhile, we first give an estimation of the condition number of the all-at-once systems and then adapt the previous work to construct two block circulant (BC) preconditioners. Both the invertibility of these two BC preconditioners and the eigenvalue distributions of preconditioned matrices are discussed in details. The efficient implementation of these BC preconditioners is also presented especially for handling the computation of dense structured Jacobi matrices. Finally, numerical experiments involving both the one- and two-dimensional Riesz fractional diffusion equations are reported to support our theoretical findings.

Circulant preconditioners for mean curvature-based image deblurring problem

The mean curvature-based image deblurring model is widely used to enhance the quality of the deblurred images. However, the discretization of the associated Euler–Lagrange equations produces a nonlinear ill-conditioned system which affects the convergence of the numerical algorithms such as Krylov subspace methods (generalized minimal residual etc.) To overcome this difficulty, in this paper, we present three new circulant preconditioners. An efficient algorithm is presented for the mean curvature-based image deblurring problem, which combines a fixed point iteration with new preconditioned matrices to handle the nonlinearity and ill-conditioned nature of the large system. The eigenvalues analysis is also presented in the paper. Fast convergence has shown in the numerical results by using the proposed new circulant preconditioners.

The Asymptotic Spectrum of Flipped Multilevel Toeplitz Matrices and of Certain Preconditionings

In this work, we perform a spectral analysis of flipped multilevel Toeplitz sequences, i.e., we study the asymptotic spectral behavior of $\{Y_{{n}} T_{{n}} (f)\}_{{n}}$, where $T_{{n}}(f)$ is a real, square multilevel Toeplitz matrix generated by a function $f\in L^1([-\pi,\pi]^d)$ and $Y_n$ is the exchange matrix, which has 1's on the main antidiagonal. In line with what we have shown for unilevel flipped Toeplitz matrix sequences, the asymptotic spectrum is determined by a 2 x 2 matrix-valued function whose eigenvalues are $\pm |f|$. Furthermore, we characterize the eigenvalue distribution of certain preconditioned flipped multilevel Toeplitz sequences with an analysis that covers both multilevel Toeplitz and circulant preconditioners. Finally, all our findings are illustrated by several numerical experiments.

An All-at-Once Preconditioner for Evolutionary Partial Differential Equations

In [McDonald, Pestana, and Wathen, SIAM J. Sci. Comput., 40 (2018), pp. A1012--A1033], a block circulant preconditioner is proposed for all-at-once linear systems arising from evolutionary partial ...

A three-level finite difference method with preconditioning technique for two-dimensional nonlinear fractional complex Ginzburg–Landau equations

In the paper, we study two-dimensional nonlinear spatial fractional complex Ginzburg–Landau equations. A centered finite difference method is exploited to discretize the spatial variables, while a three-level finite difference scheme is applied for the time integration. Theoretically, we prove the proposed method is uniquely solvable and unconditionally stable, with second order accuracy on both time and space, respectively. As the resulting discretized systems possess the block-Toeplitz structure, we proposed the preconditioned GMRES method with a block circulant preconditioner to speed up the convergence rate of the iteration. Meanwhile, fast Fourier transformation is utilized to reduce the complexity for calculating the discretized systems. Numerical experiments are carried out to verify the theoretical results and demonstrate that the proposed method enjoys the excellent computational advantage.

A circulant preconditioner for the Riesz distributed-order space-fractional diffusion equations

ABSTRACT In this paper, we study a fast algorithm for the numerical solution of the 1D distributed-order space-fractional diffusion equation. After discretization by the finite difference method, the resulting system is the symmetric positive definite Toeplitz matrix. The preconditioned conjugate gradient method with a circulant preconditioner is employed to solve the linear system. Theoretically, the spectrum of the preconditioned matrix is proved to be clustered around 1, which can guarantee the superlinear convergence rate of the proposed method. Numerical experiments are carried out to demonstrate the effectiveness of our proposed method.

Fast implicit difference schemes for time‐space fractional diffusion equations with the integral fractional Laplacian

In this paper, we develop two fast implicit difference schemes for solving a class of variable‐coefficient time–space fractional diffusion equations with integral fractional Laplacian (IFL). The proposed schemes utilize the graded L1 formula for the Caputo fractional derivative and a special finite difference discretization for IFL, where the graded mesh can capture the model problem with a weak singularity at initial time. The stability and convergence are rigorously proved via the M‐matrix analysis, which is from the spatial discretized matrix of IFL. Moreover, the proposed schemes use the fast sum‐of‐exponential approximation and Toeplitz matrix algorithms to reduce the computational cost for the nonlocal property of time and space fractional derivatives, respectively. The fast schemes greatly reduce the computational work of solving the discretized linear systems from by a direct solver to per preconditioned Krylov subspace iteration and a memory requirement from 𝒪(MN2) to 𝒪(NNexp), where N and (Nexp ≪) M are the number of spatial and temporal grid nodes, respectively. The spectrum of preconditioned matrix is also given for ensuring the acceleration benefit of circulant preconditioners. Finally, numerical results are presented to show the utility of the proposed methods.

A Fast Block $\alpha$-Circulant Preconditoner for All-at-Once Systems From Wave Equations

In this paper, we propose a fast block $\alpha$-circulant preconditioner for solving the nonsymmetric linear system arising from an all-at-once implicit discretization scheme in time for the wave equation. As a generalization of the well-known block circulant preconditioning technique, the proposed block $\alpha$-circulant preconditioner can also be efficiently inverted in a parallel-in-time manner. The complex eigenvalues of the preconditioned matrix are fully derived in explicit expression and its diagonalizability is also shown. Furthermore, a mesh-independent convergence rate of the preconditioned GMRES method is derived under certain conditions. Building upon the proposed preconditioner, two stationary iterative methods with uniform asymptotic convergence rates were also presented. The extension of our preconditioner within a simplified Newton iteration to the nonlinear wave equation is also discussed. Both linear and nonlinear numerical examples are given to illustrate the promising performance of our proposed block $\alpha$-circulant preconditioner and also validate our theoretical results on convergence analysis.

A Parallel-In-Time Block-Circulant Preconditioner for Optimal Control of Wave Equations

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 25 September 2019Accepted: 02 March 2020Published online: 06 May 2020Keywordsoptimal control, wave equation, circulant preconditioner, parallel-in-time (PinT), diagonalization, GMRESAMS Subject Headings65F08, 65F10, 65Y05Publication DataISSN (print): 1064-8275ISSN (online): 1095-7197Publisher: Society for Industrial and Applied MathematicsCODEN: sjoce3

Fast Solution Methods for Convex Quadratic Optimization of Fractional Differential Equations

In this paper, we present numerical methods suitable for solving convex quadratic Fractional Differential Equation (FDE) constrained optimization problems, with box constraints on the state and/or control variables. We develop an Alternating Direction Method of Multipliers (ADMM) framework, which uses preconditioned Krylov subspace solvers for the resulting sub-problems. The latter allows us to tackle a range of Partial Differential Equation (PDE) optimization problems with box constraints, posed on space-time domains, that were previously out of the reach of state-of-the-art preconditioners. In particular, by making use of the powerful Generalized Locally Toeplitz (GLT) sequences theory, we show that any existing GLT structure present in the problem matrices is preserved by ADMM, and we propose some preconditioning methodologies that could be used within the solver, to demonstrate the generality of the approach. Focusing on convex quadratic programs with time-dependent 2-dimensional FDE constraints, we derive multilevel circulant preconditioners, which may be embedded within Krylov subspace methods, for solving the ADMM sub-problems. Discretized versions of FDEs involve large dense linear systems. In order to overcome this difficulty, we design a recursive linear algebra, which is based on the Fast Fourier Transform (FFT). We manage to keep the storage requirements linear, with respect to the grid size $N$, while ensuring an order $N \log N$ computational complexity per iteration of the Krylov solver. We implement the proposed method, and demonstrate its scalability, generality, and efficiency, through a series of experiments over different setups of the FDE optimization problem.