Multiple Right-hand Sides Research Articles

Block minimum perturbation algorithm based on block Arnoldi process for nonsymmetric linear systems with multiple right-hand sides

In many applications in scientific computing and engineering one has to solve huge sparse linear systems of equations with several right-hand sides. Customarily, one uses the residual error as a stopping condition, but small residue does not imply accurate approximate solutions. Therefore, minimizing quantities other than the residual norm may be more adequate in this context. Based on the above consideration a block minimum perturbation Krylov subspace method (BMinPet) for the solution, Xm satisfying (A−ΔA)Xm=(B+ΔB) at each step, of nonsymmetric linear systems AX=B is shown such that the normwise backward error meets some optimality condition with minimizing the block joint backward perturbation norm ‖(ΔA, ΔB)‖F of the matrix (A, B), and a stopping criterion is given for consistent system with multiple right-hand sides to get a more robust way to stop the iterations. This process is a generalization of minimum perturbation algorithm (MinPert) by Kasenally and Simoncini (1997) (Analysis of a minimum perturbation algorithm for nonsymmetric linear systems. SIAM J. Numer. Anal., 34, 48–66) to multiple right-hand sides. For the benefit of the amount of calculation, we give lower and upper bounds about min ‖(ΔA, ΔB)‖F that are easier to compute. As a by-product, we come up with a conception of Function ψΦ(F, G) which is a generalization of Function ψΦ that is a symmetric gauge (SG) function. The algorithm is used to analyze the performance of BGMRES method by evaluating the proximity of their solutions to block joint backward perturbation optimality. The relationships between BMinPet method and relative methods are investigated. Numerical examples show us that BMinPert shows its superiority compared with BFGMRES-S(m, pf), GsGMRES, Bl-BiCG-rQ, BGMRES and BArnoldi in solving the large sparse ill-conditioned problems.

Enlarged GMRES for solving linear systems with one or multiple right-hand sides

Abstract We propose a variant of the generalized minimal residual (GMRES) method for solving linear systems of equations with one or multiple right-hand sides. Our method is based on the idea of the enlarged Krylov subspace to reduce communication. It can be interpreted as a block GMRES method. Hence, we are interested in detecting inexact breakdowns. We introduce a strategy to perform the test of detection. Furthermore, we propose a technique for deflating eigenvalues that has two benefits. The first advantage is to avoid the plateau of convergence after the end of a cycle in the restarted version. The second is to have very fast convergence when solving the same system with different right-hand sides, each given at a different time (useful in the context of a constrained pressure residual preconditioner). We test our method with these deflation techniques on academic test matrices arising from solving linear elasticity and convection–diffusion problems as well as matrices arising from two real-life applications, seismic imaging and simulations of reservoirs. With the same memory cost we obtain a saving of up to $50 \%$ in the number of iterations required to reach convergence with respect to the original method.

Correction to: A New Shifted Block GMRES Method with Inexact Breakdowns for Solving Multi-Shifted and Multiple Right-Hand Sides Linear Systems

We consider the efficient solution of linear systems with multiple shifts and multiple right-hand sides given simultaneously that arise frequently in large-scale scientific and engineering simulations. We introduce a new shifted block GMRES method that can solve the whole sequence of linear systems simultaneously, it handles effectively the situation of inexact breakdowns in the inner block Arnoldi procedure for improved robustness, and recycles spectral information at restart to achieve faster convergence. Numerical experiments are reported on a suite of sparse matrix problems and in realistic quantum chromodynamics application to show the potential of the new proposed method to solve general multi-shifted and multiple right-hand sides linear systems fast and efficiently.

Weighted and flexible versions of block CMRH method for solving nonsymmetric linear systems with multiple right-hand sides

Block Krylov subspace methods are the most popular algorithms for solving large nonsymmetric linear systems with multiple right-hand sides. One of them is the block CMRH method. This method generates a (non orthogonal) basis of the Krylov subspace through the block Hessenberg process. To accelerate the convergence of the block CMRH method, we will introduce two new methods. First, we present the block CMRH method with weighting strategy. In this method, the block CMRH method uses a different product at each restart. Second, we introduce a flexible version of the block CMRH algorithm that allows varying preconditioning at every step of the algorithm. Numerical experiments illustrate the benefits of the presented methods.

A New Shifted Block GMRES Method with Inexact Breakdowns for Solving Multi-Shifted and Multiple Right-Hand Sides Linear Systems

We consider the efficient solution of linear systems with multiple shifts and multiple right-hand sides given simultaneously that arise frequently in large-scale scientific and engineering simulations. We introduce a new shifted block GMRES method that can solve the whole sequence of linear systems simultaneously, it handles effectively the situation of inexact breakdowns in the inner block Arnoldi procedure for improved robustness, and recycles spectral information at restart to achieve faster convergence. Numerical experiments are reported on a suite of sparse matrix problems and in realistic quantum chromodynamics application to show the potential of the new proposed method to solve general multi-shifted and multiple right-hand sides linear systems fast and efficiently.

A spectrally preconditioned and initially deflated variant of the restarted block GMRES method for solving multiple right-hand sides linear systems

The solution of large linear systems with multiple right-hand sides given simultaneously is required in many large-scale scientific and engineering applications modelled by either partial differential or boundary integral equations. Block Krylov subspace methods are attractive to use for this problem class as they can overcome the memory bottleneck of direct methods and they perform block matrix-vector products, achieving high computational efficiency on modern cache-based computer architectures. In this paper we introduce variants of the block GMRES method that combine initial deflation and eigenvalue recycling strategies to remedy some typical convergence problems of block Krylov solvers. The new class of block iterative solvers has the ability to handle the approximate linear dependence of the block of right-hand sides and exploits approximate invariant subspaces recycled over the iterations to mitigate the bad effects that small eigenvalues can have on the convergence, by adapting an existing preconditioner. We illustrate the numerical behavior of the spectrally updated and initially deflated block GMRES method on a set of linear systems arising from the discretization of the Dirac equation and of boundary integral equations in electromagnetics scattering.

Flexible and deflated variants of the block shifted GMRES method

The solution of linear systems with multiple shifts and multiple right-hand sides given simultaneously is required in many large-scale scientific and engineering applications. In this paper we introduce new flexible and deflated variants of the shifted block GMRES method for this problem class. The proposed methods solve the whole sequence of linear systems simultaneously, detecting effectively the linear systems convergence and allowing the use of variable preconditioning which may be particularly useful in some applications. Numerical experiments are illustrated to show the overall significant robustness of the iterative method for solving general sparse multi-shifted and multiple right-hand-side systems, and in realistic PageRank calculations. To the best of our knowledge, this is the first Krylov subspace method that combines deflation techniques and variable preconditioning for solving sequences of multi-shifted linear systems with multiple right-hand sides simultaneously.

Block Conjugate-Gradient Method With Multilevel Preconditioning and GPU Acceleration for FEM Problems in Electromagnetics

In this letter, a graphics processing unit (GPU)-accelerated block conjugate-gradient solver with multilevel preconditioning is presented for solving a large system of sparse equations with multiple right-hand sides (RHSs), which arise in the finite-element analysis of electromagnetic problems. We demonstrate that blocking reduces the time to solution significantly and allows for better utilization of the computing power of GPUs, especially when the system matrix is complex valued. We show that, for a complex-valued sparse matrix with over $\text{1.2}$ million unknowns and 16 RHSs, the block version of the preconditioned conjugate-gradient method using a single NVIDIA Tesla P100 accelerator is almost three times faster than a nonblocked version. Numerical tests have also shown that a central processing unit (CPU)-only nonblocked complex implementation is unlikely to benefit much from blocking. Compared to an optimized CPU-only solver using an Intel Xeon E5-2680 v3 with 12 cores, the nonblocked GPU-accelerated version was $\text{4.5}$ times faster, while the blocked version was $\text{12.4}$ times faster.

An efficient algorithm for simulating ensembles of parameterized flow problems

Abstract Many applications of computational fluid dynamics require multiple simulations of a flow under different input conditions. In this paper, a numerical algorithm is developed to efficiently determine a set of such simulations in which the individually independent members of the set are subject to different viscosity coefficients, initial conditions and/or body forces. The proposed scheme, when applied to the flow ensemble, needs to solve a single linear system with multiple right-hand sides, and thus is computationally more efficient than solving for all the simulations separately. We show that the scheme is nonlinearly and long-term stable under certain conditions on the time-step size and a parameter deviation ratio. A rigorous numerical error estimate shows the scheme is of first-order accuracy in time and optimally accurate in space. Several numerical experiments are presented to illustrate the theoretical results.

Accelerated Cyclic Reduction: A distributed-memory fast solver for structured linear systems

We present Accelerated Cyclic Reduction (ACR), a distributed-memory fast direct solver for rank-compressible block tridiagonal linear systems arising from the discretization of elliptic operators, developed here for three dimensions. Algorithmic synergies between Cyclic Reduction and hierarchical matrix arithmetic operations result in a solver that has $O(k~N \log N~(\log N + k^2))$ arithmetic complexity and $O(k~N \log N)$ memory footprint, where $N$ is the number of degrees of freedom and $k$ is the rank of a typical off-diagonal block, and which exhibits substantial concurrency. We provide a baseline for performance and applicability by comparing with the multifrontal method where hierarchical semi-separable matrices are used for compressing the fronts, and with algebraic multigrid. Over a set of large-scale elliptic systems with features of nonsymmetry and indefiniteness, the robustness of the direct solvers extends beyond that of the multigrid solver, and relative to the multifrontal approach ACR has lower or comparable execution time and memory footprint. ACR exhibits good strong and weak scaling in a distributed context and, as with any direct solver, is advantageous for problems that require the solution of multiple right-hand sides.

Solution methods for linear discrete ill-posed problems for color image restoration

This work discusses four algorithms for the solution of linear discrete ill-posed problems with several right-hand side vectors. These algorithms can be applied, for instance, to multi-channel image restoration when the image degradation model is described by a linear system of equations with multiple right-hand sides that are contaminated by errors. Two of the algorithms are block generalizations of the standard Golub–Kahan bidiagonalization method with the block size equal to the number of channels. One algorithm uses standard Golub–Kahan bidiagonalization without restarts for all right-hand sides. These schemes are compared to standard Golub–Kahan bidiagonalization applied to each right-hand side independently. Tikhonov regularization is used to avoid severe error propagation. Numerical examples illustrate the performance of these algorithms. Applications include the restoration of color images.

An approach for accelerating incompressible turbulent flow simulations based on simultaneous modelling of multiple ensembles

The present paper deals with the problem of improving the efficiency of large scale turbulent flow simulations. The high-fidelity methods for modelling turbulent flows become available for a wider range of applications thanks to the constant growth of the supercomputers performance, however, they are still unattainable for lots of real-life problems. The key shortcoming of these methods is related to the need of simulating a long time integration interval to collect reliable statistics, while the time integration process is inherently sequential.The novel approach with modelling of multiple flow states is discussed in the paper. The suggested numerical procedure allows to parallelize the integration in time by the cost of additional computations. Multiple realizations of the same turbulent flow are performed simultaneously. This allows to use more efficient implementations of numerical methods for solving systems of linear algebraic equations with multiple right-hand sides, operating with blocks of vectors. The simple theoretical estimate for the expected simulation speedup, accounting the penalty of additional computations and the linear solver performance improvement, is presented. The two problems of modelling turbulent flows in a plain channel and in a channel with a matrix of wall-mounted cubes are used to demonstrate the correctness of the proposed estimates and efficiency of the suggested approach as a whole. The simulation speedup by a factor of 2 is shown.

A shifted block FOM algorithm with deflated restarting for matrix exponential computations

The approximation of exp⁡(tA)B, where A is a large matrix and B a block vector, is a key ingredient in many scientific and engineering computations. A powerful tool to manage the matrix exponential function is to resort to a suitable rational approximation, such as the Carathéodory–Fejér approximation, whose core reduces to solving some shifted linear systems with multiple right-hand sides. However, these shifted systems are often difficult to solve when tA has a large norm. In this paper, we propose to solve some alternatively shifted linear systems. The motivation is that the magnitudes of the poles of the rational approximation are often medium-sized, and they can be much smaller than the norm of tA. We then introduce a shifted block FOM algorithm with deflated restarting for solving these alternatively shifted linear systems efficiently. Our method is advantageous when one has explicit access to A, and A−1 can be computed directly. Theoretical results are given to show the rationale of the proposed strategy. The relationship between the approximations obtained from the shifted block FOM algorithm and the shifted block GMRES algorithm is also analyzed. Numerical experiments demonstrate the superiority of the proposed algorithm over many state-of-the-art algorithms for the matrix exponential.

The block CMRH method for solving nonsymmetric linear systems with multiple right-hand sides

CMRH method (Changing minimal residual with Hessenberg process) is an iterative method for solving nonsymmetric linear systems. This method is similar to QMR method but based on the Hessenberg process instead of the Lanczos process. On dense matrices, the CMRH method is less expensive and requires less storage than other Krylov methods. This paper presents a block version of the CMRH algorithm for solving linear systems with multiple right-hand sides. The new algorithm is based on the block Hessenberg process and the iterates are characterized by a block version of the quasi-minimization property. We analyze its main properties and show that under the condition of full rank of block residual the block CMRH method cannot break down. Finally, some numerical examples are presented to show the efficiency of the new method in comparison with the traditional CMRH method and a comparison with the block GMRES method is also provided.

A scalable iterative dense linear system solver for multiple right-hand sides in data analytics

Abstract We describe Parallel-Projection Block Conjugate Gradient ( pp-bcg ), a distributed iterative solver for the solution of dense and symmetric positive definite linear systems with multiple right-hand sides. In particular, we focus on linear systems appearing in the context of stochastic estimation of the diagonal of the matrix inverse in Uncertainty Quantification. pp-bcg is based on the block Conjugate Gradient algorithm combined with Galerkin projections to accelerate the convergence rate of the solution process of the linear systems. Numerical experiments on massively parallel architectures illustrate the performance of the proposed scheme in terms of efficiency and convergence rate, as well as its effectiveness relative to the (block) Conjugate Gradient and the Cholesky-based ScaLAPACK solver. In particular, on a 4 rack BG/Q with up to 65,536 processor cores using dense matrices of order as high as 524,288 and 800 right-hand sides, pp-bcg can be 2x-3x faster than the aforementioned techniques.

Equivalent Surface Impedance-Based Mixed Potential Integral Equation Accelerated by Optimized $\cal {H}$ -Matrix for 3-D Interconnects

The equivalent surface impedance (ESI)-based mixed potential integral equation (MPIE) is proposed in this paper for parameter extraction of 3-D interconnects. Boundary integral equations (BIEs) describing the conductor region and the nonconductor region are utilized to derive the ESI model, which incorporates with an MPIE to simplify the electromagnetic simulation. For large-scale problems, the solution of MPIE is accelerated by the hierarchical matrix ( $\mathcal {H}$ -matrix) algorithm. Since the interconnect problems usually have multiple ports, the method of moments discretization of MPIE leads to the matrix equation with multiple right-hand sides, which is efficiently solved by the $\mathcal {H}$ -LU-based direct solution. Procedures for $\mathcal {H}$ -matrix are optimized to improve the overall efficiency. The proposed method to optimize $\mathcal {H}$ -matrix benefits both the $\mathcal {H}$ -matrix construction and the $\mathcal {H}$ -LU procedures. The complexities of the CPU time and memory cost for the construction of the optimized $\mathcal {H}$ -matrix are of $\mathcal {O}(N\log N)$ , and the complexity for the direct $\mathcal {H}$ -LU solution is of $\mathcal {O}(N\log ^{2}N)$ . Numerical results demonstrate that the proposed method is both accurate and efficient in a broadband frequency, which is suitable for modeling of 3-D interconnects and on-chip passive structures.

Residual-Based Simpler Block GMRES for Nonsymmetric Linear Systems with Multiple Right-Hand Sides

We propose in this paper a residual-based simpler block GMRES method for solving a system of linear algebraic equations with multiple right-hand sides. We show that this method is mathematically equivalent to the block GMRES method and thus equivalent to the simpler block GMRES method. Moreover, it is shown that the residual-based method is numerically more stable than the simpler block GMRES method. Based on the deflation strategy proposed by Calandra et al. (2013), we derive a deflation strategy to detect the possible linear dependence of the residuals and a near rank deficiency occurring in the block Arnoldi procedure. Numerical experiments are conducted to illustrate the performance of the new method.

A Second Order Ensemble Timestepping Algorithm for Natural Convection

This paper presents a second-order accurate algorithm for calculating an ensemble of solutions to natural convection problems. Solutions are calculated by solving two uncoupled linear systems, each involving a shared coefficient matrix, for multiple right-hand sides at each timestep. Storage requirements and solution time are reduced. Stability and second-order accuracy of the method are proven under a CFL-type condition involving fluctuations of the velocity. Numerical tests are presented which confirm the theoretical analyses.

Enclosing the solution set of parametric interval matrix equation A(p)X = B(p)

Consider the parametric matrix equation A(p)X = B(p), where the elements of the matrices A(p) and B(p) depend linearly on a number of uncertain parameters varying within given intervals. We prove that the united parametric solution sets of the matrix equation and that of the corresponding linear system with multiple right-hand sides, although different as sets, have the same interval hull. A generalization of the parametric Krawczyk iteration with low computational complexity for the matrix equation is presented. Some details improving the implementation and the application of this method are discussed. An interval method, designed by A. Neumaier and A. Pownuk for enclosing the united solution set of parametric linear systems with particular dependency structure, is generalized for arbitrary linear dependencies between the parameters and for systems with multiple right-hand sides. A new, more powerful, sufficient condition for regularity of a parametric interval matrix is proven. An important application of the linear systems with multiple right-hand sides is presented as a key methodology for feasibility in computing the interval hull of a class of united parametric solution sets that appear in practical problems.

English

We consider solving complex symmetric linear systems with multiple right-hand sides. We assume that the coefficient matrix has indefinite real part and positive definite imaginary part. We propose a new block conjugate gradient type method based on the Schur complement of a certain 2-by-2 real block form. The algorithm of the proposed method consists of building blocks that involve only real arithmetic with real symmetric matrices of the original size. We also present the convergence property of the proposed method and an efficient algorithmic implementation. In numerical experiments, we compare our method to a complex-valued direct solver, and a preconditioned and nonpreconditioned block Krylov method that uses complex arithmetic.