Incomplete Orthogonalization Research Articles

Properties of semi-conjugate gradient methods for solving unsymmetric positive definite linear systems

The conjugate gradient (CG) method is a classic Krylov subspace method for solving symmetric positive definite linear systems. We analyze an analogous semi-conjugate gradient (SCG) method, a special case of the existing semi-conjugate direction (SCD) methods, for unsymmetric positive definite linear systems. Unlike CG, SCG requires the solution of a lower triangular linear system to produce each semi-conjugate direction. We prove that SCG is theoretically equivalent to the full orthogonalization method (FOM), which is based on the Arnoldi process and converges in a finite number of steps. Because SCG's triangular system increases in size each iteration, Dai and Yuan [Study on semi-conjugate direction methods for non-symmetric systems, Int. J. Numer. Meth. Eng. 60(8) (2004), pp. 1383–1399] proposed a sliding window implementation (SWI) to improve efficiency. We show that the directions produced are still locally semi-conjugate. A counter-example illustrates that SWI is different from the direct incomplete orthogonalization method (DIOM), which is FOM with a sliding window. Numerical experiments from the convection-diffusion equation and other applications show that SCG is robust and that the sliding window implementation SWI allows SCG to solve large systems efficiently.

High-order numerical solutions to the shallow-water equations on the rotated cubed-sphere grid

A novel numerical approach to solving the shallow-water equations on the sphere using high-order numerical discretizations in both space and time is proposed. A space-time tensor formalism is used to express the equations of motion covariantly and to describe the geometry of the rotated cubed-sphere grid. The spatial discretization is done with the direct flux reconstruction method, which is an alternative formulation to the discontinuous Galerkin approach. The equations of motion are solved in differential form and the resulting discretization is free from quadrature rules. It is well known that the time step of traditional explicit methods is limited by the phase velocity of the fastest waves. Exponential integration is employed to enable integrations with significantly larger time step sizes and improve the efficiency of the overall time integration. New multistep-type exponential propagation iterative methods of orders 4, 5 and 6 are constructed and applied to integrate the shallow-water equations in time. These new schemes enable time integration with high-order accuracy but without significant increases in computational time compared to low-order methods. The exponential matrix functions-vector products used in the exponential schemes are approximated using the complex-step approximation of the Jacobian in the Krylov-based KIOPS (Krylov with incomplete orthogonalization procedure solver) algorithm. Performance of the new numerical methods is evaluated using a set of standard benchmark tests.

KIOPS: A fast adaptive Krylov subspace solver for exponential integrators

This paper presents a new algorithm KIOPS for computing linear combinations of φ-functions that appear in exponential integrators. This algorithm is suitable for large-scale problems in computational physics where little or no information about the spectrum or norm of the Jacobian matrix is known a priori. We first show that such problems can be solved efficiently by computing a single exponential of a modified matrix. Then our approach is to compute an appropriate basis for the Krylov subspace using the incomplete orthogonalization procedure and project the matrix exponential on this subspace. We also present a novel adaptive procedure that significantly reduces the computational complexity of exponential integrators. Our numerical experiments demonstrate that KIOPS outperforms the current state-of-the-art adaptive Krylov algorithm phipm.

Approximating the large sparse matrix exponential using incomplete orthogonalization and Krylov subspaces of variable dimension

SummaryKrylov subspace approximations to the matrix exponential are popularly used with full orthogonalization instead of incomplete orthogonalization, even though the latter strategy is known to reduce the cost by truncating the recurrences of the modified Gram–Schmidt process. This study combines such a strategy with an adaptive step‐by‐step integration scheme that allows both the stepsize and the dimension of the Krylov subspace to vary. A convergence analysis is done. Numerical results on test problems drawn from systems biology and computer systems show a significant speedup over the standard implementation with full orthogonalization and fixed dimension.

An efficient exponential time integration method for the numerical solution of the shallow water equations on the sphere

The exponential propagation methods were applied in the past for accurate integration of the shallow water equations on the sphere. Despite obvious advantages related to the exact solution of the linear part of the system, their use for the solution of practical problems in geophysics has been limited because efficiency of the traditional algorithm for evaluating the exponential of Jacobian matrix is inadequate. In order to circumvent this limitation, we modify the existing scheme by using the Incomplete Orthogonalization Method instead of the Arnoldi iteration. We also propose a simple strategy to determine the initial size of the Krylov space using information from previous time instants. This strategy is ideally suited for the integration of fluid equations where the structure of the system Jacobian does not change rapidly between the subsequent time steps. A series of standard numerical tests performed with the shallow water model on a geodesic icosahedral grid shows that the new scheme achieves efficiency comparable to the semi-implicit methods. This fact, combined with the accuracy and the mass conservation of the exponential propagation scheme, makes the presented method a good candidate for solving many practical problems, including numerical weather prediction.

Parallel GMRES Incomplete Orthogonalization Auto-Tuning

Abstract Krylov linear solvers are widely used to solve various kind of scientific problems. They are entitled to many optimization in order to reduce the processing time needed to attain solution. However, these optimisations are mainly problems specific and do not always fit well in parallel. Tuning them can sometimes require and heavy work to extract sustainable performances. Auto-tuning helps the users to tune their solver with little e_ort and at a small cost. We present a general parallel auto-tuned linear solver approach in order to reduce the time of computation needed for a solver to attain solution. Our new approach is based on the tuning of the Arnoldi incomplete orthogonalization process within GMRES by monitoring the convergence thanks to a simple heuristic. We obtained promising results by reducing significantly the time of computation on di_erent cases in serial and parallel processing using a cluster, involving both real and complex valued linear systems. Our work enlight the emergence of incomplete Arnoldi orthogonalization auto-tuning as a possible optimization for iteratives methods.

Incomplete orthogonalization preconditioners for solving large and dense linear systems which arise from Semidefinite Programming

There is a strong need for the fast solution of large and dense linear systems which arise from the interior-point method for solving Semidefinite Programming with its dual problem. Often direct methods are too expensive in terms of computer memory and CPU-time requirements, then the only alternative is to use iterative methods. Here, a class of incomplete orthogonalization preconditioners for the conjugate gradient method for solving this type of linear systems will be proposed. The efficient feature of the preconditioners will be confirmed by several numerical experiments.

Some recursions on Arnoldi's method and IOM for large non-Hermitian linear systems

Arnoldi's method and the Incomplete Orthogonalization Method (IOM) for large non-Hermitian linear systems are studied. It is shown that the inverse of a general nonsingular j × j Hessenberg matrix can be updated in O( j 2) flops from that of its ( j − 1) × ( j − 1) principal submatrix. The updating recursion of inverses of the Hessenberg matrices does not need any QR or LU decomposition as commonly used in the literature. Some updating recursions of the residual norms and the approximate solutions obtained by these two methods are derived. These results are appealing because they allow one to decide when the methods converge and show one how to compute approximate solutions very cheaply and easily.

On IGMRES: An incomplete generalized minimal residual method for large unsymmetric linear systems

The truncated version of the generalized minimal residual method (GMRES), the incomplete generalized minimal residual method (IGMRES), is studied. It is based on an incomplete orthogonalization of the Krylov vectors in question, and gives an approximate or quasi-minimum residual solution over the Krylov subspace. A convergence analysis of this method is given, showing that in the non-restarted version IGMRES can behave like GMRES once the basis vectors of Krylov subspace generated by the incomplete orthogonalization are strongly linearly independent. Meanwhile, some relationships between the residual norms for IOM and IGMRES are established. Numerical experiments are reported to show convergence behavior of IGMRES and of its restarted version IGMRES(m).

REOS — A program for relaxed-orbital oscillator strength calculations

A spontaneous decay of an excited atomic state leads to a rearrangement of the spatial electron distribution in the atom. This redistribution concerns the entire electronic cloud and not only the active electron in an atomic transition. For many-electron atoms, rearrangement effects are naturally included if the electronic wavefunctions of the initial and final states are determined independently. The separate optimization of the atomic states, however, yields two sets of one-electron orbitals which are not orthogonal to each other. This incomplete orthogonality also influences the calculation of transition amplitudes since, additionally, many small contributions arise from the radial overlap of electrons of different subshells which have the same symmetry. In the framework of the GRASP92 atomic structure package we describe a program for the computation of oscillator strengths and Einstein A and B coefficients. The program is based on a determinant representation of the atomic states and allows for incomplete orthogonal radial orbital functions for the initial and final states. For large lists of configuration state functions, this module also facilitates the computation of transition arrays since the initial and final states need not to be determined in the same run.

On IOM(q): The Incomplete Orthogonalization Method for Large Unsymmetric Linear Systems

The incomplete orthogonalization method (IOM(q)), a truncated version of the full orthogonalization method (FOM) proposed by Saad, has been used for solving large unsymmetric linear systems. However, no convergence analysis has been given. In this paper, IOM(q) is analysed in detail from a theoretical point of view. A number of important results are derived showing how the departure of the matrix A from symmetric affects the basis vectors generated by IOM(q), and some relationships between the residuals for IOM(q) and FOM are established. The results show that IOM(q) behaves much like FOM once the basis vectors generated by it are well conditioned. However, it is proved that IOM(q) may generate an ill-conditioned basis for a general unsymmetric matrix such that IOM(q) may fail to converge or at least cannot behave like FOM. Owing to the mathematical equivalence between IOM(q) and the truncated ORTHORES(q) developed by Young and Jea, insights are given into the convergence of the latter. A possible strategy is proposed for choosing the parameter q involved in IOM(q). Numerical experiments are reported to show convergence behaviour of IOM(q) and of its restarted version.

DQGMRES: a Direct Quasi-minimal Residual Algorithm Based on Incomplete Orthogonalization

We describe a Krylov subspace technique, based on incomplete orthogonalization of the Krylov vectors, which can be considered as a truncated version of GMRES. Unlike GMRES(m), the restarted version of GMRES, the new method does not require restarting. Like GMRES, it does not break down. Numerical experiments show that DQGMRES(k) often performs as well as the restarted GMRES using a subspace of dimension m=2k. In addition, the algorithm is flexible to variable preconditioning, i.e., it can accommodate variations in the preconditioner at every step. In particular, this feature allows the use of any iterative solver as a right-preconditioner for DQGMRES(k). This inner-outer iterative combination often results in a robust approach for solving indefinite non-Hermitian linear systems.

DQGMRES: a Direct Quasi‐minimal Residual Algorithm Based on Incomplete Orthogonalization

DQGMRES: a Direct Quasi‐minimal Residual Algorithm Based on Incomplete Orthogonalization

DQGMRES: a Direct Quasi‐minimal Residual Algorithm Based on Incomplete Orthogonalization

DQGMRES: a Direct Quasi‐minimal Residual Algorithm Based on Incomplete Orthogonalization

A block incomplete orthogonalization method for large nonsymmetric eigenproblems

The incomplete orthogonalization method (IOM) proposed by Saad for computing a few eigenpairs of large nonsymmetric matrices is generalized into a block incomplete orthogonalization method (BIOM). It is studied how the departure from symmetry ‖A − A H ‖ affects the conditioning of the block basis vectors generated by BIOM, and some relationships are established between the approximate eigenpairs obtained by BIOM and Ritz pairs. It is proved that BIOM behaves much like generalized block Lanczos methods if the basis vectors of the block Krylov subspace generated by it are strongly linearly independent. However, it is shown that BIOM may generate a nearly linearly dependent basis for a general nonsymmetric matrix. Numerical experiments illustrate the convergence behavior of BIOM.

Ab-initio calculation of the 2s2 1So-2s3p 3P1 intercombination transition in beryllium-like ions

Transition probabilities of the 2s2 1So-2s3p 1,3P1 intercombination and resonance lines have been calculated for 14 beryllium-like ions in the atomic range 7 ⩽ Z ⩽ 36. We used multiconfiguration Dirac-Fock wave functions to explore the effects of configuration interaction, of electron rearrangement as well as of relativity in a consistent scheme along the isoelectronic sequence. To show the number of configurations needed in a proper expansion of the wave functions we systematically enlarged the basis from small to large-scale. For the low-Z elements, virtual excitations of the 2s and 3p electrons up to the 5l subshells contribute significantly to the inter-combination probability even though the transition energies remain almost unaffected by excitations beyond the 3l shells. The incomplete orthogonality of the orbital functions due to the rearrangement of the electron density clearly shifts the intercombination rates and appears to be independent of the configuration expansion. This effect decreases, as expected, at higher nuclear charges. Our ab-initio calculation agrees well with a recent measurement by Granzow et al., for ions with 11 ⩽ Z ⩽ 14. The remarkable deviations near the neutral end of the Be-sequence found in a previous relativistic ab-initio study by Kim et al. have been resolved by increasing the scale of the configuration expansion and by including electron relaxation.

The Arnoldi Method for Normal Matrices

For large Hermitian matrices the preconditionend conjugate gradient algorithm and the Lanczos algorithm are the most important methods for solving linear systems and for computing eigenvalues. There are various generalizations to the nonsymmetric case based on the Arnoldi method or on the nonsymmetric Lanczos algorithm, e.g., generalized minimal residual ((GMRES), residual minimization in a Krylov space) and conjugate gradient squared ((CGS), a biorthogonalization algorithm adapted from the biconjugate gradient method) for linear equations and the incomplete orthogonalization method and the look-ahead Lanczos algorithm for computing eigenvalues. The aim of this paper is to analyse the Arnoldi method applied to a normal matrix. It is shown that for a normal matrix A, the Arnoldi projection $H_p $ is a normal upper Hessenberg matrix if and only if A is (1)-normal. Only in this case, $H_p $ is tridiagonal and the Arnoldi method has the same properties as in the Hermitian case.

Generalized Huzinaga building-block equations for nonorthogonal electronic groups: Relation to the Adams–Gilbert theory

We show in this article that the Huzinaga building-block equations for many-electron systems, derived under the hypothesis of strong orthogonality among the system groups, can be also deduced as a particular case of the Adams–Gilbert formalism, in which intergroup overlapping is recognized from the outset. This viewpoint is more realistic because the strong orthogonality cannot be fully achieved in most practical cases due to basis-set truncation. Incomplete orthogonality leads to nonzero expectation values of the projection operators appearing in the Fock Hamiltonian (projection energy). This fact raises some conceptual problems since the projection energy has been used to interpret several important physical effects in recent investigations. The present work clarifies several practical aspects relative to the incomplete fulfillment of the strong-orthogonality hypothesis, in particular (a) the analytical relationship between the projection energy and the intergroup overlap energy involving weakly overlapping groups; (b) the required values for the projection constants in cases of incomplete orthogonality; and (c) how the effects of basis-set truncation can be analyzed unambiguously.

Deflated Krylov subspace methods for nearly singular linear systems

This paper concerns the use of Krylov subspace methods for the solution of nearly singular nonsymmetric linear systems. We show that the incomplete orthogonalization methods (IOM) in conjunction with certain deflation techniques of Stewart, Chan, and Saad can be used to solve large nonsymmetric linear systems which are nearly singular.

Reduced L1 level width and Coster-Kronig yields by relaxation and continuum interactions in atomic zinc.

Calculations of the level width \ensuremath{\Gamma}(${\mathit{L}}_{1}$) and the ${\mathit{f}}_{12}$ and ${\mathit{f}}_{13}$ Coster-Kronig yields for atomic zinc have been performed with Dirac-Fock wave functions. For \ensuremath{\Gamma}(${\mathit{L}}_{1}$), a large deviation between theory and evaluated data exists. We include the incomplete orthogonality of the electron orbitals as well as the interchannel interaction of the decaying states. Orbital relaxation reduces the total rates in all groups of the electron-emission spectrum by about 10--20 %. Different, however, is the effect of the continuum interaction. The ${\mathit{L}}_{1\mathrm{\ensuremath{-}}}$${\mathit{L}}_{23}$X Coster-Kronig part of the spectrum is definitely reduced in its intensity, whereas the MM and MN spectra are slightly enhanced. This results in a reduction of Coster-Kronig yields, where for medium and heavy elements considerable discrepancies have been found in comparison to relativistic theory. Briefly, we discuss the consequences of our calculations for heavier elements.