Bounded Condition Number Research Articles

Iterative computation of the smallest singular value and the corresponding singular vectors of a matrix

Influenced by some techniques used for computing singular points of nonlinear equations, a generalized inverse iteration method is proposed for approximating the smallest singular value σ n and the associated left and right singular vectors u, v of a matrix A∈ R n×n . In the practically relevant case σ n >0 the method is mathematically equivalent to inverse iteration with AA T. However, unlike classic inverse iteration, the new method works with matrices B k∈ R (n+1)×(n+1) obtained by bordering A in such a way that the B k have uniformly bounded condition numbers. This allows using iterative Krylov-type solvers for large problems. If σ n−1 > σ n the singular vector approximations convergence linearly with factor κ= σ n / σ n−1 <1. Moreover, a certain generalized Rayleigh quotient σ ( k) obtained as a byproduct has a relative error ( σ ( k) − σ n )/ σ n which goes to zero R-linearly with factor κ 2. Some numerical examples confirm the theoretical results and show that the algorithm works reliable also for almost singular matrices and when using Krylov solvers.

An Odyssey into Local Refinement and Multilevel Preconditioning III: Implementation and Numerical Experiments

In this paper, we examine a number of additive and multiplicative multilevel iterative methods and preconditioners in the setting of two-dimensional local mesh refinement. While standard multilevel methods are effective for uniform refinement-based discretizations of elliptic equations, they tend to be less effective for algebraic systems, which arise from discretizations on locally refined meshes, losing their optimal behavior in both storage and computational complexity. Our primary focus here is on Bramble, Pasciak, and Xu (BPX)-style additive and multiplicative multilevel preconditioners, and on various stabilizations of the additive and multiplicative hierarchical basis (HB) method, and their use in the local mesh refinement setting. In parts I and II of this trilogy, it was shown that both BPX and wavelet stabilizations of HB have uniformly bounded condition numbers on several classes of locally refined two- and three-dimensional meshes based on fairly standard (and easily implementable) red and red-green mesh refinement algorithms. In this third part of the trilogy, we describe in detail the implementation of these types of algorithms, including detailed discussions of the data structures and traversal algorithms we employ for obtaining optimal storage and computational complexity in our implementations. We show how each of the algorithms can be implemented using standard data types, available in languages such as C and FORTRAN, so that the resulting algorithms have optimal (linear) storage requirements, and so that the resulting multilevel method or preconditioner can be applied with optimal (linear) computational costs. We have successfully used these data structure ideas for both MATLAB and C implementations using the FEtk, an open source finite element software package. We finish the paper with a sequence of numerical experiments illustrating the effectiveness of a number of BPX and stabilized HB variants for several examples requiring local refinement.

A Multigrid Method for the Mortar-Type Morley Element Approximation of a Plate Bending Problem

In this paper, a multigrid algorithm is presented for the mortar element method for the Morley nonconforming plate element. Based on the multigrid theory developed by Bramble, Pasciak, and Xu in [Math. Comp., 56 (1991), pp. 1--34], we prove that the W-cycle multigrid method is optimal, i.e., the convergence rate is independent of the mesh size and mesh level. Moreover, a variable V-cycle multigrid preconditioner is constructed, which results in a preconditioned system with a uniformly bounded condition number.

Fast Iterative Solution of Saddle Point Problems in Optimal Control Based on Wavelets

In this paper, wavelet techniques are employed for the fast numerical solution of a control problem governed by an elliptic boundary value problem with boundary control. A quadratic cost functional involving natural norms of the state and the control is to be minimized. Firstly the constraint, the elliptic boundary value problem, is formulated in an appropriate weak form that allows to handle varying boundary conditions explicitly: the boundary conditions are treated by Lagrange multipliers, leading to a saddle point problem. This is combined with a fictitious domain approach in order to cover also more complicated boundaries. Deviating from standard approaches, we then use (biorthogonal) wavelets to derive an equivalent infinite discretized control problem which involves only e2-norms and -operators. Classical methods from optimization yield the corresponding optimality conditions in terms of two weakly coupled (still infinite) saddle point problems for which a unique solution exists. For deriving finite-dimensional systems which are uniformly invertible, stability of the discretizations has to be ensured. This together with the e2-setting circumvents the problem of preconditioning: all operators have uniformly bounded condition numbers independent of the discretization. In order to numerically solve the resulting (finite-dimensional) linear system of the weakly coupled saddle point problems, a fully iterative method is proposed which can be viewed as an inexact gradient scheme. It consists of a gradient algorithm as an outer iteration which alternatingly picks the two saddle point problems, and an inner iteration to solve each of the saddle point problems, exemplified in terms of the Uzawa algorithm. It is proved here that this strategy converges, provided that the inner systems are solved sufficiently well. Moreover, since the system matrix is well-conditioned, it is shown that in combination with a nested iteration strategy this iteration is asymptotically optimal in the sense that it provides the solution on discretization level J with an overall amount of arithmetic operations that is proportional to the number of unknows NJ on that level. Finally, numerical results are provided.

Multigrid for the mortar element method for P1 nonconforming element

In this paper, a multigrid algorithm is presented for the mortar element method for P1 nonconforming element. Based on the theory developed by Bramble, Pasciak, Xu in [5], we prove that the W-cycle multigrid is optimal, i.e. the convergence rate is independent of the mesh size and mesh level. Meanwhile, a variable V-cycle multigrid preconditioner is constructed, which results in a preconditioned system with uniformly bounded condition number.

Some New Search Directions for Primal-Dual Interior Point Methods in Semidefinite Programming

Search directions for primal-dual path-following methods for semidefinite programming (SDP) are proposed. These directions have the properties that (1) under certain nondegeneracy and strict complementarity assumptions, the Jacobian matrix of the associated symmetrized Newton equation has a bounded condition number along the central path in the limit as the barrier parameter $\mu$ tends to zero; and (2) the Schur complement matrix of the symmetrized Newton equation is symmetric and the cost for computing this matrix is 2mn3 + 0.5m2n2 flops, where n and m are the dimension of the matrix and vector variables of the semidefinite program, respectively. These two properties imply that a path-following method using the proposed directions can achieve the high accuracy typically attained by methods employing the direction proposed by Alizadeh, Haeberly, and Overton (currently the best search direction in terms of accuracy), but each iteration requires at most half the amount of flops (to leading order).

A Generalized Rayleigh Quotient Iteration for Computing Simple Eigenvalues of Nonnormal Matrices

A Newton type method for approximating eigenpairs of nonnormal matrices is proposed. It takes full advantage of known approximations and circumvents shortcomings of Newton's method applied to the nonlinear system Ax - xλ = 0, (x T x-1)/2=0 caused by a possibly ill-conditioned Jacobian. The eigenvalue problem is considered as a nonlinear parameter dependent system of equations F(x, λ) = Ax - xλ = 0 without a normalization condition. If λ is an eigenvalue of A then (0, λ) is a bifurcation point on the solution manifold of the system. In the paper, a bifurcation point algorithm proposed by Griewank/Reddien and modified by Allgower/Schwetlick is adapted to the case of computing a simple eigenvalue and the corresponding right and left eigenvector of a nonnormal matrix. The extended algorithm with udapted bordering vectors can be viewed as generalized Rayleigh quotient iteration. Surprisingly it turns out that, unlike Newton's method and standard Rayleigh quotient iteration, these algorithms coming from nonlinear analysis, automatically give the right bordering directions, and the linear systems to be solved per step have uniformly bounded condition numbers. The algorithms are analyzed, and numerical comparisons with other algorithms of Rayleigh quotient iteration type are given.

Well Conditioned Spherical Polynomial Systems

We investigate nonnegative spherical polynomials p, normalized to p(1)=1, which minimize the integral over the unit sphere for fixed degree. Such polynomials are useful in the construction of node systems on the sphere which support numerically stable systems of zonal functions, for instance with bounded condition numbers.

On the Local Convergence of a Predictor-Corrector Method for Semidefinite Programming

We study the local convergence of a predictor-corrector algorithm for semidefinite programming problems based on the Monteiro--Zhang unified direction whose polynomial convergence was recently established by Monteiro. Under strict complementarity and nondegeneracy assumptions superlinear convergence with Q-order 1.5 is proved if the scaling matrices in the corrector step have bounded condition number. A version of the predictor-corrector algorithm enjoys quadratic convergence if the scaling matrices in both predictor and corrector steps have bounded condition numbers. The latter results apply in particular to algorithms using the Alizadeh--Haeberly--Overton (AHO) direction since there the scaling matrix is the identity matrix.

Stable multiscale discretizations for saddle point problems and preconditioning

Stability for discretizations of saddle point problems is typically the result of satisfying the discrete Babuška-Brezzi condition. As a consequence a number of natural discretizations are ruled out and some effort is required to provide stable ones. Therefore ideas for circumventing the Babuška-Brezzi condition are interesting. Here an ansatz presented in a series of papers by Hughes et al. is described and investigated in the framework of multiscale discretizations. In particular discretizations for appending boundary conditions by Lagrange multipliers and the stationary Stokes problem are considered. Sufficient conditions for their stability are given and diagonal preconditioned which give uniformly bounded condition numbers are proposed.

Wavelet Galerkin methods for second-kind integral equations

We use vector-valued multiwavelets on compact sets to develop a Galerkin method for systems of integral equations of the second kind. We propose a compression strategy for the coefficient matrix of the linear system obtained from this method and show that the compressed scheme preserves almost optimal convergence rate of the original scheme and yields a sparse matrix with a bounded condition number.

Iterative methods for ill-conditioned Toeplitz matrices

In this paper we study the use of the Sine Transform for preconditioning linear Toeplitz systems. We consider Toeplitz matrices with a real generating function that is nonnegative with only a small number of zeros. Then we can define a preconditioner of the formSnΛSn whereSn is the matrix describing the discrete Sine transform and Λ is a diagonal matrix. If we have full knowledge aboutf then we can show that the preconditioned system is of bounded condition number independly ofn. We can obtain the same result for the case that we know only the position and order of the zeros off. If we only know the matrix and its coefficientstj, we present Sine transform preconditioners that show in many examples the same numerical behaviour.

Minimal residual algorithm and matrix-vector information

In this paper we consider the problem of finding approximate solutions to large linear systems by using general matrix-vector information. Traub and Woźniakowski have proved that the minimal residual algorithm is “almost strongly optimal” for any orthogonally invariant class of matrices when Krylov information is used. We prove that minimal residual algorithm is no longer optimal when general matrix-vector information is considered instead of Krylov information. Moreover, we analyze for which classes of matrices minimal residual algorithm preserves its optimality also for general matrix-vector information. We prove that for particular classes of matrices, including symmetric matrices, symmetric positive definite matrices, and matrices with bounded condition number, minimal residual algorithm is “strongly optimal” regardless of the information considered.

Possible bursting phenomena in self‐tuning control

AbstractIn this paper, in the case when there are no modelling errors, bursting phenomena in extended‐least‐squares‐ (ELS‐) based self‐tuning control are analysed. It is shown that in the absence of external excitation and without imposing structural conditions on the system model, bursts of the tracking error and consequently drift of the parameter estimates are possible. By analysing the possible occurrence of bursting, an upper bound on the convergence rate of the tracking error is established. It is demonstrated that it is the self‐excitation mechanism which provides global stability of the adaptive system. In other words, the possible instability of the adaptive loop generates a bounded condition number of the covariance matrix. This in turn implies stabilization of the adaptive system. In this paper we also discuss a stochastic gradient algorithm with a convergence rate similar to that presently established for the ELS‐based self‐tuning controller.

Multigrid Preconditioning for the Biharmonic Dirichlet Problem

A multigrid preconditioning scheme for solving the Ciarlet–Raviart mixed method equations for the biharmonic Dirichlet problem is presented. In particular, a Schur complement formulation for these equations which yields noninherited quadratic forms is considered. The preconditioning scheme is compared with a standard W-cycle multigrid iteration. It is proved that a Variable V-cycle preconditioner leads to problems with uniformly bounded condition numbers. However, W-cycle convergence is proved only if the number of smoothings “m is sufficiently large.” An example is given in which the W-cycle diverges unless $m \geq 8$. Divergent W-cycles are also encountered when solving the Morley equations for the biharmonic Dirichlet problem, although Brenner has proved W-cycle convergence for sufficiently large m [9]. This is illustrated with additional computations, while Variable V-cycles continue to produce excellent preconditioners in this setting. Certain approximate $L_2$-inner products are described and a modification to the Ciarlet–Raviart method is proposed which reduces the work of the multilevel schemes. Optimal order error estimates are proved for the modified method. Consideration is restricted to Ciarlet–Raviart methods of quadratic and higher degree throughout the paper.

Multilevel preconditioning

This paper is concerned with multilevel techniques for preconditioning linear systems arising from Galerkin methods for elliptic boundary value problems. A general estimate is derived which is based on the characterization of Besov spaces in terms of weighted sequence norms related to corresponding multilevel expansions. The result brings out clearly how the various ingredients of a typical multilevel setting affect the growth rate of the condition numbers. In particular, our analysis indicates how to realize even uniformly bounded condition numbers. For example, the general results are used to show that the Bramble-Pasciak-Xu preconditioner for piecewise linear finite elements gives rise to uniformly bounded condition numbers even when the refinements of the underlying triangulations are highly nonuniform. Furthermore, they are applied to a general multivariate setting of refinable shift-invariant spaces, in particular, covering those induced by various types of wavelets.

The Convergence of a Modified ELS Algorithm

In this paper the convergence analysis of a modified Extended Least Squares algorithm with forgetting factor is given. The convergence of the parameter norm, parameter difference norm, as well as the convergence for the a priori prediction error case are established without requiring bounded condition number. These properties are valid without any restriction on stability assumptions or experimental conditions and can be used in order to establish stability and convergence of explicit adaptive control schemes.Numerical examples illustrate the behaviour of the modified ELS algorithm.

On the Optimal Solution of Large Linear Systems

The information-based study of the optimal solution of large linear systems is initiated by studying the case of Krylov information. Among the algorithms that use Krylov information are minimal residual, conjugate gradient, Chebyshev, and successive approximation algorithms. A "sharp" lower bound on the number of matrix-vector multiplications required to compute an å-approximation is obtained for any orthogonally invariant class of matrices. Examples of such classes include many of practical interest such as symmetric matrices, symmetric positive definite matrices, and matrices with bounded condition number. It is shown that the minimal residual algorithm is within at most one matrix-vector multiplication of the lower bound. A similar result is obtained for the generalized minimal residual algorithm. The lower bound is computed for certain classes of orthogonally invariant matrices. How the lack of certam properties (symmetry, positive definiteness) increases the lower bound is shown. A conjecture and a number of open problems are stated.

The solution of a fourth-order partial differential equation using the method of lines

The purpose of this paper is to investigate the solution of the fourth order boundary value problem with boundary conditions where α(s) is square integrable on 0≦s≦a, bounded from above by M and below by M >0 and 0≦z≦∞. This problem arises in certain areas of fluid dynamics. If [0, a] is divided into equal intervals of length h such that and ū is a vector with ith component u(si,z) the asssociated vector differential equation is . The eigenvalues of AB are shown to be nonnegative and the matrix of eigenveciors to have bounded condition number, which assures that the solution remains bounded as h→0, and hence the solution of the boundary value problem exists.