Conjugate Gradient Type Research Articles

On the constrained Chebyshev approximation problem on ellipses

Constrained Chebyshev approximation problems of the type with minimum (p is an element of Pi(sub n):p(c)=1) and maximum (z is an element of E) with /p(z)/ are considered. Here Pi(sub n) denotes the set of all complex polynomials of degree at most n, E is any ellipse in the complex plane, and c is an element of C/E. Such approximation problems arise in the context of optimizing semi-iterative methods for the solution of large, sparse systems of linear equations Ax=b with complex non-Hermitian coefficient matrices A. The problem of obtaining optimal polynomial preconditioners for conjugate gradient type methods for Ax=b also leads to problems of this type. A new family of polynomials -- q(sub n)(z;c), n is an element of N, and c is an element of C/E -- are introduced as the polynomials which are optimal for a modified version of the Chebyshev approximation problem with Pi(sub n) replaced by a certain subfamily. Some simple properties of q(sub n) are also listed. A necessary and sufficient condition for q(sub n) to be the extremal polynomial for the approximation problem is then derived. Finally, it is shown that q(sub n) is indeed optimal for the problem for all fixed n whenever the distance between c and E is sufficiently large. Results of some numerical tests are presented.

Unsymmetric conjugate gradient methods and sparse direct methods in finite element flow simulation

AbstractA series of numerical experiments on the Cray XMP/48 and on the Cray 2 investigate the robustness and economy of direct and unsymmetric conjugate gradient (CG) type methods for the solution of matrix systems arising from a 3D FEM discretization of fluid flow problems. Computations on a Boussinesq flow model problem with either ILU preconditioned or unpreconditioned unsymmetric CG methods are presented. Such experiments seem to indicate that the unpreconditioned BICG method is robust for moderately non‐linear incompressible Navier–Stokes FEM discretizations and that the ILU preconditioned BICG method is very robust and more economic than an unsymmetric frontal solver when the generous memory of the Cray 2 is exploited to store both the matrix and its preconditioner. We cover some of the programming aspects of direct and iterative methods on a supercomputer and find that direct methods have advantages: the crucial CPU‐consuming area of code is compact but overwhelming, and its percentage of total CPU usage is independent of the spectral properties of the matrix involved. An optimal implementation of the unsymmetric CG method is more difficult because its work is related to the spectral distribution of the matrix considered and because there is no single portion of the code that overwhelmingly dominates the CPU usuage.

Stabilised bilinear-constant velocity-pressure finite elements for the conjugate gradient solution of the stokes problem

In this paper, a sufficient condition for the stability of low-order mixed finite element methods is introduced. To illustrate the possibilities, two stabilisation procedures for the popular Q 1−P 0 mixed method are theoretically analysed. The effectiveness of these procedures in practice is assessed by comparing results with those obtained using a conventional penalty formulation, for a standard test problem. It is demonstrated that with appropriate stabilisation, efficient iterative solution techniques of conjugate gradient type can be applied directly to the discrete Stokes system.

Fully vectorizable block preconditionings with approximate inverses for non-symmetric systems of equations

We used a conjugate gradient type method, with preconditioning, to solve the sparse linear systems arising from the discretization of PDEs. With such methods, the main obstacles for complete vectorization have been the preconditioning calculation and its application step within the iteration: for the matrices obtained using 5- or 9-point discretization operators, some well known existing preconditionings (like ILU) require a block-recursive procedure which prevents vectorization. Preconditioners based on nested incomplete factorization, which require the calculation of approximate inverses of tridiagonal matrices, allow complete vectorization of the application step. We present a formulation of such a preconditioning, using a Frobenius norm minimization to calculate the inverses, which also allows complete vectorization of the inverses' calculation, thus making the iterative solver completely vectorizable. Numerical experiments show that the method is robust over a range of symmetric and non-symmetric problems, and up to 4 times faster than other existing methods, such as ILU, depending on the computer and compiler being used. We also show the importance of diagonal scaling used in conjunction with other preconditionings and present some theoretical results concerning the approximate inverses of tridiagonal matrices, calculated using the Frobenius norm minimization.

Domain decomposition methods for the parallel computation of reacting flows

Domain decomposition is a natural route to parallel computing for partial differential equation solvers. In this procedure, subdomains of which the original domain of definition is comprised are assigned to independent processors at the price of periodic coordination between processors to compute global parameters and maintain the requisite degree of continuity of the solution at the subdomain interfaces. In the domain-decomposed solution of steady multidimensional systems of PDEs by finite difference methods using a pseudo-transient version of Newton iteration, the only portion of the computation which generally stands in the way of efficient parallelization is the solution of the large, sparse linear systems arising at each Newton step. For some Jacobian matrices drawn from an actual two-dimensional reacting flow problem, we make comparisons between relaxation-based linear solvers and also preconditioned iterative methods of Conjugate Gradient and Chebyshev type, focusing attention on both iteration count and global inner product count. The generalized minimum residual method with block-ILU preconditioning is judged the best serial method among those considered, and parallel numerical experiments on the Encore Multimax demostrate for it approximately 10-fold speedup on 16 processsors. The three special features of reacting flow models in relation to these linear systems are: the possibly large number of degrees of freedom per gridpoint, the dominance of dense intra-point source-term coupling over inter-point convective-diffusive coupling throughout significant portions of the flow-field and strong nonlinearities which restrict the time step to small values (independent of linear algebraic considerations) throughout significant portions of the iteration history. Though these features are exploited to advantage herein, many aspects of the paper are applicable to the modeling of general convective-diffusive systems.

Conjugate gradient type methods and preconditioning

In this paper we consider various iterative methods for the numerical solution of very large, sparse linear systems of equations, which arise in the discretization of partial differential equations. As the performance of the methods generally depends on the characteristics of the problems to be solved, a judicious choice between the methods will require knowledge about the system. The aim of this paper is to review the properties of the various iteration methods, in order to assist the user in making a deliberate selection.

Extrapolation vs. projection methods for linear systems of equations

It is shown that the four vector extrapolation methods, minimal polynomial extrapolation, reduced rank extrapolation, modified minimal polynomial extrapolation, and topological epsilon algorithm, when applied to linearly generated vector sequences, are Krylov subspace methods, and are equivalent to some well known conjugate gradient type methods. A unified recursive method that includes the conjugate gradient, conjugate residual, and generalized conjugate gradient methods is developed. Finally, the error analyses for these methods are unified, and some known and some new error bounds for them are given.

Projected gradient methods for linearly constrained problems

The aim of this paper is to study the convergence properties of the gradient projection method and to apply these results to algorithms for linearly constrained problems. The main convergence result is obtained by defining a projected gradient, and proving that the gradient projection method forces the sequence of projected gradients to zero. A consequence of this result is that if the gradient projection method converges to a nondegenerate point of a linearly constrained problem, then the active and binding constraints are identified in a finite number of iterations. As an application of our theory, we develop quadratic programming algorithms that iteratively explore a subspace defined by the active constraints. These algorithms are able to drop and add many constraints from the active set, and can either compute an accurate minimizer by a direct method, or an approximate minimizer by an iterative method of the conjugate gradient type. Thus, these algorithms are attractive for large scale problems. We show that it is possible to develop a finite terminating quadratic programming algorithm without non-degeneracy assumptions.

On a class of Chebyshev approximation problems which arise in connection with a conjugate gradient type method

We investigate approximation problems of the form $$\mathop {\min }\limits_{p:p(0) = 1} \mathop {\max |p(\lambda )|}\limits_{\lambda \in S} $$ (P) withp ranging over all polynomials of degree ≦k andS any line segment of the complex plane. Problems of this type appear in connection with error bounds for the minimum residual algorithm for solving linear systems with coefficient matrixA=eiθ(dI−N), whereN=−NH andd∈ℝ. Besides a few general results on (P), we present the explicit solution of (P) for segmentsS which are parallel to the imaginary and symmetric to the real axis. This result leads to an optimal semi-iterative method for the class of matricesA=I−N, whereN=−NT.

Reduction of pressure surges by minimax optimization

The control of pressure surges in a pipeline is an important problem in many areas of industry, including hydroelectric power generation, oil refineries and chemical processing plants. This paper deals with the control of pressure surges in a simple system consisting of a single horizontal pipe. The methodology, however, is easily extended to more complex systems and is quite easily implemented. The control problem is formulated as a nonlinear minimax optimization problem. Due to the large-scale nature of such problems, a successive linear programming (LP) method is adopted. The convergence of the method is accelerated by a conjugate gradient type search. Computational results are also provided.

A Comparison Between Some Direct and Iterative Methods for Certain Large Scale Geodetic Least Squares Problems

The purpose of this paper is to describe and compare some numerical methods for solving large dimensional linear least squares problems that arise in geodesy and, more specially, from Doppler positioning. The methods that are considered are the direct orthogonal decomposition, and the combination of conjugate gradient type algorithms with projections as well as the exploitation of “Property A”. Numerical results are given and the respective advantage of the methods are discussed with respect to such parameters as CPU time, input/output and storage requirements. Extensions of the results to more general problemsare also discussed.

Identification of a pharmacokinetic two-compartment model application to the case of salicylates administered to rabbit

The purpose of this paper is mainly the coefficient identification of a pharmacokinetic two-compartment open model with first-order absorption. This model consists of the central compartment (plasma) and the peripheral one (tissue) containing the ill site aimed by the action of the administered drug. The different phenomenons taking place between the compartments obey a first-order (linear) kinetic. This phenomenons of absorption, transfer, and elimination are characterized by their rate constants. A compartmental analysis leads to a system of coupled linear differential equations whose coefficients, the unknowns of the problem, are the above-mentioned rate constants. This system describes the transition of the drug from its administration to its action site passing through the plasma. An oral administration of a single dose is considered and the evolution of the drug concentration in the plasma is measured. These measurements, performed over time in a reduced number on the rabbit, give unique and useful information for the identification of the coefficient set appearing in the mathematical model. The identification problem is stated in terms of a constraint minimization of a cost functional, given by the quadratic error between the measured and the computed drug concentrations in the plasma. From the nonconvexity of this cost functional (existence of more than one local minimum) follows the nonuniqueness of the optimal solution of this identification problem. The minimization is performed on a minicomputer by means of a nonlinear programming method of conjugate-gradient type. The efficiency of the suggested method has been tested with measurements made on a lot of rabbits after the administration of a single dose (100 mg/kg) of aspirin in form of lysine acetylsalicylate and sodium salicylate. The administration was made, thanks to an adapted technique, by a rubber probe in two different ways: in the stomach and in the duodenum. After the model identification it was possible to study, in simulation on a computer, the effect of the metabolite/nonmetabolite drug forms as well as the effect of the drug administration site on the time course of the drug in the organism. This simulation makes it easy to have access to magnitudes, familiar to the pharmaceutical practitioner, like the half-life, the bioavailability, the clearances, the build- up of blood levels for multiple doses, etc. . . . It was possible to improve the understanding of the time course of aspirin in rabbit in comparison to the time course of its main metabolite, the sodium salicylate, and this for the same animal.

Preconditioned minimal residual methods for chebyshev spectral calculations

The problem of preconditioning the pseudospectral Chebyshev approximation of an elliptic operator is considered. The numerical sensitiveness to variations of the coefficients of the operator are investigated for two classes of preconditioning matrices: one arising from finite differences, the other from finite elements. The preconditioned system is solved by a conjugate gradient type method, and by a DuFort-Frankel method with dynamical parameters. The methods are compared on some test problems with the Richardson method and with the minimal residual Richardson method.

An attenuated projector-backprojector for iterative SPECT reconstruction

A new ray-driven projector-backprojector which can easily be adapted for hardware implementation is described and simulated in software. The projector-backprojector discretely models the attenuated Radon transform of a source distributed within an attenuating medium as line integrals of discrete pixels, obtained using the standard sampling technique of averaging the emission source of attenuation distribution over small square regions. Attenuation factors are calculated for each pixel during the projection and backprojection operations instead of using precalculated values. The calculation of the factors requires a specification of the attenuation distribution, estimated either from an assumed constant distribution and an approximate body outline or from transmission measurements. The distribution of attenuation coefficients is stored in memory for efficient access during the projection and backprojection operations. The reconstruction of the source distribution is obtained by using a conjugate gradient or SIRT type iterative algorithm which requires one projection and one backprojection operation for each iteration.

Optimal Control of a Twin-Screw Extruder by Means of a Microprocessor

We have described, in previous works (Abdallah, Lafav, 1976), the design and the implementation of an on-line quality sensor for PVC products. This has made possible to carry out an Optimal pilot operation of an industrial extruder which is intended to the non-stop production of polyvinyl chlorate (PVC) tubesThe optimal closed-loop control is the solution of the constraint minimization of a cost functional, given by the quadratic error between the desired product quality and the measured oneTo do so, we first produce a linearized static model describing the extruder and its local regulators which gives rise to a well-posed minimization problem. This mathematical model is obtained by a piecewise linearization of a nonlinear model, previously off-line identified by means of a bank of input/ output signals taken on the process in the vicinity of a given nominal working pointThen, a non-linear programming method of conjugate-gradient type is used to calculate, when required, the new and unique set noints for the local regulatorsThe complexity of the solution computed in real-time, needs the use of an evolved microprocessor. The optimal piloting algorithm is implemented on this microprocessor fitted out with industrial inputs/outputs allowing a direct process connectionVarious tests undertaken on the extrusion site have shown that the following targets have been efficiently reached :-disturbance compensation about a given nominal steady state point,-high convergence rate of the designed minimization algorithm when compared with the process time constants,-robustness of the control with regard to model errors and nominal working points

A family of methods with three‐term recursion formulae

AbstractA state‐of‐the‐art survey of a family of vector iterative methods with three‐term recursion formulae is presented. The methods can be classified under two broad categories. The Chebyshev semi‐iterative type of methods and the conjugate gradient type of methods. Relationships between the iteration parameters of the methods are established and computational techniques for estimating these parameters are presented. Numerical tests confirm the superiority of the conjugate gradient method for linear problems and the Chebyshev semi‐iterative methods for nonlinear problems.

Conjugate gradient type methods for unsymmetric and inconsistent systems of linear equations

Conjugate gradient type methods are discussed for unsymmetric and inconsistent system of equations. For unsymmetric problems, besides conjugate gradient methods based on the normal equations, we also present a (modified) minimal residual (least square) method, which converges for systems with matrices that have a positive definite symmetric part. For inconsistent problems, for completeness we discuss briefly various (well-known) versions of the conjugate gradient method. Preconditioning and rate of convergence are also discussed.

Sequential conjugate gradient‐restoration algorithm for optimal control problems with non‐differential constraints and general boundary conditions, part I

AbstractIn this paper, a new member of the family of sequential gradient‐restoration algorithms for the solution of optimal control problems is presented. This is an algorithm of the conjugate gradient type, which is designed to solve two classes of optimal control problems, called Problem P1 and Problem P2 for easy indentification.Problem P1 involves minimizing a functional I subject to differential constraints and general boundary conditions. It consists of finding the state x(t), the control u(t), and the parameter pi so that the functional I is minimized, while the constraints and the boundary conditions are satisfied to a predetermined accuracy. Problem P2 extends Problem P1 to include non‐differential constraints to be satisfied everywhere along the interval of integration.The approach taken is a sequence of two‐phase cycles, composed of a conjugate gradient phase and a restoration phase. The conjugate gradient phase involves one iteration and is designed to decrease the value of the functional, while the constraints are satisfied to first order. The restoration phase involves one or more iterations; each restorative iteration is designed to force constraint satisfaction to first order, while the norm squared of the variations of the control, the parameter, and the missing components of the initial state is minimized. The resulting algorithm has several properties: (i) it produces a sequence of feasible solutions; (ii) each feasible solution is characterized by a value of the functional I which is smaller than that associated with any previous feasible solution; and (iii) for the special case of a quadratic functional subject to linear constraints, the variations of the state, control, and parameter produced by the sequence of conjugate gradient phases satisfy various orthogonality and conjugacy conditions.The algorithm presented here differs from those of References 1‐4, in that it is not required that the state vector be given at the initial point. Instead, the initial conditions can be absolutely general. In analogy with References 1‐4, the present algorithm is capable of handling general final conditions; therefore, it is suitable for the solution of optimal control problems with general boundary conditions.The importance of the present algorithm lies in that many optimal control problems either arise naturally in the present format or can be brought to such a format by means of suitable transformations.5 Therefore, a great variety of optimal control problems can be handled. This includes: (i) problems with control equality constraints, (ii) problems with state equality constraints, (iii) problems with state‐derivative equality constraints, (iv) problems with‐control inequality constraints, (v) problems with state inequality constraints, (vi) problems with state‐derivative inequality constraints, and (vii) Chebyshev minimax problems.Several numerical examples are presented in Part 2 (Reference 6) in order to illustrate the performance of the algorithm associated with Problem P1 and Problem P2. The numerical results show the feasibility as well as the convergence characteristics of the present algorithm.

Numerical Methods for the First Biharmonic Equation and for the Two-Dimensional Stokes Problem

We describe in this report various methods, iterative and almost direct, for solving the first biharmonic problem on general two-dimensional domains once the continuous problem has been approximated by an appropriate mixed finite element method. Using the approach described in this report we recover some well known methods for solving the first biharmonic equation as a system of coupled harmonic equations, but some of the methods discussed here are completely new, including a conjugate gradient type algorithm. In the last part of this report we discuss the extension of the above methods to the numerical solution of the two dimensional Stokes problem in p- connected domains (p $\geq$ 1) through the stream function-vorticity formulation.