Row-action Methods Research Articles

Linearly convergent descent methods for the unconstrained minimization of convex quadratic splines

We propose two linearly convergent descent methods for finding a minimizer of a convex quadratic spline and establish global error estimates for the iterates. One application of such descent methods is to solve convex quadratic programs, since they can be reformulated as problems of unconstrained minimization of convex quadratic splines. In particular, we derive several new linearly convergent algorthms for solving convex quadratic programs. These algorithms could be classified as row-action methods, matrix-splitting methods, and Newton-type methods.

Primal-dual row-action method for convex programming

We present a primal-dual row-action method for the minimization of a convex function subject to general convex constraints. Constraints are used one at a time, no changes are made in the constraint functions and their Jacobian matrix (thus, the row-action nature of the algorithm), and at each iteration a subproblem is solved consisting of minimization of the objective function subject to one or two linear equations. The algorithm generates two sequences: one of them, called primal, converges to the solution of the problem; the other one, called dual, approximates a vector of optimal KKT multipliers for the problem. We prove convergence of the primal sequence for general convex constraints. In the case of linear constraints, we prove that the primal sequence converges at least linearly and obtain as a consequence the convergence of the dual sequence.

A parallel relaxation method for quadratic programming problems with interval constraints

Optimization problems with interval constraints are encountered in various fields such as network flows and computer tomography. As these problems are usually very large, they are not easy to solve without taking their sparsity into account. Recently “row-action methods”, which originate from the classical Hildreth's method for quadratic programming problems, have drawn much attention, since they are particularly useful for large and sparse problems. Various row-action methods have already been proposed for optimization problems with interval constraints, but they mostly belong to the class of sequential methods based on the Gauss-Seidel and SOR methods. In this paper, we propose a highly parallelizable method for solving those problems, which may be regarded as an application of the Jacobi method to the dual of the original problems. We prove convergence of the algorithm and report some computational results to demonstrate its effectiveness.

A row-action method for convex programming

We present a new method for minimizing a strictly convex function subject to general convex constraints. Constraints are used one at a time, no changes are made in the constraint functions (thus the row-action nature of the algorithm) and at each iteration a subproblem is solved consisting of minimization of the objective function subject to one or two linear equations. Convergence of the algorithm is established and the method is compared with other row-action algorithms for several relevant particular cases.

A Massively Parallel Algorithm for Nonlinear Stochastic Network Problems

We develop an algorithm for solving nonlinear, two-stage stochastic problems with network recourse. The algorithm is based on the framework of row-action methods. The problem is formulated by replicating the first-stage variables and then adding nonanticipativity side constraints. A series of (independent) deterministic network problems are solved at each step of the algorithm, followed by an iterative step over the nonanticipativity constraints. The solution point of the iterates over the nonanticipativity constraints is obtained analytically. The row-action nature of the algorithm makes it suitable for parallel implementations. A data representation of the problem is developed that permits the massively parallel solution of all the scenario subproblems concurrently. The algorithm is implemented on a Connection Machine CM-2 with up to 32K processing elements and achieves computing rates of 276 MFLOPS. Very large problems—8,192 scenarios with a deterministic equivalent nonlinear program with 868,367 constraints and 2,474,017 variables—are solved within a few minutes. We report extensive numerical results regarding the effects of stochasticity on the efficiency of the algorithm.

On the use of conjugate gradient to calculate the eigenvalues and singular values of large, sparse matrices

Inverse problems such as tomography involve the solution of very large, sparse linear systems. This can be done efficiently using a variety of iterative techniques such as conjugate gradient or the row action methods. But these methods give no information about the spectrum of the matrix, which would be useful in analysing convergence and the reliability of the solution. On the other hand, direct orthogonalization methods such as SVD cause unacceptable fill in the matrix. the purpose of this note is to point out a result which is reasonably well known in the numerical linear algebra community but which does not appear to be appreciated by practitioners of inverse theory, namely that the intermediate results of the conjugate gradient algorithm can be used to generate a symmetric tridiagonalization of the underlying matrix. This tridiagonalization can be exploited to give an efficient algorithm for computing the eigenvalues of symmetric matrices or the singular values of arbitrary rectangular matrices, by computing the eigenvalues of the related symmetric tridiagonal matrix—taking full advantage of the sparsity. an example will be given of the computation of the singular values of a 1500 × 400 matrix originating in seismic reflection tomography.

Numerical study of higher-order diffraction tomography via the sinc basis moment method

Behavior and limitations of the sinc basis method have been discovered during simulation studies. An iterative row-action method (ART) is compared with a full-matrix, least-squares solution (QR decomposition) and the problem of multiple solutions is discussed. Knowledge of the spatial distribution of scattered field energy can be used to speed up convergence. The first iteration is shown to be equivalent to the first Born solution. A fundamental limitation of present diffraction tomography algorithms, concerning phase shift through the scatterer, is found and related to clinical values. Effects of improvements on the initial guesses for the object function and internal field are studied. Lossy cylinder and contrived multicomponent reconstructions yield additional understanding of the phase shift problem. A sequence of simulations investigates estimated solution movement in hyperspace.

Numerical study of higher-order diffraction tomography via the sinc basis moment method.

Behavior and limitations of the sinc basis method have been discovered during simulation studies. An iterative row-action method (ART) is compared with a full-matrix, least-squares solution (QR decomposition) and the problem of multiple solutions is discussed. Knowledge of the spatial distribution of scattered field energy can be used to speed up convergence. The first iteration is shown to be equivalent to the first Born solution. A fundamental limitation of present diffraction tomography algorithms, concerning phase shift through the scatterer, is found and related to clinical values. Effects of improvements on the initial guesses for the object function and internal field are studied. Lossy cylinder and contrived multicomponent reconstructions yield additional understanding of the phase shift problem. A sequence of simulations investigates estimated solution movement in hyperspace.

Large‐scale waveform inversions of surface waves for lateral heterogeneity: 1. Theory and numerical examples

Surface wave scattering theory is presented as a new method for analyzing teleseismic surface wave data. Using surface wave scattering integrals the effect of lateral heterogeneity both on the surface wave coda generation and on the direct surface wave is described. Since the employed scattering theory for the forward problem is linear, the inverse problem can conveniently be solved in the least squares sense using an iterative matrix solver. For waveform inversions of the direct surface wave, only near forward scattering contributes. For this case the isotropic approximation is introduced, which makes it possible to retrieve phase velocity information from scattering theory. It is shown that for practical waveform inversions the resulting system of linear equations is extremely large and how row action methods can be used conveniently for carrying out the inversion on moderate size computers. The performance of the inversions is illustrated with two numerical examples. In the first example the surface wave coda generated by one point scatterer is inverted. It is shown that the reconstruction in this case is similar to Kirchhoff migration methods as used in exploration seismics. In the second example, ray geometrical effects (focusing and phase shifting) are obtained from the linear inversion with scattering theory. It follows from this example that linear waveform inversion can simultaneously fit the amplitude and the phase of surface wave data.

On the use of Cimmino's simultaneous projections method for computing a solution of the inverse problem in radiation therapy treatment planning

In radiation therapy one is confronted with the task of formulating a treatment plan which delivers a specified dose to a tumour but avoids irreparable damage to surrounding uninvolved structures. Radiation therapy treatment planning (RTTP) involves an inverse and a forward problem. The inverse problem is to devise a treatment plan, i.e. a radiation beam configuration and beam weighting, which provides a specified dose distribution to the delineated region. The forward problem is to calculate the dose distribution within the patient that results from the weighted radiation beam configuration. Since no analytic closed-form mathematical formulation of the forward operator exists, the inverse problem actually calls for computerised inversion of data. This inversion is achieved by constructing a fully discretised model that leads to a system of linear inequalities. These inequalities are solved either by a row-action method or a block-Cimmino algorithm which allows the assignment of weights within each block of inequalities.

Parallel application of block-iterative methods in medical imaging and radiation therapy

Some row-action algorithms which exploit special objective function and constraints structure have proven advantageous for solving huge and sparse feasibility or optimization problems. Recently developed block-iterative versions of such special-purpose methods enable parallel computation when the underlying problem is appropriately decomposed. This opens the door for parallel computation in image reconstruction problems of computerized tomography and in the inverse problem of radiation therapy treatment planning, all in their fully discretized modelling approach. Since there is more than one way of deriving block-iterative versions of any row-action method, the choice has to be made with reference to the underlying real-world problem.

On Block-Iterative Entropy Maximization

Abstract Entropy maximization under linear equality constraints is considered and convergence of a special-purpose block-iterative algorithm for its solution is investigated. This algorithm processes in each iterative step a group of constraints. When each block contains a single equation then a well known row-action method, called MART (for “Multiplicative Algebraic Reconstruction Technique”), is obtained. At the other extreme, when all equations are grouped into a single block, a fully simultaneous entropy maximization algorithm is obtained. In the general ease the algorithm permits assignment of weights to the equations within each block to reflect any a priori knowledge, which might be available from the modelling of some real-world process, about their relative importance. The blocks are processed in an almost cyclic order and, in contrast with Bregman’s method, the iterative step does not require inner-loop iterations but is rather represented by a closed form formula. The iterative step of the algo...

Row-Action Methods for Huge and Sparse Systems and Their Applications

This paper brings together and discusses theory and applications of methods, identified and labelled as row-action methods, for linear feasibility problems (find $x \in {\bf R}^n $, such that $Ax \leqq b$), linearly constrained optimization problems (minimize $f(x)$, subject to $Ax \leqq b$) and some interval convex programming problems (minimize $f(x)$, subject to $c \leqq Ax \leqq b$). The main feature of row-action methods is that they are iterative procedures which, without making any changes to the original matrix A, use the rows of A, one row at a time. Such methods are important and have demonstrated effectiveness for problems with large or huge matrices which do not enjoy any detectable or usable structural pattern, apart from a high degree of sparaseness. Fields of application where row-action methods are used in various ways include image reconstruction from projection, operations research and game theory, learning theory, pattern recognition and transportation theory. A row-action method for the nonlinear convex feasibility problem is also presented.

An iterative row-action method for interval convex programming

The iterative primal-dual method of Bregman for solving linearly constrained convex programming problems, which utilizes nonorthogonal projections onto hyperplanes, is represented in a compact form, and a complete proof of convergence is given for an almost cyclic control of the method. Based on this, a new algorithm for solving interval convex programming problems, i.e., problems of the form minf(x), subject to γ≤Ax≤δ, is proposed. For a certain family of functionsf(x), which includes the norm ∥x∥ and thex logx entropy function, convergence is proved. The present row-action method is particularly suitable for handling problems in which the matrixA is large (or huge) and sparse.

Extensions of Hildreth’s Row-Action Method for Quadratic Programming

An extended version of Hildreth’s iterative quadratic programming algorithm is presented, geometrically interpreted, and proved to produce a sequence of iterates that (i) converges to the solution, and (ii) has an important intermediate optimality property. This extended Hildreth algorithm is cast into a new form which more pronouncedly brings out its primal-dual nature. The application of the algorithm may be governed by an index sequence which is more general than a cyclic sequence, namely, by an almost cyclic control, and a sequence of relaxation parameters is incorporated without ruining convergence. The algorithm is a row-action method which is particularly suitable for handling large (or huge) and sparse systems.