General Convex Constraints Research Articles

Interior Gradient and Proximal Methods for Convex and Conic Optimization

Interior gradient (subgradient) and proximal methods for convex constrained minimization have been much studied, in particular for optimization problems over the nonnegative octant. These methods are using non-Euclidean projections and proximal distance functions to exploit the geometry of the constraints. In this paper, we identify a simple mechanism that allows us to derive global convergence results of the produced iterates as well as improved global rates of convergence estimates for a wide class of such methods, and with more general convex constraints. Our results are illustrated with many applications and examples, including some new explicit and simple algorithms for conic optimization problems. In particular, we derive a class of interior gradient algorithms which exhibits an $O(k^{-2})$ global convergence rate estimate.

Interior projection-like methods for monotone variational inequalities

We propose new interior projection type methods for solving monotone variational inequalities. The methods can be viewed as a natural extension of the extragradient and hyperplane projection algorithms, and are based on using non Euclidean projection-like maps. We prove global convergence results and establish rate of convergence estimates. The projection-like maps are given by analytical formulas for standard constraints such as box, simplex, and conic type constraints, and generate interior trajectories. We then demonstrate that within an appropriate primal-dual variational inequality framework, the proposed algorithms can be applied to general convex constraints resulting in methods which at each iteration entail only explicit formulas and do not require the solution of any convex optimization problem. As a consequence, the algorithms are easy to implement, with low computational cost, and naturally lead to decomposition schemes for problems with a separable structure. This is illustrated through examples for convex programming, convex-concave saddle point problems and semidefinite programming.

A specially structured nonlinear integer resource allocation problem

AbstractWe present an algorithm for solving a specially structured nonlinear integer resource allocation problem. This problem was motivated by a capacity planning study done at a large Health Maintenance Organization in Texas. Specifically, we focus on a class of nonlinear resource allocation problems that involve the minimization of a convex function over one general convex constraint, a set of block diagonal convex constraints, and bounds on the integer variables. The continuous variable problem is also considered. The continuous problem is solved by taking advantage of the structure of the Karush‐Kuhn‐Tucker (KKT) conditions. This method for solving the continuous problem is then incorporated in a branch and bound algorithm to solve the integer problem. Various reoptimization results, multiplier bounding results, and heuristics are used to improve the efficiency of the algorithms. We show how the algorithms can be extended to obtain a globally optimal solution to the nonconvex version of the problem. We further show that the methods can be applied to problems in production planning and financial optimization. Extensive computational testing of the algorithms is reported for a variety of applications on continuous problems with up to 1,000,000 variables and integer problems with up to 1000 variables. © 2003 Wiley Periodicals, Inc. Naval Research Logistics 50: 770–792, 2003.

Minimizing the total variation under a general convex constraint for image restoration

In this paper, we present a general framework for image restoration; despite its simplicity, certain variational and certain wavelet approaches can be formulated within this framework. This permits the construction of a natural model, with only one parameter, which has the advantages of both approaches. We give a mathematical analysis of this model, describe our algorithm and illustrate this by some experiments.

A new look at entropy for solving linear inverse problems

Entropy-based methods are widely used for solving inverse problems, particularly when the solution is known to be positive. Here, we address linear ill-posed and noisy inverse problems of the form z=Ax+n with a general convex constraint x/spl isin/X, where X is a convex set. Although projective methods are well adapted to this context, we study alternative methods which rely highly on some "information-based" criteria. Our goal is to clarify the role played by entropy in this field, and to present a new point of view on entropy, using general tools and results coming from convex analysis. We present then a new and broad scheme for entropic-based inversion of linear-noisy inverse problems. This scheme was introduced by Navaza in 1985 in connection with a physical modeling for crystallographic applications, and further studied by Dacunha-Castelle and Gamboa (1990). Important features of this paper are: (i) a unified presentation of many well-known reconstruction criteria, (ii) proposal of new criteria for reconstruction under various prior knowledge and with various noise statistics, (iii) a description of practical inversion of data using the aforementioned criteria, and (iv) a presentation of some reconstruction results.

Parallel proximal decomposition algorithmsfor robust estimation

In the past few years, robustness has been one problem that was given much attention inthe statistical literature. While it is now clear that no single robust regression procedure isbest (by mean square error or other adequate criteria), the LAV (least absolute value) andthe Huber‐M estimators are currently attracting considerable attention when the errors havea contaminated Gaussian or long‐tail distribution. Finding efficient algorithms to producesuch estimates in the case of large data sets is still a field of active research. In this paper,we present algorithms based on the Spingarn Partial Inverse proximal approach that takesinto account both primal and dual aspects of the related optimization problems. They can beviewed as decomposition methods. Known to be always globally convergent, such an alternativeiterative approach leads to simple computational steps and updating rules. The resultis a highly parallel algorithm particularly attractive for large‐scale problems. Its efficientimplementation on a parallel computer architecture is described. Remedies are introducedto ensure efficiency in the case of models with less than full ranks. Numerical simulationsare considered and computational performance reported. Finally, we show how the methodallows for easy handling of general convex constraints on the primal variables. We discussin detail a variety of linear and nonlinear restrictions. The case of ridge LAV and Huber‐Mregression is specifically considered.

On the Convergence of Constrained Parallel Variable Distribution Algorithms

We consider the parallel variable distribution (PVD) approach proposed by Ferris and Mangasarian [SIAM J. Optim., 4 (1994), pp. 815--832] for solving optimization problems. The problem variables are distributed among p processors with each processor having the primary responsibility for updating its block of variables while allowing the remaining "secondary" variables to change in a restricted fashion along some easily computable directions. For constrained nonlinear programs, convergence in [M. C. Ferris and O. L. Mangasarian, SIAM J. Optim., 4 (1994), pp. 815--832] was established in the special case of convex block-separable constraints. For general (inseparable) constraints, it was suggested that a dual differentiable exact penalty function reformulation of the problem be used. We propose to apply the PVD approach to problems with general convex constraints directly and show that the algorithm converges, provided certain conditions are imposed on the change of secondary variables. These conditions are both natural and practically implementable. We also show that the original requirement of exact global solution of the parallel subproblems can be replaced by a less stringent sufficient descent condition. The first rate of convergence result for the class of constrained PVD algorithms is also given.

A New Merit Function and a Successive Quadratic Programming Algorithm for Variational Inequality Problems

Recently, various merit functions for variational inequality problems have been proposed and their properties have been studied. Unfortunately, these functions may not be easy to evaluate unless the constraints of the problem have a relatively simple structure. In this paper, a new merit function for variational inequality problems with general convex constraints is proposed. At each point, the proposed function is defined as an optimal value of a quadratic programming problem whose constraints consist of a linear approximation of the given nonlinear constraints. It is shown that the set of constrained minima of the proposed merit function coincides with the set of solutions to the original variational inequality problem. It is also shown that this function is directionally differentiable in all directions and, under suitable assumptions, any stationary point of the function over the constraint set actually solves the original variational inequality problem. Finally, a descent method for solving the variational inequality problem is proposed and its convergence is proved. The method is closely related to a successive quadratic programming method for solving nonlinear programming problems.

Primal-dual row-action method for convex programming

We present a primal-dual row-action method for the minimization of a convex function subject to general convex constraints. Constraints are used one at a time, no changes are made in the constraint functions and their Jacobian matrix (thus, the row-action nature of the algorithm), and at each iteration a subproblem is solved consisting of minimization of the objective function subject to one or two linear equations. The algorithm generates two sequences: one of them, called primal, converges to the solution of the problem; the other one, called dual, approximates a vector of optimal KKT multipliers for the problem. We prove convergence of the primal sequence for general convex constraints. In the case of linear constraints, we prove that the primal sequence converges at least linearly and obtain as a consequence the convergence of the dual sequence.

A row-action method for convex programming

We present a new method for minimizing a strictly convex function subject to general convex constraints. Constraints are used one at a time, no changes are made in the constraint functions (thus the row-action nature of the algorithm) and at each iteration a subproblem is solved consisting of minimization of the objective function subject to one or two linear equations. Convergence of the algorithm is established and the method is compared with other row-action algorithms for several relevant particular cases.

Convergence properties of trust region methods for linear and convex constraints

We develop a convergence theory for convex and linearly constrained trust region methods which only requires that the step between iterates produce a sufficient reduction in the trust region subproblem. Global convergence is established for general convex constraints while the local analysis is for linearly constrained problems. The main local result establishes that if the sequence converges to a nondegenerate stationary point then the active constraints at the solution are identified in a finite number of iterations. As a consequence of the identification properties, we develop rate of convergence results by assuming that the step is a truncated Newton method. Our development is mainly geometrical; this approach allows the development of a convergence theory without any linear independence assumptions.