Solving Convex Programming Problems Research Articles

Solving the smallest ball problem based on convex programming theories

The mathematical method known to be Quadratic Programming is a branch of Convex Programming or Convex Optimization, which is then a peculiar case of Mathematical Optimization given a series of restrictions including a convex function to minimize. The Smallest Ball Problem is a problem where we seek to find the smallest enclosing ball of given points on a plane. Convex Optimization provides a solution to the Smallest Ball Problem. There are several ways to characterize a convex programming problem and its solutions, the most important of which is called the KKT conditions. By relating the Smallest Ball Problem to solving a Convex Programming Problem and using the Python packages, the radius, as well as the central point of the Smallest Ball, will be found. In addition, the underlying algorithm for solving Convex Programming Problems is studied. It can be concluded that Convex Programming, or more specifically, Quadratic Programming, gives a feasible solution to the Smallest Ball Problem.

Differential Linear Matrix Inequalities Optimization

This letter proposes a new method to solve convex programming problems with constraints expressed by differential linear matrix inequalities (DLMIs). Initially, feasible solutions of interest are characterized and a general numerical method, based on the well known outer linearization technique, is proposed and discussed from theoretical and numerical viewpoints. Feasible solutions are written as a truncated series of a given set of time valued continuous functions with symmetric matrix coefficients to be determined. The numerical method encompasses the piecewise linear solution usually adopted in the literature with lower computational burden. In the sequel, several sampled-data control design problems whose optimality conditions can be expressed in this mathematical framework are provided. They are solved in order to put in evidence the most important aspects of the proposed method as well as to evaluate and compare numerical efficiency and limitations. Moreover, it is shown that DLMIs are particularly well adapted to cope with this class of optimal control design problems.

Optimal proximal augmented Lagrangian method and its application to full Jacobian splitting for multi-block separable convex minimization problems

Abstract The augmented Lagrangian method (ALM) is fundamental in solving convex programming problems with linear constraints. The proximal version of ALM, which regularizes ALM’s subproblem over the primal variable at each iteration by an additional positive-definite quadratic proximal term, has been well studied in the literature. In this paper we show that it is not necessary to employ a positive-definite quadratic proximal term for the proximal ALM and the convergence can be still ensured if the positive definiteness is relaxed to indefiniteness by reducing the proximal parameter. An indefinite proximal version of the ALM is thus proposed for the generic setting of convex programming problems with linear constraints. We show that our relaxation is optimal in the sense that the proximal parameter cannot be further reduced. The consideration of indefinite proximal regularization is particularly meaningful for generating larger step sizes in solving ALM’s primal subproblems. When the model under discussion is separable in the sense that its objective function consists of finitely many additive function components without coupled variables, it is desired to decompose each ALM’s subproblem over the primal variable in Jacobian manner, replacing the original one by a sequence of easier and smaller decomposed subproblems, so that parallel computation can be applied. This full Jacobian splitting version of the ALM is known to be not necessarily convergent, and it has been studied in the literature that its convergence can be ensured if all the decomposed subproblems are further regularized by sufficiently large proximal terms. But how small the proximal parameter could be is still open. The other purpose of this paper is to show the smallest proximal parameter for the full Jacobian splitting version of ALM for solving multi-block separable convex minimization models.

The Proximal Alternating Minimization Algorithm for Two-Block Separable Convex Optimization Problems with Linear Constraints

The Alternating Minimization Algorithm has been proposed by Paul Tseng to solve convex programming problems with two-block separable linear constraints and objectives, whereby (at least) one of the components of the latter is assumed to be strongly convex. The fact that one of the subproblems to be solved within the iteration process of this method does not usually correspond to the calculation of a proximal operator through a closed formula affects the implementability of the algorithm. In this paper, we allow in each block of the objective a further smooth convex function and propose a proximal version of the algorithm, which is achieved by equipping the algorithm with proximal terms induced by variable metrics. For suitable choices of the latter, the solving of the two subproblems in the iterative scheme can be reduced to the computation of proximal operators. We investigate the convergence of the proposed algorithm in a real Hilbert space setting and illustrate its numerical performances on two applications in image processing and machine learning.

A Schur complement based semi-proximal ADMM for convex quadratic conic programming and extensions

This paper is devoted to the design of an efficient and convergent {semi-proximal} alternating direction method of multipliers (ADMM) for finding a solution of low to medium accuracy to convex quadratic conic programming and related problems. For this class of problems, the convergent two block semi-proximal ADMM can be employed to solve their primal form in a straightforward way. However, it is known that it is more efficient to apply the directly extended multi-block semi-proximal ADMM, though its convergence is not guaranteed, to the dual form of these problems. Naturally, one may ask the following question: can one construct a convergent multi-block semi-proximal ADMM that is more efficient than the directly extended semi-proximal ADMM? Indeed, for linear conic programming with 4-block constraints this has been shown to be achievable in a recent paper by Sun, Toh and Yang [arXiv preprint arXiv:1404.5378, (2014)]. Inspired by the aforementioned work and with the convex quadratic conic programming in mind, we propose a Schur complement based convergent semi-proximal ADMM for solving convex programming problems, with a coupling linear equality constraint, whose objective function is the sum of two proper closed convex functions plus an arbitrary number of convex quadratic or linear functions. Our convergent semi-proximal ADMM is particularly suitable for solving convex quadratic semidefinite programming (QSDP) with constraints consisting of linear equalities, a positive semidefinite cone and a simple convex polyhedral set. The efficiency of our proposed algorithm is demonstrated by numerical experiments on various examples including QSDP.

A proximal point algorithm revisit on the alternating direction method of multipliers

The alternating direction method of multipliers (ADMM) is a benchmark for solving convex programming problems with separable objective functions and linear constraints. In the literature it has been illustrated as an application of the proximal point algorithm (PPA) to the dual problem of the model under consideration. This paper shows that ADMM can also be regarded as an application of PPA to the primal model with a customized choice of the proximal parameter. This primal illustration of ADMM is thus complemental to its dual illustration in the literature. This PPA revisit on ADMM from the primal perspective also enables us to recover the generalized ADMM proposed by Eckstein and Bertsekas easily. A worst-case O(1/t) convergence rate in ergodic sense is established for a slight extension of Eckstein and Bertsekas’s generalized ADMM.

An application of a merit function for solving convex programming problems

This paper presents a gradient neural network model for solving convex nonlinear programming (CNP) problems. The main idea is to convert the CNP problem into an equivalent unconstrained minimization problem with objective energy function. A gradient model is then defined directly using the derivatives of the energy function. It is also shown that the proposed neural network is stable in the sense of Lyapunov and can converge to an exact optimal solution of the original problem. It is also found that a larger scaling factor leads to a better convergence rate of the trajectory. The validity and transient behavior of the neural network are demonstrated by using various examples.

Delayed Lagrangian neural networks for solving convex programming problems

In this paper, the global convergence is further studied for the Lagrangian programming neural networks. A delayed Lagrangian programming neural network is proposed for solving a class of convex programming problems with equality constraints. Based on Lyapunov method, it is proved that the delayed Lagrangian networks are stable and globally convergent under some conditions. Simulation examples are provided to show that the Lagrangian networks with delay are more effective than that without delays by choosing proper delays.

Convergence of a numerical abstract convexity algorithm

We propose a series of algorithms for solving abstract convex programming problems and prove convergence to the global solution of the problem. The algorithms use an approximation of the objective function by piecewise-linear minorants.

Two methods for minimizing convex functions in a class of nonconvex sets

The conditional gradient method and the steepest descent method, which are conventionally used for solving convex programming problems, are extended to the case where the feasible set is the set-theoretic difference between a convex set and the union of several convex sets. Iterative algorithms are proposed, and their convergence is examined.

A note on finite termination of iterative algorithms in mathematical programming

Two well-known results on the unified treatment of finite termination of a class of algorithms for solving convex programming problems and for solving variational inequality problems are reconsidered. In particular, some of the underlying assumptions employed in the existing literature are shown to be redundant.

Application of projection neural network in solving convex programming problems

In this paper we present that solution of convex programming problems is equivalent with solution of projection formulation, then we introduce neural network models for solving projection formulation and analysis stability conditions and convergence. Simulation shows that the introduced neural network is effective in solving convex programming problems.

Logic-Based Solution Methods for Optimal Control of Hybrid Systems

Combinatorial optimization over continuous and integer variables is a useful tool for solving complex optimal control problems of dynamical systems formulated in discrete-time. Current approaches are based on mixed-integer linear (or quadratic) programming (MIP), which provides the solution after solving a sequence of relaxed linear (or quadratic) programs. MIP formulations require the translation of the discrete/logic part of the problem into mixed-integer inequalities. Although this operation can be done automatically, most of the original symbolic structure of the problem (e.g., transition functions of finite state machines, logic constraints, symbolic variables, etc.) is lost during the conversion, with a consequent loss of computational performance. In this paper, we attempt to overcome such a difficulty by combining numerical techniques for solving convex programming problems with symbolic techniques for solving constraint satisfaction problems (CSP). The resulting hybrid solver proposed here takes advantage of CSP solvers for dealing with satisfiability of logic constraints very efficiently. We propose a suitable model of the dynamics and a class of optimal control problems that embrace both symbolic and continuous variables/functions, and that are tailored to the use of the new solver. The superiority in terms of computational performance with respect to commercial MIP solvers is shown on a centralized supply chain management problem with uncertain forecast demand.

Solving limit analysis problems: an interior-point method

This paper exposes an interior-point method used to solve convex programming problems raised by limit analysis in mechanics. First we explain the main features of this method, describing in particular its typical iteration. Secondly, we show and study the results of its application to a concrete limit analysis problem, for a large range of sizes, and we compare them for validation with existing results and with those of linearized versions of the problem. As one of the objectives of the work, another classical problem is analysed for a Gurson material, to which linearization or conic programming does not apply. Copyright © 2005 John Wiley & Sons, Ltd.

A Recurrent Neural Network for Nonlinear Convex Optimization Subject to Nonlinear Inequality Constraints

This paper presents a novel recurrent neural network for solving nonlinear convex programming problems subject to nonlinear inequality constraints. Under the condition that the objective function is convex and all constraint functions are strictly convex or that the objective function is strictly convex and the constraint function is convex, the proposed neural network is proved to be stable in the sense of Lyapunov and globally convergent to an exact optimal solution. Compared with the existing neural networks for solving such nonlinear optimization problems, the proposed neural network has two major advantages. One is that it can solve convex programming problems with general convex inequality constraints. Another is that it does not require a Lipschitz condition on the objective function and constraint function. Simulation results are given to illustrate further the global convergence and performance of the proposed neural network for constrained nonlinear optimization.

A combination of potential reduction steps and steepest descent steps for solving convex programming problems

A combination of potential reduction steps and steepest descent steps for solving convex programming problems

A combination of potential reduction steps and steepest descent steps for solving convex programming problems

AbstractThis paper studies the possibility of combining interior point strategy with a steepest descent method when solving convex programming problems, in such a way that the convergence property of the interior point method remains valid but many iterations do not request the solution of a system of equations. Motivated by this general idea, we propose a hybrid algorithm which combines a primal–dual potential reduction algorithm with the use of the steepest descent direction of the potential function. The $O(\sqrt{n}\left|{\ln {\epsilon}} \right|)$ complexity of the potential reduction algorithm remains valid but the overall computational cost can be reduced. Our numerical experiments are also reported. Copyright © 2002 John Wiley & Sons, Ltd.

Nonlinear Proximal Decomposition Method for Convex Programming

In this paper, we propose a new decomposition method for solving convex programming problems with separable structure. The proposed method is based on the decomposition method proposed by Chen and Teboulle and the nonlinear proximal point algorithm using the Bregman function. An advantage of the proposed method is that, by a suitable choice of the Bregman function, each subproblem becomes essentially the unconstrained minimization of a finite-valued convex function. Under appropriate assumptions, the method is globally convergent to a solution of the problem.

Solving convex programs with infinitely many linear constraints by a relaxed cutting plane method

One of the major computational bottlenecks of using the conventional cutting plane approach to solve convex programming problems with infinitely many linear constraints lies in finding a global optimizer of a nonlinear and nonconvex program. This paper presents a relaxed scheme to generate a new cut. In each iteration, the proposed scheme chooses a point at which the constraints are violated to a degree rather than at which the violation is maximized. A convergence proof is provided. The proposed scheme also exhibits the capability of generating an approximate solution to any level of accuracy in a finite number of iterations.

Solving convex programming problems with equality constraints by neural networks

This paper presents a neural network approach for solving convex programming problems with equality constraints. After defining the energy function and neural dynamics of the proposed neural network, we show the existence of an equilibrium point at which the neural dynamics becomes asymptotically stable. It is shown that under proper conditions, an optimal solution of the underlying convex programming problems is an equilibrium point of the neural dynamics, and vise versa. The configuration of the proposed neural network with an exact layout is provided for solving linear programming problems. The operational characteristics of the neural network are demonstrated by numerical examples.