Separable Convex Programming Research Articles

An alternating direction-based contraction method for linearly constrained separable convex programming problems

The classical alternating direction method (ADM) has been well studied in the context of linearly constrained convex programming and variational inequalities where the involved operator is formed as the sum of two individual functions without crossed variables. Recently, ADM has found many novel applications in diversified areas, such as image processing and statistics. However, it is still not clear whether ADM can be extended to the case where the operator is the sum of more than two individual functions. In this article, we extend the spirit of ADM to solve the general case of the linearly constrained separable convex programming problems whose involved operator is separable into finitely many individual functions. As a result, an alternating direction-based contraction-type method is developed. The realization of tackling this class of problems broadens the applicable scope of ADM substantially.

A Parallel Splitting Method for Separable Convex Programs

In this paper, we propose a new parallel splitting augmented Lagrangian method for solving the nonlinear programs where the objective function is separable with three operators and the constraint is linear. The method is an improvement of the method of He (Comput. Optim. Appl., 2(42):195–212, 2009), where we generate a predictor using the same parallel splitting augmented Lagrangian scheme as that in He (Comput. Optim. Appl., 2(42):195–212, 2009), while adopting a new strategy to get the next iterate. Under the mild assumptions of convexity of the underlying mappings and the non-emptiness of the solution set, we prove that the proposed algorithm is globally convergent. We apply the new method in the area of image processing and to solve some quadratic programming problems. The preliminary numerical results indicate that the new method is efficient.

Convergence Analysis of Alternating Direction Method of Multipliers for a Class of Separable Convex Programming

The purpose of this paper is extending the convergence analysis of Han and Yuan (2012) for alternating direction method of multipliers (ADMM) from the strongly convex to a more general case. Under the assumption that the individual functions are composites of strongly convex functions and linear functions, we prove that the classical ADMM for separable convex programming with two blocks can be extended to the case with more than three blocks. The problems, although still very special, arise naturally from some important applications, for example, route-based traffic assignment problems.

An Inexact Perturbed Path-Following Method for Lagrangian Decomposition in Large-Scale Separable Convex Optimization

This paper studies an inexact perturbed path-following algorithm in the framework of Lagrangian dual decomposition for solving large-scale separable convex programming problems. Unlike the exact versions considered in the literature, we propose solving the primal subproblems inexactly up to a given accuracy. This leads to an inexactness of the gradient vector and the Hessian matrix of the smoothed dual function. Then an inexact perturbed algorithm is applied to minimize the smoothed dual function. The algorithm consists of two phases, and both make use of the inexact derivative information of the smoothed dual problem. The convergence of the algorithm is analyzed, and the worst-case complexity is estimated. As a special case, an exact path-following decomposition algorithm is obtained and its worst-case complexity is given. Implementation details are discussed, and preliminary numerical results are reported.

Linearized alternating direction method of multipliers with Gaussian back substitution for separable convex programming

Recently, we have proposed combining the alternating direction method of multipliers (ADMM) with a Gaussian back substitution procedure for solving the convex minimization model with linear constraints and a general separable objective function, i.e., the objective function is the sum of many functions without coupled variables. In this paper, we further study this topic and show that the decomposed subproblems in the ADMM procedure can be substantially alleviated by linearizing the involved quadratic terms arising from the augmented Lagrangian penalty. When the resolvent operators of the separable functions in the objective have closed-form representations, embedding the linearization into the ADMM subproblems becomes necessary to yield easy subproblems with closed-form solutions. We thus show theoretically that the blend of ADMM, Gaussian back substitution and linearization works effectively for the separable convex minimization model under consideration.

On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks

We consider a class of linearly constrained separable convex programming problems whose objective functions are the sum of three convex functions without coupled variables. For those problems, Han and Yuan (2012) have shown that the sequence generated by the alternating direction method of multipliers (ADMM) with three blocks converges globally to their KKT points under some technical conditions. In this paper, a new proof of this result is found under new conditions which are much weaker than Han and Yuan’s assumptions. Moreover, in order to accelerate the ADMM with three blocks, we also propose a relaxed ADMM involving an additional computation of optimal step size and establish its global convergence under mild conditions.

A new partial splitting augmented Lagrangian method for minimizing the sum of three convex functions

In this paper, we propose a new partial splitting augmented Lagrangian method for solving the separable constrained convex programming problem where the objective function is the sum of three separable convex functions and the constraint set is also separable into three parts. The proposed algorithm combines the alternating direction method (ADM) and parallel splitting augmented Lagrangian method (PSALM), where two operators are handled by a parallel method, while the third operator and the former two are dealt with by an alternating manner. Under mild conditions, we prove the global convergence of the new method. We also report some preliminary numerical results on constrained matrix optimization problem, illustrating the advantage of the new algorithm over the most recently PADALM of Peng and Wu (2010) [12].

An ADM-based splitting method for separable convex programming

We consider the convex minimization problem with linear constraints and a block-separable objective function which is represented as the sum of three functions without coupled variables. To solve this model, it is empirically effective to extend straightforwardly the alternating direction method of multipliers (ADM for short). But, the convergence of this straightforward extension of ADM is still not proved theoretically. Based on ADM's straightforward extension, this paper presents a new splitting method for the model under consideration, which is empirically competitive to the straightforward extension of ADM and meanwhile the global convergence can be proved under standard assumptions. At each iteration, the new method corrects the output of the straightforward extension of ADM by some slight correction computation to generate a new iterate. Thus, the implementation of the new method is almost as easy as that of ADM's straightforward extension. We show the numerical efficiency of the new method by some applications in the areas of image processing and statistics.

Efficient methods for selfish network design

Intuitively, Braess’s paradox states that destroying a part of a network may improve the common latency of selfish flows at Nash equilibrium. Such a paradox is a pervasive phenomenon in real-world networks. Any administrator who wants to improve equilibrium delays in selfish networks, is facing some basic questions:–Is the network paradox-ridden?–How can we delete some edges to optimize equilibrium flow delays?–How can we modify edge latencies to optimize equilibrium flow delays? Unfortunately, such questions lead to NP-hard problems in general. In this work, we impose some natural restrictions on our networks, e.g. we assume strictly increasing linear latencies. Our target is to formulate efficient algorithms for the three questions above. We manage to provide: –A polynomial-time algorithm that decides if a network is paradox-ridden, when latencies are linear and strictly increasing.–A reduction of the problem of deciding if a network with (arbitrary) linear latencies is paradox-ridden to the problem of generating all optimal basic feasible solutions of a Linear Program that describes the optimal traffic allocations to the edges with constant latency.–An algorithm for finding a subnetwork that is almost optimal wrt equilibrium latency. Our algorithm is subexponential when the number of paths is polynomial and each path is of polylogarithmic length.–A polynomial-time algorithm for the problem of finding the best subnetwork which outperforms any known approximation for the case of strictly increasing linear latencies.–A polynomial-time method that turns the optimal flow into a Nash flow by deleting the edges not used by the optimal flow, and performing minimal modifications on the latencies of the remaining ones. Our results provide a deeper understanding of the computational complexity of recognizing the most severe manifestations of Braess’s paradox, and our techniques show novel ways of using the probabilistic method and of exploiting convex separable quadratic programs.

A Note on the Alternating Direction Method of Multipliers

We consider the linearly constrained separable convex programming, whose objective function is separable into m individual convex functions without coupled variables. The alternating direction method of multipliers has been well studied in the literature for the special case m=2, while it remains open whether its convergence can be extended to the general case m≥3. This note shows the global convergence of this extension when the involved functions are further assumed to be strongly convex.

Inverse optimization for linearly constrained convex separable programming problems

Abstract In this paper, we study inverse optimization for linearly constrained convex separable programming problems that have wide applications in industrial and managerial areas. For a given feasible point of a convex separable program, the inverse optimization is to determine whether the feasible point can be made optimal by adjusting the parameter values in the problem, and when the answer is positive, find the parameter values that have the smallest adjustments. A sufficient and necessary condition is given for a feasible point to be able to become optimal by adjusting parameter values. Inverse optimization formulations are presented with l 1 and l 2 norms. These inverse optimization problems are either linear programming when l 1 norm is used in the formulation, or convex quadratic separable programming when l 2 norm is used.

Using separable programming to solve the multi-product multiple ex-ante constraint newsvendor problem and extensions

This paper provides an approximating programming technique to solve the multi-product newsvendor model in which product demands are independent and stocking quantities are subject to two or more ex-ante linear contraints, such as budget or volume constraints. Previous research has attempted to solve this problem with Lagrange relaxation techniques or by limiting the distribution of demand. However, by taking advantage of the separable nature of the problem, a close approximation of the optimal solution can be found using convex separable programming for any demand distribution in the traditional newsvendor model and extensions. Sensitivity analysis of the linear program provides managerial insight into the effects of parameters of the problem on the optimal solution and future decisions.

Convex programming with single separable constraint and bounded variables

In this paper a minimization problem with convex objective function subject to a separable convex inequality constraint "?" and bounded variables (box constraints) is considered. We propose an iterative algorithm for solving this problem based on line search and convergence of this algorithm is proved. At each iteration, a separable convex programming problem with the same constraint set is solved using Karush-Kuhn-Tucker conditions. Convex minimization problems subject to linear equality/ linear inequality "?" constraint and bounds on the variables are also considered. Numerical illustration is included in support of theory.

Effective Linear Calculational Method for Nonlinear Optimization with a Convex Polyhedral Objective Function and Linear Constraints

This research proposes an effective linear calculational method based on convex cone concept for solving non-linear optimization problems with a convex polyhedral objective function and linear constraints. One familiar type of convex polyhedral objective functions is a triangular type, which can be normally solved by weighted goal programming (WGP). The necessary preference information of WGP is weights of positive and negative deviational variables. Alternatively, the linear calculational method prefers the other operational way in designing of an objective function by deviational constants, which is practical for the decision maker. For convex polyhedral type objective function problems, conventionally separable convex programming and goal programming (GP) are applied. By separable convex programming, it needs to separate the objective function into line segments before solving the problem, which means increasing of variables and constraints. In case of GP each breakpoint is determined as a goal so the number of constraints and the deviational variables are drastically increased. By the effective linear calculational method proposed in this paper, the problem could be simply formulated to the linear programming problem, which is easy for the decision maker to apply. Moreover this method has lower number of constraints and variables than existing methods so the calculational time can also be reduced.

Global and Superlinear Convergence of Inexact Uzawa Methods for Saddle Point Problems with Nondifferentiable Mappings

This paper investigates inexact Uzawa methods for nonlinear saddle point problems. We prove that the inexact Uzawa method converges globally and superlinearly even if the derivative of the nonlinear mapping does not exist. We show that the Newton-type decomposition method for saddle point problems is a special case of a Newton--Uzawa method. We discuss applications of inexact Uzawa methods to separable convex programming problems and coupling of finite elements/boundary elements for nonlinear interface problems.

An extension of predictor-corrector algorithm to a class of convex separable program

Predictor-corrector algorithm for linear programming, proposed by Mizuno et al. [1], becomes the best-known in the interior point methods. In this paper it is modified and then extended to solving a class of convex separable programming problems.

Risk Management in Herd Sire Portfolio Selection: A Comparison of Rounded Quadratic and Separable Convex Programming

Quadratic programming, which does not guarantee integer solutions, may be used to consider risk when herd sires are selected. Because semen cannot be purchased in partial units, two alternatives were investigated: separable convex programming (a linear approximation of quadratic programming) and rounded quadratic programming. Expected net revenue and its variance were calculated for 383 Holstein bulls. Expected net revenue was based on semen price; PTA dollars for milk, fat, and protein; and three linear type traits: foot angle, udder depth, and teat placement. Maximization of utility (expected net revenue less its variance times a risk aversion factor) was subject to constraints: lots of 5 units, maximum average price per lot, minimum number of lots from bulls that transmit calving ease, and minimum number of different sires of selected bulls. Multiple formulations for expected net revenue, constraint combinations, and risk aversion factors were used. Twenty-two of 60 pairs of sire portfolios from rounded quadratic and separable convex programming were identical. Semen price per lot and lots from bulls that transmit calving ease were influential constraints. At the two lowest amounts of risk aversion examined, results supported common recommendations of selection of three to seven herd sires, regardless of herd size.

Methods for job configuration in semiconductor manufacturing

Job configuration in semiconductor manufacturing refers to the process of configuring silicon wafers in terms of chip types and volumes, organizing collections of wafers into jobs, and determining the volume of jobs to be released into the production line. The difficulties of job configuration lie in the random yield loss, the numerous technological constraints, and the rigid set serviceability requirement (i.e., demand must be met in terms of all chip types). Motivated by the configuration problems in the early-user hardware development programs at some IBM plants, we develop here a systematic approach to job configuration. The centerpiece of the approach is a careful treatment of the set serviceability constraint, in terms of both probability and expectation. It solves the job configuration problem as separable convex programs using marginal allocation algorithms. Through numerical examples, we demonstrate that by allowing diversity of chip types at either the job- or the wafer-level, higher serviceability can be achieved using fewer wafers.

Refined proximity and sensitivity results in linearly constrained convex separable integer programming

We define a new measure of the structure of a linear constraint matrix and establish some propertie. We then use the measure to refine proximity and sensitivity results in the literature. In particular, we refine bounds on the distance between continuous and discrete optima of linearly constrained convex separable programming problems and on the size of the feasible region for a linear program. We apply the results to improve the computational complexity of Hochbaum and Shanthikumar's algorithm for integer nonlinear programming, reducing the computational effort for certain problems from exponential to polynomial.

Separable Quadratic Programming via a Primal-Dual Interior Point Method and its Use in a Sequential Procedure

This paper extends a primal-dual interior point procedure for linear programs to the case of convex separable quadratic objectives. Included are efficient procedures for: attaining primal and dual feasibility, variable upper bounding, and free variables. A sequential procedure that invokes the quadratic solver is proposed and implemented for solving linearly constrained convex separable nonlinear programs. Computational results are provided for several large test cases from stochastic programming. The proposed methods compare favorably with MINOS, especially for the larger examples. The nonlinear programs range in size up to 8,700 constraints and 22,000 variables. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.