Subgradient Optimization Method Research Articles

A two-point heuristic to calculate the stepsize in subgradient method with application to a network design problem

We introduce a heuristic rule for calculating the stepsize in the subgradient method for unconstrained convex nonsmooth optimization which, unlike the classic approach, is based on retaining some information from previous iteration. The rule is inspired by the well known two-point stepsize by Barzilai and Borwein (BB) [6] for smooth optimization and it coincides with (BB) in case the function to be minimised is convex quadratic.Under the use of appropriate safeguards we demonstrate that the method terminates at a point that satisfies an approximate optimality condition.The proposed approach is tested in the framework of Lagrangian relaxation for integer linear programming where the Lagrangian dual requires maximization of a concave and nonsmooth (piecewise affine) function. In particular we focus on the relaxation of the Minimum Spanning Tree problem with Conflicting Edge Pairs (MSTC). Comparison with classic subgradient method is presented. The results on some widely used academic test problems are provided too.

Solving the multi-compartment vehicle routing problem by an augmented Lagrangian relaxation method

In real-world routing applications such as waste collection or fuel distribution, multiple products that cannot be mixed should be jointly collected or delivered. Multi-compartment vehicles are applied with great flexibility in routing and assignment decisions for these real-world applications. This study concentrates on the multi-compartment vehicle routing problem with fixed compartment size and fixed assignment products to compartments. For this problem, we develop two new formulations in the space–time network and state-space–time network. For the state-space–time network formulation, we dualize the task allocation constraints to the objective function and decompose the Lagrangian relaxed problem (LR) into a series of identical routing subproblems. To produce high-quality feasible solutions, the augmented Lagrangian relaxed problem (ALR) with superior convergence and feasibility is further applied to this problem. The ALR is decomposed into many solvable routing subproblems by alternating direction method of multipliers. A time-dependent dynamic programming algorithm is developed to settle the obtained subproblems of the LR and ALR. The subgradient optimization method solves the Lagrangian dual problems. The solution quality can be evaluated by the relative gap between the global lower and upper bounds at each iteration. In numerical experiments, the proposed method produces solutions with good integrality gaps. Based on Lagrangian relaxation and decomposition techniques, this research provides a novel solving scheme for the multi-compartment vehicle routing problem.

Charging station location problem for maximizing the space-time-electricity accessibility: A Lagrangian relaxation-based decomposition scheme

The driving range of electric vehicles limits the accessibility of their users. Charging infrastructures such as charging stations are essential to improve the accessibility of electric vehicles. Existing charging station location studies neglected accessibility-based indicators in their optimization models. For the charging station location problem, this paper proposes a novel objective function that maximizes the space–time-electricity accessibility of electric vehicles. Then we formulate an integer-programming model in the space–time-electricity network. A Lagrangian relaxation-based decomposition scheme is developed to solve this problem. The constraints that couple flow with location variables are dualized to the objective function, resulting in a set of independent routing subproblems and a knapsack subproblem. At each iteration, a primal heuristic utilizes the result of the intermediate knapsack subproblem to generate a feasible solution and Lagrangian multipliers are updated by the subgradient optimization method. The numerical experiments are conducted on three networks, showing that the proposed method achieves good integrality gaps.

Quantized and Distributed Subgradient Optimization Method With Malicious Attack

This letter considers a distributed optimization problem in a multi-agent system where a fraction of the agents act in an adversarial manner. Specifically, the malicious agents steer the network of agents away from the optimal solution by sending false information to their neighbors and consume significant bandwidth in the communication process. We propose a distributed gradient-based optimization algorithm in which the non-malicious agents exchange quantized information with one another. We prove convergence of the solution to a neighborhood of the optimal solution, and characterize the solutions obtained in the communication-constrained environment and presence of malicious agents. Numerical simulations to illustrate the results are also presented.

Lagrangian relaxation-based decomposition approaches for the capacitated arc routing problem in the state-space-time network

ABSTRACT We study the capacitated arc routing problem in the state-space-time network. A multi-commodity network flow model is formulated in the state-space-time network. Two Lagrangian relaxation-based decomposition methods are developed to tackle this problem. By dualizing the coupling constraints to the objective function, the Lagrangian relaxed problem (LR) can be decomposed into several routing subproblems in the state-space-time network. Then we extend quadratic penalty terms to the LR and obtain an augmented Lagrangian relaxed problem (ALR). By applying the alternating direction method of multipliers and linearization techniques, we decompose the ALR into a sequence of routing subproblems. We solve these subproblems of LR and ALR by a time-dependent dynamic programming algorithm. Feasible solutions are produced according to the result of the LR or ALR. The Lagrangian multipliers are updated by the subgradient optimization method. We implement the proposed methods in three networks, showing that the ALR-based method has better performance.

Space planning considering congestion in container terminal yards

This paper proposes a nonlinear mathematical model for solving the problem of allocating storage space to future container arrivals in container terminal yards by considering the waiting times of trucks in the yard, called stochastic space allocation model (SSAM). Three heuristic procedures based on the Frank-Wolfe algorithm and the subgradient optimization method are proposed to efficiently solve the problem. From the numerical experiment, it is found that the proposed algorithms for SSAM provide solutions with the gaps between the objective values and the optimal values are no greater than 0.69%. It was also found that the best algorithm among three proposed algorithms (FWMP-S) obtained the average 7.89% improvement in the objective value compared with a previous model with a handling capacity constraint instead of considering waiting time (DSAM). It is found that the SSAM reduces the average system time of transporters for ship operation and that of road trucks by 17.39 and 7.14%, respectively, compared with the DSAM. This study proposes the most appropriate heuristic algorithm depending on the problem characteristics of the SSAM.

Lagrangian heuristic algorithm for green multi-product production routing problem with reverse logistics and remanufacturing

Today, due to increased market competition, the integration of production and distribution decisions into the supply chain leads to efficiency improvements. Therefore, production routing models have been developed to optimize production and distribution, simultaneously. In recent years, since, the product life cycle has become shorter than in the past, product return policies with fast response times, emphasis on return management, deformation and restorage of finished goods have become very significant. In this paper, the multi-product production routing problem with startup costs and environmental considerations has been studied. Furthermore, reverse logistics and remanufacturing decisions have been integrated. After modeling the problem as mixed-integer linear programming, due to its NP-hardness and the successful application of the Lagrangian Relaxation algorithm (LR) in solving complex supply chain problems, this algorithm has been chosen as the solution method. After applying the standard LR algorithm and the improved LR algorithm that its subgradient optimization method was modified, as a heuristic algorithm, the feasiblizer algorithm is also proposed for feasibilization of the solution obtained from the LR algorithms. To validate the model and solution method, firstly, test problems are solved by GAMS, and then the proposed algorithm is applied to test problems. The numerical results show the good performance of the LR algorithm in medium-size test problems. Finally, based on the computational experiments, managerial insights on the problem have been provided.

Lagrangian Duality for Robust Problems with Decomposable Functions: The Case of a Robust Inventory Problem

We consider a class of min-max robust problems in which the functions that need to be “robustified” can be decomposed as the sum of arbitrary functions. This class of problems includes many practical problems, such as the lot-sizing problem under demand uncertainty. By considering a Lagrangian relaxation of the uncertainty set, we derive a tractable approximation, called the dual Lagrangian approach, that we relate with both the classical dualization approximation approach and an exact approach. Moreover, we show that the dual Lagrangian approach coincides with the affine decision rule approximation approach. The dual Lagrangian approach is applied to a lot-sizing problem, in which demands are assumed to be uncertain and to belong to the uncertainty set with a budget constraint for each time period. Using the insights provided by the interpretation of the Lagrangian multipliers as penalties in the proposed approach, two heuristic strategies, a new guided iterated local search heuristic, and a subgradient optimization method are designed to solve more complex lot-sizing problems in which additional practical aspects, such as setup costs, are considered. Computational results show the efficiency of the proposed heuristics that provide a good compromise between the quality of the robust solutions and the running time required in their computation. Summary of Contribution: The paper includes both theoretical and algorithmic contributions for a class of min-max robust optimization problems where the objective function includes the maximum of a sum of affine functions. From the theoretical point of view, a tractable Lagrangian dual model resulting from a relaxation of the well-known adversarial problem is proposed, providing a new perspective of well-known models, such as the affinely adjustable robust counterpart (AARC) and the dualization technique introduced by Bertsimas and Sim. These results are particularized to lot-sizing problems. From the algorithm point of view, efficient heuristic schemes—which exploit the information based on the interpretation of the Lagrangian multipliers to solve large size robust problems—are proposed, and their performance is evaluated through extensive computational results based on the lot-sizing problem. In particular, a guided iterated local search and a subgradient optimization method are proposed and compared against the dualization approach proposed by Bertsimas and Sim and with several heuristics based on the AARC approach, which include an iterated local search heuristic and a Benders decomposition approach. Computational results show the efficiency of the proposed heuristics, which provide a good compromise between the quality of the robust solutions and the running time.

A unitary distributed subgradient method for multi-agent optimization with different coupling sources

Distributed optimization techniques offer high quality solutions to various engineering problems, such as resource allocation and distributed estimation and control. In this work, we first consider distributed convex constrained optimization problems where the objective function is encoded by multiple local and possibly nonsmooth objectives privately held by a group of agents, and propose a distributed subgradient method with double averaging (abbreviated as DSA2) that only requires peer-to-peer communication and local computation to solve the global problem. The algorithmic framework builds on dual methods and dynamic average consensus; the sequence of test points is formed by iteratively minimizing a local dual model of the overall objective where the coefficients, i.e., approximated subgradients of the objective, are supplied by the dynamic average consensus scheme. We theoretically show that DSA2 enjoys non-ergodic convergence properties, i.e., the local minimizing sequence itself is convergent, a distinct feature that cannot be found in existing results. Specifically, we establish a convergence rate of O(1t) in terms of objective function error. Then, extensions are made to tackle distributed optimization problems with coupled functional constraints by combining DSA2 and dual decomposition. This is made possible by Lagrangian relaxation that transforms the coupling in constraints of the primal problem into that in cost functions of the dual, thus allowing us to solve the dual problem via DSA2. Both the dual objective error and the quadratic penalty for the coupled constraint are proved to converge at a rate of O(1t), and the primal objective error asymptotically vanishes. Numerical experiments and comparisons are conducted to illustrate the advantage of the proposed algorithms and validate our theoretical findings.

Communication-Efficient Distributed Demand Response: A Randomized ADMM Approach

In this paper, we consider the distributed demand response (DDR) problem for achieving the real-time power balance in a neighborhood with a large number of load customers and renewable energy sources (RES). While most of the existing DDR schemes require iterative information exchange between the customers and the load aggregator through two-way communications, this paper studies the DDR schemes that rely on neighbor-wise communication between customers only. Such DDR schemes can be realized by low-cost wireless networks. To this end, we propose the use of a randomized alternating direction method of multipliers to develop a fully DDR algorithm. Notably, the proposed DDR algorithm is communication-efficient because it can yield promising power balance performance using a few times of neighbor-wise message exchanges. Moreover, the proposed DDR algorithm does not need synchronization between customers and is robust against random communication errors. For performing online demand response control, we combine the proposed DDR algorithm with the rolling-window-based model-predictive control method, and simple load and RES forecasting methods. By using real solar power data, we demonstrate via simulations that the proposed DDR algorithm improves the real-time power balance substantially and outperforms the existing DDR schemes that use the subgradient method for optimization.

Variant of Constants in Subgradient Optimization Method over Planar 3-Index Assignment Problems

A planar 3-index assignment problem (P3AP) of size n is an NP-complete problem. Its global optimal solution can be determined by a branch and bound algorithm. The efficiency of the algorithm depends on the best lower and upper bound of the problem. The subgradient optimization method, an iterative method, can provide a good lower bound of the problem. This method can be applied to the root node or a leaf of the branch and bound tree. Some conditions used in this method may result in one of those becoming optimal. The formulas used in this method contain some constants that can be evaluated by computational experiments. In this paper, we show a variety of initial step length constants whose values have an effect on the lower bound of the problem. The results show that, for small problem sizes, when n < 20, the most suitable constants are best chosen in the interval [0.1, 1]. Meanwhile, the interval [0.05, 0.1] is the best interval chosen for the larger problem sizes, when n ≥ 20.

Planning Online Advertising Using Lorenz Curves

We study the online display advertising problem in which advertisers’ demands for ad exposures (impressions) of various types compete for slices of shared resources. In general, advertisers prefer to receive impressions that are evenly-spread across the audience segments they target, as well as evenly-spread across time. In order to accomplish this, we use the Gini coefficient measure and formulate single- and multi-period optimization planning problems that maximize the spreading of impressions across targeted audience segments while limiting demand shortfalls. The contributions of this study are manifold. First, we show how Gini-based metrics can be used to measure several dimensions of spreading that publishers of online advertising care about. Second, we provide adaptations of existing ad planning models to measure Gini-based performance. Third, we quantify the potential benefits of using Gini-based metrics. Fourth, we propose a visualization approach to display the results obtained with the Gini-based performance metrics. We graph the Lorenz curves, describe their practical interpretation, and explain how managers can use the Gini coefficients and the corresponding Lorenz curves to effectively monitor several performance dimensions of a publisher’s ad delivery system. Fifth, we introduce a novel optimization-based decomposition scheme to efficiently solve the proposed Gini-based problem formulations. Our decomposition scheme incorporates ingredients from the Dantzig-Wolfe decomposition and subgradient optimization methods, and exploits the special structure of the Gini-based ad planning problems. Finally, we show that our proposed decomposition method is on average 60 times faster than solving the Gini-based formulation directly.

A SUBGRADIENT METHODS FOR NON-SMOOTH VECTOR OPTIMIZATION PROBLEMS

Vector optimization problems are a significant extension of scalar optimization and have wide range of application in various fields of economics, decision theory, game theory, information theory and optimal control theory. In this paper, unlike general subgradient methods, bundle methods and gradient sampling methods used to solve nonsmooth vector optimization problems which include scalarization approach, a subgradient method without usual scalariza- tion approached is proposed for minimizing a non-differentiable convex func- tion which works directly with vector-valued function. A general sub-gradient method for non-smooth convex optimization that includes regularization and interior point variants of Newton's Method are proposed. This algorithm builds a sequence of efficient points at the interior of epigraph of objective function which satisfies KKT conditions. In this paper, under the suitable conditions it is proved that the sequence generated by algorithm converges to ǫ-efficient point.

Parallel computing subgradient method for nonsmooth convex optimization over the intersection of fixed point sets of nonexpansive mappings

Nonsmooth convex optimization problems are solved over fixed point sets of nonexpansive mappings by using a distributed optimization technique. This is done for a networked system with an operator, who manages the system, and a finite number of users, by solving the problem of minimizing the sum of the operator’s and users’ nondifferentiable, convex objective functions over the intersection of the operator’s and users’ convex constraint sets in a real Hilbert space. We assume that each of their constraint sets can be expressed as the fixed point set of an implementable nonexpansive mapping. This setting allows us to discuss nonsmooth convex optimization problems in which the metric projection onto the constraint set cannot be calculated explicitly. We propose a parallel subgradient algorithm for solving the problem by using the operator’s attribution such that it can communicate with all users. The proposed algorithm does not use any proximity operators, in contrast to conventional parallel algorithms for nonsmooth convex optimization. We first study its convergence property for a constant step-size rule. The analysis indicates that the proposed algorithm with a small constant step size approximates a solution to the problem. We next consider the case of a diminishing step-size sequence and prove that there exists a subsequence of the sequence generated by the algorithm which weakly converges to a solution to the problem. We also give numerical examples to support the convergence analyses.

A Lagrangian relaxation approach for expansion of a highway network

This paper deals with the problem of improving an existing road network in the context of strategic planning through the creation of new highway corridors. To address this problem we analyse three mixed integer programming models. The so-called [P1] is the classical capacitated multicommodity network design model. The model named [P2] imposes on [P1] the location of a single (main path) highway corridor in the road network and [P3] adds to [P2] a set of sub-tour breaking constraints. The stated goal is to minimize the total travel time for a known origin-destination demand matrix with a given budget. In this paper we propose an efficient method for [P3], based on a Lagrangian relaxation, to obtain easily-solved sub-problems. A cutting-plane method for solving the Lagrangian sub-problems is proposed. This method generates valid cuts until an optimal solution is found. The Lagrangian dual problem is solved using the sub-gradient optimization method. A case study has been carried out for the region of Castilla-La Mancha (Spain). Computational comparisons between the proposed method and a state-of-the-art mixed-integer code are presented. The Lagrangian relaxation approach is found to be capable of generating good feasible solutions to the case study within a reasonable computational time.

THE PROBABILISTIC NEURAL NET NEURON’S NUMBER CALCULATIONS

The sub-gradient method of estimation of the number of the hidden layer neurons of a probabilistic neural network is suggested. This method allows evaluating the data compactness violation in λ-space. This evaluation based on the noise stability sub-gradient iterative optimization method. This method allows reducing the number of the hidden layer neurons and classification time.

Primal convergence from dual subgradient methods for convex optimization

When solving a convex optimization problem through a Lagrangian dual reformulation subgradient optimization methods are favorably utilized, since they often find near-optimal dual solutions quickly. However, an optimal primal solution is generally not obtained directly through such a subgradient approach unless the Lagrangian dual function is differentiable at an optimal solution. We construct a sequence of convex combinations of primal subproblem solutions, a so called ergodic sequence, which is shown to converge to an optimal primal solution when the convexity weights are appropriately chosen. We generalize previous convergence results from linear to convex optimization and present a new set of rules for constructing the convexity weights that define the ergodic sequence of primal solutions. In contrast to previously proposed rules, they exploit more information from later subproblem solutions than from earlier ones. We evaluate the proposed rules on a set of nonlinear multicommodity flow problems and demonstrate that they clearly outperform the ones previously proposed.

Reformulation and a Lagrangian heuristic for lot sizing problem on parallel machines

We consider the capacitated lot sizing problem with multiple items, setup time and unrelated parallel machines. The aim of the article is to develop a Lagrangian heuristic to obtain good solutions to this problem and good lower bounds to certify the quality of solutions. Based on a strong reformulation of the problem as a shortest path problem, the Lagrangian relaxation is applied to the demand constraints (flow constraint) and the relaxed problem is decomposed per period and per machine. The subgradient optimization method is used to update the Lagrangian multipliers. A primal heuristic, based on transfers of production, is designed to generate feasible solutions (upper bounds). Computational results using data from the literature are presented and show that our method is efficient, produces lower bounds of good quality and competitive upper bounds, when compared with the bounds produced by another method from the literature and by high-performance MIP software.

A Direct Splitting Method for Nonsmooth Variational Inequalities

We propose a direct splitting method for solving a nonsmooth variational inequality in Hilbert spaces. The weak convergence is established when the operator is the sum of two point-to-set and monotone operators. The proposed method is a natural extension of the incremental subgradient method for nondifferentiable optimization, which strongly explores the structure of the operator using projected subgradient-like techniques. The advantage of our method is that any nontrivial subproblem must be solved, like the evaluation of the resolvent operator. The necessity to compute proximal iterations is the main difficulty of other schemes for solving this kind of problem.

An Improved Subgradiend Optimization Technique for Solving IPs with Lagrangean Relaxation

The objective of this paper is to improve the subgradient optimization method which is used to solve non-differentiable optimization problems in the Lagrangian dual problem. One of the main drawbacks of the subgradient method is the tuning process to determine the sequence of step-lengths to update successive iterates. In this paper, we propose a modified subgradient optimization method with various step size rules to compute a tuning- free subgradient step-length that is geometrically motivated and algebraically deduced. It is well known that the dual function is a concave function over its domain (regardless of the structure of the cost and constraints of the primal problem), but not necessarily differentiable. We solve the dual problem whenever it is easier to solve than the primal problem with no duality gap. However, even if there is a duality gap the solution of the dual problem provides a lower bound to the primal optimum that can be useful in combinatorial optimization. Numerical examples are illustrated to demonstrate the method. DOI: http://dx.doi.org/10.3329/dujs.v61i2.17059 Dhaka Univ. J. Sci. 61(2): 135-140, 2013 (July)