Lagrangian Dual Problem Research Articles

Fully distributed convex hull pricing based on alternating direction method of multipliers

In certain power markets, due to the non-convex operation characteristics, generators adhering to the decisions of Independent System Operators (ISO) may struggle to recover costs through local marginal energy sales. ISOs impose discriminatory additional payments as incentives for generator compliance. Convex hull pricing is a unified scheme that significantly reduces these supplementary payments. The Lagrangian dual problem of the Unit Commitment (UC) problem is solved within the dual space to determine convex hull prices. To navigate the computational challenges posed by the real large-scale power systems, we propose a methodology for a global convergence distributed solution, which addresses the Lagrangian dual problem. This methodology is based on the Alternating Direction Method of Multipliers (ADMM) algorithm and incorporates convex hull cut planes to enhance computational efficiency. Moreover, a tight and compact UC model is employed to reduce the number of iterations. Numerical results indicate that if the convex hull descriptions of units can be obtained, our algorithm is capable of providing precise convex hull prices and high-quality solutions within a feasible timeframe, while also maintaining the confidentiality of individual subset unit information.

Spatiotemporal Transformation-Based Neural Network With Interpretable Structure for Modeling Distributed Parameter Systems.

Many industrial processes can be described by distributed parameter systems (DPSs) governed by partial differential equations (PDEs). In this research, a spatiotemporal network is proposed for DPS modeling without any process knowledge. Since traditional linear modeling methods may not work well for nonlinear DPSs, the proposed method considers the nonlinear space-time separation, which is transformed into a Lagrange dual optimization problem under the orthogonal constraint. The optimization problem can be solved by the proposed neural network with good structural interpretability. The spatial construction method is employed to derive the continuous spatial basis functions (SBFs) based on the discrete spatial features. The nonlinear temporal model is derived by the Gaussian process regression (GPR). Benefiting from spatial construction and GPR, the proposed method enables spatially continuous modeling and provides a reliable output range under the given confidence level. Experiments on a catalytic reaction process and a battery thermal process demonstrate the effectiveness and superiority of the proposed method.

A Customized Augmented Lagrangian Method for Block-Structured Integer Programming.

Integer programming with block structures has received considerable attention recently and is widely used in many practical applications such as train timetabling and vehicle routing problems. It is known to be NP-hard due to the presence of integer variables. We define a novel augmented Lagrangian function by directly penalizing the inequality constraints and establish the strong duality between the primal problem and the augmented Lagrangian dual problem. Then, a customized augmented Lagrangian method is proposed to address the block-structures. In particular, the minimization of the augmented Lagrangian function is decomposed into multiple subproblems by decoupling the linking constraints and these subproblems can be efficiently solved using the block coordinate descent method. We also establish the convergence property of the proposed method. To make the algorithm more practical, we further introduce several refinement techniques to identify high-quality feasible solutions. Numerical experiments on a few interesting scenarios show that our proposed algorithm often achieves a satisfactory solution and is quite effective.

CSAT‐FRMET: An energy‐efficient hybrid FiWi network‐based 5G model

AbstractHybrid fiber wireless (FiWi) access networks seek to combine the vast amount of bandwidth that is available for optical networks and wireless network mobility; hence, it favors larger bandwidth request but cause the highly nonlinear lagrangian dual problem that decrease energy‐efficient network resilience. Hence, a novel Ceplato‐SAT3 model has been proposed in which the SAT3 algorithm locates the optical network unit (ONU) in an optimal position with high‐speed cost‐efficient placement and weight factor introduced into the position update to improve the exploitation and exploration capability of the algorithm. To prioritize the large bandwidth request during resource allocation, gradient priority justifier requests (GPJR) has been utilized and a novel messaging technique termed flow‐random‐message‐exchanger‐topology (FRMET) provides the index for these prioritized requests depending on QoS parameters such as notification control, reflective QoS attribute (RQA), flow bit rates and maximum packet loss rate thereby decreasing the overhead and eliminate nonlinear langrage dualand perform O2U scheduling to reduce network resilience however synchronization between different ONU clocks may cause upstream data fatal crash at the optical line terminal (OLT). Hence, curl‐trio‐opt control mechanism has been proposed in which the minimization problem is transformed into a joint optimization problem of energy consumption, traffic demand, network requests, and service flow requests by the Lyapunov technique thereby minimizing the upstream data fatal lash. The proposed model has been implemented in Python and the output obtained showed better performance in terms of ONU arrangement, throughput, energy consumption, computation, latency, and error.

ADMM‐Based Distributed Algorithm for Energy Management in Multi‐Microgrid System

This article focuses on Energy Management Problem (EMP) in Muti‐microgrid systems. Each microgrid (MG) has four basic equipments including Renewable Generator (RG), Fuel Generator (FG), Storage Device (SD) and Smart Load (SL). In consideration of equipment capacity, confidence interval for forecasts of RG output, supply–demand balance and other factors, an optimization model with the objective of minimizing operation cost was established. Through Lagrange dual problem and variable substitution, we transform the centralized problem into an equivalent distributed form. Then, a completely distributed algorithm based on Alternating Direction Method of Multipliers (ADMM) and Average Consensus (AC) is proposed to attain a global optimal schedule plan. The algorithm overcomes the defect that each microgrid needs to know the total must‐run load in advance. At the same time, the electricity clearing price is related to the objective function through Lagrange dual variables, which can be obtained while the optimal plan is determined. Finally, the simulation verifies the convergence of the algorithm, and the calculated optimal cost is the same as that of the centralized method, ensuring its effectiveness. Besides, the algorithm also has good performance in plug‐and‐play scenarios. © 2023 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

Lagrangian dual theory and stability analysis for fuzzy optimization problems

This paper investigates the Lagrangian dual theory for a class of fuzzy optimization problems with equality and inequality constraints. To begin with, the Lagrangian dual problem corresponding to the primal fuzzy optimization problem is established by introducing the Lagrangian fuzzy-valued function, and the weak dual theorem is derived. Subsequently, a necessary and sufficient condition for the existence of saddle-points is formulated, which serves as the fundamental basis for proving the strong dual theorem. The stability of the optimization model when its constraints are perturbed by parameters is then analyzed. Finally, the developed Lagrangian dual theory is applied to support vector machines to address the binary classification problem of triangular fuzzy number datasets.

Solving the multi-compartment vehicle routing problem by an augmented Lagrangian relaxation method

In real-world routing applications such as waste collection or fuel distribution, multiple products that cannot be mixed should be jointly collected or delivered. Multi-compartment vehicles are applied with great flexibility in routing and assignment decisions for these real-world applications. This study concentrates on the multi-compartment vehicle routing problem with fixed compartment size and fixed assignment products to compartments. For this problem, we develop two new formulations in the space–time network and state-space–time network. For the state-space–time network formulation, we dualize the task allocation constraints to the objective function and decompose the Lagrangian relaxed problem (LR) into a series of identical routing subproblems. To produce high-quality feasible solutions, the augmented Lagrangian relaxed problem (ALR) with superior convergence and feasibility is further applied to this problem. The ALR is decomposed into many solvable routing subproblems by alternating direction method of multipliers. A time-dependent dynamic programming algorithm is developed to settle the obtained subproblems of the LR and ALR. The subgradient optimization method solves the Lagrangian dual problems. The solution quality can be evaluated by the relative gap between the global lower and upper bounds at each iteration. In numerical experiments, the proposed method produces solutions with good integrality gaps. Based on Lagrangian relaxation and decomposition techniques, this research provides a novel solving scheme for the multi-compartment vehicle routing problem.

D-Optimal Data Fusion: Exact and Approximation Algorithms

We study the D-optimal Data Fusion (DDF) problem, which aims to select new data points, given an existing Fisher information matrix, so as to maximize the logarithm of the determinant of the overall Fisher information matrix. We show that the DDF problem is NP-hard and has no constant-factor polynomial-time approximation algorithm unless P = NP. Therefore, to solve the DDF problem effectively, we propose two convex integer-programming formulations and investigate their corresponding complementary and Lagrangian-dual problems. Leveraging the concavity of the objective functions in the two proposed convex integer-programming formulations, we design an exact algorithm, aimed at solving the DDF problem to optimality. We further derive a family of submodular valid inequalities and optimality cuts, which can significantly enhance the algorithm performance. We also develop scalable randomized-sampling and local-search algorithms with provable performance guarantees. Finally, we test our algorithms using real-world data on the new phasor-measurement-units placement problem for modern power grids, considering the existing conventional sensors. Our numerical study demonstrates the efficiency of our exact algorithm and the scalability and high-quality outputs of our approximation algorithms. History: Accepted by Andrea Lodi, Area Editor for Design & Analysis of Algorithms—Discrete. Funding: Y. Li and W. Xie were supported in part by Division of Civil, Mechanical and Manufacturing Innovation [Grant 2046414] and Division of Computing and Communication Foundations [Grant 2246417]. J. Lee was supported in part by Air Force Office of Scientific Research [Grants FA9550-19-1-0175 and FA9550-22-1-0172]. M. Fampa was supported in part by Conselho Nacional de Desenvolvimento Científico e Tecnológico [Grants 305444/2019-0 and 434683/2018-3]. F. Qiu and R. Yao were supported in part by the U.S. Department of Energy Advanced Grid Modeling Program under [Grant DE-OE0000875]. Supplemental Material: The e-companion is available at https://doi.org/10.1287/ijoc.2022.0235 .

Reducing the low-wavenumber dispersion error by building the Lagrange dual problem with a powerful local restriction

Abstract The broadband finite-difference (FD) coefficients computed by a cost function have been widely applied to suppress of numerical dispersion. Under the same conditions, the FD coefficients with small low-wavenumber dispersion error will produce a more accurate numerical solution in the long-time seismic wave simulation. Thus, how to reduce the low-wavenumber dispersion error becomes a crucial problem. According to the research into the zero-point position at the dispersion curve for three types of cost function, we found that the more zero points concentrate on the low-wavenumber region, the less the dispersion error. Therefore, the concentration of zero points is a good way to reduce dispersion error, which can be implemented by modified wavenumbers of zero points. Then, we design a Lagrange dual problem with a restriction based on the modified wavenumbers. The requirements for constructing the Lagrange dual problem are the optimization function and restricted condition, where the former is based on the dispersion relation, and the latter comprises the modified wavenumbers. The solution of this optimization problem, calculated by the dual ascent method, affords a less low-wavenumber dispersion error than the solution yielded by the alternating direction method of multipliers (ADMM). The theoretical analysis and numerical modeling suggest that the proposed method is superior to the existing optimal FD coefficients in reducing numerical error accumulation in low-frequency simulation.

On duality gap with polynomial multipliers for polynomial optimization problems

In this paper, we examine a zero-duality gap with polynomial multipliers between a non-convex polynomial program and its associated generalized Lagrangian dual problem. To this end, we introduce a sum of squares certificate of non-negativity, abbreviated as (SC), that allows us to verify the non-negativity of a polynomial over a feasible set of a system of given polynomials. With the help of (SC), we show that the zero-duality gap with polynomial multipliers coincides with the convergent Lasserre hierarchy of semidefinite programming relaxations. This zero-duality gap with polynomial multipliers reduces to the classical zero-duality gap with constant multipliers when considering it in the setting of SOS-convex polynomial programs or quadratic programs with the hidden convexity. We also show that, in the framework of truncated polynomial systems, the zero-duality gap with polynomial multipliers can be checked via the classical Karush–Kuhn–Tucker condition under an additional convexity condition.

Stable strong duality for cone-constrained set-valued optimization problems: A perturbation approach

In this paper we consider a general set-valued otimimization problem of the model Winf F(x), s.t. x in X where $F\colon X\rightrightarrows Y\cup\{+\infty_{Y}\}$ is a proper mapping. The problem is then embed into a parametric problem and can be express as (P) $\mathop{\winf}\bigcup\limits_{x\in X}\Phi(x,0_Z)$ where $\Phi\colon X\times Z\rightrightarrows Y^\bullet$ is a proper set-valued {\it perturbation mapping} such that $ \Phi(x,0_Z) = F(x) $. A representation of the epigraph of the conjugate mapping $\Phi^\ast$ is established (Theorem 1) and it is used as the basic tool for establishing a general stable strong duality for the problem (P) (Theorem 3). As applications, the mentioned general strong duality result is then applied to a cone-constrained set-valued optimization problem (CSP) to derive three dual problems for (CSP): The Lagrange dual problem and two forms of Fenchel-Lagrange dual problems for (CSP). Consequently, three stable strong duality results for the three primal-dual pairs of problems are derived (Theorems 4 and 5), among them, one is entirely new while the others extend some known ones in the literature.

Bid price controls for car rental network revenue management

We consider a car rental network revenue management (RM) problem, accounting for the key operational characteristics of car rental services such as the varying length of rentals and mobility of inventories, which imply the intertemporal and spatial correlations of rental demands for inventories across different locations and days. The problem is formulated as an infinite‐horizon cyclic stochastic dynamic program to account for the time‐varying and cyclic nature of car rental businesses. To tackle the curse of dimensionality, we propose a Lagrangian relaxation (LR) approach with product‐ and time‐dependent Lagrangian multipliers to decomposing the dynamic network problem into multiple single‐station single‐day subproblems. We show that the Lagrangian dual problem is a convex program and then develop a subgradient‐based algorithm to solve the dual problem and derive an LR‐based bid price policy. To improve the scalability of the LR approach, we further propose three simpler LR‐based bid price policy variants with either location‐dependent or leadtime‐dependent Lagrangian multipliers, or both. Our numerical study indicates that the LR‐based bid price policies can outperform some commonly used heuristics. Using a set of real‐world booking data, we provide a case study in which we empirically demonstrate the operational characteristics of car rental services, calibrate the arrival process of booking requests using a Poisson regression model, and demonstrate that the LR‐based bid price policies indeed outperform other heuristics consistently in both in‐sample and out‐of‐sample horizons.

Global optimization for non-convex programs via convex proximal point method

<p style='text-indent:20px;'>In this study, a convex proximal point algorithm (CPPA) is considered for solving constrained non-convex problems, and new theoretical results are proposed. It is proved that every cluster point of CPPA is a stationary point, and the initial point of CPPA is key to global optimization. Several sufficient conditions for the initial point selection are provided for CPPA to find the global minimum. Motivated by these results, numerical experiments were conducted on non-convex quadratic programming problems with convex quadratic constraints. The performance of CPPAs was compared, with the initial point randomly selected or obtained through the Lagrangian dual problem. The numerical results demonstrate that the quality of the CPPA with the computed Lagrangian dual initial point is much better than that with the random initial point, in terms of the objective function value.</p>

A Hybrid Stochastic/Robust Model for Transmission Expansion Planning under an Ellipsoidal Uncertainty Set

Over the last few years, the concept of robust optimization (RO) and its application to power system problems have been at the center of much recent research due to its successful implementation in handling uncertainties. In this regard, this paper presents a new approach for hybrid stochastic/robust transmission expansion planning (TEP), considering three sources of uncertainties, including demand growth, generation capacity, and thermal capacity of transmission lines. These uncertain parameters are modeled using an ellipsoidal uncertainty set, which can capture the correlation efficiently. The TEP problem is modeled as a bilevel mathematical optimization where the lower level is replaced by its Lagrange dual optimization problem using the duality theorem (DT). One main benefit of using the dual problem is that all uncertainties will appear in the objective function and thus can be converted to convex second-order cone constraints. The entire bilevel problem is then solved using the standard column-and-constraint generation (C&CG) technique by decomposing the mathematical problem into a master and a binary variable-free subproblem. Numerical studies on two test power systems validate the effectiveness of the proposed model. The results indicate that the presented methodology is more efficient than the existing models from the computational complexity perspective.

Vehicle Selection and Resource Optimization for Federated Learning in Vehicular Edge Computing

As a distributed deep learning paradigm, federated learning (FL) provides a powerful tool for the accurate and efficient processing of on-board data in vehicular edge computing (VEC). However, FL involves the training and transmission of model parameters, which consumes the vehicles’ precious energy resources and takes up much time. It is a departure from many applications with severe real-time requirements in VEC. And the capabilities and data quality of each vehicle are distinct that will affect the performance of training the model. Therefore, it is crucial to select the appropriate vehicles to participate in learning tasks and optimize resource allocation under learning time and energy consumption constraints. In this paper, taking the vehicle position and velocity into consideration, we formulate a min-max optimization problem to jointly optimize the on-board computation capability, transmission power, and local model accuracy to achieve the minimum cost in the worst case of FL. Specifically, we propose a greedy algorithm to select vehicles with higher image quality dynamically, and it keeps the system’s overall cost to a minimum in FL. The formulated optimization problem is a nonlinear programming problem, so we decompose it into two subproblems. For the resource allocation problem, we use the Lagrangian dual problem and the subgradient projection method to approximate the optimal value iteratively. For the local model accuracy problem, we develop an adaptive harmony algorithm for heuristic search. The simulation results show that our proposed algorithms have well convergence and effectiveness and achieve a tradeoff between cost and fairness.

Strong and total duality for constrained composed optimization via a coupling conjugation scheme

ABSTRACT Based on a coupling conjugation scheme and the perturbational approach, we build Fenchel–Lagrange dual problem of a composed optimization model with infinite constraints in separated locally convex spaces. This paper has mainly two targets. One is to establish strong duality under a new regularity condition ( RC A ) and an extension closed-type condition ( ECRC A ). The e-convex counterpart of Fenchel–Moreau theorem plays a key role in analysing the relation between them. The other aim is to achieve the sufficient and necessary characterizations for total duality in terms of c-subdifferentials. For this purpose, a formula for ε-c-subdifferentials of a proper function composed with a linear continuous operator is proved and applied.

Cooperative Distributed GPU Power Capping for Deep Learning Clusters

The recent GPU-based clusters that handle deep learning (DL) tasks have the features of GPU device heterogeneity, a variety of deep neural network (DNN) models, and high computational complexity. Thus, the traditional power capping methods for CPU-based clusters or small-scale GPU devices cannot be applied to the GPU-based clusters handling DL tasks. This article develops a cooperative distributed GPU power capping (CD-GPC) system for GPU-based clusters, aiming to minimize the training completion time of invoked DL tasks without exceeding the limited power budget. Specifically, we first design the frequency scaling approach using the online model estimation based on the recursive least square method. This approach achieves the accurate tuning for DL task training time and power usage of GPU devices without needing offline profiling. Then, we formulate the proposed FS problem as a Lagrangian dual decomposition-based economic model predictive control problem for large-scale heterogeneous GPU clusters. We conduct both the NVIDIA GPU-based lab-scale real experiments and real job trace-based simulation experiments for performance evaluation. Experimental results validate that the proposed system improves the power capping accuracy to have a mean absolute error of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$&lt; \!1\%$</tex-math></inline-formula> , and reduces the deadline violation ratio of invoked DL tasks by 21.5% compared with other recent counterparts.

A Simplified Solution Method for End-of-Term Storage Energy Maximization Model of Cascaded Reservoirs

In medium-term scheduling, the end-of-term storage energy maximization model is proposed to create conditions for the safety, stability and economic operation of the hydropower system after control term, which satisfies the system load demand undertaken by the cascaded system in a given scheduling period. This paper presents a simplified solution method based on the Lagrangian relaxation method (LR) to solve the end-of-term storage energy maximization model. The original Lagrange dual problem with multiple Lagrange multipliers is converted to that with only one Lagrange multiplier by an entropy-based aggregate function method, which relaxes the complex cascaded hydropower system load balance constraints. The subgradient method and successive approximation of dynamic programming (DPSA) are adopted to update the Lagrange multiplier iteratively and solve the subproblem of the Lagrange dual problem, respectively. The Wujiang cascaded hydropower system is studied, and the result shows that the simplified solution method for the end-of-term storage energy maximization model both improves solving efficiency and ensures solving accuracy to a great extent.

Pricing-Based Deep Reinforcement Learning for Live Video Streaming With Joint User Association and Resource Management in Mobile Edge Computing

Mobile Edge Computing (MEC) is a promising technique in the 5G Era to improve the Quality of Experience (QoE) for online video streaming due to its ability to reduce the backhaul transmission by caching certain content. However, it still takes effort to address the user association and video quality selection problem under the limited resource of MEC to fully support the low-latency demand for live video streaming. We found the optimization problem to be a non-linear integer programming, which is impossible to obtain a globally optimal solution under polynomial time. In this paper, we formulate the problem and derive the closed-form solution in the form of Lagrangian multipliers; the searching of the optimal variables is formulated as a Multi-Arm Bandit (MAB) and we propose a Deep Deterministic Policy Gradient (DDPG) based algorithm exploiting the supply-demand interpretation of the Lagrange dual problem. Simulation results show that our proposed approach achieves significant QoE improvement, especially in the low wireless resource and high user number scenario compared to other baselines.

Mixed degree regression function estimation using weighted lasso penalty

We consider a nonparametric function estimation method with mixed degree using weighted lasso penalty. The ℓ₁ norm penalty controls linear and cubic trends depending on the value of the parameter. In the proposed estimator, we introduce one computational algorithm for constrained convex optimization problems corresponding to the Lagrangian dual problem based on quadratic programming. Subsequently, using simulations and two real data analyses, numerical studies are conducted to verify the performance of the estimator of the proposed method by identifying the relationship between the data.