Lagrangian Relaxation Research Articles

A Lagrangian heuristic for integrated production planning, material ordering and investment multi-project scheduling in a project-driven supply chain

Effective decision making in large project planning presents a significant challenge. Despite progress in developing mathematical models, few studies comprehensively address project and supply-chain management aspects in an integrated framework. This article presents an integrated approach for multi-project resource investment, material ordering and production planning in a project-driven supply chain. A mixed-integer programming model is developed to optimize the total cost of renewable resource investment, production, transportation, and material ordering and holding, with an opportunity to share mobile renewable resources across multiple projects. A Lagrangian relaxation heuristic algorithm is proposed to determine the upper and lower bounds for the objective function, with an additional set of valid cuts to reduce the distance between bounds. The results of numerical experiments demonstrate the favourable performance of the proposed algorithm compared to the GAMS solver. This article provides insights into supply-chain participant expenses and activity scheduling based on these findings.

A Lagrangian relaxation approach to an electricity system investment model with a high temporal resolution

AbstractThe global production of electricity contributes significantly to the release of carbon dioxide emissions. Therefore, a transformation of the electricity system is of vital importance in order to restrict global warming. This paper proposes a modelling methodology for electricity systems with a large share of variable renewable electricity generation, such as wind and solar power. The model developed addresses the capacity expansion problem, i.e. identifying optimal long-term investments in the electricity system. Optimal investments are defined by minimum investment and production costs under electricity production constraints—having different spatial resolutions and technical detail—while meeting the electricity demand. Our model is able to capture a range of strategies to manage variations and to facilitate the integration of variable renewable electricity; it is very large due to the high temporal resolution required to capture the variations in wind and solar power production and the chronological time representation needed to model energy storage. Moreover, the model can be further extended—making it even larger—to capture a large geographical scope, accounting for the trade of electricity between regions with different conditions for wind and solar power. Models of this nature thus typically need to be solved using some decomposition method to reduce solution times. In this paper, we develop a decomposition method using so-called variable splitting and Lagrangian relaxation; the dual problem is solved by a deflected subgradient algorithm. Our decomposition regards the temporal resolution by defining 2-week periods throughout the year and relaxing the overlapping constraints. The method is tested and evaluated on some real-world cases containing regions with different energy mixes and conditions for wind power. Numerical results show shorter computation times as compared with the non-decomposed model and capacity investment options similar to the optimal solution provided by the latter model.

Dual r-rank decomposition and its applications

Dual r-rank decomposition and its applications

Surrogate Lagrangian Relaxation: A Path to Retrain-Free Deep Neural Network Pruning

Network pruning is a widely used technique to reduce computation cost and model size for deep neural networks. However, the typical three-stage pipeline (i.e., training, pruning, and retraining (fine-tuning)) significantly increases the overall training time. In this article, we develop a systematic weight-pruning optimization approach based on surrogate Lagrangian relaxation (SLR), which is tailored to overcome difficulties caused by the discrete nature of the weight-pruning problem. We further prove that our method ensures fast convergence of the model compression problem, and the convergence of the SLR is accelerated by using quadratic penalties. Model parameters obtained by SLR during the training phase are much closer to their optimal values as compared to those obtained by other state-of-the-art methods. We evaluate our method on image classification tasks using CIFAR-10 and ImageNet with state-of-the-art multi-layer perceptron based networks such as MLP-Mixer; attention-based networks such as Swin Transformer; and convolutional neural network based models such as VGG-16, ResNet-18, ResNet-50, ResNet-110, and MobileNetV2. We also evaluate object detection and segmentation tasks on COCO, the KITTI benchmark, and the TuSimple lane detection dataset using a variety of models. Experimental results demonstrate that our SLR-based weight-pruning optimization approach achieves a higher compression rate than state-of-the-art methods under the same accuracy requirement and also can achieve higher accuracy under the same compression rate requirement. Under classification tasks, our SLR approach converges to the desired accuracy × faster on both of the datasets. Under object detection and segmentation tasks, SLR also converges 2× faster to the desired accuracy. Further, our SLR achieves high model accuracy even at the hardpruning stage without retraining, which reduces the traditional three-stage pruning into a two-stage process. Given a limited budget of retraining epochs, our approach quickly recovers the model’s accuracy.

Decomposition Is All You Need: Single-Objective to Multi-Objective Optimization towards Artificial General Intelligence

As a new abstract computational model in evolutionary transfer optimization (ETO), single-objective to multi-objective optimization (SMO) is conducted at the macroscopic level rather than the intermediate level for specific algorithms or the microscopic level for specific operators; this method aims to develop systems with a profound grasp of evolutionary dynamic and learning mechanism similar to human intelligence via a “decomposition” style (in the abstract of the well-known “Transformer” article “Attention is All You Need”, they use “attention” instead). To the best of our knowledge, it is the first work of SMO for discrete cases because we extend our conference paper and inherit its originality status. In this paper, by implementing the abstract SMO in specialized memetic algorithms, key knowledge from single-objective problems/tasks to the multi-objective core problem/task can be transferred or “gathered” for permutation flow shop scheduling problems, which will reduce the notorious complexity in combinatorial spaces for multi-objective settings in a straight method; this is because single-objective tasks are easier to complete than their multi-objective versions. Extensive experimental studies and theoretical results on benchmarks (1) emphasize our decomposition root in mathematical programming, such as Lagrangian relaxation and column generation; (2) provide two “where to go” strategies for both SMO and ETO; and (3) contribute to the mission of building safe and beneficial artificial general intelligence for manufacturing via evolutionary computation.

Joint operation planning of drivers and trucks for semi-autonomous truck platooning

Semi-autonomous truck platooning, a futuristic and promising traveling mode of trucks on highways, has received extensive attention as autonomous driving gains more maturity. Under the semi-autonomous platooning mode, rather than requiring a human driver for each truck in the platoon, only one human driver is needed for the leading truck, and all following trucks can be controlled by autonomous driving and wireless communication technology. Therefore, in addition to saving energy, such a technology can help reduce labor costs by using fewer drivers to fulfill the same demand. Compared to traditional truck platooning, the schedules of drivers under semi-autonomous truck platooning may not be exactly the same as trucks, which are usually neglected in recent studies. Therefore, in this paper, we make the first attempt to develop a mathematical modeling framework to optimise the operation plan of drivers and trucks interdependently. To be mentioned, the truck fleet is equipped with the semi-autonomous platooning function and time windows. Specifically, the operation plan will dictate the number of drivers to be dispatched, and traveling routes and time schedules of drivers and trucks interdependently with the objective to minimise the total operation cost, including drivers' fixed dispatch cost, on-road labor cost, and energy consumption cost while fulfilling all delivery demands. To tackle the large-scale problem in a timely manner, a tailored Lagrangian Relaxation approach is proposed to solve the model. Numerical experiments are conducted to demonstrate the performance of the proposed modeling framework and validate the feasibility and efficiency of the proposed solution algorithm.

Energy-Efficient Joint Collaborative and Passive Beamforming for Intelligent-Reflecting-Surface-Assisted Wireless Sensor Networks

Intelligent reflecting surface (IRS) provides a promising technology that can improve the energy efficiency of wireless communications, by building the controllable wireless propagation environment with the utilization of massive passive reflecting elements. Motivated by this, we apply the IRS to assist the data collection based on collaborative beamforming in Wireless Sensor Networks (WSNs). With the objective to maximize the network lifetime, we present an energy efficient joint collaborative and passive beamforming design for an IRS-assisted WSN, which jointly optimizes the beamforming vector for all nodes and phase shifts for IRS’s reflecting elements. To efficiently resolve the joint optimization problem, we develop a penalty dual decomposition (PDD) based algorithm combined with augmented Lagrangian method, penalty-based method and block coordinate descent algorithm, which decomposes the joint optimization problem into four simplified subproblems. Especially in solving IRS’s phase shifts optimization subproblem, we formulate this subproblem as a quadratically constrained quadratic programming (QCQP) problem and further propose a low computational complexity approximate iteration algorithm to resolve this QCQP problem, which iteratively linearizes this QCQP problem to a sequence of approximate programming with linear equality constraints, whose closed-form solution is presented. Theoretical analysis indicates the proposed algorithm can converge to a KKT solution. Simulation results indicate that the IRS-assisted design and proposed algorithm improve the energy efficiency and prolong the network lifetime as compared to alternative algorithms.

STARS Enabled Integrated Sensing and Communications

A simultaneously transmitting and reflecting surface (STARS) enabled integrated sensing and communications (ISAC) framework is proposed, where the entire space is partitioned by STARS into a sensing space and a communication space. A novel sensing-at-STARS structure is proposed, where dedicated sensors are mounted at STARS to address the significant path loss and clutter interference of sensing. The Cramér-Rao bound (CRB) of the two-dimensional (2D) direction-of-arrivals (DOAs) estimation of the sensing target is derived, which is then minimized subject to the minimum communication requirement. A novel approach is proposed to transform the complicated CRB minimization problem into a trackable modified Fisher information matrix (FIM) optimization problem. Both independent and coupled phase-shift models of STARS are investigated: 1) For the independent phase-shift model, to address the coupling problem of ISAC waveform and STARS coefficient, an efficient double-loop iterative algorithm based on the penalty dual decomposition (PDD) framework is conceived; 2) For the coupled phase-shift model, based on the PDD framework, a low complexity alternating optimization algorithm is proposed to tackle the coupled phase-shift constraint by alternately optimizing the amplitude and phase-shift coefficients of STARS with closed-form expressions. Finally, the numerical results demonstrate that: 1) STARS significantly outperforms conventional RIS in terms of CRB under the communication constraints; 2) The coupled phase-shift model achieves comparable performance to the independent one for low communication requirements or sufficient STARS elements; 3) It is more efficient to increase the number of passive elements of STARS than the active elements of the sensor; 4) Higher sensing accuracy can be achieved by STARS using the practical 2D maximum likelihood estimator compared with the conventional RIS.

Decentralized frequency regulation service provision for virtual power plants: A best response potential game approach

The high-renewable-penetrated power system frequently requires frequency regulation services. By aggregating heterogeneous demand-side flexible resources, the virtual power plants (VPP) are able to quickly respond to the frequency regulation signal, enabling them promising frequency regulation service providers. However, due to the difference in ownership, it is unlikely that the self-interested demand-side resources are willing to follow the VPP-oriented dispatch results. To bridge this gap, this paper proposes a decentralized game-based mechanism for the frequency regulation service provision inside the VPP. Firstly, the supply function-based bidding models of aggregated demand-side resources, namely, demand-side flexible resources and renewable distributed generations, are established. Then, a non-cooperative generalized Nash game for frequency regulation service provision is formulated. The existence, uniqueness, and competitive efficiency loss of the game are analyzed by designing an equivalent best response potential game of the original game. Additionally, a decentralized look-ahead real-time market clearing mechanism for frequency regulation service provision inside the VPP is proposed based on dual decomposition algorithm. Finally, the effectiveness and computation efficiency of the proposed method are validated in the case studies. Meanwhile, it is shown that the competition efficiency loss can be effectively reduced from around 80% to 6% when increasing the demand-side resource population from 20 to 2000 as well as balancing the demand-side resource composition.

Solution strategies for integrated distribution, production, and relocation problems arising in modular manufacturing

Recently, there has been a paradigm shift by certain energy companies towards modular manufacturing, whereby transportable modular production units can be relocated between production facilities to meet the spatial and temporal changes in the availabilities, demands, and prices of the underlying commodities. We refer to the optimal distribution, production, and storage of commodities, including intermediary commodities, and the relocation and operation of the modular production units as the dynamic multiple commodity supply chain problem with modular production units. To this end, we present a “flow-based” and a “path-based” mixed-integer linear programming formulation to model the problem. In an effort to solve large-scale instances of the problem, we propose an iterative three-stage matheuristic for the “path-based” formulation. In the first stage of the matheuristic, a feasible solution to the problem is generated by an integrated column generation and Lagrangian relaxation based heuristic. In the second stage of the matheuristic, a path-relinking procedure is utilized as a local search heuristic to further improve the solution. And in the final stage of the matheuristic, the Lagrangian multipliers are updated via a subgradient method. The effectiveness of the matheuristic is illustrated through numerical experiments with a set of randomly generated test instances. For the large-scale test instances, the results show that the matheuristic produces quality solutions orders of magnitude faster than a “state-of-the-art” mixed-integer linear programming solver.

Freight transport using additional railcars attached to intercity passenger trains with transshipment and railcar circulation: Tabu-search-based Lagrangian heuristic

Developing under-exploited passenger rail networks is emerging as an alternative for freight transport. In light of this, we investigate a long-haul freight mode in which additional railcars for carrying shipments are attached to passenger trains, and the shipments can transfer among the trains at intermediate stops. To coordinate space–time schedules of shipments and railcars, we construct a two-level space–time network: one level captures the departure, transshipment, and arrival of shipments; whereas the other level depicts the holding, reshuffling, and repositioning of railcars. Incorporating variables of space–time arc selection and constraints concerning shipment (un-) loading, locomotive pulling capacity, supply–demand coupling, and flow conservation on networks, a binary integer model is established to minimize the total travel cost of both railcars and shipments. The primal problem is decomposed using Lagrangian relaxation. Lagrangian multipliers and dual bounds are approximated by solving the Lagrangian dual. We devise a primal heuristic based on tabu search for expediting high-quality solutions. Numerical experiments demonstrate computational performance and managerial insights.

FDNet: An end-to-end fusion decomposition network for infrared and visible images.

Infrared and visible image fusion can generate a fusion image with clear texture and prominent goals under extreme conditions. This capability is important for all-day climate detection and other tasks. However, most existing fusion methods for extracting features from infrared and visible images are based on convolutional neural networks (CNNs). These methods often fail to make full use of the salient objects and texture features in the raw image, leading to problems such as insufficient texture details and low contrast in the fused images. To this end, we propose an unsupervised end-to-end Fusion Decomposition Network (FDNet) for infrared and visible image fusion. Firstly, we construct a fusion network that extracts gradient and intensity information from raw images, using multi-scale layers, depthwise separable convolution, and improved convolution block attention module (I-CBAM). Secondly, as the FDNet network is based on the gradient and intensity information of the image for feature extraction, gradient and intensity loss are designed accordingly. Intensity loss adopts the improved Frobenius norm to adjust the weighing values between the fused image and the two raw to select more effective information. The gradient loss introduces an adaptive weight block that determines the optimized objective based on the richness of texture information at the pixel scale, ultimately guiding the fused image to generate more abundant texture information. Finally, we design a single and dual channel convolutional layer decomposition network, which keeps the decomposed image as possible with the input raw image, forcing the fused image to contain richer detail information. Compared with various other representative image fusion methods, our proposed method not only has good subjective vision, but also achieves advanced fusion performance in objective evaluation.

Constrained DRL for Energy Efficiency Optimization in RSMA-Based Integrated Satellite Terrestrial Network.

To accommodate the requirements of extensive coverage and ubiquitous connectivity in 6G communications, satellite plays a more significant role in it. As users and devices explosively grow, new multiple access technologies are called for. Among the new candidates, rate splitting multiple access (RSMA) shows great potential. Since satellites are power-limited, we investigate the energy-efficient resource allocation in the integrated satellite terrestrial network (ISTN)-adopting RSMA scheme in this paper. However, this non-convex problem is challenging to solve using conventional model-based methods. Because this optimization task has a quality of service (QoS) requirement and continuous action/state space, we propose to use constrained soft actor-critic (SAC) to tackle it. This policy-gradient algorithm incorporates the Lagrangian relaxation technique to convert the original constrained problem into a penalized unconstrained one. The reward is maximized while the requirements are satisfied. Moreover, the learning process is time-consuming and unnecessary when little changes in the network. So, an on-off mechanism is introduced to avoid this situation. By calculating the difference between the current state and the last one, the system will decide to learn a new action or take the last one. The simulation results show that the proposed algorithm can outperform other benchmark algorithms in terms of energy efficiency while satisfying the QoS constraint. In addition, the time consumption is lowered because of the on-off design.

Solving the multi-compartment vehicle routing problem by an augmented Lagrangian relaxation method

In real-world routing applications such as waste collection or fuel distribution, multiple products that cannot be mixed should be jointly collected or delivered. Multi-compartment vehicles are applied with great flexibility in routing and assignment decisions for these real-world applications. This study concentrates on the multi-compartment vehicle routing problem with fixed compartment size and fixed assignment products to compartments. For this problem, we develop two new formulations in the space–time network and state-space–time network. For the state-space–time network formulation, we dualize the task allocation constraints to the objective function and decompose the Lagrangian relaxed problem (LR) into a series of identical routing subproblems. To produce high-quality feasible solutions, the augmented Lagrangian relaxed problem (ALR) with superior convergence and feasibility is further applied to this problem. The ALR is decomposed into many solvable routing subproblems by alternating direction method of multipliers. A time-dependent dynamic programming algorithm is developed to settle the obtained subproblems of the LR and ALR. The subgradient optimization method solves the Lagrangian dual problems. The solution quality can be evaluated by the relative gap between the global lower and upper bounds at each iteration. In numerical experiments, the proposed method produces solutions with good integrality gaps. Based on Lagrangian relaxation and decomposition techniques, this research provides a novel solving scheme for the multi-compartment vehicle routing problem.

A multi-objective closed-loop supply chain under uncertainty: An efficient Lagrangian relaxation reformulation using a neighborhood-based algorithm

Population growth and unprecedented industrialization have contributed to a significant surge in global waste generation, underscoring the need for efficient waste management practices such as recycling, disassembling, restoration, and disposal. Closed-loop supply chain (CLSC) networks offer a solution for effective waste management by enabling the recycling, reassembly, and reuse of waste products. However, designing an optimal CLSC network presents challenges due to uncertainties associated with various parameters, including customer demand, sustainable development goals, and the complexity of network design optimization. This paper addresses these challenges by proposing a comprehensive CLSC network that optimizes environmental, economic, and social footprints through a multi-objective optimization approach. To account for parameter uncertainties, we employ scenario generation using a scenario-based stochastic programming procedure. To solve the proposed model, we transform the multi-objective formulation into a single-objective model using a weighted sum method considering a set of problem-specific heuristics. Furthermore, we leverage Lagrangian relaxation theory to formulate various problem reformulations, aimed at attaining an optimal lower bound for the CLSC problem. We complement this with a neighborhood-based algorithm that helps identify a feasible upper bound. By utilizing a sub-gradient framework, we iteratively reoptimize both the selected reformulation and the neighborhood-based algorithm, culminating in the achievement of an optimal solution. To exemplify the practical applicability of our research, we present a case study focusing on the light engineering industry in Bangladesh. Through rigorous analyses, we showcase that our proposed algorithm can deliver an optimal solution with an optimality gap of 0.06. By offering insights into the successful implementation of our CLSC network in a real-world context, we aim to provide valuable guidance for practitioners seeking to establish efficient and sustainable waste management systems.

Research on Resource Allocation Strategy of Indoor Visible Light Communication and Radio Frequency Systems Integrating Orthogonal Frequency-Division Multiple Access Technology

Aiming at the problem of resource allocation strategies limiting the system transmission rate in indoor visible light communication and radio frequency (VLC/RF) systems, a new resource allocation method is proposed, and orthogonal frequency-division multiple access (OFDMA) technology is introduced. The capacity of the communication system is effectively increased. In the VLC/RF system model based on OFDMA, the Lyapunov optimization method is used to transform the time averaging problem into a series of single-slot online problems to reduce its computational complexity, the optimization problem is decomposed into three independent subproblems, and the Lagrange relaxation method and convex optimization theory are used to solve the subproblems to maximize the average transmission rate of the system, and the Lyapunov drift is used to ensure the stability of the system. Simulation verifies that the Lyapunov optimization algorithm does not require iteration, which improves the optimization speed to a high extent. The simulation results show that the proposed resource allocation strategy effectively balances the system queue length and transmission rate, improves the average transmission rate of the system to the greatest extent under the premise of ensuring the stability of the system, and compares with other algorithms from the aspects of system stability and transmission rate, which proves the effectiveness of the Lyapunov optimization algorithm.

Mixed-model moving assembly line material placement optimization for a shorter time-dependent worker walking time

Car mass production commonly involves a moving assembly line that mixes several car models. This requires plenty of material supplies at the line side, but available space is scarce. Thus, material is placed apart from ideal positions. Then, picking it up involves walking along the line. This time is non-productive and can encompass 10–15% of total production time. Thus, it is important to estimate and minimize it during production planning. However, the calculations are difficult because the conveyor continuously moves. Therefore, most literature bounds walking time by a constant, but this discards valuable potential. To better approximate it, we use a time-dependent V-shaped function. A comparison indicates that for a majority of instances, constant walking time estimates of 95% confidence are at least 51% higher. Then, we introduce a model to optimize material positions such that the model-mix walking time is minimized. This poses an NP-hard sequencing problem with a recursive and nonlinear objective function. Our key discovery is a lower bound on the objective of partial solutions, established by a Lagrangian relaxation that can be solved in quadratic time. Resulting branch and bound based algorithms allow to quickly and reliably optimize up to the largest real-world sized instances.

A market mechanism for multiple air traffic resources

We introduce a model that extends the concept of air traffic flow management slot to the concept of time window, allowing to effectively deal with a network of interacting regulations. The model aims at minimising the total cost of delay of a time window allocation to flights and is based on an integer programming problem. It consists in a market-based mechanism between flights and a central authority to trade time windows, which fulfils the properties of individual rationality (every participating airline has a non-negative profit from the mechanism) and weak budget-balance (the mechanism requires no external subsidisation). Equity is assumed to be respected because the First Planned First Served allocation is an endowment guaranteed to all flights and allocated for free. The proposed market mechanism can be implemented in a distributed manner preventing the disclosure of confidential information by airlines, and is based on the Lagrangian relaxation of the integer optimisation problem, solved through the subgradient algorithm. We present some computational experiments conducted to test the model on some real instances of air traffic data.

Blind video watermarking scheme for medical video authentication

Medical video watermarking is one of the beneficial and efficient tools to prohibit important patients' data from illicit enrollment and redistribution. In this paper, a new blind watermarking scheme has been proposed to improve the confidentiality, integrity, authenticity, and perceptual quality of a medical video with minimum distortion. The proposed scheme is based on 2D-DWT and dual Hessenberg-QR decomposition, where the input medical video is initially processed into frames. Then, the processed frames are transformed into sub-bands using 2D-DWT, followed by applying Hessenberg-QR decomposition on the selected wavelet HL2 sub-band. The watermark is scrambled via Arnold cat map to raise confidentiality and then concealed in the modified selected features. The watermark is extracted in a fully blind mode without referencing the original video, which reduces the extraction time. The proposed scheme maintained a fundamental tradeoff between robustness and visual imperceptibility compared to existing methods against many commonly encountered attacks. The visual imperceptibility has been evaluated using well-known metrics PSNR, SSIM, Q-index, and histogram analysis. The proposed scheme achieves a high PSNR value of (70.6899 dB) with minimal distortion and a high robustness level with an average NC value of (0.9998) and BER value of (0.0023) while conserving a large payload capacity. The obtained results show superior performance over similar video watermarking methods. The limitation of this scheme is the elapsed time during the embedding process since we utilized dual Hessenberg-QR decomposition. One possible solution to reduce time consumption is simple decompositions like bound-constrained SVM or similar decompositions.

A Lagrangian Relaxation Method for an Online Decentralized Assignment of Electric Vehicles to Charging Stations

A Lagrangian Relaxation Method for an Online Decentralized Assignment of Electric Vehicles to Charging Stations