Distributed Optimization Algorithm Research Articles

Optimal Whole Slide Image Segmentation Using Generalized Normal Distribution Optimization

Whole slide image (WSI) segmentation is a crucial task aiding tumour and cancerous cell diagnosis. Generalized Normal Distribution Optimization (GNDO) algorithm is adopted for whole slide image segmentation based on thresholding in this paper. GNDO algorithm utilizes the generalized normal distribution's properties to determine the ideal thresholds for image segmentation. Through various metrics, the efficacy of GNDO in comparison to traditional Otsu thresholding methods is demonstrated. As demonstrated by the results, it can offer reliable and flexible solutions for different histopathology images.

Multi-objective generalized normal distribution optimization: a novel algorithm for multi-objective problems

AbstractThis study introduces the Multi-objective Generalized Normal Distribution Optimization (MOGNDO) algorithm, an advancement of the Generalized Normal Distribution Optimization (GNDO) algorithm, now adapted for multi-objective optimization tasks. The GNDO algorithm, previously known for its effectiveness in single-objective optimization, has been enhanced with two key features for multi-objective optimization. The first is the addition of an archival mechanism to store non-dominated Pareto optimal solutions, ensuring a detailed record of the best outcomes. The second enhancement is a new leader selection mechanism, designed to strategically identify and select the best solutions from the archive to guide the optimization process. This enhancement positions MOGNDO as a cutting-edge solution in multi-objective optimization, setting a new benchmark for evaluating its performance against leading algorithms in the field. The algorithm's effectiveness is rigorously tested across 35 varied case studies, encompassing both mathematical and engineering challenges, and benchmarked against prominent algorithms like MOPSO, MOGWO, MOHHO, MSSA, MOALO, MOMVO, and MOAOS. Utilizing metrics such as Generational Distance (GD), Inverted Generational Distance (IGD), and Maximum Spread (MS), the study underscores MOGNDO's ability to produce Pareto fronts of high quality, marked by exceptional precision and diversity. The results affirm MOGNDO's superior performance and versatility, not only in theoretical tests but also in addressing complex real-world engineering problems, showcasing its high convergence and coverage capabilities. The source codes of the MOGNDO algorithm are publicly available at https://nimakhodadadi.com/algorithms-%2B-codes.

Thermodynamic study and the development of a support vector machine model for predicting adsorption behavior of orange peel-derived beads in wastewater treatment

This study investigates the use of orange peels as a precursor for synthesizing sodium alginate-encapsulated beads for methylene blue (MB) removal. The prepared beads (BOP1 and BOP2) underwent characterization through FTIR, XRF, SEM and TGA. Subsequently, the impacts of various factors, including temperature, the initial pH, initial concentration, salt and humic acid, are studied. The adsorption isotherms show high adsorbed quantities of 764.92 and 659.78 mg/g for BOP1 and BOP2 respectively, while the obtained data are best described by the monolayer with two energies (MMTE) model, which is then used to perform a thermodynamic study of the MB adsorption mechanism. Additionally, the adsorption kinetics data are modeled using three models, with the PFO model identified as the most appropriate. The regenerated beads demonstrate the ability to be reused up to 7 cycles, The effects of NaCl and humic acid on MB adsorption reveal that NaCl inhibits adsorption due to competition with Na+, while humic acid has no effect. Finally, a support vector machine (SVM) model optimized by the Levy Flight Distribution Optimization (LFD) algorithm is developed and found to be capable of accurately predicting the adsorption behavior of the prepared beads. This model is then used in optimizing the process conditions for maximal MB removal. Overall, this study demonstrates that the prepared beads could be potential low-cost and environmentally friendly adsorbents for wastewater treatment applications.

Implicit Posteriori Parameter Distribution Optimization in Reinforcement Learning.

Efficient and intelligent exploration remains a major challenge in the field of deep reinforcement learning (DRL). Bayesian inference with a distributional representation is usually an effective way to improve the exploration ability of the RL agent. However, when optimizing Bayesian neural networks (BNNs), most algorithms need to specify an explicit parameter distribution such as a multivariate Gaussian distribution. This may reduce the flexibility of model representation and affect the algorithm performance. Therefore, to improve sample efficiency and exploration based on Bayesian methods, we propose a novel implicit posteriori parameter distribution optimization (IPPDO) algorithm. First, we adopt a distributional perspective on the parameter and model it with an implicit distribution, which is approximated by generative models. Each model corresponds to a learned latent space, providing structured stochasticity for each layer in the network. Next, to make it possible to optimize an implicit posteriori parameter distribution, we build an energy-based model (EBM) with value function to represent the implicit distribution which is not constrained by any analytic density function. Then, we design a training algorithm based on amortized Stein variational gradient descent (SVGD) to improve the model learning efficiency. We compare IPPDO with other prevailing DRL algorithms on the OpenAI Gym, MuJoCo, and Box2D platforms. Experiments on various tasks demonstrate that the proposed algorithm can represent the parameter uncertainty implicitly for a learned policy and can consistently outperform competing approaches.

Adaptive multi-view subspace learning based on distributed optimization

As the rapid development of Internet of Things (IoT), information is collected from different sensors and stored in distributed devices which can be regarded as the multi-view data. There are currently numerous clustering algorithms designed to handle multi-view data. However, most of these algorithms still suffer from the following problems: They are designed to operate directly on raw data, which preserves excessive redundant information. They primarily focus on pairwise relationships between views, neglecting the intricate high-order connections among multiple views. The prior information of singular values is not taken into account in multiple views and different views are considered to have equal contributions for clustering. To efficiently address above problems, adaptive multi-view subspace learning based on distributed optimization (AMSLDO) is proposed in this paper. Specifically, the original multi-view data is projected to a low-dimensional space for subspace representation, and multiple representation matrices are stacked in a tensor with weighted tensor nuclear norm to obtain high-order correlations and discover the prior information of singular values. Furthermore, the consensus matrix is learned from different representation matrices through adaptive weights. Meanwhile, samples are partitioned into the ideal number of clusters through Laplacian rank constraint. An efficient distributed optimization algorithm based on the Alternating Direction Method of Multipliers (ADMM) framework is designed to solve the proposed model. Extensive experiments are conducted on six datasets, demonstrating the superiority of the proposed model compared with eleven state-of-the-art methods.

Resilient distributed optimization for cyber–physical systems under adversarial environments: An event-based method

This work presents a resilient distributed optimization algorithm based on the event-triggering mechanism for cyber–physical systems (CPSs) to optimize an average of convex cost functions corresponding to multiple agents under adversarial environments. Two attack scenarios, including the f-total (each agent is affected by at most f malicious agents in the whole network) and the f-local (each agent is affected by at most f malicious agents in its in-neighbor set) attacks are considered. Subsequently, the convergence conditions under these two attack scenarios are provided, respectively, both of which guarantee that the state values of benign agents converge to a bounded error range. The optimality conditions are also presented by theoretical analysis, which guarantee that the state values of benign agents converge to a safety interval constructed by local optimal values under certain graph conditions, despite the misbehavior of malicious agents. In addition, four numerical examples are presented to show the effectiveness and superiority of the event-triggering resilient distributed optimization (RDO-E) algorithm. Compared to existing resilient algorithms, the proposed method achieves resilient distributed optimization with higher accuracy and less demanding communication overheads. Finally, by applying the proposed method to the multi-microgrid system, a resilient economic dispatch problem (REDP) is successfully solved, which validates the practical viability of the RDO-E algorithm.

Output feedback distributed optimization algorithms of higher‐order uncertain nonlinear multi‐agent systems

AbstractOutput feedback distributed optimization problem is studied for higher‐order multi‐agent systems with uncertain nonlinearities, which are assumed to satisfy linear growth conditions. The agents dynamics are permitted to be heterogenous with different nonlinearities. The problem is solved by embedded control approach based output feedback distributed optimization algorithms. First, a first‐order optimal signal generator is designed, of which outputs reach the minimizer of the global cost function. Second, embedding the generator into the feedback loop and taking the outputs of the generator as the reference outputs of the agents, output feedback tracking controllers are designed for the higher‐order multi‐agent systems on the basis of nonseparation principle and feedback domination technique. Under the proposed control algorithms, all the agents outputs asymptotically approach a bounded neighbor region of the global minimizer. Simulations demonstrate the effectiveness of the proposed output feedback distributed optimization algorithms.

Distributed Discrete-Time Convex Optimization With Closed Convex Set Constraints: Linearly Convergent Algorithm Design.

The convergence rate and applicability to directed graphs with interaction topologies are two important features for practical applications of distributed optimization algorithms. In this article, a new kind of fast distributed discrete-time algorithms is developed for solving convex optimization problems with closed convex set constraints over directed interaction networks. Under the gradient tracking framework, two distributed algorithms are, respectively, designed over balanced and unbalanced graphs, where momentum terms and two time-scales are involved. Furthermore, it is demonstrated that the designed distributed algorithms attain linear speedup convergence rates provided that the momentum coefficients and the step size are appropriately selected. Finally, numerical simulations verify the effectiveness and the global accelerated effect of the designed algorithms.

Distributed Multi-Area Intraday Economic Dispatch Using Modified Critical Region Projection Algorithm

Convergence acceleration is always a critical issue in distributed multi-area scheduling. The critical region projection (CRP) algorithm based on multi-parametric quadratic programming (MPQP) shows better convergence performance for multi-area static economic dispatch. However, its imperfect decomposition framework cannot be directly applied to dynamic economic dispatch. Therefore, this paper modifies the CRP algorithm to achieve fast distributed multi-area intra-day economic dispatch (MAIDED). First, we introduce slack variables to each area and add the corresponding penalty term into their objective functions to present a primal decomposition framework with penalty relaxation. It is more realistic than the decomposition framework of traditional CRP (TCRP). Then, an iterative algorithm of double spatial scale search (DSSS) is developed to improve the convergence rate of distributed solving based on the similar optimal value function in adjacent critical regions (CRs). Moreover, we design an initial value selection method based on data fitting to further reduce the number of iterations. Finally, three interconnected power systems of different sizes are used for numerical testing to demonstrate that the proposed modified CRP (MCRP) algorithm can meet the practical application and has higher convergence efficiency. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is motivated by the problem of distributed MAIDED for power systems but it also applies to other multi-agent networks with a coordinator. The convergence speed of the existing distributed optimization algorithm is slow, which increases the risk of communication failure and attack. Meanwhile, more iterations will result in increased communication and computing costs. This paper modified the TCRP algorithm for high-efficient distributed MAIDED. The penalty relaxation is used to ensure that the decomposition framework is consistent with the actual system operation. The DSSS algorithm and the initial value selection method based on data fitting are designed to effectively reduce the number of iterations. In this paper, the DSSS is that the coordinator enlarges the CRs uploaded from each area to perform a rough optimization for approaching the global optimal solution fast, and then in the next iteration, conducts a precise optimization like the TCRP algorithm to determine the precise optimal solution. The numerical testing of the interconnected power systems of different sizes indicates that the MCRP algorithm has fewer iterations and calculation times than the TCRP algorithm. Future work will extend the MCRP algorithm to solve the distributed multi-area unit commitment problem.

Distributed Optimization of Islanded Microgrids Integrating Multi-Type VSG Frequency Regulation and Integrated Economic Dispatch

The question of how to simultaneously perform frequency regulation and integrated economic scheduling for microgrids with low-inertia islanding operation under communication constraints is a difficult problem that needs to be solved for many current applications. To this end, this paper establishes a microgrid scheduling control model containing a virtual synchronous generator (VSG) with multiple types of power sources and proposes a distributed optimization algorithm that integrates frequency regulation and comprehensive economic scheduling to simultaneously realize frequency regulation and economic scheduling in a microgrid. Firstly, a distributed economic dispatch problem is proposed based on a comprehensive consideration of the costs and benefits of various types of power VSGs, as well as the overall inertia and standby capacity requirements of the microgrid, which minimizes the integrated costs incurred by the participation of each type of VSG in the frequency regulation and improves the stable operation of the microgrid in terms of frequency under perturbation. Then, the optimal scheduling problem is solved by reconstructing the optimization problem based on considering the dynamic characteristics of microgrid inverters and using event-triggered communication to sense and compensate for the supply-demand imbalance online. The proposed method can avoid inter-layer coordination across time scales, improve the inertia, frequency regulation capability, and economy of the system, and enhance its robustness to short-term communication failures. Finally, simulation results are used to verify the effectiveness of the method.

Prescribed-time distributed optimization problem with constraints

In recent years, distributed optimization problem have a wide range of applications in various fields. This paper considers the prescribed-time distributed optimization problem with/without constraints. Firstly, we assume the state of each agent is constrained, and the prescribed-time distributed optimization algorithm with constraints is designed on the basis of gradient projection algorithm and consensus algorithm. Secondly, the constrained distributed optimization problem is transformed into the unconstrained distributed optimization problem, and according to the gradient descent algorithm and consensus algorithm, we also propose the prescribed-time distributed optimization algorithm without constraints. By designing the appropriate objective functions, we prove the multi-agent system can converge to the optimal solution within any prescribed-time, and the convergence time is fully independent of the initial conditions and system parameters. Finally, three simulation examples are provided to verify the validity of the designed algorithms.

Robust Beamforming for Downlink Multi-Cell Systems: A Bilevel Optimization Perspective

Utilization of inter-base station cooperation for information processing has shown great potential in enhancing the overall quality of communication services (QoS) in wireless communication networks. Nevertheless, such cooperations require the knowledge of channel state information (CSI) at base stations (BSs), which is assumed to be perfectly known. However, CSI errors are inevitable in practice which necessitates beamforming technique that can achieve robust performance in the presence of channel estimation errors. Existing approaches relax the robust beamforming design problems into semidefinite programming (SDP), which can only achieve a solution that is far from being optimal. To this end, this paper views robust beamforming design problems from a bilevel optimization perspective. In particular, we focus on maximizing the worst-case weighted sum-rate (WSR) in the downlink multi-cell multi-user multiple-input single-output (MISO) system considering bounded CSI errors. We first reformulate this problem into a bilevel optimization problem and then develop an efficient algorithm based on the cutting plane method. A distributed optimization algorithm has also been developed to facilitate the parallel processing in practical settings. Numerical results are provided to confirm the effectiveness of the proposed algorithm in terms of performance and complexity, particularly in the presence of CSI uncertainties.

Fixed‐time distributed adaptive optimization for third‐order nonlinear fully heterogeneous vehicular platoon systems

SummaryIn this article, the problem of fixed‐time distributed optimization is researched for third‐order fully heterogeneous nonlinear connected and autonomous vehicles. To address this problem, a fixed‐time distributed optimization algorithm is proposed via a two‐step strategy. First, an optimization algorithm is proposed to generate a virtual reference signal that can converge to the optimal solution. Then, we further construct an adaptive tracking controller to track the virtual signal based on the first step. In the design process of the controller, the virtual control variables and the actual control input are obtained by using the backstepping technique. Furthermore, a fast fixed‐time filter is introduced to avoid the issue of discontinuous gradient functions. Fuzzy logic systems are employed to address the unknown part of the nonlinear function. By using tools from the fixed‐time stability criterion, convex optimization theory, and Lyapunov stability theory, it can be demonstrated that the provided strategy drives all optimization variables to the optimal solution within a fixed time and guarantees the closed‐loop system signals are bounded. Finally, the effectiveness of control strategy is verified via a numerical simulation.

Development of AVR controller performance using exponential distribution and transit search optimization techniques

This paper attempts to obtain the optimal solution to enhance the performance of the Automatic Voltage Regulator (AVR) Controller, as it is an essential tool to control the synchronous generator output voltage. The controller improves AVR system stability and response time; moreover, it is demonstrated that the Proportional Integral Derivative (PID) controller achieves the goal by applying two artificial intelligence techniques to design the optimal values of the Automatic Voltage Regulator (AVR) PID controller for a single area model. The first is the Exponential Distribution Optimization Algorithm (EDO), and the second is the Transit Search Optimization Algorithm (TS). EDO and TS are used to determine the best PID controller parameters and have also recently been developed in the breadth of optimization problems and associated computational complexities field. The objective function, Integrated Square Error (ISE), minimizes the error voltage for improved stability and response. The outcomes are compared to various optimization techniques to prove the validation of the two proposed methods. The results show that the EDO and TS proved their superiority through their stability level to the AVR system and their steady-state error improvement. Moreover, the dominant effect of damping frequency decreases the oscillation and the reduced maximum overshoot that protects the loads from being subjected to non-permissible over-voltage levels. Finally, a robustness test is applied to the two proposed optimization methods to prove their reliability and effectiveness.

Accelerated Distributed Optimization Algorithm With Malicious Nodes

Accelerated Distributed Optimization Algorithm With Malicious Nodes

Optimization-Based Ramping Reserve Allocation of BESS for AGC Enhancement

The transient behavior of Automatic Generation Control (AGC) systems is a critical aspect of power system operation. Therefore, fully extracting the potential of Battery Energy Storage Systems (BESSs) for AGC enhancement is of paramount importance. In light of the challenges posed by diverse resource interconnections and the variability associated, we propose an online optimization scheme that can adapt to changes in an unknown and variable environment. To leverage the synergy between BESSs and Conventional Generators (CGs), we devise a variant of the Area Injection Error (AIE) as a measure to quantify the ramping needs. Based on this measure, we develop a distributed optimization algorithm with adaptive learning rates for the allocation of the ramping reserve. The algorithm restores a larger learning rate for compliance with the ramping needs upon detecting a potentially destabilizing event. We demonstrate the effectiveness and scalability of the proposed scheme through comprehensive case studies. It is shown that the proposed scheme can improve the transient behavior of the AGC system by bridging the gap in ramping capability.

Communication-efficient distributed cubic Newton with compressed lazy Hessian

Recently, second-order distributed optimization algorithms have been becoming a research hot in distributed learning, due to their faster convergence rate than the first-order algorithms. However, second-order algorithms always suffer from serious communication bottleneck. To conquer such challenge, we propose communication-efficient second-order distributed optimization algorithms in the parameter-server framework, by incorporating cubic Newton methods with compressed lazy Hessian. Specifically, our algorithms require each worker communicate compressed Hessians with the server only at some particular iterations, which can save both communication bits and communication rounds. For non-convex problems, we theoretically prove that our algorithms can reduce the communication cost comparing to the state-of-the-art second-order algorithms, while maintaining the same iteration complexity order O(ϵ−3/2) as the centralized cubic Newton methods. By further using gradient regularization technique, our algorithms can achieve global convergence for convex problems. Moreover, for strongly convex problems, our algorithms can achieve local superlinear convergence rate without any requirement on initial conditions. Finally, numerical experiments are conducted to show the high efficiency of the proposed algorithms.

A Three-Stage Optimal Operation Strategy of Interconnected Microgrids With Rule-Based Deep Deterministic Policy Gradient Algorithm.

The ever-increasing requirements of demand response dynamics, competition among different stakeholders, and information privacy protection intensify the challenge of the optimal operation of microgrids. To tackle the above problems, this article proposes a three-stage optimization strategy with a deep reinforcement learning (DRL)-based distributed privacy optimization. In the upper layer of the model, the rule-based deep deterministic policy gradient (DDPG) algorithm is proposed to optimize the load migration problem with demand response, which enhances dynamic characteristics with the interaction between electricity prices and consumer behavior. Due to the competition among different stakeholders and the information privacy requirement in the middle layer of the model, a potential game-based distributed privacy optimization algorithm is improved to seek Nash equilibriums (NEs) with encoded exchange information by a distributed privacy-preserving optimization algorithm, which can ensure the convergence as well as protect privacy information of each stakeholder. In the lower layer of the model of each stakeholder, economic cost and emission rate are both taken as operation objectives, and a gradient descent-based multiobjective optimization method is employed to approach this objective. The simulation results confirm that the proposed three-stage optimization strategy can be a viable and efficient way for the optimal operation of microgrids.

Efficient and choreographed quality-of- service management in dense 6G verticals with high-speed mobility requirements

Future 6G networks are envisioned to support very heterogeneous and extreme applications (known as verticals). Some examples are further-enhanced mobile broadband communications, where bitrates could go above one terabit per second, or extremely reliable and low-latency communications, whose end-to-end delay must be below one hundred microseconds. To achieve that ultra-high Quality-of-Service, 6G networks are commonly provided with redundant resources and intelligent management mechanisms to ensure that all devices get the expected performance. But this approach is not feasible or scalable for all verticals. Specifically, in 6G scenarios, mobile devices are expected to have speeds greater than 500 kilometers per hour, and device density will exceed ten million devices per square kilometer. In those verticals, resources cannot be redundant as, because of such a huge number of devices, Quality-of-Service requirements are pushing the effective performance of technologies at physical level. And, on the other hand, high-speed mobility prevents intelligent mechanisms to be useful, as devices move around and evolve faster than the usual convergence time of those intelligent solutions. New technologies are needed to fill this unexplored gap. Therefore, in this paper we propose a choreographed Quality-of-Service management solution, where 6G base stations predict the evolution of verticals at real-time, and run a lightweight distributed optimization algorithm in advance, so they can manage the resource consumption and ensure all devices get the required Quality-of-Service. Prediction mechanism includes mobility models (Markov, Bayesian, etc.) and models for time-variant communication channels. Besides, a traffic prediction solution is also considered to explore the achieved Quality-of-Service in advance. The optimization algorithm calculates an efficient resource distribution according to the predicted future vertical situation, so devices achieve the expected Quality-of-Service according to the proposed traffic models. An experimental validation based on simulation tools is also provided. Results show that the proposed approach reduces up to 12% of the network resource consumption for a given Quality-of-Service.

Evaluation of axis straightness error in the machining of hole and shaft parts based on improved exponential distribution optimizer

The straightness error of hole and shaft parts is one of the important parameters to reflect machining quality. The modeling process of traditional mathematical methods is complex, and the solution precision is not high. The intelligent optimization algorithm has significant advantages in solving this kind of problem. It can depend on random operators jumping out of local optima and does not need to calculate a large gradient information. Therefore, this paper proposes an improved exponential distribution optimization (IEDO) algorithm to achieve the minimum zone evaluation of straightness error. Firstly, the mathematical model of minimum zone method for axis straightness evaluation is established as the objective function. Secondly, the principle of the basic exponential distribution optimization (EDO) algorithm is described, and the exponential distribution optimizer is improved in three aspects: in the initialization, the interval shortening strategy is introduced to solve the problem of uneven initial population distribution; the adaptive switch probability is proposed to replace the constant value (0.5) to balance the ability of global exploration and local exploitation; a guidance strategy based on weight is proposed to guide the search process to reach the global optimal quickly. Then, nine typical benchmark functions are utilized to test the performance of the improved algorithm, which reveals satisfactory results. Finally, IEDO successfully applies to evaluation of axis straightness error with good accuracy.