Alternating Direction Method Of Multipliers Algorithm Research Articles

Sequential inertial linear ADMM algorithm for nonconvex and nonsmooth multiblock problems with nonseparable structure

The alternating direction method of multipliers (ADMM) has been widely used to solve linear constrained problems in signal processing, matrix decomposition, machine learning, and many other fields. This paper introduces two linearized ADMM algorithms, namely sequential partial linear inertial ADMM (SPLI-ADMM) and sequential complete linear inertial ADMM (SCLI-ADMM), which integrate linearized ADMM approach with inertial technique in the full nonconvex framework with nonseparable structure. Iterative schemes are formulated using either partial or full linearization while also incorporating the sequential gradient of the composite term in each subproblem’s update. This adaptation ensures that each iteration utilizes the latest information to improve the efficiency of the algorithms. Under some mild conditions, we prove that the sequences generated by two proposed algorithms converge to the critical points of the problem with the help of KŁ property. Finally, some numerical results are reported to show the effectiveness of the proposed algorithms.

A decentralized HC-ADMM approach for large antenna arrays

This work addresses the evolving landscape of internet of things (IoT) applications and large antenna array systems, where optimizing spectral efficiency and simplifying design complexities are crucial. Focusing on two key challenges, the study introduces a novel hybrid analog-digital transceiver strategy tailored for frequency-selective channels. By integrating Shannon and Hartley theorems, the approach enhances data transfer rates, thereby optimizing radio frequency (RF) chain utilization in large-scale antennas. To achieve a balance between transceiver performance and hardware complexity, the study employs a decentralized alternating direction method of multipliers (ADMM) framework. The proposed hybrid consensus ADMM algorithm (HC-ADMM) ensures efficient convergence in decentralized optimization scenarios. Comparative analyses with ADMM and existing system transceiver optimization (ESTO) models highlight HC-ADMM's superior performance across key metrics such as spectral efficiency, efficiency per cell, total efficiency, and optimal scheduling of user equipment (UEs). Particularly notable is HC-ADMM's advanced optimization capabilities as the number of transmit antennas increases, positioning it as a promising approach for enhancing overall communication network performance.

Efficient decentralized optimization for edge-enabled smart manufacturing: A federated learning-based framework

The volume of industrial data of smart manufacturing is growing rapidly. Edge computing has emerged as an advanced technique that provides scalable resources for Industrial Internet of Things (IIoT) devices to analyze industrial data and enhance productivity. However, real-time and privacy-protecting data services are crucial for comprehensive decision-making, particularly utilizing machine learning in edge-enabled smart manufacturing. Therefore, how to optimize efficiently the data processing has become a critical concern for manufacturers seeking to upgrade the overall manufacturing operations. In this paper, we propose a decentralized federated learning-based framework to ensure the processing of industrial data in a low-latency and secure manner. Our solution designs the combination between edge nodes and IIoT devices as a global consensus problem with equilibrium constraints. Moreover, we adopt the distributed Alternating Direction Method of Multipliers (ADMM) algorithm to optimize the proposed data processing model, considering its advantages of decomposability and parallelism. To flexibly handle various types of industrial data, such as low-rank and high-dimensional data, we present two types of inexact ADMM algorithms to provide efficient model training services, respectively. Furthermore, an integrated optimization flow is designed for data processing in edge-enabled smart manufacturing. Finally, using industrial datasets from a thermal power plant for steam prediction case, we show that the presented inexact algorithms can decrease the response time by up to 17.2% and 58% respectively compared to other existing algorithms, respectively, while achieving the comparable level of statistical accuracy.

Optimal querying for communication-efficient ADMM using Gaussian process regression

In distributed optimization schemes consisting of a group of agents connected to a central coordinator, the optimization algorithm often involves the agents solving private local sub-problems and exchanging data frequently with the coordinator to solve the global distributed problem. In those cases, the query-response mechanism usually causes excessive communication costs to the system, necessitating communication reduction in scenarios where communication is costly. Integrating Gaussian processes (GP) as a learning component to the Alternating Direction Method of Multipliers (ADMM) has proven effective in learning each agent’s local proximal operator to reduce the required communication exchange. A key element for integrating GP into the ADMM algorithm is the querying mechanism upon which the coordinator decides when communication with an agent is required. In this paper, we formulate a general querying decision framework as an optimization problem that balances reducing the communication cost and decreasing the prediction error. Under this framework, we propose a joint query strategy that takes into account the joint statistics of the query and ADMM variables and the total communication cost of all agents in the presence of uncertainty caused by the GP regression. In addition, we derive three different decision mechanisms that simplify the general framework by making the communication decision for each agent individually. We integrate multiple measures to quantify the trade-off between the communication cost reduction and the optimization solution’s accuracy/optimality. The proposed methods can achieve significant communication reduction and good optimization solution accuracy for distributed optimization, as demonstrated by extensive simulations of a distributed sharing problem.

A Machine Learning-Assisted Distributed Optimization Method for Inverter-Based Volt-VAR Control in Active Distribution Networks

The number of smart inverters in active distribution networks is growing rapidly, making it challenging to realize a fast, distributed Volt/Var control (VVC). This work proposes a machine learning-assisted distributed algorithm to accelerate the solution of the VVC strategy. We first observe the convergence process of the Alternating Direction Method of Multipliers (ADMM)-based VVC problem and explore the potential relationships between the convergence and time-series regression. Then, the long short-term memory (LSTM) technique is applied to learn the convergence process and regress the converged values of the dual and global variables with previous ADMM observations. After that, the LSTM-assisted ADMM algorithm is proposed, where the regressions are used for ADMM parameter updates. In this algorithm, the inputs of the LSTM model are carefully designed since the complementary conditions implied in the conventional ADMM should be considered. Unlike existing methods, the proposed method does not use the LSTM to determine the VVC strategy directly, indicating that it is non-intrusive and can satisfy all safety constraints during operations. The proof of its optimality and convergence is also given. The numerical simulations on the 33-bus distribution system demonstrate the effectiveness and efficiency of the proposed method.

A distributed robust state estimation method based on alternating direction method of multipliers for integrated electricity‐heat system

AbstractIntegrated electricity‐heat system (IEHS) has been paid more and more attention in recent years for its advantage in improving energy efficiency, reducing carbon emissions and increasing renewable energy penetration. To ensure the safety, reliability and economic operation of IEHS, several centralised state estimation (SE) methods for IEHS have been proposed. However, power systems and heat systems often belong to different management entities, and there are industrial barriers such as information privacy, operational differences, and target differences between them, which leads to less applicability of centralised SE methods. In addition, the robustness of existing distributed SE methods for IEHS is not satisfactory. To this end, a distributed robust state estimation (DRSE) model for IEHS based on the alternating direction method of multipliers (ADMM) is proposed. Firstly, by introducing auxiliary state variables and measurements, a robust linear SE model based on weighted least absolute values (WLAV) is proposed. Then, second‐order cone constraints composed of auxiliary state variables are added to the SE model, leading a SOCP‐based robust SE model. Finally, the ADMM algorithm is used to solve the proposed SOCP‐based robust SE model. Simulations demonstrate that the proposed method has higher estimation accuracy in both general and strongly correlated adverse data tests and also can ensure data privacy, good robustness and high estimation accuracy. This indicates that the method proposed has good robustness and solves the problem of weak robustness of existing distributed static state estimation methods.

Asynchronous ADMM for nonlinear continuous‐time systems

AbstractThis paper presents synchronous as well as asynchronous formulations of the alternating direction method of multipliers (ADMM) for solving continuous‐time nonlinear distributed model predictive control (DMPC) problems. It is shown that the optimal control problems of certain system classes can be transformed to fit the consensus‐based ADMM variant problem formulation. The arising subproblems are solved locally on the agent level while the consensus step is solved centrally by a coordinator. Furthermore, the convergence of the synchronous and asynchronous ADMM algorithms to their respective first‐order optimality conditions is presented in a continuous‐time setting. The algorithm is applied to different example systems for which the convergence behavior and influence of the individual algorithmic parameters are investigated. The computation time of the agents remains unaffected by the system size and thus demonstrates the applicability to high‐scaled systems. Moreover, results show that the asynchronous algorithm performs better in terms of execution time when compared to its synchronous counterpart.

Distributed ADMM power optimal control for standalone hybrid generation systems

With the rapid development and increased demand for renewable energy sources, standalone hybrid generation systems have become an essential energy solution. Power optimization control is thus critical to achieving the efficient operation and stability of this system. The distributed ADMM (alternating direction method of multipliers)-based approach has the full potential to deal with the power optimization problem of standalone hybrid generation systems. This study uses an optimization algorithm with a Gaussian penalty function, ADMM-ρ, to alternately optimize the power reference values of wind, light, and battery-containing power generation subsystems. The local controller regulates the output power of the converter according to this reference value. This ensures that the wind and photovoltaic power generation subsystem work in load-tracking or maximum power-tracking modes so that the optimal operation of hybrid power generation meets the balance of supply and demand while prolonging the service life of the batteries. Simulation experiments show that the distributed ADMM algorithm can reliably address the power optimization challenge of hybrid power generation systems.

Privacy-Preserving Distributed ADMM With Event-Triggered Communication.

This article addresses distributed optimization problems, in which a group of agents cooperatively minimize the sum of their private objective functions via information exchanging. Building on alternating direction method of multipliers (ADMM), we propose a privacy-preserving and communication-efficient decentralized quadratically approximated ADMM algorithm, termed PC-DQM, for solving such type of problems under the scenario of limited communication. In PC-DQM, an event-triggered mechanism is designed to schedule the communication instants for reducing communication cost. Simultaneously, for privacy preservation, a Hessian matrix with perturbed noise is introduced to quadratically approximate the objective function, which results in a closed form of primal vector update and then avoids solving a subproblem at each iteration with possible high computation cost. In addition, the triggered scheme is also utilized to schedule the update of Hessian, which can also reduce computation cost. We theoretically show that PC-DQM can protect privacy but without losing accuracy. In addition, we rigorously prove that PC-DQM converges linearly to the exact optimal solution for strongly convex and smooth objective functions. Finally, numerical simulation is presented to illustrate the effectiveness and efficiency of our algorithm.

Imputed quantile vector autoregressive model for multivariate spatial–temporal data

AbstractImputing missing values in multivariate spatial–temporal data is important in many fields. Existing low rank tensor learning methods are popular for handling this task but are sensitive to high level of skewness. The aim of this paper is to develop an alternative method with robustness and high imputation accuracy for multivariate spatial–temporal data. In view of the fact that quantile regression is robust to noises and outliers, we propose an imputed quantile vector autoregressive (IQVAR) model. IQVAR can simultaneously impute missing values and estimate parameters of quantile vector autoregressive model. The objective function includes check loss and nuclear norm penalization. We develop an ADMM (Alternating Direction Method of Multipliers) algorithm to solve the resulting optimization problem. Simulation studies and real data analysis are conducted to verify the efficiency of IQVAR. Compared with other approaches, IQVAR is more robust and accurate.

The ADMM algorithm for audio signal recovery and performance modification with the dual Douglas-Rachford dynamical system

<abstract><p>Practitioners employ operator splitting methods—such as alternating direction method of multipliers (ADMM) and its "dual" Douglas-Rachford method (DR)—to solve many kinds of optimization problems. We provide a gentle introduction to these algorithms, and illustrations of their duality-like relationship in the context of solving basis pursuit problems for audio signal recovery. Recently, researchers have used the dynamical systems associated with the iterates of splitting methods to motivate the development of schemes to improve performance. These developments include a class of methods that act by iteratively minimizing surrogates for a Lyapunov function for the dynamical system. An exemplar of this class is currently state-of-the-art for the feasibility problem of finding wavelets with special structure. Early experimental evidence has also suggested that, when implemented in a primal-dual (ADMM and DR) framework, this exemplar may provide improved performance for basis pursuit problems. We provide a reasonable way to compute the updates for this exemplar, and we study the application of this method to the aforementioned basis pursuit audio problems. We provide experimental results and visualizations of the dynamical system for the dual DR sequence. We observe that for highly structured problems with real data, the algorithmic behavior is noticeably different than for randomly generated problems.</p></abstract>

Research on power sharing optimization strategy based on hybrid multi-microgrid

To alleviate the increasing distributed energy penetration problem in the distribution network under the “double carbon”, a P2P multi-microgrid power sharing and trading method based on the alternating direction method of multipliers (ADMM) are proposed in the hybrid multi-microgrid. The paper constructs the optimization model of each sub-microgrid and a multi-microgrid model for P2P power sharing transactions based on the ADMM algorithm. Finally, the proposed model is simulated and verified using a multi-microgrid composed of three microgrids. The results show that the electric energy trading tariff between microgrids obtained from the model in this paper is reasonable, which can improve the utilization rate of new energy and reduce the penetration rate of distributed energy in the distribution network.

Adaptive Trend Filtering for ECG Denoising and Delineation.

Standard recordings of electrocardiograhic signals are contaminated by a large variety of noises and interferences, which impair their analysis and the further related diagnosis. In this article, we propose a method, based on compressive sensing techniques, to remove the main noise artifacts and to locate the main features of the pulses in the electrocardiogram (ECG). The motivation is to use trend filtering with a varying proximal parameter, in order to sequentially capture the peaks of the ECG, which have different functional regularities. The practical implementation is based on an adaptive version of the alternating direction method of multiplier (ADMM) algorithm. We present results obtained on simulated signals and on real data illustrating the validity of this approach, showing that results in peak localization are very good in both cases and comparable to state of the art approaches.

Multi-block alternating direction method of multipliers for ultrahigh dimensional quantile fused regression

In this paper, we consider a quantile fused LASSO regression model that combines quantile regression loss with the fused LASSO penalty. Intuitively, this model offers robustness to outliers, thanks to the quantile regression, while also effectively recovering sparse and block coefficients through the fused LASSO penalty. To adapt our proposed method for ultrahigh dimensional datasets, we introduce an iterative algorithm based on the multi-block alternating direction method of multipliers (ADMM). Moreover, we demonstrate the global convergence of the algorithm and derive comparable convergence rates. Importantly, our ADMM algorithm can be easily applied to solve various existing fused LASSO models. In terms of theoretical analysis, we establish that the quantile fused LASSO can achieve near oracle properties with a practical penalty parameter, and additionally, it possesses a sure screening property under a wide class of error distributions. The numerical experimental results support our claims, showing that the quantile fused LASSO outperforms existing fused regression models in robustness, particularly under heavy-tailed distributions.

Activating demand side flexibility market in a fully decentralized P2P transactive energy trading framework using ADMM algorithm

The communication network development and rising penetration of prosumers in distribution networks pave the way to emerge a fully decentralized energy transactions framework. Prosumers can share their excess energy production in a new market called the peer-to-peer (P2P) market. This paper proposes a decentralized P2P energy trading market that facilitates electrical energy trading between customers. In this new market, customers act as individual players and decide on their own willingness to maximize their social welfare by selling the extra energy production in the P2P market. Other participants can compensate for their energy deficiency in the P2P market at lower prices. In the decentralized framework, the participant's privacy is protected by sharing a limited amount of data between peers. The P2P market paved the way for another technology named the DSF market. The DSF market is activated by the distribution system operator (DSO) in the local P2P market framework. When congestion is detected in the following day, the DSO requests flexibility from the demand-side flexibility (DSF) and publishes a flexibility requirement table (FRT). Such market structure can incentivize customers to participate in DSF programs and help the main grid by mitigating congestion. This paper employed the alternating direction method of multipliers (ADMM) algorithm for solving the decentralized market. Numerical studies are carried out for several peers in a local community. Simulation results demonstrate that the customers participating in the P2P market supply 15.9% of their demand within the P2P market. Additionally, the findings indicate that customers exhibit a willingness to engage in the DSF program. Moreover, it is observed that 99% of the demand changes requested from the DSO in the FRT are adhered to by the peers.

Fast alternating direction algorithms for sparse portfolio model via sorted ℓ1-norm

In this paper, we propose two novel Alternating Direction Method of Multipliers (ADMM) algorithms for the sparse portfolio problem via sorted ℓ1-norm penalization (SLOPE). The first algorithm (FADMM) is presented by adding a prediction-correction step to the classic ADMM framework. Since the problem is not strongly convex, the second fast ADMM (FADMMR) is proposed by utilizing both prediction-correction step and restarting rules. Numerical experiments show that the FADMMR algorithm converges faster than the FADMM algorithm and ADMM algorithm when tuning parameters are relatively small. On the other hand, when tuning parameters are relative large, the FADMM algorithm performs better than the FADMMR algorithm and ADMM algorithm. The FADMM algorithm and FADMMR algorithm converge faster than the ADMM algorithm in terms of convergence time for different sizes of tuning parameters. For large-scale portfolio problem, the proposed algorithms have highly performance as well. Finally, empirical analysis on five datasets of stocks index show that the proposed algorithms are efficient and superior for solving sparse portfolio problems via SLOPE.

DC-DistADMM: ADMM Algorithm for Constrained Optimization Over Directed Graphs

This article reports an algorithm for multi-agent distributed optimization problems with a common decision variable, local linear equality and inequality constraints and set constraints with convergence rate guarantees. \textcolor{black}{The algorithm accrues all the benefits of the Alternating Direction Method of Multipliers (ADMM) approach}. It also overcomes the limitations of existing methods on convex optimization problems with linear inequality, equality and set constraints by allowing directed communication topologies. Moreover, the algorithm can be synthesized distributively. The developed algorithm has: (i) a $O(1/k)$ rate of convergence, where $k$ is the iteration counter, when individual functions are convex but not-necessarily differentiable, and (ii) a geometric rate of convergence to any arbitrary small neighborhood of the optimal solution, when the objective functions are smooth and restricted strongly convex at the optimal solution. The efficacy of the algorithm is evaluated by a comparison with state-of-the-art constrained optimization algorithms in solving a constrained distributed $\ell_1$-regularized logistic regression problem, and unconstrained optimization algorithms in solving a $\ell_1$-regularized Huber loss minimization problem. Additionally, a comparison of the algorithm's performance with other algorithms in the literature that utilize multiple communication steps is provided.

Extended ADMM for general penalized quantile regression with linear constraints in big data

Quantile regression offers a powerful means of understanding the comprehensive relationship between response variables and predictors. By formulating prior domain knowledge and assumptions as constraints on parameters, the estimation efficiency can be enhanced. This paper studies some methods based on multi-block ADMM (Alternating Direction Method of Multipliers) to fit general penalized quantile regression models with linear constraints on regression coefficients. Different formulations for handling linear constraints and general penalties are explored and compared. Among these formulations, the most efficient one is identified, which provides an explicit expression for each parameter during iterations and eliminates the nested-loop in existing algorithms. Furthermore, this work addresses the challenges posed by big data by developing a parallel ADMM algorithm suitable for distributed data storage. The algorithm’s convergence and a robust stopping criterion are established. To demonstrate the excellent performance of the proposed algorithms, extensive numerical experiments and a real data example are presented. These empirical validations showcase the effectiveness of the methods in handling complex datasets. The details of theoretical proofs and different algorithm variations are provided in the Appendix.

Multispectral and hyperspectral image fusion based on low-rank unfolding network

Recently, deep unfolding networks (DUNs) have been applied to the fusion of low spatial resolution hyperspectral (LR HS) and high spatial resolution multispectral (HR MS) images and achieved satisfactory high spatial resolution hyperspectral (HR HS) images. However, the low-rank and sparse priors in these networks are not exploited sufficiently. In this paper, we establish an LR HS and HR MS image fusion model based on robust principal component analysis (RPCA), which simultaneously captures the low-rank and sparse properties in HS images. Then, the fusion model is optimized with the alternating direction method of multipliers (ADMM). To make full use of the representation capacity of DUNs, we unfold the derived ADMM algorithm as a network, named low-rank unfolding network (LRU-Net). Specifically, each iteration in ADMM is unfolded as one stage in LRU-Net, in which the low-rank and sparse priors are learned by singular value thresholding (SVT) and sparse module, respectively. Finally, all features from all stages are integrated to produce the desired HR HS image. Three benchmark datasets were chosen for comparison to demonstrate the effectiveness of the proposed LRU-Net. The experimental results demonstrate that LRU-Net performs better in terms of both qualitative and quantitative results compared to state-of-the-art fusion methods. The source code is publicly available at https://github.com/RSMagneto/LRU-Net.

Integrating alternating direction method of multipliers and bush for solving the traffic assignment problem

This paper introduces two novel parallel algorithmic frameworks to address the user equilibrium traffic assignment problem (UE-TAP). Most of the existing solution algorithms for the UE-TAP are executed in a sequential manner. This study endeavors to explore parallel computing methods based on model decomposition. Considering that the TAPs can be decomposed based on their origins, thus, following the spirit of the alternating direction method of multipliers (ADMM), a new parallel algorithm B-ADMM is proposed, which integrates the concept of the bush. Subsequently, the convergence of the proposed algorithm is rigorously proven. To enhance the algorithmic parallelism while maintaining the convergence efficiency of the B-ADMM algorithm, this paper further employs the parallel block coordinate descent (PBCD) method to improve the B-ADMM algorithm. We develop a bi-level parallel algorithm PBCD-ADMM, in which the independent origins/bushes are separated into several blocks, and the origin/bush-based restricted subproblems in each block can be solved in parallel. Furthermore, for the given bush belonging to a block, the bush links can also be grouped into several sub-blocks based on the original link-blocking scheme. Thus, these link-based subproblems in each sub-block can also be solved in parallel. A numerical experiment is conducted to validate the proposed algorithms, which indicates that the two new parallel algorithms perform better in terms of convergence speed compared with the original ADMM algorithm.