Alternating Direction Method Of Multipliers Framework Research Articles

Simultaneous Activity and Attenuation Estimation in TOF-PET With TV-Constrained Nonconvex Optimization.

An alternating direction method of multipliers (ADMM) framework is developed for nonsmooth biconvex optimization for inverse problems in imaging. In particular, the simultaneous estimation of activity and attenuation (SAA) problem in time-of-flight positron emission tomography (TOF-PET) has such a structure when maximum likelihood estimation (MLE) is employed. The ADMM framework is applied to MLE for SAA in TOF-PET, resulting in the ADMM-SAA algorithm. This algorithm is extended by imposing total variation (TV) constraints on both the activity and attenuation map, resulting in the ADMM-TVSAA algorithm. The performance of this algorithm is illustrated using the penalized maximum likelihood activity and attenuation estimation (P-MLAA) algorithm as a reference.

Poisson image deblurring with frame-based nonconvex regularization

Poisson image deblurring, which aims to restore the latent image from its blurred and noisy observation, has drawn significant attention in image processing. Due to its ill-posed nature, enhancing image quality often involves incorporating a well-defined prior to effectively regularize the ill-posed inverse problem. Building upon the framelet system, we propose a frame-based nonconvex regularization method for Poisson image deblurring. The method is formulated by combining a data-fitting term with the difference of two norms, namely ℓ1 and ℓ2, on the latent image. We solve the optimization problem by combining the difference of convex functions algorithm (DCA) with the alternating direction method of multipliers (ADMM), establishing its convergence. We further employ a simulated annealing procedure and show that proposed algorithm almost certainly converges to a global minimum. Two different approaches are employed to handle the frame-based ℓ1 norm within the ADMM framework. In particular, the frame-based nonconvex regularization method is also considered for the blind Poisson problem. An effective recovery model and its algorithm are presented. Experimental results demonstrate the effectiveness of our proposed models compared with other models in terms of quantitative metrics and visual quality.

Fast alternating direction algorithms for sparse portfolio model via sorted ℓ1-norm

In this paper, we propose two novel Alternating Direction Method of Multipliers (ADMM) algorithms for the sparse portfolio problem via sorted ℓ1-norm penalization (SLOPE). The first algorithm (FADMM) is presented by adding a prediction-correction step to the classic ADMM framework. Since the problem is not strongly convex, the second fast ADMM (FADMMR) is proposed by utilizing both prediction-correction step and restarting rules. Numerical experiments show that the FADMMR algorithm converges faster than the FADMM algorithm and ADMM algorithm when tuning parameters are relatively small. On the other hand, when tuning parameters are relative large, the FADMM algorithm performs better than the FADMMR algorithm and ADMM algorithm. The FADMM algorithm and FADMMR algorithm converge faster than the ADMM algorithm in terms of convergence time for different sizes of tuning parameters. For large-scale portfolio problem, the proposed algorithms have highly performance as well. Finally, empirical analysis on five datasets of stocks index show that the proposed algorithms are efficient and superior for solving sparse portfolio problems via SLOPE.

GCN-based proximal unrolling matrix completion for piecewise smooth signal recovery

Time-varying piecewise-smooth signal recovery is an important task in computer vision, image processing and environmental monitoring. The performance of many methods degrades dramatically in extreme scenario of sparse observations, a common case in these fields. In this extreme case, how to accurately recover time-varying piecewise-smooth signals is still a challenge. In this paper, we aim to batch recovery of the time-varying piecewise-smooth signals accurately. We first propose a graph signal matrix completion model based on piecewise smoothness, and use alternating direction method of multipliers (ADMM) to solve it, which is Low Rank and Piecewise Smoothness based recovery method (LRPS). To accelerate convergence, we further propose a proximal unrolling method based on LRPS (Pro-Un LRPS), where a denoising graph convolutional network (GCN) is used to replace the proximal operation in the ADMM framework. The Pro-Un LRPS can converge to a critical-point of the original optimization model. We interpret the embedded network as a subgradient-estimator from the optimization perspective. Finally, experimental results on several datasets demonstrate superior recovery performance of our methods when known entries are sparse, compared with existing matrix completion approaches. The convergence rate of Pro-Un LRPS is faster than that of LRPS, illustrating the effectiveness of the unrolling scheme.

Network Identification Using μ-PMU and Smart Meter Measurements

The network identification plays a very prominent role for the network operator to accomplish the various objectives such as state-estimation, monitoring, control, planning, and real-time analytics. The network structure varies from time-to-time and its details are often not available with the network operator. To address this issue, in this article, an alternating direction method of multipliers (ADMM) based framework is presented herein to identify the network topology and line parameters using smart meter and microphasor measurement unit (μ) measurements. The presented algorithm is divided into two sections 1) approximate parameter evaluation through regression, to extract the partial topology information and 2) complete network topology identification through the ADMM framework. This algorithm accomplishes the objectives of identifying the network configuration, branch parameters (e.g., conductance and susceptance), and change in branch parameters. Simulation results demonstrate the effectiveness of the presented algorithm on the benchmarked IEEE 13-bus and IEEE 123-bus feeders under various operating scenarios. Furthermore, the presented framework illustrates excellent network identification even with the presence of the stochastic nature of renewable power generation. The presented algorithm exhibits an excellent performance even with the consideration of noise in both measurements. In addition, the comparative performance is carried out on the benchmarked unbalanced IEEE 13-bus and balanced IEEE 33-bus feeders to highlight the efficacy of the presented framework over the state-of-art framework.

Joint design of detecting and screening waveforms against interrupted sampling repeater jamming with inaccurate jamming knowledge

The interrupted sampling repeater jamming (ISRJ) is a coherent interference, and it will result in a large amount of intra-pulse false targets in pulse compression (PC) radar. If the intercepted slices of the ISRJ, named as the screening waveform (SW), are orthogonal to the un-intercepted transmit waveform, named as the detection waveform (DW), the ISRJ can be suppressed by filtering with DW. In this paper, we investigate the problem of ISRJ suppression by jointly designing the DW and SW. Based on the inaccuracy of ISRJ parameter estimation, the joint design problem is formulated by minimizing the integrated sidelobe level (ISL) of the cross-correlation between the ISRJ and DW. The resulting design problem, which involves constraints of constant modulus and orthogonality, is NP-hard, and an alternating optimization like (AltOpt-like) algorithm is developed. Concretely, the SW is first obtained with the aid of the block coordinate descent (BCD) method, then the DW is optimized based on the alternating direction method of multipliers (ADMM) framework. Moreover, in the ADMM framework, low complexity algorithms are devised to solve the subproblems of the primal variables. Numerical simulations are provided to demonstrate the effectiveness of the proposed scheme.

Robust Multi-Robot Trajectory Optimization Using Alternating Direction Method of Multiplier

Wepropose a variant of alternating direction method of multiplier (ADMM) to solve constrained trajectory optimization problems. Our ADMM framework breaks a joint optimization into small sub-problems, leading to a low iteration cost and decentralized parameter updates. Starting from a collision-free initial trajectory, our method inherits the theoretical properties of primal interior point method (P-IPM), i.e., guaranteed collision avoidance and homotopy preservation throughout optimization, while being orders of magnitude faster. We have analyzed the convergence and evaluated our method for time-optimal multi-UAV trajectory optimizations and simultaneous goal-reaching of multiple robot arms, where we take into consider kinematics-, dynamics-limits, and homotopy-preserving collision constraints. Our method highlights an order of magnitude’s speedup, while generating trajectories of comparable qualities as state-of-the-art P-IPM solver.

A hierarchical control scheme with bi-level communication networks for the interconnected DC microgrids cluster

This paper proposes a hierarchical control scheme based on distributed alternating direction method of multipliers (ADMM) for the interconnected DC microgrids cluster. This scheme is fully distributed and consists of three coordinated control levels through bi-level distributed communication networks. The constraints of the economic dispatch problem are decoupled for tertiary control and decomposed into global and local optimization problems using the ADMM framework. To realize parallel computing of ADMM, we designed a fast second-order distributed average consensus algorithm to estimate the global average exchanged value and the average output power of a single microgrid, thus removing the coupling within the microgrids cluster and individual microgrid. Considering the voltage deviation caused by the traditional droop control, voltage control without droop based on the distributed algorithm is proposed to realize voltage restoration and power distribution among dispatchable generators for individual microgrid, combined with traditional primary and secondary control. Numerical simulations of an off-grid microgrids cluster are designed to validate the effectiveness of the proposed control method. Results show that the proposed hierarchical control scheme can ensure the independence and privacy of information, which is beneficial to solve technical and economic challenges brought by centralized optimization. In addition, the proposed algorithm can be applied to more realistic scenarios, such as conditions for time-varying loads and cost functions.

GPU‐accelerated and mixed norm regularized online extreme learning machine

SummaryExtreme learning machine (ELM) is a prominent example of neural network with its fast training speed, and good prediction performance. An online version of ELM called online sequential extreme learning machine (OS‐ELM) has also been proposed for the sequential training. Combined with the need for regularization to prevent over‐fitting in addition to the large number of neurons required in the hidden layer, OS‐ELM demands huge amount of computation power for the large‐scale data. In this article, a mixed norm () regularized online machine learning algorithm (MRO‐ELM) that is based on alternating direction method of multipliers (ADMM) is proposed. A linear combination of the mixed norm and the Frobenius norm regularization is applied using the ADMM framework and update formulas are derived. Graphics processing unit (GPU) accelerated version of MRO‐ELM (GPU‐MRO‐ELM) is also proposed to reduce the training time by processing appropriate parts in parallel using the implemented custom kernels. In addition, a novel automatic hyper‐parameter tuning method is incorporated to GPU‐MRO‐ELM using progressive validation with GPU acceleration. The experimental results show that the MRO‐ELM algorithm and its GPU version outperform OS‐ELM in terms of training speed, and testing accuracy. Also, compared to the cross validation, the proposed automatic hyper‐parameter tuning demonstrates dramatical reduction in the tuning time.

Structure-Awareness SAR Imagery by Exploiting Structure Tensor TV Regularization Under Multitask Learning Framework

Conventional sparsity-driven synthetic aperture radar (SAR) imagery often encounters the problem of loss of structural features in weak scattering. Although there are algorithms that focus on structure enhancement, no proper balance between accuracy and efficiency can be achieved. In this article, a novel feature enhancement algorithm, named structure-awareness SAR (SA-SAR), is proposed by exploiting an emerging regularizer of structure tensor total variation (STV). By imposing the STV norm onto the prior of the scenes or targets to be imaged, the intended structure feature can be analytically solved under the proximal algorithm. More specifically, the regularization method is developed within the alternating direction method of multipliers (ADMM) framework, where closed-form proximity operators can be derived. Due to the ADMM framework, it facilitates to incorporate with more features to be enhanced in a fully synergistic way. Therefore, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> and entropy norms are involved so that the sparse and focusing features can be enhanced accordingly. Considering the coherence of the SAR data, a linear proximal operator is developed within the multitask learning framework. The unavoidable error propagations can be alleviated in a great extent. In such cases, the proposed algorithm is superior in terms of convergence and efficiency. To facilitate the implementation and computation, the STV proximal mapping is optimized under the Vieta theorem. Finally, both simulated and raw SAR data are applied to verify the effectiveness of the proposed algorithm. Comparisons with conventional algorithms are carried out to show the superiority of the proposed algorithm.

Efficient ADMM Framework Based on Functional Measurement Model for mmW 3-D SAR Imaging

Compressed sensing (CS) shows significant potential in the field of active millimeter-wave (mmW) synthetic aperture radar (SAR) imaging due to the merits of reducing system complexity and achieving high-speed sensing. However, most CS-driven imaging methods suffer from the excessive computational burden, since the calculative steps always rely on vectorization and consequently lead to extremely large-scale matrix operations. To address this issue, we propose an efficient alternating direction method of multipliers (ADMMs) framework for mmW 3-D SAR imaging. In our scheme, we utilize the single-frequency holographic (SFH) technique and construct SFH-based forward/inverse sensing operators rather than converting the imaging process into a special case of “linear inverse problems,” by which the large-scale matrix inversions are avoided and consequently the computational complexity is reduced. Based on the SFH functional measurement model, the SFH-ADMM is derived to reconstruct the 3-D image from sparsely sampled measurement echo while suppressing noisy clutters and ambiguities. Besides, the SFH-ADMM iteration steps undergird a neural network design, yielding a tailored SFH-ADMM-Net with trainable parameters and layer-fixed structures, which further shorten the execution time and improve reconstruction performance. The network is trained by simulated data, which are generated according to the radar signal model. Extensive experiments, including simulations and laboratory tests, demonstrate the superiority of the proposed algorithms in terms of both reconstruction accuracy and computational speed.

Joint ptycho-tomography with deep generative priors

Joint ptycho-tomography is a powerful computational imaging framework to recover the refractive properties of a 3D object while relaxing the requirements for probe overlap that is common in conventional phase retrieval. We use an augmented Lagrangian scheme for formulating the constrained optimization problem and employ an alternating direction method of multipliers (ADMM) for the joint solution. ADMM allows the problem to be split into smaller and computationally more efficient subproblems: ptychographic phase retrieval, tomographic reconstruction, and regularization of the solution. We extend our ADMM framework with plug-and-play (PnP) denoisers by replacing the regularization subproblem with a general denoising operator based on machine learning. While the PnP framework enables integrating such learned priors as denoising operators, tuning of the denoiser prior remains challenging. To overcome this challenge, we propose a denoiser parameter to control the effect of the denoiser and to accelerate the solution. In our simulations, we demonstrate that our proposed framework with parameter tuning and learned priors generates high-quality reconstructions under limited and noisy measurement data.

Transmit Sequence Design for Dual-Function Radar-Communication System With One-Bit DACs

In this paper, the problem of transmit sequence design for a dual-function radar-communication (DFRC) system equipped with one-bit digital-to-analog converters (DACs) is investigated. More specifically, a nonconvex problem is formulated by minimizing the symbol mean-square error, while ensuring the target localization performance for radar. A direct one-bit sequence design tackled by an alternating minimization (AM) approach, which involves a simple unconstrained quadratic sub-problem with closed-form solution and a quadratically constrained nonconvex sub-problem tackled by the alternating direction method of multipliers (ADMM) algorithm, is developed. The solutions of primal variables under the ADMM framework are provided and the convergence is discussed. Additionally, an indirect but computationally more efficient design, which is realized by transmit beamforming based on an accelerated primal gradient (APG) method, is presented. For better understanding of this design, both the Cramér-Rao Bound (CRB) and symbol error probability of the resulting beamformer with one-bit quantization using the Bussgang theorem are analyzed. Numerical simulations are provided to demonstrate the effectiveness of the proposed schemes.

The application of plug-and-play ADMM framework and BM3D denoiser for compressed sensing MR image reconstruction

Compressed Sensing Magnetic Resonance Imaging (CS-MRI) is an effective technique to reduce MRI data acquisition time. There is currently growing interest in using Alternating Direction Method of Multiplier (ADMM) for CS-MRI reconstruction. In this paper, we propose a flexible plug-and-play framework to incorporate the block matching 3D (BM3D) denoising algorithm as prior into the plug-and-play ADMM reconstruction procedure for CS-MRI reconstruction, termed BM3D Plug-and-play ADMM (BPA) method. We investigated the performance of the proposed BPA method for the construction of highly under-sampled MRI data of two different sampling masks. Compared with other widely used CS-MRI reconstruction methods, such as, PANO, BM3D-IT, BM3D-MRI and BM3D-AMP-MRI, the proposed framework can reconstruct highly under-sampled CS-MRI data with improved gains in Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM).

Synchronous and asynchronous parallel computation for large-scale optimal control of connected vehicles

Connected vehicles is an important intelligent transportation system to improve the traffic performance. This paper proposes two parallel computation algorithms to solve a large-scale optimal control problem in the coordination of multiple connected vehicles. The coordination is formulated as a centralized optimization problem in the receding horizon fashion. A decentralized computation network is designed to facilitate the development of parallel algorithms. We use Taylor series to linearize non-convex constraints, and introduce a set of consensus constraints to transform the centralized problem to a standard consensus optimization problem. A synchronous parallel algorithm is firstly proposed to solve the consensus optimization problem by applying the alternating direction method of multipliers (ADMM). The ADMM framework allows us to decompose the coupling constraints and decision variables, leading to parallel iterations for each vehicle in a synchronous fashion. We then propose an asynchronous version of the parallel algorithm that allows the vehicles to update their variables asynchronously in the computation network. The effectiveness and efficiency of the proposed algorithms are validated by extensive numerical simulations.

Quaternion block sparse representation for signal recovery and classification

This paper presents a quaternion block sparse representation (QBSR) method for structural sparse signal recovery in quaternion space. Due to the noncommutativity of quaternion multiplication, conventional optimization algorithms originally designed for real-valued and complex-valued signal recovery problems are not applicable to the optimization problem of QBSR. To combat this problem, we leverage several quaternion operators and devise an effective algorithm for QBSR within the ADMM (Alternating Direction Method of Multipliers) framework. The second contribution of this work is to develop a QBSR based classifier referred to as QBSRC for quaternion data classification with application to color face recognition. Compared with real-valued representation based classifiers handling multiple color channels of a color image independently, QBSRC treats a color image as a quaternion signal and represents it in a holistic manner. The third contribution is to provide the theoretical analysis of QBSRC and rigorously prove that QBSRC is guaranteed to succeed in classification of any new test sample under appropriate condition. Experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed method for quaternion signal recovery and classification.

Lower and upper bound limit analysis via the alternating direction method of multipliers

Abstract Computational limit analysis methods invariably lead to the need to solve a mathematical programming problem. The alternating direction method of multipliers (ADMM) is one versatile and robust technique to solve non-linear convex optimization problems that has recently found applications in a wide range of fields. Its solution scheme, based on an operator splitting algorithm, is not only easy to implement but also suitable to efficiently solve large-scale variational problems. Starting from the ADMM framework, we derive a strict upper bound finite element formulation using a two-(primal)-field approximation, one for the velocity field and the other for the plastic strain rate field. Next, following a similar approach, we develop a novel strict lower bound formulation. Here, the two-(primal)-field model is based on a redundant approximation of the stress field. Duality principles are then explored in order to unify these two formulations.The effectiveness of this approach is demonstrated on test problems and, to conclude, some considerations are made about the performance results.

VERTICOX: Vertically Distributed Cox Proportional Hazards Model Using the Alternating Direction Method of Multipliers.

The Cox proportional hazards model is a popular semi-parametric model for survival analysis. In this paper, we aim at developing a federated algorithm for the Cox proportional hazards model over vertically partitioned data (i.e., data from the same patient are stored at different institutions). We propose a novel algorithm, namely VERTICOX, to obtain the global model parameters in a distributed fashion based on the Alternating Direction Method of Multipliers (ADMM) framework. The proposed model computes intermediary statistics and exchanges them to calculate the global model without collecting individual patient-level data. We demonstrate that our algorithm achieves equivalent accuracy for the estimation of model parameters and statistics to that of its centralized realization. The proposed algorithm converges linearly under the ADMM framework. Its computational complexity and communication costs are polynomially and linearly associated with the number of subjects, respectively. Experimental results show that VERTICOX can achieve accurate model parameter estimation to support federated survival analysis over vertically distributed data by saving bandwidth and avoiding exchange of information about individual patients. The source code for VERTICOX is available at: https://github.com/daiwenrui/VERTICOX.

Improving Rate of Convergence via Gain Adaptation in Multi-Agent Distributed ADMM Framework

In this paper, the Alternating Direction Method of Multipliers (ADMM) is investigated for distributed optimization problems in a networked multi-agent system. In particular, a new adaptive-gain ADMM algorithm is derived in a closed form and under the standard convex property in order to greatly speed up convergence of ADMM-based distributed optimization. Using Lyapunov direct approach, the proposed solution embeds control gains into weighted network matrix among the agents and uses those weights as adaptive penalty gains in the augmented Lagrangian. It is shown that the proposed closed loop gain adaptation scheme significantly improves the convergence time of underlying ADMM optimization. Convergence analysis is provided and simulation results are included to demonstrate the effectiveness of the proposed scheme.

ADMM Empowered Distributed Computational Intelligence for Internet of Energy

Internet of Energy (IoE), the paradigm of applying Internet of Things (IoT) to energy management systems, aims to improve energy systems' efficiency and reliability by enhancing connectivity and interoperability among geographically distributed energy devices. This requires distributed computational intelligence responsible for data processing and decision making of the energy devices. To achieve distributed, scalable, and privacy-protected energy management in IoE, this article proposes using Alternating Direction Method of Multipliers (ADMM) as the theoretical framework to design the distributed computational intelligence in IoE. In the first place, a brief introduction of ADMM for solving energy management problems is given. Based on the ADMM framework, a distributed intelligence system is designed for each decision maker in energy systems. With the distributed intelligence, decision makers can interact with each other to achieve system-wide goals of energy management without disclosing their private data. Moreover, we provide some examples of ADMM applications in practical distributed energy management in IoE and discuss the challenges of ADMM implementation in IoE. Lastly, a joint computing and networking resources management architecture is proposed to meet the challenges. The result of a case study shows that this architecture can reduce communications and computing costs of ADMM implementation.