ADMM Algorithm Research Articles

A novel privacy‐preserved voluntary outage management model on a transactive energy scheme

Unpredictable occurrence of natural disasters in modern power systems has significantly increased the attentions to the outage management of multi-microgrid (MMG) systems. In this regard, a multi-objective two-stage outage management scheme is presented in this paper to protect undamaged microgrids (MGs) against load curtailment and minimize the curtailed loads simultaneously. Considering the importance of power system economic, this scheme is designed in a privacy-preserved transactive energy (TE) market and enables all MGs to participate in outage management voluntarily. In detail, a DC optimization is autonomously operated in the first stage in each MG to supply its demand loads. Then, the second stage is designed to exchange the surplus power of the last stage with the damaged MGs. These voluntary power transactions are pre-specified in contracts, adjusted between each MG and distributed system operator (DSO). Moreover, in this stage, the ADMM algorithm based on DC-OPF has prepared a privacy protection platform and proved the results’ accuracy. The presented test grid is the IEEE 34-node test system and the numeric results demonstrate the effectiveness of the proposed multi-objective scheme.

ADMM algorithm for some regularized Perona–Malik equation and applications to image denoising

ADMM algorithm for some regularized Perona–Malik equation and applications to image denoising

Subgroup analysis for functional partial linear regression model

AbstractIn a functional partial linear regression (FPLR) model, where the response variable is scalar while the explanatory variables involve both infinite‐dimensional functional predictors and finite‐dimensional scalar covariates, the relationships between the response and the explanatory variables are often assumed to be the same for all subjects. This article relaxes this assumption and considers a subgroup analysis for the FPLR model, which allows the intercepts to vary for different subgroups from a heterogeneous population. By projecting the functional predictors onto the corresponding eigenspace, the subgroup analysis based on the FPLR model can be simplified to a framework that is similar to the classical subgroup analysis problem. To automatically identify subgroups among observations and estimate the regression parameters of interest, we combine the functional principal component analysis with the concave pairwise penalized approach and develop an ADMM algorithm for functional subgroup analysis. We also establish the consistency of the proposed estimators under mild conditions. Simulation experiments demonstrate that the concave penalized subgroup approach could potentially achieve substantial gains over the ordinary FPLR model. The analysis of data from a creative achievement study is used to illustrate the practical performance of the subgroup analysis for the FPLR model.

High-Resolution ISAR Imaging Based on Plug-and-Play 2D ADMM-Net

We propose a deep learning architecture, dubbed Plug-and-play 2D ADMM-Net (PAN), by combining model-driven deep networks and data-driven deep networks for effective high-resolution 2D inverse synthetic aperture radar (ISAR) imaging with various signal-to-noise ratios (SNR) and incomplete data scenarios. First, a sparse observation model of 2D ISAR imaging is established, and a 2D ADMM algorithm is presented. On this basis, using the plug and play (PnP) technique, PnP 2D ADMM is proposed, by combining the 2D ADMM algorithm and the deep denoising network DnCNN. Then, we unroll and generalize the PnP 2D ADMM to the PAN architecture, in which all adjustable parameters in the reconstruction layers, denoising layers, and multiplier update layers are learned by end-to-end training through back-propagation. Experimental results showed that the PAN with a single parameter set can achieve noise-robust ISAR imaging with superior reconstruction performance on incomplete simulated and measured data under different SNRs.

An inexact accelerated stochastic ADMM for separable convex optimization

An inexact accelerated stochastic Alternating Direction Method of Multipliers (AS-ADMM) scheme is developed for solving structured separable convex optimization problems with linear constraints. The objective function is the sum of a possibly nonsmooth convex function and a smooth function which is an average of many component convex functions. Problems having this structure often arise in machine learning and data mining applications. AS-ADMM combines the ideas of both ADMM and the stochastic gradient methods using variance reduction techniques. One of the ADMM subproblems employs a linearization technique while a similar linearization could be introduced for the other subproblem. For a specified choice of the algorithm parameters, it is shown that the objective error and the constraint violation are $$\mathcal {O}(1/k)$$ relative to the number of outer iterations k. Under a strong convexity assumption, the expected iterate error converges to zero linearly. A linearized variant of AS-ADMM and incremental sampling strategies are also discussed. Numerical experiments with both stochastic and deterministic ADMM algorithms show that AS-ADMM can be particularly effective for structured optimization arising in big data applications.

An efficient and economical storage and energy sharing model for multiple multi-energy microgrids

Multi-energy microgrids are facing a dilemma that realizing high local energy efficiency requires large-capacity ESS with hefty investment costs. To address the dilemma, an efficient and economic hybrid storage and energy sharing model for multiple microgrids is proposed. Specificly, a hybrid energy storage system (HESS) is introduced, which contains an electrical battery and a heat storage tank and is able to realize energy conversion. The multiple microgrids can share energy through the HESS in a collaborative way. An energy optimization problem is formulated to minimize the overall energy costs including energy purchase cost and HESS operating cost. ADMM algorithm is used to solve the problem in a distributed manner to avoid privacy concerns. The storage and energy sharing benefits of the microgrids and the HESS are determined by Nash bargaining solution. Simulation results show that the model can effectively improve the utilization of the renewable energy, and lead to considerable economic benefits for both the microgrids and the HESS.

Hyperspectral Unmixing Using Nonlocal Similarity-Regularized Low-Rank Tensor Factorization

Recently, methods based on nonnegative tensor factorization (NTF), which benefits from the tensor representation of hyperspectral imagery (HSI) without any information loss, have attracted increasing attention. However, most existing methods fail to explore the internal spatial structure of data, resulting in low unmixing performance. Moreover, when the algorithm is optimized, the solution is unstable. In this article, a regularizer based on nonlocal tensor similarity is proposed, which can not only fully preserve the global information of HSI but also mine the internal information of data in the spatial domain. HSI is regarded as a 3-D tensor and is directly subjected to endmember extraction and abundance estimation. To fully explore the structural characteristics of data, we simultaneously use the local smoothing and low tensor rank prior of the data to constrain the unmixing model. First, several 4-D tensor groups can be obtained after the nonlocal similarity structure of HSI is learned. Subsequently, a low tensor rank prior is applied to each 4-D tensor, which can fully simulate the nonlocal similarity in the image. In addition, total variation (TV) is also used to explore the local spatial relationship of data, which can generate a smooth abundance map through edge preservation. The optimization is solved by the ADMM algorithm. Experiments on synthetic and real data illustrate the superiority of the proposed method.

Laplacian Smoothing Stochastic ADMMs With Differential Privacy Guarantees

Many machine learning tasks such as structured sparse coding and multi-task learning can be converted into an equality constrained optimization problem. The stochastic alternating direction method of multipliers (SADMM) is a popular algorithm to solve such large-scale problems, and has been successfully used in many real-world applications. However, existing SADMMs fail to take into consideration an important issue in their designs, i.e., protecting sensitive information. To address this challenging issue, this paper proposes a novel differential privacy stochastic ADMM framework for solving equality constrained machine learning problems. In particular, to further lift the utility in privacy-preserving equality constrained optimization, a Laplacian smoothing operation is also introduced into our differential privacy ADMM framework, and it can smooth out the Gaussian noise used in the Gaussian mechanism. Then we propose an efficient differentially private variance reduced stochastic ADMM (DP-VRADMM) algorithm with Laplacian smoothing for both strongly convex and general convex objectives. As a by-product, we also present a new differentially private stochastic ADMM algorithm with DP guarantees. In theory, we provide both private guarantees and utility guarantees for the proposed algorithms, which show that Laplacian smoothing can improve the utility bounds of our algorithms. Experimental results on real-world datasets verify our theoretical results and the effectiveness of our algorithms.

Truncated Residual Based Plug-and-Play ADMM Algorithm for MRI Reconstruction

Plug-and-play alternating direction method of multiplier (PnP-ADMM) can be used to solve the magnetic resonance imaging (MRI) reconstruction problem, which allows plugging the pre-trained denoiser to solve the denoising-like subproblem in ADMM. Many researchers explore the property of the denoiser to ensure the convergence of PnP-ADMM. However, it is hard to prove that the pre-trained deep learning based denoiser satisfies the assumption. In this paper, we propose a truncated residual based PnP-ADMM that solves the denoising-like subproblem by subtracting the predicted residual of the denoising neural network. Instead of establishing the assumption for the denoiser, we truncate the residual with a variable that tends to zero. In this way, the iterative sequence of the proposed algorithm can strictly converge to a fixed point. This truncated residual based plug-and-play ADMM algorithm is called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TRPA</i> . Moreover, to allow the denoiser to deal with the continuous noise level, we design a continuous conditional instance normalization (CCIN) layer. With the truncated residual and the powerful neural network with CCIN, the TRPA has strict convergence property as well as great image restoration ability. Qualitative and quantitative results show that our proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TRPA</i> is comparable with state-of-the-art methods. It is worth noticing that TRPA as a plug-and-play method is comparable with the end-to-end model at the leaderboard of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fastMRI</i> challenge.

Hyperspectral Fusion Using Weighted Nonlocal Vector Total Variation

Hyperspectral (HS) images have high spectral but low spatial resolutions, while multispectral (MS) images, on the other hand, have low spectral but high spatial resolutions. In HS–MS fusion, the HS and MS images are combined to obtain a single image with high spatial and spectral resolutions. Such images are typically textured and exhibit repetitive structures. To exploit this prior, we propose a nonlocal weighted total-variation regularizer. The novelty of the design is that the following hold: 1) we use a weighted norm, where the weights are derived from the MS image and 2) pixel variations over nonlocal neighborhoods are considered. We incorporate the regularizer into a standard convex optimization framework involving quadratic data-fidelity terms. We develop an efficient ADMM algorithm for solving this optimization problem—the novelty in this regard is that we use a variable splitting technique that results in the closed-form solutions of the ADMM subproblems. We report results on standard datasets demonstrating that the proposed regularizer can recover fine textures (as opposed to local pixel-based methods) and outperform the state-of-the-art methods.

Exact Decomposition of Joint Low Rankness and Local Smoothness Plus Sparse Matrices.

It is known that the decomposition in low-rank and sparse matrices (L+S for short) can be achieved by several Robust PCA techniques. Besides the low rankness, the local smoothness (LSS) is a vitally essential prior for many real-world matrix data such as hyperspectral images and surveillance videos, which makes such matrices have low-rankness and local smoothness property at the same time. This poses an interesting question: Can we make a matrix decomposition in terms of L&LSS +S form exactly? To address this issue, we propose in this paper a new RPCA model based on three-dimensional correlated total variation regularization (3DCTV-RPCA for short) by fully exploiting and encoding the prior expression underlying such joint low-rank and local smoothness matrices. Specifically, using a modification of Golfing scheme, we prove that under some mild assumptions, the proposed 3DCTV-RPCA model can decompose both components exactly, which should be the first theoretical guarantee among all such related methods combining low rankness and local smoothness. In addition, by utilizing Fast Fourier Transform (FFT), we propose an efficient ADMM algorithm with a solid convergence guarantee for solving the resulting optimization problem. Finally, a series of experiments on both simulations and real applications are carried out to demonstrate the general validity of the proposed 3DCTV-RPCA model.

Guided Nonlocal Patch Regularization and Efficient Filtering-Based Inversion for Multiband Fusion

In multiband fusion, an image with a high spatial and low spectral resolution is combined with an image with a low spatial but high spectral resolution to produce a single multiband image having high spatial and spectral resolutions. This comes up in remote sensing applications such as pansharpening (MS+PAN), hyperspectral sharpening (HS+PAN), and HS-MS fusion (HS+MS). Remote sensing images are textured and have repetitive structures. Motivated by nonlocal patch-based methods for image restoration, we propose a convex regularizer that (i) takes into account long-distance correlations, (ii) penalizes patch variation, which is more effective than pixel variation for capturing texture information, and (iii) uses the higher spatial resolution image as a guide image for weight computation. We come up with an efficient ADMM algorithm for optimizing the regularizer along with a standard least-squares loss function derived from the imaging model. The novelty of our algorithm is that by expressing patch variation as filtering operations and by judiciously splitting the original variables and introducing latent variables, we are able to solve the ADMM subproblems efficiently using FFT-based convolution and soft-thresholding. As far as the reconstruction quality is concerned, our method is shown to outperform state-of-the-art variational and deep learning techniques.

Optimized Cpu-Gpu Partitioning for an Admm Algorithm Applied to a Peer to Peer Energy Market

Optimized Cpu-Gpu Partitioning for an Admm Algorithm Applied to a Peer to Peer Energy Market

Learning the Proximity Operator in Unfolded ADMM for Phase Retrieval

This paper considers the phase retrieval (PR) problem, which aims to reconstruct a signal from phaseless measurements such as magnitude or power spectrograms. PR is generally handled as a minimization problem involving a quadratic loss. Recent works have considered alternative discrepancy measures, such as the Bregman divergences, but it is still challenging to tailor the optimal loss for a given setting. In this paper we propose a novel strategy to automatically learn the optimal metric for PR. We unfold a recently introduced ADMM algorithm into a neural network, and we emphasize that the information about the loss used to formulate the PR problem is conveyed by the proximity operator involved in the ADMM updates. Therefore, we replace this proximity operator with trainable activation functions: learning these in a supervised setting is then equivalent to learning an optimal metric for PR. Experiments conducted with speech signals show that our approach outperforms the baseline ADMM, using a light and interpretable neural architecture.

Hyperspectral Image Denoising With Weighted Nonlocal Low-Rank Model and Adaptive Total Variation Regularization

Hyperspectral image (HSI) is always corrupted by various types of noise during image capturing, such as Gaussian noise, stripe noise, deadline noise, impulse noise, and more. Such complicated noise significantly degrades imaging quality and thus limits the performance of downstream vision tasks. Current HSI denoising methods tackle this problem by modeling either the spectral-spatial prior of HSI or the noise characteristic of HSI, and few work consider the two aspects simultaneously. In this paper, we propose a new HSI denoising method by simultaneously modeling the HSI prior and the HSI noise characteristic. Specifically, we firstly utilize the non independent and identically distributed (non i.i.d.) mixture of Gaussian (MoG) assumption to characterize the complex noise, which corresponds to optimize a weighted fidelity function. Secondly, we exploit HSI’s non-local similarity and spatial-spectral correlation priors by applying non-local low rank model. Thirdly, we design an adaptive edge preserving total variation regularization term to characterize the non-local smooth property of HSI. Finally, we propose a new denoising model and develop effective ADMM algorithm to solve it. Extensive experiments on simulated data and real data substantiate the superiority of the proposed method beyond state-of-the-arts.

Forecast Research on Multidimensional Influencing Factors of Global Offshore Wind Power Investment Based on Random Forest and Elastic Net

Recently, countries around the world have begun to develop low-carbon energy sources to alleviate energy shortage and cope with climate change. The offshore wind power has become a new direction for clean energy exploration. However, the accuracy of offshore wind power investment is still an urgent problem due to its complexity. Therefore, this paper investigates offshore wind power investment to improve the investment forecasting accuracy. In this study, the random forest (RF) algorithm was used to screen out the key factors influencing multi-dimensional global offshore wind power investment, and the elastic net (EN) was optimized using the ADMM algorithm and used in the global offshore wind power investment forecast model. The results show that the adoption of the random forest algorithm can effectively screen out the key influencing factors of global offshore wind power investment. Water depth, offshore distance and sweeping area have the most influence on the investment. Moreover, compared with other models, the elastic net optimized by ADMM can better reflect the changing trend of global offshore wind power investment, with smaller errors and a higher regression accuracy. The application of the RF–EN combined model can screen out effective factors from complex multi-dimensional influencing factors, and perform high-precision regression analysis, which is conducive to improving the global offshore wind power investment forecast. The conclusion obtained can set a more reasonable plan for the future construction and investment of global offshore wind power projects.

Joint Resource Optimization for IRS-Assisted Mmwave MIMO Under QoS Constraints

This paper focuses on the non-convex joint optimization with a dynamic resource of multi-user for an intelligent reflecting surface-enhanced mmWave system, where all users have individual rates or quality of service requirements. Firstly, the objective function of the above non-linear problem is converted into a quadratic programming form under the quality of service constraints. Further, a multi-blocks alternating optimization framework with dynamic power allocation is proposed to obtain the maximum sum-rate, where the relaxed ADMM algorithm is adopted to tackle the optimal full-digital precoder and the corresponding passive reflecting matrix is obtained by the gradient-projection. The numerical results verify that beam optimization should be emphasized in high SNR, but joint dynamic resource allocation can further improve system performance even if the hardware dimensions reaches the limit.

Alternating Direction Method of Multipliers Network for Bioluminescence Tomography Reconstruction.

Bioluminescence tomography (BLT) is an effective noninvasive molecular imaging modality for three dimensional visualization of in vivo tumor research in small animals. The approaches of deep learning have shown great potential in the field of optical molecular imaging in recent years. However, the common problem with these existing end-to-end networks is the black box technology, whose solving process is not theoretically proven. In this work, we proposed a novel Alternating Direction Method of Multipliers Network (ADMM-Net) to solve the poor interpretation problem of internal process. The ADMM-Net combines the framework of deep learning on the basis of traditional ADMM algorithm to dynamically learn various parameters of the algorithm in the form of network. To evaluate the performance of our proposed network, we implemented numerical simulation experiments. The results show that the ADMM-Net can accurately reconstruct the location of the source, and the morphological similarity with the real source is also higher.

Biclustering analysis of functionals via penalized fusion

In biomedical data analysis, clustering is commonly conducted. Biclustering analysis conducts clustering in both the sample and covariate dimensions and can more comprehensively describe data heterogeneity. In most of the existing biclustering analyses, scalar measurements are considered. In this study, motivated by time-course gene expression data and other examples, we take the “natural next step” and consider the biclustering analysis of functionals under which, for each covariate of each sample, a function (to be exact, its values at discrete measurement points) is present. We develop a doubly penalized fusion approach, which includes a smoothness penalty for estimating functionals and, more importantly, a fusion penalty for clustering. Statistical properties are rigorously established, providing the proposed approach a strong ground. We also develop an effective ADMM algorithm and accompanying R code. Numerical analysis, including simulations, comparisons, and the analysis of two time-course gene expression data, demonstrates the practical effectiveness of the proposed approach.

Modified ADMM algorithm for solving proximal bound formulation of multi-delay optimal control problem with bounded control

This study presents an algorithm for solving Optimal Control Problems(OCPs) with objective function of the Lagrange - type and multiple delayson both the state and control variables of the constraints; with bounds on thecontrol variable. The full discretization of the objective functional and the multipledelay constraints was carried out using the Simpson numerical scheme.The discrete recurrence relations generated from the discretization of both theobjective functional and constraints were used to develop the matrix operatorswhich satisfy the basic spectral properties The primal-dual residuals of the algorithmwere derived in order to ascertain the rate of convergence of the algorithm;which performs faster when relaxed with an accelerator variant in the sense ofNesterov. The direct numerical approach for handling the multi-delay controlproblem was observed to obtain an accurate result at a faster rate of convergencewhen over-relaxed with an accelerator variant. This research problemis limited to linear constraints and objective functional of the Lagrange-typeand can address real-life models with multiple delays as applicable to quadraticoptimization of Intensity Modulated Radiation Theory (IMRT) planning. Thenovelty of this research paper lies in the method of discretization and its adaptationto handle linearly and proximal bound constrained program formulatedfrom the multiple delay optimal control problems.