Alternating Minimization Method Research Articles

Image reconstruction based on TV-L1 model for planar array capacitance defect detection

Planar array capacitive imaging is an emerging nondestructive testing (NDT) technology with broad application prospects. The severe ill-posed inverse problem of planar array capacitive imaging reduces quality of reconstructed images. In this paper, a total variation regularization model with an L1-norm (TV-L1 model) of 3 × 4 planar array capacitance imaging is established for imaging the defects detected in the inner layers of multi-layer composite components. Considering the solution instability caused by the non-differentiability of the TV-L1 model, a variable splitting self-adaptive augmented Lagrange alternating direction algorithm (VS-SALAD) is proposed. An augmented Lagrange method enhances model splitability, and a series of subproblems are obtained by variable splitting operation. The subproblems are solved iteratively using alternating minimization method with self-adaptive penalty parameter. Experimental results demonstrate that TV-L1 model effectively alleviates the ill-posed problem, and the proposed algorithm significantly improves both the quality of reconstructed images and solving speed of the TV-L1 model.

Imbalanced and missing multi-label data learning with global and local structure

Label missing and class imbalance problems are two hot research topics in machine learning, and they have been impeding the improvement of model performance, especially in the multi-label learning. Although some existing methods have proven to be effective, they are suitable for only one case. How to effectively address above two issues simultaneously is a challenging problem. In this paper, we propose a novel model named Imbalanced and Missing multi-Label data learning with Global and Local structure (IMLGL) to address the aforementioned challenge. There are following three advantages. At the empirical risk level, we introduce the label correlation matrix C into the loss function and devise a dynamic weighting method to address the aforementioned challenge. At the data level, we analyze the structural characteristics of the data, and introduce local low-rank and global high-rank term to enhance the generalization performance of the model. At the label level, a smoothing term is also introduced for learning the constraint classifier coefficient matrix W. Our method utilizes alternative optimization technique and alternating minimization method for solving. Extensive experiments on six datasets demonstrate the competitiveness of our approach.

Proximal alternating minimization method for Poisson noise removal

Proximal alternating minimization method for Poisson noise removal

Mixed Gaussian-impulse noise removal using non-convex high-order TV penalty

To restore images with clear edge details and no staircase artifacts from degraded versions, this paper incorporates the ℓ2 plus ℓ0 data fidelity and non-convex high-order total variation regularizer to establish an optimization model for eliminating mixed Gaussian-impulse noise. Among them, the ℓ2 fidelity is adopted to suppress Gaussian noise, while the ℓ0-norm is more suitable for detecting and removing impulse noise. In addition, the non-convex regularization displays excellent performance in overcoming the staircase effect and preserving edge details. Computationally, by using the iteratively reweighted ℓ1 algorithm and variable splitting method, this work designs a modified alternating minimization method to solve the optimization problem we construct. In theory, the convergence proof of our resulting algorithm is also presented. Finally, in the experimental section, we conduct extensive numerical experiments on degraded images and compare with other existing techniques. From the intuitive effects and restoration accuracy, it follows that our newly proposed method is effective and competitive for image deblurring and mixed noise removal.

Adaptive integral alternating minimization method for robust learning of nonlinear dynamical systems from highly corrupted data

This paper proposes an adaptive integral alternating minimization method (AIAMM) for learning nonlinear dynamical systems using highly corrupted measured data. This approach selects and identifies the system directly from noisy data using the integral model, encompassing unknown sparse coefficients, initial values, and outlier noisy data within the learning problem. It is defined as a sparse robust linear regression problem. An adaptive threshold parameter selection method is proposed to constrain model fitting errors and select appropriate threshold parameters for sparsity. The robustness and accuracy of the proposed AIAMM are demonstrated through several numerical experiments on typical nonlinear dynamical systems, including the van der Pol oscillator, Mathieu oscillator, Lorenz system, and 5D self-exciting homopolar disc dynamo. The proposed method is also compared to several advanced methods for sparse recovery, with the results indicating that the AIAMM demonstrates superior performance in processing highly corrupted data.

Hybrid distributed online and alternating convex optimization‐based initial value accelerated hybrid precoding for millimeter wave multiple‐input‐multiple‐output

SummaryThe hybrid precoding problem is considered a Frobenius norm reduction problem for the narrowband channel in a millimeter wave (mmWave) with multiple inputs and outputs (mmWave MIMO). This work proposes a hybrid distributed online and alternating convex optimization (HDO‐ACO) algorithm to improve hybrid precoding (HP), interpreted as a trace minimization problem. HDO‐ACO alternately determines the digital and analog precoders to reduce the trace by keeping the other constant. Initially, HDO‐ACO uses Lagrange's method to determine the digital precoding subproblem. Then, it uses the integrated distributed online convex optimization and alternating minimization algorithm in the analog precoder design. In the HP method, the digital and analog precoders are iteratively updated until the highest number of iterations or convergence is reached. But this hybrid precoding method requires an initial analog precoding matrix input to begin the iteration. The algorithms converge gradually and fall into a suboptimal solution when the initial analog precoding matrix is set randomly. Hence, an initial value acceleration‐based heuristic approach is used in the HDO‐ACO alternating minimization algorithm that calculates the initial feasible value of an alternating minimization method using channel conditions. The simulation results of the proposed HDO‐ACO algorithm are presented under bit error rate (BER), spectral efficiency (SE), and convergence behavior by comparing it with modified block coordinate descent–HP (MBCD‐HP), manifold optimization–alternating minimization (MO‐AltMin), and semi‐definite relaxation‐based alternating optimization (SDR‐AO). The proposed HDO‐ACO attains maximum SE and less BER than other hyper‐precoding designs.

Tensor-Train Decomposition-Based Hybrid Beamforming for Millimeter-Wave Massive Multiple-Input Multiple-Output/Free-Space Optics in Unmanned Aerial Vehicles with Reconfigurable Intelligent Surface Networks

Unmanned aerial vehicles (UAVs) can support low-cost, highly mobile communications while making it possible to establish dedicated terrestrial networks. To overcome the pointing error (PE) and beam misalignment of millimeter-wave large-scale multiple-input multiple-output/free-space optics (MIMO/FSO) caused by UAV jitter, a millimeter-wave massive MIMO/FSO hybrid beamforming method based on tensor train decomposition is proposed. This approach is used for reconfigurable intelligent surface (RIS) network-assisted UAV millimeter-wave massive MIMO/FSO to improve system spectral efficiency. Firstly, the high-dimensional channel of RIS-assisted millimeter-wave massive MIMO/FSO in UAV is represented as a low-dimensional channel by tensor training decomposition. Secondly, the two-way gated recursive unit attention neural network model can effectively solve the FSO PE caused by UAV jitter, and the fast fading channel and Doppler frequency shift are estimated by the Fast Cyclic Tensor Power Method (FCTPM) based on tensor training decomposition. Finally, the RIS phase shift matrix is optimized by singular value decomposition. The hybrid beamforming and RIS phase-shift matrices were estimated by using the low-complexity phase extraction alternating minimization method to solve the beam misalignment problem. Simulation results show that by using the proposed method, the spectrum utilization rate is improved by 23.6% compared with other methods.

Iterative Variational Bayesian Inference Based Channel Estimation for TWR mmWave Systems

Millimeter wave (mmWave) with two way relaying (TWR) is an emerging paradigm towards cellular fifth/sixth generation (5G/6G) technology that can support various data hungry applications with improve coverage, network throughput, and reliability. The foreseen potential of mmWave TWR system can be further alleviated by using large-scale multiple input multiple output (MIMO) architecture. However, high power consumption, hardware complexity and cost of fully digital large-scale MIMO architecture restricts its application. As a countermeasure, hybrid precoding structures with reduced RF chains are utilized. Therefore, the amalgamation of TWR MIMO mmWave system with hybrid precoding structure demands an accurate channel state information (CSI) to avail the maximum gain of two way MIMO communications. Furthermore, the inherent self-interference in TWR systems in collusion with sparse mmWave channels imposes severe challenges in acquiring accurate CSI. The challenge is further aggravated in case of frequency-selective mmWave channel. To solve this problem, we propose a novel iterative variational Bayesian inference (IVBI)-based channel estimation (CE) scheme in the time domain for TWR system with reduced RF chain. In the proposed method, we follow the alternative minimization method involving variational Bayesian inference (VBI) to estimate all the channels of mmWave TWR system. The performance of the proposed algorithm are evaluated over both flat fading and frequency-selective channel. Extensive simulation results for symmetric and non-symmetric MIMO structure validate the proposed algorithm. The Bayesian Cramer-Rao bound (BCRB) of the proposed estimator is also derived. This novel scheme converges fast and achieves significant improvement in terms of normalized mean square error (NMSE) and bit error rate (BER) as compared to state-of-art.

Adaptive Total-Variation and Nonconvex Low-Rank Model for Image Denoising

In recent years, image denoising methods based on total variational regularization have attracted extensive attention. However, the traditional total variational regularization method is an approximate solution based on convex method, and does not consider the particularity of the region with rich details. In this paper, the adaptive total-variation and nonconvex low-rank model for image denoising is proposed, which is a hybrid regularization model. First, the image is decomposed into sparse terms and low rank terms, and then the total variational regularization is used to denoise. At the same time, an adaptive coefficient based on gradient is constructed to adaptively judge the flat area and detail texture area, slow down the denoising intensity of detail area, and then play the role of preserving detail information. Finally, by constructing a nonconvex function, the optimal solution of the function is obtained by using the alternating minimization method. This method not only effectively removes the image noise, but also retains the detailed information of the image. The experimental results show the effectiveness of the proposed model, and SNR and SSIM of the denoised image are improved.

Convergence rates of training deep neural networks via alternating minimization methods

Convergence rates of training deep neural networks via alternating minimization methods

Linear Convergence of a Proximal Alternating Minimization Method with Extrapolation for \(\boldsymbol{\ell_1}\) -Norm Principal Component Analysis

A popular robust alternative of the classic principal component analysis (PCA) is the -norm PCA (L1-PCA), which aims to find a subspace that captures the most variation in a dataset as measured by the -norm. L1-PCA has shown great promise in alleviating the effect of outliers in data analytic applications. However, it gives rise to a challenging nonsmooth, nonconvex optimization problem, for which existing algorithms are either not scalable or lack strong theoretical guarantees on their convergence behavior. In this paper, we propose a proximal alternating minimization method with extrapolation (PAMe) for solving a two-block reformulation of the L1-PCA problem. We then show that for both the L1-PCA problem and its two-block reformulation, the Kurdyka–Łojasiewicz exponent at any of the limiting critical points is . This allows us to establish the linear convergence of the sequence of iterates generated by PAMe and to determine the criticality of the limit of the sequence with respect to both the L1-PCA problem and its two-block reformulation. To complement our theoretical development, we show via numerical experiments on both synthetic and real-world datasets that PAMe is competitive with a host of existing methods. Our results not only significantly advance the convergence theory of iterative methods for L1-PCA but also demonstrate the potential of our proposed method in applications.

Nonlocal low-rank plus deep denoising prior for robust image compressed sensing reconstruction

It is challenging for current compressive sensing (CS) approaches to reconstruct image from compressed observations with impulsive noise and outliers, termed robust image CS problem. In this paper, we propose a novel reconstruction model for the robust image CS reconstruction problem in the presence of impulsive noise. To ensure high-quality image reconstruction, we develop a nonlocal low-rank plus deep denoising prior model to simultaneously capture the nonlocal self-similarity (NSS) and deep prior, leading to a complementary reconstruction effect. Moreover, the robust M-estimator is utilized to suppress the outliers, which can strongly improve the robustness to impulsive noise. Considering the nonconvexity and complexity, both the half-quadratic (HQ) strategy and the alternating minimization method are employed to minimize the resulting large-scale optimization problem. Extensive experiments have demonstrated the effectiveness and superiority of our method significantly than the existing state-of-the art CS methods in terms of both visual quality and quantitative indexes under impulsive noise.

NON-CONVEX HYBRID TOTAL VARIATION FOR RESTORING MEDICAL IMAGE CORRUPTED BY POISSON NOISE

Abstract. In this work, we proposed the hybrid non-convex regularizers for Poisson noise removal on medical images. The model is built by a combination of non-convex total variation and non-convex fractional total variation. The proposed model allows for avoiding the annoying staircase artifacts and obtaining the reconstruction results with sharp and neat edges during the noise removal process. For handling the minimization problem, we employ the alternating minimization method associated with the iteratively reweighted l1 algorithm. Numerical experiments illustrate the efficiency of the proposed model and corresponding algorithm.

Hybrid Precoding Based on Active Learning for mmWave Massive MIMO Communication Systems

In this paper, a cost-effective and high-accuracy precoding technique based on ϵ-Fuzzy pareto active learning (FPAL) is proposed for millimeter wave (mmWave) massive MIMO communications. The proposed method achieves a low iteration convergence with low complexity. Two practical structures, namely fully-connected and partially-connected structures are considered for hybrid precoding. Furthermore, the effect of high and low-resolution quantization in the digital-to-analog converter, phase shifter, and imperfect channel state information are discussed. The performance results of the proposed technique and alternating minimization methods beside fully digital techniques are compared and discussed in the terms of the spectral efficiency (SE), bit error rate (BER), normalized mean square error (NMSE), energy efficiency (EE) for phase-shifter (PS) with low bit quantization, and imperfect channel state information (CSI). To address the applicability of the proposed method, experimental results are obtained from a real mmWave hardware setup compliant with 3GPP standards, and verify the simulated ones for the proposed ϵ-FPAL hybrid precoding scheme. The reduced complexity, higher performance and EE make the proposed method suitable for 5G and beyond communication systems.

A Non-Convex Hybrid Overlapping Group Sparsity Model with Hyper-Laplacian Prior for Multiplicative Noise

Multiplicative noise removal is a quite challenging problem in image denoising. In recent years, hyper-Laplacian prior information has been successfully introduced in the image denoising problem and significant denoising effects have been achieved. In this paper, we propose a new hybrid regularizer model for removing multiplicative noise. The proposed model consists of the non-convex higher-order total variation and overlapping group sparsity on a hyper-Laplacian prior regularizer. It combines the advantages of the non-convex regularization and the hybrid regularization, which may simultaneously preserve the fine-edge information and reduce the staircase effect at the same time. We develop an effective alternating minimization method for the proposed nonconvex model via an alternating direction method of multipliers framework, where the majorization–minimization algorithm and the iteratively reweighted algorithm are adopted to solve the corresponding subproblems. Numerical experiments show that the proposed model outperforms the most advanced model in terms of visual quality and certain image quality measurements.

Accelerated matrix completion algorithm using continuation strategy and randomized SVD

The matrix completion (MC) problem aims to recover the complete matrix from the observed matrix with missing entries. The unknown matrix is generally assumed to have a low-rank structure. It is well known that the low-rank minimization problem is NP-hard, so the rank function of the matrix is approximated by a surrogate function. In this paper, we firs t apply the optimization transfer technique to formulate the minimization problem into a bi-variate minimization problem by adding an auxiliary variable. We apply the alternating minimization method to find the minimizer of the bi-variate minimization problem. For the auxiliary variable, the minimum can be formulated as a linear combination of the observation data and the original variable. For the original variable, the minimum is a generalized singular thresholding (GSVT) operator of the auxiliary variable. However, most existing iteration methods for matrix completion problem use fixed thresholding value and also suffer from full singular value decomposition (SVD), which need more computation time for iteration. Thus, they become inefficient for large-scale problems. In order to handle these limitations, we apply the continuation strategy and the randomized singular value decomposition (RSVD) to reduce the computational costs. The threshold value of the continuous strategy is reduced in each iteration until a given minimum thresholding value is reached, while the RSVD only needs to decompose small-scale matrices. Hence, using both strategies simultaneously can effectively reduce the computation time for iteration. Numerical experiments demonstrate that the proposed algorithm can effectively recover the missing entries both in synthetic and real data.

An adaptive image restoration algorithm based on hybrid total variation regularization

In imaging systems, the mixed Poisson-Gaussian noise (MPGN) model can accurately describe the noise present. Total variation (TV) regularization-based methods have been widely utilized for Poisson-Gaussian removal with edge-preserving. However, TV regularization sometimes causes staircase artifacts with piecewise constants. To overcome this issue, we propose a new model in which the regularization term is represented by a combination of total variation and high-order total variation. We study the existence and uniqueness of the minimizer for the considered model. Numerically, the minimization problem can be efficiently solved by the alternating minimization method. Furthermore, we give rigorous convergence analyses of our algorithm. Experiments results are provided to demonstrate the superiority of our proposed hybrid model and algorithm for deblurring and denoising images simultaneously, in comparison with several state-of-the-art numerical algorithms.

An accelerated alternating directional method with non-monotone technique for matrix recovery

<abstract><p>Low-rank and sparse structures have been frequently exploited in matrix recovery and robust PCA problems. In this paper, we develop an alternating directional method and its variant equipped with the non-monotone search procedure for solving a non-convex optimization model of low-rank and sparse matrix recovery problems, where the concerned matrix with incomplete data is separable into a low-rank part and a sparse part. The main idea is to use the alternating minimization method for the low-rank matrix part, and to use the non-monotone line search technique for the sparse matrix part to iteratively update, respectively. To some extent, the non-monotone strategy relaxes the single-step descent into a multi-step descent and then greatly improves the performance of the alternating directional method. Theoretically, we prove the global convergence of the two proposed algorithms under some mild conditions. Finally, the comparison of numerical experiments shows that the alternate directional method with non-monotone technique is more effective than the original monotone method and the previous method. The efficiency and effectiveness of the proposed algorithms are demonstrated by solving some instances of random incomplete matrix recovery problems and some problems of the background modeling in video processing.</p></abstract>

Non-Convex High-Order TV and ℓ 0-Norm Wavelet Frame-Based Speckle Noise Reduction

To obtain natural restorations from the noisy images contaminated by speckle noise, this brief presents a novel hybrid non-convex regularizers model for image denoising. The proposed new variational model closely combines the superiorities of non-convex high-order total variation function and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{0}$ </tex-math></inline-formula> -norm wavelet frame. This combination helps to avoid the staircase artifacts and maintain discontinuities while removing noise. Numerically, by integrating two popular tools: iteratively reweighted <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> algorithm and variable splitting method, a modified alternating minimization method is adopted to optimize the resulting minimization problem. Finally, compared with several despeckling methods, numerical experiments indicate the competitive performance of our solver in visual improvement and objective measurement.

Hybrid non-convex regularizers model for removing multiplicative noise

Obtaining natural and realistic restorations from the noisy images contaminated by multiplicative noise is a challenging task in image processing. To get over this conundrum, by introducing the non-convex potential functions into the total variation and high-order total variation regularizers, we investigate a novel hybrid non-convex optimization model for image restoration. Numerically, to optimize the resulting high-order PDE system, a proximal linearized alternating minimization method, based on the classical iteratively reweighted ℓ 1 algorithm and variable splitting technique, is designed in detail. Meanwhile, the convergence of the constructed algorithm is also established on the basis of convex analysis. The provided numerical experiments point out that our new scheme shows superiorities in both visual effects and quantitative comparison, especially in terms of the staircase aspects suppression and edge details preservation, compared with some popular denoising methods.