Nonconvex Penalty Functions Research Articles

NG-RED:Nonconvex group-matrix residual denoising learning for image restoration

Group-matrix based prior modeling has demonstrated superior performance in various image restoration (IR) applications. Though the joint prior of nonlocal self-similarity (NSS) and local sparsity of images are integrated into restoration model through sparse coding, the gap between nonlocal group-matrix denoising and IR task still exist. Inspired by the success of denoising prior driven plug-and-play strategy, this paper designs a novel nonconvex group-matrix residual denoising (NG-RED) learning model. Towards this end, we define two group-matrix denoising models based on nonlocal group-matrix residual, namely SCGD and LRGD. Through introducing the significant theoretical relationship of SCGD and LRGD, the optimization problem of NG-RED is further solved via generalized singular value thresholding (GSVT) operator. Moreover, our NG-RED are integrated into IR framework that serves as image prior via HQS scheme. We evaluate the proposed NG-RED by using typical nonconvex penalty functions on image denoising and restoration problem. Extensive experiments on typical restoration problems demonstrate that the proposed method is comparable to state-of-the-art denoising methods and outperforms several testing IR methods in terms of both objective and perceptual quality metrics.

Balance Sparse Decomposition Method with Nonconvex Regularization for Gearbox Fault Diagnosis

As an important part of rotating machinery, gearboxes often fail due to their complex working conditions and harsh working environment. Therefore, it is very necessary to effectively extract the fault features of the gearboxes. Gearbox fault signals usually contain multiple characteristic components and are accompanied by strong noise interference. Traditional sparse modeling methods are based on synthesis models, and there are few studies on analysis and balance models. In this paper, a balance nonconvex regularized sparse decomposition method is proposed, which based on a balance model and an arctangent nonconvex penalty function. The sparse dictionary is constructed by using Tunable Q-Factor Wavelet Transform (TQWT) that satisfies the tight frame condition, which can achieve efficient and fast solution. It is optimized and solved by alternating direction method of multipliers (ADMM) algorithm, and the non-convex regularized sparse decomposition algorithm of synthetic and analytical models are given. Through simulation experiments, the determination methods of regularization parameters and balance parameters are given, and compared with the L1 norm regularization sparse decomposition method under the three models. Simulation analysis and engineering experimental signal analysis verify the effectiveness and superiority of the proposed method.

Adaptive Huber trace regression with low-rank matrix parameter via nonconvex regularization

In this paper, we consider the adaptive Huber trace regression model with matrix covariates. A non-convex penalty function is employed to account for the low-rank structure of the unknown parameter. Under some mild conditions, we establish an upper bound for the statistical rate of convergence of the regularized matrix estimator. Theoretically, we can deal with heavy-tailed distributions with bounded (1+δ)-th moment for any δ>0. Furthermore, we derive the effect of the adaptive parameter on the final estimator. Some simulations, as well as a real data example, are designed to show the finite sample performance of the proposed method.

Hyperbolic tangent penalty function for edge-preserving image filtering

Edge-preserving image filtering provides a common interface for a variety of edge-aware image processing tasks. It is essential to the field of image processing and computational photography. Recent attempts at optimization models with non-convex regularization have shown promising performance in edge-preserving image filtering. In this paper, we propose a novel non-convex regularization term based on a hyperbolic tangent penalty function, which is shown to be more edge-preserving than popular non-convex penalty functions. We embed the proposed regularization term in an optimization model for edge-preserving image filtering. To solve the model with the non-convex regularization, we propose an efficient solution based on the additive half quadratic minimization and Fourier domain optimization. We have conducted extensive experiments to evaluate the proposed filter. Both quantitative and qualitative results demonstrate that our filter benefits various applications. Furthermore, the proposed filter is highly efficient and renders real-time processing of 720P color images on a modern GPU.

3D sparse SAR imaging based on complex nonconvex penalty function

3D sparse SAR imaging based on complex nonconvex penalty function

A novel non-convex penalty function-based STAP algorithm for airborne passive radar systems

In conventional sparse Space-Time Adaptive Processing (STAP) algorithms, the l0-norm is often used instead of the l1-norm to relax convexity constraints. The approach presented in this paper can be mathematically reformulated as a linear problem, which can be efficiently solved using various convex optimization techniques. While this method effectively circumvents the NP-Hard complexity associated with the l0-norm, it does come with certain potential challenges. For example, the observation matrix must meet the criteria of the Restricted Isometry Property (RIP) and other stringent conditions. In this research, a novel STAP algorithm is introduced, which is based on a non-convex penalty function. This innovative algorithm substitutes the traditional l0-norm with a custom-designed non-convex penalty function and is applied to the Direct Filter Processor (DFP), solved using the recursive least squares (RLS) algorithm. The resulting algorithm, known as gp-RLS, is demonstrated through Monte Carlo simulations to outperform other l1-based and reduced-rank STAP algorithms in terms of faster convergence, improved signal-to-clutter-plus-noise ratio (SCNR), and enhanced target detection performance.

A fusion non-convex group sparsity difference method and its application in rolling bearing fault diagnosis

Bearing fault is a common factor leading to machine failures. How to extract the periodic transient signal due to bearing faults submerged in strong noise is a challenging problem for bearing fault diagnosis. Total variation denoising is a method used to remove noise and extract features. However, it solely relies on the sparsity of the first-order difference of the signal, resulting in the loss of important features and underestimation of amplitude. Additionally, it fails to capture the periodicity of the signal. To address these issues, a novel approach called fusion non-convex group sparsity difference (FNC-GSD) method is proposed for bearing fault diagnosis. Firstly, it is recognized that noise does not exhibit sparsity in the difference domain, while transient signal exhibits group sparsity in the difference domain. This grouping property enhances the feature selection ability of sparse model. Inspired by this, the group sparsity of the transient signal in the difference domain is used to preserve the fault features as much as possible. Additionally, in order to promote sparsity of the signal itself in the time domain to preserve the potential impulse component, a l 1-norm regularization term is introduced. Furthermore, a non-convex sparsity-inducing penalty function strategy is employed to prevent amplitude underestimation. The proposed sparse model considers both the group sparsity in the difference domain and the sparsity in the time domain of the transient signal, and its solution is derived according to the majorization–minimization algorithm. And the particle swarm optimization algorithm is used to adaptively search the regularization parameters of FNC-GSD. Finally, multiple bearing fault diagnosis experiments are conducted to demonstrate the performance of the FNC-GSD. The results show that it has advantages in fault feature extraction compared with some other methods.

Efficient indoor localization with wireless sensor networks via learnable sparse coding

In this study, the indoor localization problem for human movement is built as a sparse representation, and a learnable sparse coding algorithm is proposed by introducing a nonconvex penalty function as the sparse constraints. The good properties of the penalty are able to help reduce the iteration cost to find a sparse solution indicating the target locations. The sparse code can achieve its optimal value from the trained neural network model with parametrization. It is instead of a piece of algorithmic processing in the conventional methods requiring a large iteration. A real-world experiment using a Bluetooth low energy wireless sensor network validates the proposed method and shows the improvement of the error reduction at the first five iteration points on average compared to iterative shrinkage and thresholding algorithm (ISTA), and achieves competitive performance in the localization tasks for learned-ISTA-based algorithms.

Morphological component analysis under non-convex smoothing penalty framework for gearbox fault diagnosis

The sparse representation methodology has been identified to be a promising tool for gearbox fault diagnosis. The core is how to precisely reconstruct the fault signal from noisy monitoring signals. The non-convex penalty has the ability to induce sparsity more efficiently than convex penalty. However, the introduction of non-convex penalty usually influences the convexity of the model, resulting in the unstable or sub-optimal solution. In this paper, we propose the non-convex smoothing penalty framework (NSPF) and combine it with morphological component analysis (MCA) for gearbox fault diagnosis. The proposed NSPF is a unify penalty construction framework, which contains many classical penalty while a new set of non-convex smoothing penalty functions can be generated. These non-convex penalty can guarantee the convexity of the objective function while enhancing the sparsity, thus the global optimal solution can be acquired. The simulation and engineering experiments validate that the NSPF enjoys more reconstruction precision compared to the existing penalties.

Rolling bearing incipient fault feature extraction using impulse-enhanced sparse time-frequency representation

Incipient faults features are often extremely weak and susceptible to heavy noise, making it challenging to obtain the concentrated faulty energy ridges in the time-frequency domain. Thus, a novel impulse-enhanced sparse time-frequency representation (IESTFR) method is proposed in this paper. First, the time-rearranged multisynchrosqueezing transform is utilized to produce a time-frequency representation (TFR) with a high energy concentration for faulty impulses. Next, a new non-convex penalty function is constructed by the hyperbolic tangent function, which can enhance the periodic impulsivity of sparse TFR for more obvious fault characteristic frequency. Moreover, the time-frequency transform is evaluated and compared by simulated signals and a selection strategy for the regularization parameter is designed. Simulated signals and two experimental signals are applied to verify the effectiveness of IESTFR, and the results show that IESTFR is effective and superior in bearing incipient fault feature extraction.

Background oriented schlieren image displacement estimation method based on global optical flow

Background oriented schlieren (BOS) technology is a recent non-contact optical diagnostic technology, which can be used for the visualization of variable density flow and the quantitative measurement of the refractive index field. The accuracy of BOS image displacement estimation directly determines the accuracy of refractive index field reconstruction. For BOS images, the optical path construction method, vibration and uneven illumination, blurring and defocusing effects, and the refocusing effect of volume refraction make it difficult for the BOS image sequence to satisfy the constant brightness assumption. In this study, the optical flow algorithm based on the assumption of a constant gradient in PIV is improved and applied to BOS image displacement estimation. A data item based on the assumption of constant brightness combined with the assumption of constant gradient is proposed. The first-order div-curl regularization constraint is used as the spatial smoothing condition, and the non-convex Charbonnier penalty function is used to determine the displacement field by the variational method to minimize the energy function. The proposed method is tested on the synthetic images obtained by two-dimensional turbulent direct numerical simulation (DNS), the BOS synthetic images with brightness changes, and the actual experimental BOS images of the flow field above a premixed butane/air combustion flame with different equivalence ratios. Compared with the current advanced optical flow algorithm, this method has higher accuracy and robustness. Overall, the proposed method can not only obtain more accurate results when the brightness of the BOS synthetic image and experimental image changes, but also helps in retaining the flow details such as small divergence and vorticity structure of the heat flow field while reducing the outliers.

Deep and Low-Rank Quaternion Priors for Color Image Processing

Due to the physical nature of color images, color image processing such as denoising and inpainting has shown extensive and versatile possibilities over grayscale image processing. The monochromatic and the concatenation model have been widely used to process color images by processing each color channel independently or concatenating three color channels as one unified one and then used existing grayscale image processing methods directly without specific operations. These above schemes, however, have some limitations: (1) they would destroy the inherent correlation among three color channels since they cannot represent color images holistically; (2) they usually focus on one specific handcrafted prior such as smoothness, low-rankness, or even deep prior and thus failing to fuse deep and handcrafted priors of color images flexibly. To conquer these limitations, we propose one unified model to integrate deep prior and low-rank quaternion prior (DLRQP) for color image processing under the plug-and-play (PnP) framework. Specifically, the quaternion representation with low-rank constraint is introduced to denote the color image in a holistic way and one advanced denoiser is adopted to explore the deep prior in an iterative process. To tightly approximate the quaternion rank, one nonconvex penalty function is further utilized. We derive an alternate iterative approach to tackle the proposed model. We empirically demonstrate that our model can achieve superior performance over existing methods on both color image denoising and inpainting tasks.

Gearbox fault diagnosis based on generalized multivariate logarithmic regularization

Feature extraction based on sparse representation is widely applied in the field of mechanical fault diagnosis. L1 norm regularization is a classical sparse regularization method, but this method has sparse underestimation for large-value features. A signal sparse representation method based on generalized multivariate logarithmic regularization is proposed in this paper. First, the sparse penalty term in the proposed method is designed according to the minimum convolution and logarithmic function, namely the generalized logarithmic non-convex penalty function. Then, the convexity condition of the objective function is studied to verify the feasibility of the method. The applicability of the method is also improved by analyzing the parameter constraint relation in the objective function. Finally, the sparse optimal solution is obtained by the forward-back splitting algorithm. Experiments show that compared with other non-convex sparse models, the proposed method can solve the problem of sparse underestimation more effectively and improve the reliability of gearbox fault diagnosis.

Wavelet-domain group-sparse denoising method for ECG signals

In this paper, a group-sparse signal denoising approach is proposed, and both non-convex regularization and sparsity characteristics in wavelet domain are incorporated to estimate the electrocardiogram (ECG) signals with noise. To strongly promote wavelet sparsity, a parameterized non-convex penalty function is introduced, and the interval for the parameter is identified to guarantee the strict convexity of the total cost function. To retain the details of ECG signals, all the wavelet coefficients are estimated by minimizing certain single objective function, and thus the insignificant coefficients that do not survive wavelet thresholding can be maintained. The algorithm is solved based on the majorization–minimization optimization method, the alternating direction method of multipliers, and proximal method. The real collected ECG signals and MIT-BIH arrhythmia database are used to evaluate the effectiveness of the proposed wavelet-domain group-sparse method (WDGS) for ECG signal enhancement. By qualitative and quantitative analysis, it shows that our method can effectively suppress the undesired noise and keep the important morphology of ECG signals.

One-step sparse estimates in the reverse penalty for high-dimensional correlated data

Variable selection methods are widely used in modeling high-dimensional data, such as portfolios, gene selection, etc. But strong correlations exist in high-dimensional data leading to poor model estimation and prediction. In this paper, inspired by Yang and Yang (2021), we propose generalized adaptive smooth adjustment for correlated effects estimator to deal with high-dimensional correlated data, reducing estimation bias and encouraging grouping, but also improving the correct rate of model identification. Furthermore, we use the one-step estimation algorithm to convert the non-convex penalty function to L1 penalty, greatly reducing the computational cost. Finally, we give some finite sample performance and case analysis of the proposed method and demonstrate the consistency of variable selection and oracle property under some conditions.

Estimation of high-dimensional vector autoregression via sparse precision matrix

Summary We consider the problem of estimating sparse vector autoregression (VAR) via penalized precision matrices. This matrix is the output of the underlying directed acyclic graph of the VAR process, whose zero components correspond to the zero coefficients of the graphical representation of the VAR. The sparsity-based precision matrix estimator is deduced from the D-trace loss with convex and nonconvex penalty functions. We establish the consistency of the penalized estimator and provide the conditions for which all true zero entries of the precision matrix are actually estimated as zero with probability tending to one. The relevance of the method is supported by simulated experiments and a real data application.

Asymmetry total variation and framelet regularized nonconvex low-rank tensor completion

The low-rank tensor representation has shown enormous potential and advantages in diverse applications, such as image inpainting, image restoration. However, (1) due to the inherent limitations of the low-rank tensor model, the tensor nuclear norm is utilized yet as a biased estimate of the tensor rank, which limits the image recovery performance. (2) most current low-rank approximation-based methods focus on the preservation of global information, while ignoring the local details preservation such as the spatial piece-wise smoothness and sharp edges. To overcome the above obstacles, we propose a novel asymmetry three-dimensional total variation and framelet regularized nonconvex low-rank tensor completion (ATV-FNTC) model, which integrates the nonconvex penalty function and asymmetric three-dimensional total variation into one unified model. Specifically, ATV-FNTC introduces one nonconvex penalty function as a tighter regularizer to approximate the tensor rank. Different from existing methods, the asymmetric three-dimensional total variation (ATV) regularization is developed to achieve more accurate recovery of detailed information and flexibly control the smoothing strength of different dimensions of tensor data. Furthermore, we design an iterative algorithm to solve the nonconvex ATV-FNTC model by the alternating direction methods of multipliers. Experimental results reveal the superiority of the proposed method compared with several state-of-the-art methods when handling different completion tasks.

Inference for low‐ and high‐dimensional inhomogeneous Gibbs point processes

AbstractGibbs point processes (GPPs) constitute a large and flexible class of spatial point processes with explicit dependence between the points. They can model attractive as well as repulsive point patterns. Feature selection procedures are an important topic in high‐dimensional statistical modeling. In this paper, a composite likelihood (in particular pseudo‐likelihood) approach regularized with convex and nonconvex penalty functions is proposed to handle statistical inference for possibly high‐dimensional inhomogeneous GPPs. We particularly investigate the setting where the number of covariates diverges as the domain of observation increases. Under some conditions provided on the spatial GPP and on penalty functions, we show that the oracle property, consistency and asymptotic normality hold. Our results also cover the low‐dimensional case which fills a large gap in the literature. Through simulation experiments, we validate our theoretical results and finally, an application to a tropical forestry dataset illustrates the use of the proposed approach.

Low-rank matrix factorization with nonconvex regularization and bilinear decomposition

Low-rank matrix factorization (LRMF) is a powerful approach that can recover useful information with low-rank features from corrupted data and has been broadly applied in various scenarios such as computer vision and statistics. Existing convex relaxation-based LRMF models' performance and solutions are often limited by unsatisfied recovery accuracy and prolonged execution time due to excessive relaxation bias and heavy computational burden. In this paper, we first explore one class of parameterized non-convex penalty function and generalize it as a non-convex relaxation substitution term for the original rank function in the LRMF, which can effectively enhance the recovery capability for low-rank data. In order to reduce the computational burden of the nuclear-norm-based optimization scheme and make LRMF applicable to large-scale data. We propose to perform bilinear decomposition on the low-rank matrix and derive the bilinear factorization formula under the parameterized non-convex alternative. With the above considering, a low-rank matrix recovery model via bilinear decomposition and non-convex regularization, denoted as NCBF, is formulated. Further, the optimal solution algorithm for the NCBF model is developed in the framework of the block coordinate descent, combining simultaneously three strategies of decoupling, proximal gradient descent, and block prox-linear. Also, the convergence condition for the proposed algorithm has been given. We perform extensive experiments using synthetic data and real-world image datasets. The synthetic data test results demonstrate the dual superiority of the bilinear decomposition scheme in terms of operation time and estimation accuracy. Application of denoising to both generic and CT image datasets, experimental results show that NCBF outperforms several existing methods under both subjective and objective metrics evaluations.

Nonconvex variational approach for simultaneously recovering cartoon and texture images

Many real-world applications in image processing and computer vision require splitting an input image into a cartoon component and a texture component. We propose a nonconvex variational image decomposition model for simultaneously recovering cartoon and texture images. To induce the sparsity of gradient norms of the cartoon image more strongly than the classical total variation regularization, we applied the nonconvex firm penalty function as a regularizer for the cartoon image. The nonconvex firm penalty regularizer function has a better ability to separate the piecewise constant component with neat edges. The G-norm was used as an oscillating prior for the texture image. Converting the proposed optimization model to a constrained problem by variable splitting, we addressed it with the alternating direction method of multipliers. Experimental results and comparisons were given to verify the superiority of existing state-of-the-art methods in terms of correlation, peak signal-to-noise ratio, structural similarity, and visual quality. Finally, we demonstrated the effectiveness of the proposed model by several applications such as image abstraction and pencil sketching, artifact removal, image denoising, image composition, and detail enhancement.