Nonconvex Sparse Regularization Research Articles

Generalized Shannon entropy sparse wavelet packet transform for fault detection of traction motor bearings in high-speed trains

An effective structural health monitoring method of traction motor bearings is a powerful guarantee for the safety operation of high-speed trains. However, it is exceptionally difficult to detect bearing fault characteristics from the vibration signals of traction motor bearings operating at high rotational speeds. In this scenario, a generalized Shannon entropy sparse wavelet packet transform (GSWPT) for fault detection of motor bearings is proposed in this paper. Firstly, a generalized Shannon entropy sparse regularization method is proposed to obtain sparse wavelet reconstruction coefficients by extending the definition of the Shannon information entropy, and the non-convex sparse regularization function is minimized by synergistic swarm optimization algorithm. Then, the wavelet node coefficients are weighted according to the second-order cyclostationarity index of the wavelet packet node to further enhance the sparsity of the reconstructed signal. Moreover, the optimal decomposition level of GSWPT is adaptively selected by the maximum sparsity and cyclostationarity criterion. Particularly, in order to verify the bearing fault detection performance of GSWPT in practical engineering, a bearing fault dynamic model of traction motor in high-speed train was established based on Hertz contact theory and the fourth-order Runge-Kutta method to obtain simulated data under strong Gaussian white noise, and a corresponding test platform was constructed to collect experimental data under different operating conditions. Finally, the applications on the simulated and experimental signals of traction motor bearings in high-speed trains demonstrate that GSWPT significantly outperforms the conventional wavelet packet transform, dual-tree complex wavelet packet transform, blind deconvolution, modal decomposition, and Infogram methods to some extent for fault detection.

Image Deblurring Based on Convex Non-Convex Sparse Regularization and Plug-and-Play Algorithm

Image deblurring based on sparse regularization has garnered significant attention, but there are still certain limitations that need to be addressed. For instance, convex sparse regularization tends to exhibit biased estimation, which can adversely impact the deblurring performance, while non-convex sparse regularization poses challenges in terms of solving techniques. Furthermore, the performance of the traditional iterative algorithm also needs to be improved. In this paper, we propose an image deblurring method based on convex non-convex (CNC) sparse regularization and a plug-and-play (PnP) algorithm. The utilization of CNC sparse regularization not only mitigates estimation bias but also guarantees the overall convexity of the image deblurring model. The PnP algorithm is an advanced learning-based optimization algorithm that surpasses traditional optimization algorithms in terms of efficiency and performance by utilizing the state-of-the-art denoiser to replace the proximal operator. Numerical experiments verify the performance of our proposed algorithm in image deblurring.

Cauchy non-convex sparse feature selection method for the high-dimensional small-sample problem in motor imagery EEG decoding.

The time, frequency, and space information of electroencephalogram (EEG) signals is crucial for motor imagery decoding. However, these temporal-frequency-spatial features are high-dimensional small-sample data, which poses significant challenges for motor imagery decoding. Sparse regularization is an effective method for addressing this issue. However, the most commonly employed sparse regularization models in motor imagery decoding, such as the least absolute shrinkage and selection operator (LASSO), is a biased estimation method and leads to the loss of target feature information. In this paper, we propose a non-convex sparse regularization model that employs the Cauchy function. By designing a proximal gradient algorithm, our proposed model achieves closer-to-unbiased estimation than existing sparse models. Therefore, it can learn more accurate, discriminative, and effective feature information. Additionally, the proposed method can perform feature selection and classification simultaneously, without requiring additional classifiers. We conducted experiments on two publicly available motor imagery EEG datasets. The proposed method achieved an average classification accuracy of 82.98% and 64.45% in subject-dependent and subject-independent decoding assessment methods, respectively. The experimental results show that the proposed method can significantly improve the performance of motor imagery decoding, with better classification performance than existing feature selection and deep learning methods. Furthermore, the proposed model shows better generalization capability, with parameter consistency over different datasets and robust classification across different training sample sizes. Compared with existing sparse regularization methods, the proposed method converges faster, and with shorter model training time.

Robust Nonconvex Sparse Optimization for Impact Force Identification

The inherent sparse structure of impact forces has garnered considerable attention in the field of impact force identification. However, conventional convex sparse regularization methods, including the widely used [Formula: see text] regularization, often encounter challenges such as underestimation of impact amplitudes and biased estimations. To address these limitations, we propose a robust nonconvex sparse regularization method for impact force identification. The key advantage of our method is the simultaneous retention of robustness and unbiasedness. The robustness of our method is primarily achieved through an efficient solver developed within the alternating direction method of multipliers (ADMM) framework. By combining convex and nonconvex strategies, the ADMM solver separates the intractable nonconvex problem into more manageable convex sub-problems. Additionally, the ADMM solver incorporates the firm-thresholding operator, which ensures an unbiased amplitude distribution and preserves the impact amplitudes. With a sparse and under-determined sensor configuration, our proposed method enables simultaneous impact localization and time-history reconstruction. We comprehensively demonstrate the algorithmic performance through a series of numerical simulations and laboratory experiments on typical composite structures. The comparative results clearly indicate that our proposed approach achieves significant improvements in identification accuracy compared to classical sparse regularization methods, such as [Formula: see text] and [Formula: see text] regularization.

Group sparsity extension of “Non-convex sparse regularization via convex optimization for impact force identification”

We recently proposed a non-convex sparse regularization method in the ADMM framework to reconstruct and localize unknown impact forces. This method surpasses the convex ℓ1 regularization not only in inducing sparsity but also in avoiding the high-amplitude underestimation of solutions. In this work, we aim to improve the identification performance and enable the monitoring of unknown impact forces with fewer sensors by incorporating prior information in the form of natural grouping of the solution components. To achieve this, we extend our previous work on non-convex sparse regularization by incorporating group information into the method. The resulting group sparsity problem is challenging to solve due to the mixed structure and possible grouping irregularity. To address this, we develop an efficient ADMM solver in a grouped manner, featuring a novel shrinkage operator. We validate our approach both numerically and experimentally on aircraft-like composite laminated plates. Our case studies demonstrate that the proposed method achieves high accuracy and strong robustness in impact localization and time–history reconstruction from single-sensor-based measurements.

Ultrasonic defect detection in noisy signals by a nonconvex sparse regularization approach

Ultrasonic testing has been an important tool for the detection of hidden defects in materials yet its effectiveness is usually compromised by the noise originated from both the testing system and the material being evaluated. Motivated by the prior knowledge that only a small number of defects or reflectors exist within the tested material thus the measured signal can be viewed as a linear combination of a few echoes, a signal processing method is proposed in this study where a nonconvex sparse regularization method based on lp-norm (0 < p < 1) penalty is employed to suppress noises in ultrasonic NDT signals thus enhancing the effectiveness and accuracy of flaw detection. Based on a specially designed over-complete Gabor dictionary, the nonconvex sparse regularization method is introduced to sparsely represent the noisy signal. After signal representation, noise is suppressed by a pruning operator, facilitating the identification and reconstruction of flaw-reflected echoes. The performance of the proposed method is quantitatively evaluated and compared with competing algorithms using simulated noisy ultrasonic signals in a statistical manner. Experimental results are also presented to demonstrate the effectiveness of our method.

Dual-graph with non-convex sparse regularization for multi-label feature selection

Dual-graph with non-convex sparse regularization for multi-label feature selection

Non-convex sparse regularization via convex optimization for impact force identification

Instead of traditional Tikhonov regularization, sparse regularization methods like ℓ1 regularization have been a popular choice for impact force identification because it can encode sparse prior information into the inverse problem solving. However, ℓ1 regularization tends to have a limited ability to promote sparsity and underestimate high-amplitude components in the solution. Non-convex regularization holds the potential to promote sparsity more efficiently, and then further improves the accuracy of solutions. In this paper, we consider a family of non-convex regularizers with convexity-preserving characteristics to promote sparsity. By virtue of the non-convex regularizers, we propose a non-convex sparse regularization method to simultaneously localize and reconstruct impact events with a highly under-determined sensor setting. To avoid the inherent difficulties concerning non-convex optimization, we turn to the Alternating Direction Method of Multipliers(ADMM) algorithm. ADMM helps separate the primal non-convex problem into a sequence of convex sub-problems, each of which is easy-to-solve. The non-convex exponential penalty is chosen as the representative of this class of regularizers through theoretical analysis. Several simulations and experiments are conducted on a stiffened composite laminated panel to validate the proposed method. The proposed method outperforms the standard ℓ1 regularization in both localization accuracy and time-history reconstruction accuracy.

Sparse regularization based feature selection: A survey

Feature selection, as an essential preprocessing tool, aims to identify a subset of crucial features by eliminating redundant and noisy features according to a predefined criterion. In recent years, sparse learning has received considerable attention. As an important component of sparse learning, sparse regularization has been widely used in feature selection to enforce irrelevant feature coefficients to be small or exactly zero, which is especially suitable for selecting the discriminative and relevant features. In this paper, we attempt to provide a survey on various sparse regularization based methods. We group the existing sparse regularization based methods into two categories, i.e., convex sparse regularization based feature selection and non-convex sparse regularization based feature selection. This survey not only compares the common and different characters of sparse regularizations based feature selection methods, but also offers useful guidance for practitioners of feature selection.

An adaptive generalized logarithm sparse regularization method and its application in rolling bearing fault diagnosis

The generalized logarithm sparse regularization method (G-log) for fault diagnosis of rotating devices can effectively reconstruct repetitive transient shocks from noise-disturbed signals, but its reconstruction accuracy frequently becomes inferior due to unsuitable regularization parameters. Moreover, conventional sparse regularization methods perform nothing on the input signals to guarantee that the impulse characteristics remain constant during the entire iteration process, which exacerbates the influence of noise on the reconstruction accuracy. To overcome these challenges, an adaptive generalized logarithm sparse regularization method (AG-log) based on the second-order cyclostationary indicator (ICS2) and the improved maximum correlation Pearson correlation coefficient deconvolution (IMCPCCD) method is proposed in this paper. Firstly, the optimal threshold parameter k for each iteration of AG-log is determined based on the ICS2 criterion to ensure the optimal reconstruction accuracy, while the optimal combination of iteration numbers N and k is established. Secondly, the original signal and the IMCPCCD filtered signal are alternately used as the input signal of AG-log according to the parity of the iterative steps to reduce the interference of noise. Finally, the application on simulated and two engineering case signals demonstrates that AG-log has better reconstruction accuracy compared with conventional nonconvex sparse regularization methods.

Phase coherent noise reduction in digital holographic microscopy based on adaptive non-convex sparse regularization

Digital holographic microscopy is a new type of quantitative phase measurement tool. When a coherent light source is used for illumination, the phase coherent noise seriously affects the imaging quality and measurement accuracy. A method of phase coherent noise reduction based on adaptive non-convex sparse regularization is proposed in this paper. This denoising problem is considered as a sparse regularization model, which uses a non-convex penalty called the generalized minimax-concave; the penalty can lead to a more accurate estimation of the phase image amplitude and reduce most of the noise. Meanwhile, the threshold of the algorithm to solve this problem is guided by the edge credibility map adaptively. In edge regions, the threshold value is small so that edge details are preserved; however, in other regions, the threshold value is big so that the phase coherent noise is removed. The experimental results confirmed the validity of the proposed method.

Transient-extracting transform based on non-convex sparse regularization: A useful scheme for bearing fault feature enhancement

Periodic transient shocks of vibration signals often reflect the failure of mechanical components. With the time-frequency (TF) distribution (TFD), Transient-extracting transform (TET) can descript and extract the fault-related transient component effectively. Nevertheless, vibrations collected from practical sources are always contaminated by strong background noise, thus the transient shocks are often submerged by noise components. Therefore, how to effectively extract the weak transient component from noise is an important task to achieve the bearing fault diagnosis. Considering the above issue, this paper developed a generalized mini-max concave (GMC) penalty based TET (GMCTET) for bearing fault feature enhancement. Specifically, the GMC penalty based-sparse regularization is firstly applied to the raw signal. Benefiting from the good ability of GMC penalty-based sparse regularization in both sparsity and signal fidelity. The transient shocks are initially presented, and the loss of energy amplitude of the raw signal during the processing is reduced simultaneously. Next, TET can well characterize these transient shocks via a highly readable TFD, and the transient component is extracted based on the TF coefficients. The shock structures in transient component are further highlighted after TET treatment. Finally, the bearing fault frequencies are identified through the spectrum of the extracted transient component. The effectiveness of GMCTET is verified from the comparative analysis results of the simulation and experimental studies.

Nonconvex Sparse Regularization for Deep Neural Networks and Its Optimality.

Recent theoretical studies proved that deep neural network (DNN) estimators obtained by minimizing empirical risk with a certain sparsity constraint can attain optimal convergence rates for regression and classification problems. However, the sparsity constraint requires knowing certain properties of the true model, which are not available in practice. Moreover, computation is difficult due to the discrete nature of the sparsity constraint. In this letter, we propose a novel penalized estimation method for sparse DNNs that resolves the problems existing in the sparsity constraint. We establish an oracle inequality for the excess risk of the proposed sparse-penalized DNN estimator and derive convergence rates for several learning tasks. In particular, we prove that the sparse-penalized estimator can adaptively attain minimax convergence rates for various nonparametric regression problems. For computation, we develop an efficient gradient-based optimization algorithm that guarantees the monotonic reduction of the objective function.

Convex compressive beamforming with nonconvex sparse regularization.

The convex sparse penalty based compressive beamforming technique can achieve robust high resolution in direction-of-arrival (DOA) estimation tasks, but it often leads to an insufficient sparsity-inducing problem due to its convex loose approximation to ideal ℓ0 nonconvex penalty. On the contrary, the nonconvex sparse penalty can tightly approximate ℓ0 penalty to effectively enhance DOA estimation accuracy, but it incurs an initialization sensitivity problem due to its multiple local minimas. Leveraging their individual advantages, a minimax-concave penalty (MCP) regularized DOA estimation algorithm is proposed to achieve a maximally sparse level while maintaining the convex property of the overall objective function. Moreover, an accelerated block gradient descent-ascent algorithm with convergence guarantee is developed to rapidly achieve its one optimal point. Simulation results demonstrate that MCP penalty improves DOA estimation accuracy compared with popular sparse compressive beamforming techniques in strong noise scenarios and weak source confirmation. Ocean experimental results also validate that it retains more stable DOA estimation accuracy and incurs less artificial interferences.

A Sparsity-Assisted Fault Diagnosis Method Based on Nonconvex Sparse Regularization

Sparse representation theory can be adopted for fault feature extraction and classification. Inspired by these two capabilities of sparse representation theory, this paper proposes a novel collaborative sparsity-assisted fault diagnosis (CSFD) method. Specifically, due to the repeatability and sparsity of fault feature signal in the whole signal, the feature extraction capability of sparse representation is utilized to extract fault features and construct a feature matrix. Subsequently, owing to the sparsity of fault classification problem itself, the classification capability of sparse representation is employed to achieve fault classification. Moreover, fault feature extraction is a key issue for fault diagnosis. Therefore, in order to improve the effectiveness of fault feature extraction, a RC-adjustment strategy with the ability to adaptively adjust regularization parameter is designed to assist the generalized minimax-concave (GMC) penalty for feature extraction. The advantage of the GMC penalty is that it can establish a nonconvex sparse regularization model with convex optimization solution. Finally, the proposed CSFD method is tested and verified by Case Western Reserve University (CWRU) bearing dataset and actual engineering dataset. Especially, the diagnostic accuracy of this method can reach 99.92%, which is nearly 7.68% higher than the traditional method based on L1 norm penalty. Enormous experiment results have thoroughly demonstrated the effectiveness of the proposed CSFD method for fault diagnosis.

Sparsity-Oriented Nonconvex Nonseparable Regularization for Rolling Bearing Compound Fault under Noisy Environment

Rolling bearing is widely used in rotating machinery and, at the same time, it is easy to be damaged due to harsh operating environments and conditions. As a result, rolling bearing is critical to the safe operation of the machinery devices. Compound fault of rolling bearing is not a simple superimposition of multiple single faults, but the coupling of multiple fault features, making the vibration signal, becomes complicated. In our study, sparsity-oriented nonconvex nonseparable regularization (SONNR) method is proposed to rolling bearing compound fault diagnosis under noisy environment. Firstly, a theoretical model of rolling bearing compound fault is established, and the vibration characteristics of rolling bearing compound fault are analyzed. Secondly, four-layer structure of the SONNR method is proposed: input layer, nonconvex sparse regularization layer, signal reconstruction layer, and compound faults isolation layer. Finally, the validity of the method is verified by simulation data and actual data, and it is compared with the traditional time domain diagnostic methods and artificial intelligence methods.

Impact force identification via sparse regularization with generalized minimax-concave penalty

As a typical inverse problem, impact force identification tends to be a challenge owing to its ill-posedness. Recently, the sparse characteristic of the impact force in time domain are taken into consideration to convert the ill-posed problem into a sparse recovery task. The classic way to obtain sparse approximate solutions of impact force is to use the L1-norm regularization. However, underestimating the exact solution inevitably happens when applying the L1-norm regularization to impact force identification. In this paper, a novel sparse regularization method with generalized minimax-concave (GMC) penalty is proposed to deal with the impact force identification problem. Even if the GMC penalty turns out to be a nonconvex regularizer to promote sparsity in the sparse estimation, it retains the convexity of the sparsity-regularized least squares cost function and the global optimal solution can be found by convex optimization. Additionally, an approach to adaptively set regularization parameters is presented. Finally, simulations and experiments are conducted to verify the performances of the proposed method, and the results are compared with those of the L1-norm regularization method. Results demonstrate that the nonconvex sparse regularization based on GMC can produce more accurate estimations for impact force identification.

Compressed sensing with nonconvex sparse regularization and convex analysis for duct mode detection

Recently, research interests are increasing in mode detection methods based on compressed sensing, as it can reduce the number of sensors required by the classical Shannon-Nyquist sampling theory. This paper proposes a nonconvex sparse regularization method for azimuthal mode detection for aero engine fan noise. The nonconvex sparse regularization is based on the generalized minimax-concave (GMC) penalty, which can maintain the convexity of the sparse-regularized least squares cost function, and thus the global optimal solution can be solved by convex optimization algorithms. The main advantage of the GMC method over conventional compressed sensing method in mode detection is that the GMC method can better recover the mode amplitudes with a small number of sensors. Besides, the GMC method can suppress effectively the irrelated modes induced by the background noise or sensor installation errors. Therefore, the proposed method for duct mode detection can significantly improve the accuracy of the detected modes. Numerical simulations and experimental tests verify the effectiveness of the GMC method in mode detection for aero engine fan noise, and comparison studies show that the GMC method provides more accurate mode detection results than l1 minimization in the category of compressed sensing, as well as traditional mode detection methods.

Non-convex sparse regularization for impact force identification

Many convex regularization methods, such as the classical Tikhonov regularization based on l2-norm penalty and the standard sparse regularization method based on l1-norm penalty, have been widely investigated for impact force identification. However, in many practical applications, these regularization methods commonly underestimate the true solution. In this paper, we propose a non-convex sparse regularization method based on lp-norm (0 < p < 1) penalty, to seek the sufficiently sparse and highly accurate solution of impact force identification. Firstly, a non-convex optimization model based on lp-norm penalty instead of l2-norm penalty or l1-norm penalty is developed for regularizing inverse problems of impact force identification to overcome the mismatch between l0-norm and l1-norm regularizations. Secondly, an iteratively reweighed l1-norm algorithm is introduced to solve such a non-convex model through transforming it into a series of l1-norm regularizations. Finally, numerical simulation and experimental validation are carried out to evaluate the effectiveness and performance of lp-norm regularization when 0 = p ≤ 1. Our numerical results demonstrate that, compared to the state-of-the-art Tikhonov regularization and l1-norm regularization, the solution of lp-norm regularization achieves more stability, sparseness and accuracy when dealing with the heavily noisy response. Additionally, both numerical and experimental results demonstrate that the peak relative error of the identified impact force using lp-norm regularization has a decreasing tendency as p is approaching 0; while the identification of lp-norm regularization with 0 = p ≤ 1/2 bears no significant difference, which always outperforms that the identification with 1/2 = p ≤ 1.

Prognosis of Bearing Degeneration Using Adaptive Quaternion Least Mean Biquadrate Under Framework of Hypercomplex Data

Prognostics of bearing degeneration play a crucial role in implementation of systems maintenance strategies. Although numerous classical prognostic models have been developed, these models are carried out based on channel-wise processing and therefore fail to capture the inherent nonlinear and coupling nature of multi-dimensional or multi-channel data. To address this issue, a novel prognostics method based on adaptive quaternion-valued least-mean biquadrate (AQ-LMB) algorithm is proposed for a unified processing of hypercomplex data by the virtue of quaternion algebra. The cost function of the proposed AQ-LMB algorithm is designed via the biquadrate form of system output error to adapt to the more common nonlinear and non-Gaussian data, and the update weight vector is derived through Hamilton calculus instead of traditional complex gradient calculation. First, the time series of health indicators (e.g., root-mean-square, RMS) derived from historical data are decomposed by a nonconvex sparse regularization (SR) algorithm associated with a nonconvex penalty, that is, the low frequency trend component (LFC) and high frequency noise component (HFC) are obtained. Then both LFC and HFC are respectively predicted by the AQ-LMB algorithm. The final predicted health indicators can be obtained by integrating the correspondingly predicted LFC and HFC. The separate analysis of both sub-components makes it possible to distinguish their respective contributions to the entire degeneration process, thus avoiding false deviation and improving the prediction accuracy. Finally, the effectiveness of the proposed nonconvex SR and AQ-LMB approach in improving prognostic accuracy is illustrated via three-and four-dimensional run-to-failure datasets of the rolling bearings.