Split Augmented Lagrangian Shrinkage Algorithm Research Articles

Local regularization assisted split augmented Lagrangian shrinkage algorithm for feature selection in condition monitoring

Feature selection plays a vital role in improving the efficiency and accuracy of condition monitoring by constructing sparse but effective models. In this study, an advanced feature selection algorithm named the local regularization assisted split augmented Lagrangian shrinkage algorithm (LR-SALSA) is proposed. The feature selection is realized by solving a l1-norm optimization problem, which usually selects more sparse and representative features at less computational costs. The proposed algorithm operates in two stages, namely variable selection and coefficient estimation. In the stage of variable selection, the primal problem is converted into three subproblems which can be solved separately. Then individual penalty parameters are applied to every coefficient of the model when dealing with the first subproblem. Under the Bayesian evidence framework, an iterative algorithm is derived to optimize these hyperparameters. During the optimization process, redundant variables will be pruned to guarantee model sparsity and improve computational efficiency at the same time. In the second stage, the coefficients for the selected model terms are determined using the least squares technique. The superior performance and efficiency of the proposed LR-SALSA method are validated through two numerical examples and a real-world cutting tool wear prediction case study. Compared with the existing methods, the proposed method can generate a sparse model and ensure a good trade-off between estimation accuracy and computational efficiency.

SALSA-Net: Explainable Deep Unrolling Networks for Compressed Sensing.

Deep unrolling networks (DUNs) have emerged as a promising approach for solving compressed sensing (CS) problems due to their superior explainability, speed, and performance compared to classical deep network models. However, the CS performance in terms of efficiency and accuracy remains a principal challenge for approaching further improvements. In this paper, we propose a novel deep unrolling model, SALSA-Net, to solve the image CS problem. The network architecture of SALSA-Net is inspired by unrolling and truncating the split augmented Lagrangian shrinkage algorithm (SALSA) which is used to solve sparsity-induced CS reconstruction problems. SALSA-Net inherits the interpretability of the SALSA algorithm while incorporating the learning ability and fast reconstruction speed of deep neural networks. By converting the SALSA algorithm into a deep network structure, SALSA-Net consists of a gradient update module, a threshold denoising module, and an auxiliary update module. All parameters, including the shrinkage thresholds and gradient steps, are optimized through end-to-end learning and are subject to forward constraints to ensure faster convergence. Furthermore, we introduce learned sampling to replace traditional sampling methods so that the sampling matrix can better preserve the feature information of the original signal and improve sampling efficiency. Experimental results demonstrate that SALSA-Net achieves significant reconstruction performance compared to state-of-the-art methods while inheriting the advantages of explainable recovery and high speed from the DUNs paradigm.

OPK_SNCA: Optimized prior knowledge via sparse non-convex approach for cone-beam X-ray luminescence computed tomography imaging

BackgroundThe development of Cone-beam X-ray luminescence computed tomography (CB-XLCT) has allowed the quantitative in-depth biological imaging, but with a greatly ill-posed and ill-conditioned inverse problem. Although the predefined permissible source region (PSR) is a widely used way to alleviate the problem for CB-XLCT imaging, how to obtain the accurate PSR is still a challenge for the process of inverse reconstruction. MethodsWe proposed an optimized prior knowledge via a sparse non-convex approach (OPK_SNCA) for CB-XLCT imaging. Firstly, non-convex Lp-norm optimization model was employed for copying with the inverse problem, and an iteratively reweighted split augmented lagrangian shrinkage algorithm was developed to obtain a group of sparse solutions based on different non-convex p values. Secondly, a series of permissible regions (PRs) with different discretized mesh was further achieved, and the intersection operation was implemented on the group of PRs to get a reasonable PSR. After that, the final PSR was adopted as an optimized prior knowledge to enhance the reconstruction quality of inverse reconstruction. ResultsBoth simulation experiments and in vivo experiment were performed to evaluate the efficiency and robustness of the proposed method. ConclusionsThe experimental results demonstrated that our proposed method could significantly improve the imaging quality of the distribution of X-ray-excitable nanophosphors for CB-XLCT.

An Interference Mitigation Technique for Automotive Millimeter Wave Radars in the Tunable Q-Factor Wavelet Transform Domain

With the widespread use of millimeter-wave radar in advanced driver assistance systems (ADASs) and higher level autonomous driving, the probability of mutual interference increases. Radar echoes are distorted by the presence of mutual interferences, leading to the increase of the noise level and even producing ghost targets. Therefore, mitigating interference is one of the most important and pressing challenges for automotive radars today. This article presents an interference mitigation technique in the tunable <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -factor wavelet transform (TQWT) domain. It is discovered that the target’s signal exhibits sparsity in the high <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -factor wavelet transform. Conversely, the interference signal has better representation in the low <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -factor wavelet transform. Then, using morphological components analysis and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{1}$ </tex-math></inline-formula> -norm penalized least squares, one sparsity-based nonlinear signal separation model is further derived to reduce the interferences. Finally, the split augmented Lagrangian shrinkage algorithm is applied to optimize the proposed model. Experiments demonstrate that the performance of the proposed algorithm is comparable to three state-of-the-art methods.

Reconstruction of conductivity distribution with electrical impedance tomography based on hybrid regularization method.

Purpose: Physiological or pathological variation would cause a change of conductivity. Electrical impedance tomography (EIT) is favorable in reconstructing conductivity distribution inside the detected area. However, the reconstruction is an ill-posed inverse problem and the spatial resolution of the reconstructed image is relatively poor. Approach: To deal with the problem, a regularization method is commonly applied. Traditional regularization methods have their own disadvantages. In this work, we develop an innovative hybrid regularization method to determine the conductivity distribution from the boundary measurement. To address the unwanted artifact observed in the total variation (TV) method, the proposed approach incorporates the TV method with the non-convex sparse penalty term-based wavelet transform. In the reconstruction, the sensitivity matrix is also normalized to increase the sensitivity of the measurement to the variation of the conductivity. The objective function is minimized with the split augmented Lagrangian shrinkage algorithm. Results: The feasibility of the proposed method is evaluated by numerical simulation and phantom experiment. The results verify that the reconstruction with the proposed method is more advantageous, as obvious improvement is observed in the reconstructed image. Conclusions: With the proposed method, the artifact can be effectively suppressed and the reconstructed image of conductivity distribution is improved. It has great potential in medical imaging, which would be helpful for the accurate diagnosis of disease.

Adaptive Sparse Detector for Suppressing Powerline Component in EEG Measurements.

Powerline interference (PLI) is a major source of interference in the acquisition of electroencephalogram (EEG) signal. Digital notch filters (DNFs) have been widely used to remove the PLI such that actual features, which are weak in energy and strongly connected to brain states, can be extracted explicitly. However, DNFs are mathematically implemented via discrete Fourier analysis, the problem of overlapping between spectral counterparts of PLI and those of EEG features is inevitable. In spite of their effectiveness, DNFs usually cause distortions on the extracted EEG features, which may lead to incorrect diagnostic results. To address this problem, we investigate an adaptive sparse detector for reducing PLI. This novel approach is proposed based on sparse representation inspired by self-adaptive machine learning. In the coding phase, an overcomplete dictionary, which consists of redundant harmonic waves with equally spaced frequencies, is employed to represent the corrupted EEG signal. A strategy based on the split augmented Lagrangian shrinkage algorithm is employed to optimize the associated representation coefficients. It is verified that spectral components related to PLI are compressed into a narrow area in the frequency domain, thus reducing overlapping with features of interest. In the decoding phase, eliminating of coefficients within the narrow band area can remove the PLI from the reconstructed signal. The sparsity of the signal in the dictionary domain is determined by the redundancy factor. A selection criteria of the redundancy factor is suggested via numerical simulations. Experiments have shown the proposed approach can ensure less distortions on actual EEG features.

Nonnegative Blind Source Separation for Ill-Conditioned Mixtures via John Ellipsoid.

Nonnegative blind source separation (nBSS) is often a challenging inverse problem, namely, when the mixing system is ill-conditioned. In this work, we focus on an important nBSS instance, known as hyperspectral unmixing (HU) in remote sensing. HU is a matrix factorization problem aimed at factoring the so-called endmember matrix, holding the material hyperspectral signatures, and the abundance matrix, holding the material fractions at each image pixel. The hyperspectral signatures are usually highly correlated, leading to a fast decay of the singular values (and, hence, high condition number) of the endmember matrix, so HU often introduces an ill-conditioned nBSS scenario. We introduce a new theoretical framework to attack such tough scenarios via the John ellipsoid (JE) in functional analysis. The idea is to identify the maximum volume ellipsoid inscribed in the data convex hull, followed by affinely mapping such ellipsoid into a Euclidean ball. By applying the same affine mapping to the data mixtures, we prove that the endmember matrix associated with the mapped data has condition number 1, the lowest possible, and that these (preconditioned) endmembers form a regular simplex. Exploiting this regular structure, we design a novel nBSS criterion with a provable identifiability guarantee and devise an algorithm to realize the criterion. Moreover, for the first time, the optimization problem for computing JE is exactly solved for a large-scale instance; our solver employs a split augmented Lagrangian shrinkage algorithm with all proximal operators solved by closed-form solutions. The competitiveness of the proposed method is illustrated by numerical simulations and real data experiments.

Sparse-based defect detection of weld feature guided waves with a fusion of shear wave characteristics

Shear wave is suitable for long weld defect detection in offshore platforms with its advantages of long-distance propagation, small attenuation, and high accuracy. Owing to dispersion, multi-mode, and strong background noises, the defect signal can generally be overwhelmed. To solve this problem, a new sparse-based defect detection method is proposed for weld feature guided waves with a fusion of shear wave characteristics. First, an over-complete dictionary is established combining wavenumber and scattering characteristics of shear wave. Then, the split augmented Lagrangian shrinkage algorithm is introduced in the basis pursuit algorithm for the sparse solution. The defect features of the signal can be sparsely extracted. The effectiveness is evaluated via simulation studies and further verified through the practical experiment data. Compared with the wavelet atom method, the location error of the proposed method reduces by about 1.0% to only 0.564% in simulation and about 2.0% to only 0.671% in practical experiment.

Dictionary adaptation and variational mode decomposition for gyroscope signal enhancement

The paper proposes an approach to signal denoising based on a combination of Variational Mode Decomposition with the Split Augmented Lagrangian Shrinkage Algorithm.In our research, we found that the proposed approach gives a great improvement of denoising gyroscopic signals. In turn, the results for the synthetic signals are not straightforward. For the bumps synthetic signals, the proposed algorithm gives the best results for different levels of signal degradation. While for the Doppler and blocks synthetic signals the reference methods give better results. However, for heavisine test signal the proposed algorithm gives better results in almost all cases.A weak point of the presented algorithm is its time complexity. The proposed approach is based on the Split Augmented Lagrangian Shrinkage Algorithm, which is the iterative optimization method since the time of computation strongly depends on the number of iterations.The presented results show that the proposed approach gives a great improvement in signal denoising and it is a promising direction of future research.

Feature Extraction of Echo Signal of Weld Defect Guided Waves Based on Sparse Representation

In order to accurately detect the location of weld defect from strong background noises, a new method incorporating Morlet wavelet basis atom into sparse representation theory is investigated and it is suitable for detecting the weld defect feature from the guided wave echo signal. After determining the matching basis function, the split augmented Lagrangian shrinkage algorithm is introduced to solve the basis pursuit denoising problem. The echo signal can be intuitively represented in the sparse coefficients, then the defect signal can be effectively extracted from strong background noises. Both the simulated studies and the real applications of experimental signals show that this method is precise and stable, and can effectively suppress the influence of noises.

Oscillatory-Plus-Transient Signal Decomposition Using TQWT and MCA

This paper describes a method for decomposing a signal into the sum of an oscillatory component and a transient component. The process uses the tunable Q-factor wavelet transform (TQWT): The oscillatory component is modeled as a signal that can be sparsely denoted by high Q-factor TQWT; similarly, the transient component is modeled as a piecewise smooth signal that can be sparsely denoted using low Q-factor TQWT. Since the low and high Q-factor TQWT has low coherence, the morphological component analysis (MCA) can effectively decompose the signal into oscillatory and transient components. The corresponding optimization problem of MCA is resolved by the split augmented Lagrangian shrinkage algorithm (SALSA). The applications of the proposed method to speech, electroencephalo-graph (EEG), and electrocardiograph (ECG) signals are included.

Transient extraction based on minimax concave regularized sparse representation for gear fault diagnosis

Extraction of fault-induced transients is one of the difficulties in gear fault diagnosis as the vibration signal is composed of multiple components. Sparse representation is one of the effective methods for extracting transients from such noisy and multicomponent vibration signals. However, conventional sparse representation approaches have the problems of amplitude underestimation, computational inefficiency and mono-component orientation. To address such problems, this paper first proposes to use minimax concave penalty function to construct the objective function as the minimax concave penalty function outperforms other penalty functions in preserving high-amplitude components based on the comparisons among different non-convex penalty functions. Then, the multiple over-complete dictionaries are established based on Fourier bases, where every two elements of each dictionary form a tight frame, enabling to simplify calculation and avoid computing high-dimensional inverse matrix when performing iterative optimization. Since multiple dictionaries are constructed, the multi-components of gear vibration signal can be extracted, respectively. With the constructed dictionaries, the sparse representation coefficients can be calculated using split augmented Lagrangian shrinkage algorithm and the repetitive transients can then be extracted. The simulation study and experimental analysis show that the fault transients and harmonic components can be accurately as well as computational-effectively extracted without underestimation of high amplitude components. Comparison results between minimax concave, arctan and L1 norm also prove the superior high-amplitude preserving ability of the proposed method.

Separation of Multicomponent Chirp Signals Using Morphological Component Analysis and Fractional Fourier Transform

A chirp signal is a large-bandwidth signal which is widely used in engineering. In many applications, it is necessary to decompose a mixed chirp signal into its components. However, the traditional Fourier transform method cannot process a mixed chirp signal when its components intersect in the joint time–frequency domain. Combining the advantages of the morphological component analysis (MCA) with multicomponent signal processing and the fractional Fourier transform (FrFT) in chirp signal processing, this letter proposes the MCA-FrFT method to decompose a multicomponent chirp signal. First, the cost function is defined using the FrFT and optimized by the split augmented Lagrangian shrinkage algorithm (SALSA), and then, all the morphological components are obtained. The proposed method is verified by simulations, and simulation results show that the proposed method has good performance in separating the multicomponent chirp signals into components. Besides, the proposed method is evaluated experimentally in the sea target detection, and the experimental results confirm that the proposed method can not only extract the low observable targets from a heavy sea-clutter environment but also separate them from each other.

LSALSA: accelerated source separation via learned sparse coding

We propose an efficient algorithm for the generalized sparse coding (SC) inference problem. The proposed framework applies to both the single dictionary setting, where each data point is represented as a sparse combination of the columns of one dictionary matrix, as well as the multiple dictionary setting as given in morphological component analysis (MCA), where the goal is to separate a signal into additive parts such that each part has distinct sparse representation within a corresponding dictionary. Both the SC task and its generalization via MCA have been cast as $\ell_1$-regularized least-squares optimization problems. To accelerate traditional acquisition of sparse codes, we propose a deep learning architecture that constitutes a trainable time-unfolded version of the Split Augmented Lagrangian Shrinkage Algorithm (SALSA), a special case of the Alternating Direction Method of Multipliers (ADMM). We empirically validate both variants of the algorithm, that we refer to as LSALSA (learned-SALSA), on image vision tasks and demonstrate that at inference our networks achieve vast improvements in terms of the running time, the quality of estimated sparse codes, and visual clarity on both classic SC and MCA problems. Finally, we present a theoretical framework for analyzing LSALSA network: we show that the proposed approach exactly implements a truncated ADMM applied to a new, learned cost function with curvature modified by one of the learned parameterized matrices. We extend a very recent Stochastic Alternating Optimization analysis framework to show that a gradient descent step along this learned loss landscape is equivalent to a modified gradient descent step along the original loss landscape. In this framework, the acceleration achieved by LSALSA could potentially be explained by the network's ability to learn a correction to the gradient direction of steeper descent.

Sparse augmented Lagrangian algorithm for system identification

A huge class of nonlinear dynamic systems can be approximated by the Nonlinear AutoRegressive with eXogenous inputs (NARX) models. This paper proposes a novel method, Sparse Augmented Lagrangian (SAL), for NARX model variable selection and parameter estimation. Firstly, Split Augmented Lagrangian Shrinkage Algorithm (SALSA) is applied to produce some intermediate models with subsampling technique, and then only the model terms with high selecting probability are chosen into the final model, followed by the model parameter estimation via SALSA. The model sparsity and algorithm convergence can be guaranteed through theoretical analysis. Two nonlinear examples and one real-world application from the process industry are used to demonstrate the effectiveness and advantages of the proposed method in comparison to several popular methods.

FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors.

Rain streaks removal is an important issue in outdoor vision systems and has recently been investigated extensively. In this paper, we propose a novel video rain streak removal approach FastDeRain, which fully considers the discriminative characteristics of rain streaks and the clean video in the gradient domain. Specifically, on the one hand, rain streaks are sparse and smooth along the direction of the raindrops, whereas on the other hand, clean videos exhibit piecewise smoothness along the rain-perpendicular direction and continuity along the temporal direction. Theses smoothness and continuity results in the sparse distribution in the different directional gradient domain, respectively. Thus, we minimize 1) the ℓ1 norm to enhance the sparsity of the underlying rain streaks, 2) two ℓ1 norm of unidirectional Total Variation (TV) regularizers to guarantee the anisotropic spatial smoothness, and 3) an ℓ1 norm of the time-directional difference operator to characterize the temporal continuity. A split augmented Lagrangian shrinkage algorithm (SALSA) based algorithm is designed to solve the proposed minimization model. Experiments conducted on synthetic and real data demonstrate the effectiveness and efficiency of the proposed method. According to comprehensive quantitative performance measures, our approach outperforms other state-of-the-art methods, especially on account of the running time. The code of FastDeRain can be downloaded at https://github.com/TaiXiangJiang/FastDeRain.

Sparse non-convex Lp regularization for cone-beam X-ray luminescence computed tomography

ABSTRACTCone-beam X-ray luminescence computed tomography (CB-XLCT) is an attractive hybrid imaging modality, and it has the potential of monitoring the metabolic processes of nanophosphors-based drugs in vivo. However, the XLCT imaging suffers from a severe ill-posed problem. In this work, a sparse nonconvex Lp (0 < p < 1) regularization was utilized for the efficient reconstruction for early detection of small tumour in CB-XLCT imaging. Specifically, we transformed the non-convex optimization problem into an iteratively reweighted scheme based on the L1 regularization. Further, an iteratively reweighted split augmented lagrangian shrinkage algorithm (IRW_SALSA-Lp) was proposed to efficiently solve the non-convex Lp (0 < p < 1) model. We studied eight different non-convex p-values (1/16, 1/8, 1/4, 3/8, 1/2, 5/8, 3/4, 7/8) in both 3D digital mouse experiments and in vivo experiments. The results demonstrate that the proposed non-convex methods outperform L2 and L1 regularization in accurately recovering sparse targets in CB-XLCT. And among all the non-convex p-values, our Lp(1/4 < p < 1/2) methods give the best performance.

Double-dictionary signal decomposition method based on split augmented Lagrangian shrinkage algorithm and its application in gearbox hybrid faults diagnosis

Gearbox with hybrid distributed and localized faults usually generates coupled modulation vibration signal. It is hard to decompose the coupled signal for precise diagnosis. To solve this problem, a novel signal decomposition method is proposed on the basis of double-dictionary and split augmented Lagrangian shrinkage algorithm (SALSA). The dictionary possessing high similarity to fault features consists of steady modulation sub-dictionary and impact modulation sub-dictionary. The SALSA is improved by adding a hard threshold de-noising to obtain optimal sparse coefficients of steady modulation and impact modulation components. Key parameters including Lagrange multipliers (λs, λp), penalty factor μ of SALSA and hard threshold ε are studied to determine their optimal value ranges. Kinds of simulation signals show the effectiveness of the proposed method, and experimental tests on fixed-shaft gearbox and planetary gearbox further verify the reliability. Comparative analyses with methods respectively based on matching pursuit and tunable Q-factor wavelet transform indicate that the proposed method is superior to the other two methods in calculation efficiency and anti-noise performance, especially when these two kinds of modulation components are completely coupled in some resonance bands.

Sparsity-based signal extraction using dual Q-factors for gearbox fault detection

Early detection of faults developed in gearboxes is of great importance to prevent catastrophic accidents. In this paper, a sparsity-based feature extraction method using the tunable Q-factor wavelet transform with dual Q-factors is proposed for gearbox fault detection. Specifically, the proposed method addresses the problem of simultaneously extracting periodic transients and high-resonance component from noisy data for the gearboxes fault detection purpose. Firstly, a sparse optimization problem is formulated to jointly estimate the useful components from the noisy observation. In order to promote wavelet sparsity, non-convex regularizations are employed in the cost function of the optimization problem. Then, a fast converging, computationally efficient iterative algorithm which termed SpaEdualQA (the sparsity-based signal extraction algorithm using dual Q-factors) is developed to solve the formulated optimization problem. The derivation of the proposed fast algorithm combines the split augmented Lagrangian shrinkage algorithm (SALSA) with majorization–minimization (MM). Finally, the effectiveness of the proposed SpaEdualQA is validated by analyzing numerical signals and real data collected from engineering fields. The results demonstrated that the proposed SpaEdualQA can effectively extract periodic transients and high-resonance component from noisy vibration signals.

Convex Formulation for Multiband Image Classification With Superpixel-Based Spatial Regularization

Superpixels are a powerful device to characterize the spatial–contextual information in remotely sensed hyperspectral image (HSI) interpretation. However, the exploitation of superpixels in classification problems is not straightforward, often leading to unbearable NP-hard discrete integer optimization problems. In this paper, we attack this hurdle by leveraging on a convex relaxation of the original integer optimization problem, which opens the door to include oversegmented superpixel-based regularizers. Specifically, we develop a new method for generating oversegmented superpixels. Then, we introduce a family of convex regularizers in the form of graph total variation, which promotes the same labeling in each superpixel. Vectorial total variation is also included in order to promote piecewise smoothness and align discontinuities along the class boundaries. The solution of the obtained convex optimization problem is computed with the split-augmented Lagrangian shrinkage algorithm. Experiments on HSIs yield classification maps with precise boundaries and inner consistency inside oversegmented superpixels, leading to the state-of-the-art classification accuracies.