Hard Shrinkage Research Articles

Smooth hard shrinkage operator for tensor completion based on self-adaptive transforms

The tensor completion(TC) problem has attracted wide attention due to its significant practical significance and broad application background. Various approaches have been used to solve this problem, including approximating the rank function through its convex envelope, the tensor nuclear norm. However, due to the gap between the rank function and its convex envelope, this approach often fails to achieve satisfactory recovery. Recent developments have highlighted that non-explicit non-convex functions, especially the one induced by the Smooth Hard(SH) shrinkage operator, can approximate the l0 norm more effectively compared to other methods, and this function facilitates the reconstruction of noisy MRI data with minimal sampling rate requirements. In our paper, we implement the SH shrinkage operator within the TC framework, achieving accurate tensor data recovery at lower sampling rates. Moreover, we adopt a method that adaptively generates transformation matrices based on specific features of the data, which, compared to traditional TC models, can better exploit the inherent low-rank properties of tensor data. Additionally, our approach integrates a reweighting strategy to efficiently merge tensors recovered from various dimensions. Finally, a feasible minimization algorithm is proposed, which combines the non-explicit non-convex regularizer and self-adaptive transforms. It is proven that the constructed sequence of the proposed algorithm is a Cauchy sequence and can converge to a stable point. Extensive experimental results demonstrate that our proposed approach can achieve accurate tensor recovery at lower sampling rates compared to other methods.

Patch-based weighted SCAD prior for compressive sensing

The nuclear norm-based convex surrogate of the rank function has been widely used in compressive sensing (CS) to exploit the sparsity of nonlocal similar patches in an image. However, this method treats different singular values equally and thus may produce a result far from the optimum one. In order to alleviate the limitations of the nuclear norm, different singular values should be treated differently. The reason is that large singular values can be used to retrieve substantial contents of an image, while small ones may contain noisy information. In this paper, we propose a model via non-convex weighted Smoothly Clipped Absolute Deviation (SCAD) prior. Our motivation is that SCAD shrinkage behaves like a soft shrinkage operator for small enough inputs, whereas for large enough ones, it leaves the input intact and behaves like hard shrinkage. For moderate input values, SCAD makes a good balance between soft shrinkage and hard shrinkage. Numerically, the alternating direction method of multiplier (ADMM) is adopted to split the original problem into several sub-problems with closed-form solutions. We further analyze the convergence of the proposed method under mild conditions. Various experimental results demonstrate that the proposed model outperforms many existing state-of-the-art CS methods.

REMOVAL OF ELECTRODE MOTION ARTIFACT IN ECG SIGNALS USING WAVELET BASED THRESHOLD METHODS WITH GREY INCIDENCE DEGREE

Cardiovascular diseases are one of the most frequent and dangerous problems in modern society in nowadays. Unfortunately electrocardiograms (ECG) signals, during their acquisition process, are affected by various types of noise and artifacts due to the movement, or breathing of the patient, electrode contact, power-line interferences, etc. The aim of this study was to develop an algorithm to remove electrode motion artifact in ECG signals. Donoho and Johnstone proposed Wavelet thresholding de-noising method based on discrete wavelet transform (DWT) is suitable for non-stationary signals. The wavelet transform coefficient is processed by using grey relation analysis of the grey theory, and a new wavelet threshold method namely wavelet threshold method with grey incidence degree (GID) (or the GID threshold method) based is introduced. It shows that the signal smoothness and similarity of the two signal criteria have been greatly improved by the GID threshold method compared with existing threshold methods. According to the characteristics of different ECG signals, GID threshold method gets better results than it can adaptively deal with noise separation and details remaining of the two opposing signal problems, so as to provide a better choice for wavelet threshold methods of signal processing. Performance analysis was performed by evaluating Mean Square Error (MSE), Signal-to-noise ratio (SNR) and visual inspection over the denoised signal from each algorithm. The experimental result shows that GID hard shrinkage method with sub-band or level dependent thresholding gives the best denoising performance on ECG signal. The result shows that soft threshold not always gives better denoising performance; it depends on which wavelet thresholding algorithm was chosen.

Iterated Hard Shrinkage for Minimization Problems with Sparsity Constraints

A new iterative algorithm for the solution of minimization problems in infinite-dimensional Hilbert spaces which involve sparsity constraints in form of $\ell^{p}$-penalties is proposed. In contrast to the well-known algorithm considered by Daubechies, Defrise, and De Mol, it uses hard instead of soft shrinkage. It is shown that the hard shrinkage algorithm is a special case of the generalized conditional gradient method. Convergence properties of the generalized conditional gradient method with quadratic discrepancy term are analyzed. This leads to strong convergence of the iterates with convergence rates $\mathcal{O}(n^{-1/2})$ and $\mathcal{O}(\lambda^n)$ for $p=1$ and $1 < p \leq 2$, respectively. Numerical experiments on image deblurring, backwards heat conduction, and inverse integration are given.

Iterative Regularization and Nonlinear Inverse Scale Space Applied to Wavelet-Based Denoising

In this paper, we generalize the iterative regularization method and the inverse scale space method, recently developed for total-variation (TV) based image restoration, to wavelet-based image restoration. This continues our earlier joint work with others where we applied these techniques to variational-based image restoration, obtaining significant improvement over the Rudin-Osher-Fatemi TV-based restoration. Here, we apply these techniques to soft shrinkage and obtain the somewhat surprising result that a) the iterative procedure applied to soft shrinkage gives firm shrinkage and converges to hard shrinkage and b) that these procedures enhance the noise-removal capability both theoretically, in the sense of generalized Bregman distance, and for some examples, experimentally, in terms of the signal-to-noise ratio, leaving less signal in the residual.

Integration of PET compartmental models and wavelet analysis techniques: A feasibility study

Wavelet analysis has become routinely used in neuroimaging, especially in PET image analysis. Techniques have been developed which combine spatial wavelet transforms and linear temporal models 1, 2. However, in PET, often the temporal model can be well described using a compartmental model 3. Compartmental models are non-linear models and as such some of the assumptions made when wavelet analysis is used are no longer justified. The advantage of wavelet analysis is that spatially correlated data can be analyzed easily and the resulting estimates will have better mean squared error properties. Here, we investigate the feasibility of combining PET compartmental models with wavelet spatial models, in order to integrate the advantages of both underlying techniques. We have investigated using the Simplified Reference Tissue Model (SRTM) 4 both in the image domain and in the wavelet domain, where the data had been transformed using Battle-Lemarie wavelets. A simulation based on several blocks in a small artificial image was considered, where each block had a time activity curve (TAC) associated with it. These TACs were generated from the underlying true model using parameter values based on real data. In addition a simulated PET image was generated using a PET simulator and analyzed. In the wavelet domain, in both studies, the data was analyzed firstly without any shrinkage being performed, and subsequently using three different shrinkage operators; hard shrinkage, soft shrinkage and linear shrinkage. The variances used in the shrinkage were estimated in two different ways, block by block using the parameter values themselves, and point-by-point using the time series data 5. The results of our simulation studies suggest that it is indeed possible to combine the two techniques of PET compartmental modeling and spatial wavelet analysis. The small block simulation showed that in the presence of spatially correlated noise, the wavelet analysis results have better power signal to noise ratio (pSNR) properties than the corresponding image domain results. While there was a small improvement in the overall image, a greater improvement was seen in the regions containing true signal (as opposed to the background). It was also found that linear shrinkage with point-by-point variance estimation gave the greatest improvement in pSNR. As can be seen in the figure 1, the result of the wavelet shrinkage analysis of the PET simulator data is much smoother than the image domain result, and does not contain spurious pixels of large intensity. This work demonstrates, through simulation, that the integration of PET compartmental models and wavelet analysis techniques is feasible, certainly for the SRTM. It would be of interest to examine other compartmental models, and of course, apply this method to real data sets.

Wavelet Shrinkage Denoising Using the Non-Negative Garrote

In this article, we combine Donoho and Johnstone's wavelet shrinkage denoising technique (known as WaveShrink) with Breiman's non-negative garrote. We show that the non-negative garrote shrinkage estimate enjoys the same asymptotic convergence rate as the hard and the soft shrinkage estimates. Simulations are used to demonstrate that garrote shrinkage offers advantages over both hard shrinkage (generally smaller mean-square-error and less sensitivity to small perturbations in the data) and soft shrinkage (generally smaller bias and overall mean-square-error). The minimax thresholds for the non-negative garrote are derived and the threshold selection procedure based on Stein's unbiased risk estimate (SURE) is studied. We also propose a threshold selection procedure based on combining Coifman and Donoho's cycle-spinning and SURE. The procedure is called SPINSURE. We use examples to show that SPINSURE is more stable than SURE: smaller standard deviation and smaller range.

Understanding WaveShrink: variance and bias estimation

Donoho & Johnstone's WaveShrink procedure has proved valuable for function estimation and nonparametric regression. WaveShrink is based on the principle of shrinking wavelet coefficients towards zero to remove noise. WaveShrink has very broad asymptotic near-optimality properties and achieves the optimal risk to within a factor of log n. In this paper, we derive computationally efficient formulae for computing the exact bias, variance and L2 risk of WaveShrink estimates in finite sample situations. We use these formulae to understand the behaviour of WaveShrink estimators, construct approximate confidence intervals and bias estimates for WaveShrink and compute ideal thresholds for a given function. We show that hard shrinkage has smaller bias but larger variance than soft shrinkage, and that significantly smaller thresholds should be used for soft shrinkage. We also compute minimax thresholds for WaveShrink estimators and demonstrate that the minimax thresholds can nearly achieve the ideal rank for a range of functions.