Stein's Unbiased Risk Estimate Research Articles

A sharp boundary for SURE-based admissibility for the normal means problem under unknown scale

We consider quasi-admissibility/inadmissibility of Stein-type shrinkage estimators of the mean of a multivariate normal distribution with covariance matrix an unknown multiple of the identity. Quasi-admissibility/inadmissibility is defined in terms of non-existence/existence of a solution to a differential inequality based on Stein’s unbiased risk estimate (SURE). We find a sharp boundary between quasi-admissible and quasi-inadmissible estimators related to the optimal James–Stein estimator. We also find a class of priors related to the Strawderman class in the known variance case where the boundary between quasi-admissibility and quasi-inadmissibility corresponds to the boundary between admissibility and inadmissibility in the known variance case. Additionally, we also briefly consider generalization to the case of general spherically symmetric distributions with a residual vector.

Impulse interference processing for MT data based on a new adaptive wavelet threshold de-noising method

Wavelet de-noising method is often used in the processing of magnetotelluric (MT) signal, and the wavelet hard and soft threshold de-noising method are the most popular although there is much room for improvement in the threshold function selection and threshold determination. A new adaptive wavelet threshold de-noising method was proposed by selecting a new threshold function and presenting an adaptive method for obtaining optimal threshold based on the multi-resolution Stein unbiased risk estimation. New threshold function and an adaptive method to determine threshold were discussed, and the principle and implementation of the algorithm were given. The simulated signal and the measured MT data contaminated by impulse interference were analyzed, and the obtained results were compared with those of the conventional wavelet hard and soft threshold de-noising methods. The results show that the proposed method overcomes the defects of the traditional wavelet soft and hard threshold due to a new threshold function, and a new method to determine the threshold of each layer is applied and provides an adaptive method for filtering MT data in the wavelet domain that requires a minimum of human intervention. The presented de-noising method is very suitable for suppressing the impulse interference for MT data and can get higher signal-to-noise ratio than the traditional wavelet threshold de-noising methods. After de-noising, the accurate data is loaded for further impedance estimation and geological interpretation.

The complex data denoising in MR images based on the directional extension for the undecimated wavelet transform

Abstract Magnetic resonance (MR) images are commonly affected by noises. Denoising is an important issue that has been frequently discussed in recent years. In this paper, an interesting phenomenon is found that the directional information is abundant in MR images. Therefore, high-quality reconstructed MR images could be obtained if the related directional information is considered. To address the issue, the directional extension for the undecimated wavelet transform (DEUWT), an effective tool that is able to handle the directional information and provides the translation-invariant (TI) property as well, is employed to process MR images. Based on the DEUWT, we present a novel and fast wavelet domain complex data denoising algorithm for MR images. In the presented algorithm, we combine the DEUWT with the stein’s unbiased risk estimator (SURE) thresholding, and treat the real and imaginary components of the MR image as a single complex entity. The experimental results show that the proposed algorithm outperforms existing state-of-the-art methods on both simulated complex images and complex phantoms.

SURE estimates under dependence and heteroscedasticity

The multivariate Bayesian hierarchical model with independent means has been studied extensively and is widely used in practice. In contrast, the case of dependent means has received scant attention, even though multivariate observations are often correlated. In this paper, we investigate a multivariate heteroscedastic Bayesian hierarchical model in which an informative prior with equicorrelated means is assumed. We estimate the mean vector by the shrinkage estimator based on Stein’s unbiased risk estimation (SURE). It is shown that the squared error loss of the SURE estimator is close to that of an oracle estimator as the number of means grows. Our SURE estimator includes the SURE estimator under independence considered by Xie et al. (2012) as a special case. The finite-sample performance of our estimator is explored via simulations and two real data sets are used for illustration purposes.

Denoising portal images by minimizing the SURE estimator on a parameterized family of shrinkage functions

The number of verification portal images in radiotherapy has increased in the last years. On the other hand, radiation delivered during imaging is not confined to the treatment volumes, but also affects the surrounding organs and tissues. In order to reduce the overall radiation dose due to imaging, one approach would be to reduce the dose per image, but noise would increase and the quality of portal images would reduce. The limited quality of portal images makes it difficult to propose a reduction of dose if there is no way to effectively reduce noise. Denoising algorithms could be the solution if the quality of the restored image can match the image obtained with a standard dose. In this work the statistical properties of noise in a portal imaging system and the statistical properties of portal images are used to develop an efficient denoising method. The result is a method that minimizes the Stein’s unbiased risk estimator (SURE) in the image domain over a parametric family of shrinkage functions operating in the wavelet domain. The presented denoising method shows a better performance than the adaptive Wiener estimator for different portal images and noise energies.

A passive forensic method for video: Exposing dynamic object removal and frame duplication in the digital video using sensor noise features

The purpose of this paper is to detect tampering in the video using passive forensic method. With the extensive availability of sophisticated video editing software and tools, it has become easy to forge the video using different tampering techniques. A well-known video tampering technique is to du plicate the frames or manipulate the contents of the frames to remove and hide the objects or person in a video. Video forensic is necessary because people tampered the video to get justice from court of law on the basis of video evidence, disgrace the important celebrity and to hide or expose some unwanted object. In this paper, we propose a passive forensic method to expose dynamic object removal and frames duplication by detecting the noise variance between original and tampered video frames. The proposed method exploits sensor noise features, extracted from each frame of the video using Discrete Wavelet Transform (DWT) and nonlinear thresholding such as Hard and Soft with Stein’s Unbiased Risk Estimator (SURE) shrinkage. Gaussian Mixture Density (GMD) is used as Bayesian classifier and Expectation-Maximization (EM) algorithm set the parameters of the GMD. Implementation results demonstrate that we are getting commendable processing speed, accuracy 97.36-96.26%, recall 99.80-98.94% and precision 97.34-87.95% respectively for object removal and frame duplication detection in comparison to existing methods. Proposed method successfully detect and localize forgery in the given video.

Automated detection of focal EEG signals using features extracted from flexible analytic wavelet transform

Epilepsy is a neurological disease which is difficult to diagnose accurately. An authentic detection of focal epilepsy will help the clinicians to provide proper treatment for the patients. Generally, focal electroencephalogram (EEG) signals are used to diagnose the epilepsy. In this paper, we have developed an automated system for the detection of focal EEG signals using differencing and flexible analytic wavelet transform (FAWT) methods. The differenced EEG signals are subjected to 15 levels of FAWT. Various entropies namely cross correntropy, Stein’s unbiased risk estimate (SURE) entropy, and log energy entropy are extracted from the reconstructed original signal and 16 sub-band signals. The statistically significant features are obtained from Kruskal–Wallis test based on (p < 0.05). K-nearest neighbor (KNN) and least squares support vector machine (LS-SVM) classifiers with different distances and kernels respectively are used for automated diagnosis. In the proposed methodology, we have achieved classification accuracy of 94.41% in detecting focal EEG signals using LS-SVM classifier with ten-fold cross validation strategy.

An Approach for External Preference Mapping Improvement by Denoising Consumer Rating Data

In this study, denoising data was advocated in sensory analysis field to remove the existing noise in consumer rating data before processing to External Preference Mapping (EPM). This technique is a data visualization used to understand consumers sensory profiles by relating their preferences towards products to external information about sensory characteristics of the perceived products. The output is a perceptual map which visualizes the optimal products and aspects that maximize consumers likings. Hence, EPM is considered as a decision tool to support the development or improvement of products and respond to market requirements. In fact, the stability of the map is affected by the high variability of judgments that make consumer rating data very noisy. This may lead to mismatch between products features and consumers’ preferences then distorted results and wrong decisions. To remove the existing noise, the use of some filtering methods is proposed. Regularized Principal Component Analysis (RPCA) and Stein’s Unbiased Risk Estimate (SURE), based respectively on hard and soft thresholding rules, were applied to consumer rating data to separate the signal to noise and maintain only useful information about the given liking scores. As a way to compare the EPM obtained from each strategy, a sampling process was conducted to randomly select samples from noisy and cleaned data, then perform their corresponding EPM. The stability of the obtained maps was evaluated using an indicator that computes and compares distances between them before and after denoising. The effectiveness of this methodology was evaluated by a simulation study and a potential application was shown on real dataset. Results show that recorded distances after denoising are lower than those before in almost cases for both RPCA and SURE. However, RPCA outperforms SURE. The corresponding map is made more stable where level lines are seen smoothed and products are better located on liking zones. Hence, noise removal reduces variability in data and brings closer preferences which improves the quality of the visualized map.

Real Oriented Dual Tree Wavelet Transform with an Optimal Threshold using Neighbor Coefficients and Gaussian Bilateral Filter for Image Denoising

A novel method for image denoising using real oriented 2-D Dual Tree Wavelet Transform (DTWT) is proposed. In this approach image is contaminated by white Gaussian noise. The optimal threshold which is data driven is determined using the Neigh Sure that uses Stein's Unbiased Risk Estimator (SURE) with minimum risk for the whole noisy image. Real oriented 2-D DTWT of a noisy image results in six sub-bands that is non-separable. The transformed image is soft thresholded using the optimal threshold value. Further the denoising performance is improved by using the Gaussian bilateral filter. The experimental result shows that the proposed method is better in terms of Peak Signal to Noise Ratio (PSNR) than other existing techniques.

Adaptive higher-order spectral estimators

Many applications involve estimation of a signal matrix from a noisy data matrix. In such cases, it has been observed that estimators that shrink or truncate the singular values of the data matrix perform well when the signal matrix has approximately low rank. In this article, we generalize this approach to the estimation of a tensor of parameters from noisy tensor data. We develop new classes of estimators that shrink or threshold the mode-specific singular values from the higher-order singular value decomposition. These classes of estimators are indexed by tuning parameters, which we adaptively choose from the data by minimizing Stein's unbiased risk estimate. In particular, this procedure provides a way to estimate the multilinear rank of the underlying signal tensor. Using simulation studies under a variety of conditions, we show that our estimators perform well when the mean tensor has approximately low multilinear rank, and perform competitively when the signal tensor does not have approximately low multilinear rank. We illustrate the use of these methods in an application to multivariate relational data.

Estimation of Kullback-Leibler losses for noisy recovery problems within the exponential family

We address the question of estimating Kullback-Leibler losses rather than squared losses in recovery problems where the noise is distributed within the exponential family. Inspired by Stein unbiased risk estimator (SURE), we exhibit conditions under which these losses can be unbiasedly estimated or estimated with a controlled bias. Simulations on parameter selection problems in applications to image denoising and variable selection with Gamma and Poisson noises illustrate the interest of Kullback-Leibler losses and the proposed estimators.

Enhancement and Edge-Preserving Denoising: An OpenCL-Based Approach for Remote Sensing Imagery

Image enhancement and edge-preserving denoising are relevant steps before classification or other postprocessing techniques for remote sensing images. However, multisensor array systems are able to simultaneously capture several low-resolution images from the same area on different wavelengths, forming a high spatial/spectral resolution image and raising a series of new challenges. In this paper, an open computing language based parallel implementation approach is presented for near real-time enhancement based on Bayesian maximum entropy (BME), as well as an edge-preserving denoising algorithm for remote sensing imagery, which uses the local linear Stein's unbiased risk estimate (LLSURE). BME was selected for its results on synthetic aperture radar image enhancement, whereas LLSURE has shown better noise removal properties than other commonly used methods. Within this context, image processing methods are algorithmically adapted via parallel computing techniques and efficiently implemented using CPUs and commodity graphics processing units (GPUs). Experimental results demonstrate the reduction of computational load of real-world image processing for near real-time GPU adapted implementation.

Online filtering using piecewise smoothness priors: Application to normal and abnormal electrocardiogram denoising

In this work, a block-wise extension of Tikhonov regularization is proposed for denoising smooth signals contaminated by wide-band noise. The proposed method is derived from a constrained least squares problem in two forms: 1) a block-wise fixed-lag smoother with smooth inter-block transitions applied in matrix form, and 2) a fixed-interval smoother applied as a forward-backward zero-phase filter. The filter response is maximally flat and monotonically decreasing, without any ripples in its pass-band. The method is also extended to smoothness of multiple smoothness orders, and its relationship with Lipschitz regularity and block-wise Wiener smoothing is also studied.The denoising of normal and abnormal electrocardiogram (ECG) signals in different stationary and non-stationary noise levels is studied as case study. While most ECG denoising techniques benefit from the pseudo-periodicity of the ECG, the developed technique is merely based on the smoothness assumption, which makes it a powerful method for both normal and abnormal ECG. The performance of the method is assessed by Monte-Carlo simulations over three standard normal and abnormal ECG databases of different sampling rates, in comparison with bandpass filtering, wavelet denoising with various parameters, and Savitzky-Golay filters using Stein's unbiased risk estimate shrinkage scheme.

On estimation and prediction of geostatistical regression models via a corrected Stein's unbiased risk estimator

We consider geostatistical regression models to predict spatial variables of interest, where likelihood‐based methods are used to estimate model parameters. It is known that parameters in the Matérn covariogram cannot be estimated well, even when increasing amounts of data are collected densely in a fixed domain. Although a best linear unbiased predictor has been proposed when model parameters are known, a predictor with estimated parameters is nonlinear and may be not the best in practice. Therefore, we propose an adjusted procedure for the likelihood‐based estimates to improve the predicted ability of the nonlinear spatial predictor. The adjusted parameter estimators based on minimizing a corrected Stein's unbiased risk estimator tend to have less bias than the conventional likelihood‐based estimators, and the resulting spatial predictor is more accurate and more stable. Statistical inference for the proposed method is justified both theoretically and numerically. To verify the practicability of the proposed method, a groundwater data set in Bangladesh is analyzed.

Bilateral linear SURE-based SAR interferogram filter

ABSTRACTThis article proposes a new filter for interferometric synthetic aperture radar (InSAR) phase denoising. Traditional phase filters generally face two major challenges: to preserve texture details while reducing noise and to perform well in less time. The local linear model-based guided filter and Stein’s unbiased risk estimate (SURE)-based filter, in contrast, have a high quality of edge-preserving performance and high efficiency owing to the feature of SURE formula and simplicity. Nevertheless, as these filters are designed for general digital images, they are not suitable for periodic and high-noise-level interferometric phase images. In this article, we modified the original filters by considering the coherence coefficient and features of the interferometric phase image, creating a new patch-based filter adapted to areas characterized by different coherences. Moreover, after obtaining the solution of a patch, considering the geometric closeness and the phasic similarity, we used a bilateral filter combining the pixels in the patch to obtain the estimate. Experimental results based on the simulated and real data confirmed the effectiveness of the proposed algorithm.

Multispectral image denoising with optimized vector non-local mean filter

Nowadays, many applications rely on images of high quality to ensure good performance in conducting their tasks. However, noise goes against this objective as it is an unavoidable issue in most applications. Therefore, it is essential to develop techniques to attenuate the impact of noise, while maintaining the integrity of relevant information in images. We propose in this work to extend the application of the Non-Local Means filter (NLM) to the vector case and apply it for denoising multispectral images. The objective is to benefit from the additional information brought by multispectral imaging systems. The NLM filter exploits the redundancy of information in an image to remove noise. A restored pixel is a weighted average of all pixels in the image. In our contribution, we propose an optimization framework where we dynamically fine tune the NLM filter parameters and attenuate its computational complexity by considering only pixels which are most similar to each other in computing a restored pixel. Filter parameters are optimized using Stein's Unbiased Risk Estimator (SURE) rather than using ad hoc means. Experiments have been conducted on multispectral images corrupted with additive white Gaussian noise. PSNR and similarity comparison with other approaches are provided to illustrate the efficiency of our approach in terms of both denoising performance and computation complexity.

The SURE-LET approach using hybrid thresholding function for image denoising

The Stein's unbiased risk estimate and the linear expansion of thresholds (SURE-LET) approach proposed by Luisier et al. is very efficient for image denoising. But the SURE-LET approach adopts the pointwise thresholding function excluding the intrascale information in the wavelet transform, which limits the denoising ability of the technique. In this paper, to improve the SURE-LET approach, a hybrid thresholding function is designed by incorporating the local Wiener filter into the pointwise thresholding function. Experimental results show that the new hybrid thresholding function gets better denoising performance objectively and subjectively compared to the related SURE-LET approaches and the state-of-the-art wavelet-based methods.

Wavelet-domain TI Wiener-like filtering for complex MR data denoising

Magnetic resonance (MR) images are affected by random noises, which degrade many image processing and analysis tasks. It has been shown that the noise in magnitude MR images follows a Rician distribution. Unlike additive Gaussian noise, the noise is signal-dependent, and consequently difficult to reduce, especially in low signal-to-noise ratio (SNR) images. Wirestam et al. in [20] proposed a Wiener-like filtering technique in wavelet-domain to reduce noise before construction of the magnitude MR image. Based on Wirestam's study, we propose a wavelet-domain translation-invariant (TI) Wiener-like filtering algorithm for noise reduction in complex MR data. The proposed denoising algorithm shows the following improvements compared with Wirestam's method: (1) we introduce TI property into the Wiener-like filtering in wavelet-domain to suppress artifacts caused by translations of the signal; (2) we integrate one Stein's Unbiased Risk Estimator (SURE) thresholding with two Wiener-like filters to make the hard-thresholding scale adaptive; and (3) the first Wiener-like filtering is used to filter the original noisy image in which the noise obeys Gaussian distribution and it provides more reasonable results. The proposed algorithm is applied to denoise the real and imaginary parts of complex MR images. To evaluate our proposed algorithm, we conduct extensive denoising experiments using T1-weighted simulated MR images, diffusion-weighted (DW) phantom and in vivo data. We compare our algorithm with other popular denoising methods. The results demonstrate that our algorithm outperforms others in term of both efficiency and robustness.

Optimization of wavelet- and curvelet-based denoising algorithms by multivariate SURE and GCV

One of the most crucial challenges in seismic data processing is the reduction of noise in the data or improving the signal-to-noise ratio (SNR). Wavelet- and curvelet-based denoising algorithms have become popular to address random noise attenuation for seismic sections. Wavelet basis, thresholding function, and threshold value are three key factors of such algorithms, having a profound effect on the quality of the denoised section. Therefore, given a signal, it is necessary to optimize the denoising operator over these factors to achieve the best performance. In this paper a general denoising algorithm is developed as a multi-variant (variable) filter which performs in multi-scale transform domains (e.g. wavelet and curvelet). In the wavelet domain this general filter is a function of the type of wavelet, characterized by its smoothness, thresholding rule, and threshold value, while in the curvelet domain it is only a function of thresholding rule and threshold value. Also, two methods, Stein's unbiased risk estimate (SURE) and generalized cross validation (GCV), evaluated using a Monte Carlo technique, are utilized to optimize the algorithm in both wavelet and curvelet domains for a given seismic signal. The best wavelet function is selected from a family of fractional B-spline wavelets. The optimum thresholding rule is selected from general thresholding functions which contain the most well known thresholding functions, and the threshold value is chosen from a set of possible values. The results obtained from numerical tests show high performance of the proposed method in both wavelet and curvelet domains in comparison to conventional methods when denoising seismic data.

Block thresholding image denoising with dual-tree complex wavelet transform

As a new tool of signal and image analysis, the dual-tree complex wavelet transform (DTCWT) has been found very useful for a lot of applications in machine learning and pattern recognition. In this paper, we present a method for denoising of images corrupted with additive white Gaussian noise, which employs DTCWT on noisy image to obtain complex coefficients with properties of approximate shift invariance and directional selection. Furthermore, our method adopts block thresholding scheme in denoising procedure, which can empirically choose the optimal block size and threshold at each resolution level by minimizing Stein's unbiased risk estimate. Experimental results on several benchmark images show that our method outperforms most conventional term by term or block thresholding based methods for eliminating different levels of noise. Moreover, its denoising result is compared to state-of-the-art denoising methods with finer structures preservation.