Unbiased Risk Estimate Research Articles

Large-Scale Graph Signal Denoising: A Heuristic Approach

Graph signal denoising aims to extract a clean signal from a noisy graph while preserving its intrinsic structure, which is particularly challenging in large-scale graphs. This study presents an efficient method utilizing spectral kernels to create a dictionary of atoms within the graph spectrum and applies analysis coefficients to Stein's unbiased risk estimator (SURE) for denoising. Given the computational difficulty of exhaustive search on large-scale graphs, we enhance this process with spectral clustering for parallel processing and the Imperialist competitive algorithm (ICA) for the search strategy. Our approach divides the initial graph using spectral clustering into k-clusters via the first k-eigenvectors of the graph Laplacian matrix with the k-means algorithm, then applies the SURE scheme to each cluster separately. ICA optimizes the search within each cluster. Results indicate that independently applying SURE and ICA to each subgraph, combined with parallel processing, significantly reduces computational complexity compared to processing the entire graph. This efficiency gain is due to easier parallel processing of subgraphs and more effective ICA execution. Finally, we aggregate the denoised signals from different subgraphs into a unified denoised signal, minimizing mean square error (MSE). Extensive evaluation of various graphs demonstrates the scheme's effectiveness, especially on large graphs.

Enhancing diffusion-weighted prostate MRI through self-supervised denoising and evaluation

Diffusion-weighted imaging (DWI) is a magnetic resonance imaging (MRI) technique that provides information about the Brownian motion of water molecules within biological tissues. DWI plays a crucial role in stroke imaging and oncology, but its diagnostic value can be compromised by the inherently low signal-to-noise ratio (SNR). Conventional supervised deep learning-based denoising techniques encounter challenges in this domain as they necessitate noise-free target images for training. This work presents a novel approach for denoising and evaluating DWI scans in a self-supervised manner, eliminating the need for ground-truth data. By leveraging an adapted version of Stein’s unbiased risk estimator (SURE) and exploiting a phase-corrected combination of repeated acquisitions, we outperform both state-of-the-art self-supervised denoising methods and conventional non-learning-based approaches. Additionally, we demonstrate the applicability of our proposed approach in accelerating DWI scans by acquiring fewer image repetitions. To evaluate denoising performance, we introduce a self-supervised methodology that relies on analyzing the characteristics of the residual signal removed by the denoising approaches.

An Unbiased Risk Estimator for Partial Label Learning with Augmented Classes

Partial Label Learning (PLL) is a typical weakly supervised learning task, which assumes each training instance is annotated with a set of candidate labels containing the ground-truth label. Recent PLL methods adopt identification-based disambiguation to alleviate the influence of false positive labels and achieve promising performance. However, they require all classes in the test set to have appeared in the training set, ignoring the fact that new classes will keep emerging in real applications. To address this issue, in this paper, we focus on the problem of Partial Label Learning with Augmented Class (PLLAC), where one or more augmented classes are not visible in the training stage but appear in the inference stage. Specifically, we propose an unbiased risk estimator with theoretical guarantees for PLLAC, which estimates the distribution of augmented classes by differentiating the distribution of known classes from unlabeled data and can be equipped with arbitrary PLL loss functions. Besides, we provide a theoretical analysis of the estimation error bound of the estimator, which guarantees the convergence of the empirical risk minimizer to the true risk minimizer as the number of training data tends to infinity. Furthermore, we add a risk-penalty regularization term in the optimization objective to alleviate the influence of the over-fitting issue caused by negative empirical risk. Extensive experiments on benchmark, UCI and real-world datasets demonstrate the effectiveness of the proposed approach.

UIR-ES: An unsupervised underwater image restoration framework with equivariance and stein unbiased risk estimator

Underwater imaging faces challenges for enhancing object visibility and restoring true colors due to the absorptive and scattering characteristics of water. Underwater image restoration (UIR) seeks solutions to restore clean images from degraded ones, providing significant utility in downstream tasks. Recently, data-driven UIR has garnered much attention due to the potent expressive capabilities of deep neural networks (DNNs). These DNNs are supervised, relying on a large amount of labeled training samples. However, acquiring such data is expensive or even impossible in real-world underwater scenarios. While recent researches suggest that unsupervised learning is effective in UIR, none of these frameworks consider signal physical priors. In this work, we present a novel physics-inspired unsupervised UIR framework empowered by equivariance and unbiased estimation techniques. Specifically, equivariance stems from the invariance, inherent in natural signals to enhance data-efficient learning. Given that degraded images invariably contain noise, we propose a noise-tolerant loss for unsupervised UIR based on the Stein unbiased risk estimator to achieve an accurate estimation of the data consistency. Extensive experiments on the benchmark UIR datasets, including the UIEB and RUIE datasets, validate the superiority of the proposed method in terms of quantitative scores, visual outcomes, and generalization ability, compared to state-of-the-art counterparts. Moreover, our method demonstrates even comparable performance with the supervised model.

Bayesian estimation for mean vector of multivariate normal distribution on the linear and nonlinear exponential balanced loss based on wavelet decomposition

This paper addresses the problem of Bayesian wavelet estimating the mean vector of multivariate normal distribution under a multivariate normal prior distribution based on linear and nonlinear exponential balanced loss functions. The covariance matrix of multivariate normal distribution is considered known. Bayes estimators of mean vector parameter of multivariate normal distribution are achieved. Then two soft shrinkage wavelet threshold estimators based on Stein’s unbiased risk estimate (SURE) and Bayes estimators are provided. Finally, the performance of the soft shrinkage wavelet estimators was checked through simulation study and Electrical Grid Stability Simulated data set. Simulation and real data results showed the better performance of SURE thresholds based on linear and nonlinear exponential balanced loss functions compared to other classical wavelet methods. Also, they showed better performance for SURE threshold based on nonlinear exponential balanced loss function in multivariate normal distribution with small dimensions.

Adaptive Truncation Threshold Determination for Multimode Fiber Single-Pixel Imaging

Truncated singular value decomposition (TSVD) is a popular recovery algorithm for multimode fiber single-pixel imaging (MMF-SPI), and it uses truncation thresholds to suppress noise influences. However, due to the sensitivity of MMF relative to stochastic disturbances, the threshold requires frequent re-determination as noise levels dynamically fluctuate. In response, we design an adaptive truncation threshold determination (ATTD) method for TSVD-based MMF-SPI in disturbed environments. Simulations and experiments reveal that ATTD approaches the performance of ideal clairvoyant benchmarks, and it corresponds to the best possible image recovery under certain noise levels and surpasses both traditional truncation threshold determination methods with less computation—fixed threshold and Stein’s unbiased risk estimator (SURE)—specifically under high noise levels. Moreover, target insensitivity is demonstrated via numerical simulations, and the robustness of the self-contained parameters is explored. Finally, we also compare and discuss the performance of TSVD-based MMF-SPI, which uses ATTD, and machine learning-based MMF-SPI, which uses diffusion models, to provide a comprehensive understanding of ATTD.

Population-Based Risk of Psychiatric Disorders Associated With Recurrent Copy Number Variants

Recurrent copy number variants (rCNVs) have been associated with increased risk of psychiatric disorders in case-control studies, but their population-level impact is unknown. To provide unbiased population-based estimates of prevalence and risk associated with psychiatric disorders for rCNVs and to compare risks across outcomes, rCNV dosage type (deletions or duplications), and locus features. This genetic association study is an analysis of data from the Lundbeck Foundation Initiative for Integrative Psychiatric Research (iPSYCH) case-cohort sample of individuals born in Denmark in 1981-2008 and followed up until 2015, including (1) all individuals (n = 92 531) with a hospital discharge diagnosis of attention-deficit/hyperactivity disorder (ADHD), autism spectrum disorder (ASD), bipolar disorder, major depressive disorder (MDD), or schizophrenia spectrum disorder (SSD) and (2) a subcohort (n = 50 625) randomly drawn from the source population. Data were analyzed from January 2021 to August 2023. Carrier status of deletions and duplications at 27 autosomal rCNV loci was determined from neonatal blood samples genotyped on single-nucleotide variant microarrays. Population-based rCNV prevalence was estimated with a survey model using finite population correction to account for oversampling of cases. Hazard ratio (HR) estimates and 95% CIs for psychiatric disorders were derived using weighted Cox proportional hazard models. Risks were compared across outcomes, dosage type, and locus features using generalized estimating equation models. A total of 3547 rCNVs were identified in 64 735 individuals assigned male at birth (53.8%) and 55 512 individuals assigned female at birth (46.2%) whose age at the end of follow-up ranged from 7.0 to 34.7 years (mean, 21.8 years). Most observed increases in rCNV-associated risk for ADHD, ASD, or SSD were moderate, and risk estimates were highly correlated across these disorders. Notable exceptions included high ASD-associated risk observed for Prader-Willi/Angelman syndrome duplications (HR, 20.8; 95% CI, 7.9-55). No rCNV was associated with increased MDD risk. Also, rCNV-associated risk was positively correlated with locus size and gene constraint but not with dosage type. Comparison with published case-control and community-based studies revealed a higher prevalence of deletions and lower associated increase in risk for several rCNVs in iPSYCH2015. This study found that several rCNVs were more prevalent and conferred less risk of psychiatric disorders than estimated previously. Most case-control studies overestimate rCNV-associated risk of psychiatric disorders, likely because of selection bias. In an era where genetics is increasingly being clinically applied, these results highlight the importance of population-based risk estimates for genetics-based predictions.

Estimation of the efficiency of unbiased predictive risk estimator in the inversion of 2D magnetotelluric data

Estimation of the efficiency of unbiased predictive risk estimator in the inversion of 2D magnetotelluric data

Low and high dimensional wavelet thresholds for matrix-variate normal distribution

The matrix-variate normal distribution is a probability distribution that is a generalization of the multivariate normal distribution to matrix-valued random variables. In this paper, we introduce a wavelet shrinkage estimator based on Stein’s unbiased risk estimate (SURE) threshold for matrix-variate normal distribution. We find a new SURE threshold for soft thresholding wavelet shrinkage estimator under the reflected normal balanced loss function in low and high dimensional cases. Also, we obtain the restricted wavelet shrinkage estimator based on non-negative sub matrix of the mean matrix. Finally, we present a simulation study to test the validity of the wavelet shrinkage estimator and two real examples for low and high dimensional data sets.

Predicting absolute risk for a person with missing risk factors.

We compared methods to project absolute risk, the probability of experiencing the outcome of interest in a given projection interval accommodating competing risks, for a person from the target population with missing predictors. Without missing data, a perfectly calibrated model gives unbiased absolute risk estimates in a new target population, even if the predictor distribution differs from the training data. However, if predictors are missing in target population members, a reference dataset with complete data is needed to impute them and to estimate absolute risk, conditional only on the observed predictors. If the predictor distributions of the reference data and the target population differ, this approach yields biased estimates. We compared the bias and mean squared error of absolute risk predictions for seven methods that assume predictors are missing at random (MAR). Some methods imputed individual missing predictors, others imputed linear predictor combinations (risk scores). Simulations were based on real breast cancer predictor distributions and outcome data. We also analyzed a real breast cancer dataset. The largest bias for all methods resulted from different predictor distributions of the reference and target populations. No method was unbiased in this situation. Surprisingly, violating the MAR assumption did not induce severe biases. Most multiple imputation methods performed similarly and were less biased (but more variable) than a method that used a single expected risk score. Our work shows the importance of selecting predictor reference datasets similar to the target population to reduce bias of absolute risk predictions with missing risk factors.

Deep Network Regularization for Phase-based Magnetic Resonance Electrical Properties Tomography with Stein's Unbiased Risk Estimator.

Magnetic resonance imaging (MRI) can extract the tissue conductivity values from in vivo data using the so-called phase-based magnetic resonance electrical properties tomography (MR-EPT). However, this procedure suffers from noise amplification caused by the use of the Laplacian operator. To counter this issue, we propose a novel preprocessing denoiser for magnetic resonance transceive phase images, operating in an unsupervised manner. Inspired by the deep image prior approach, we apply the random initialization of a convolutional neural network, which enforces an implicit regularization. Additionally, we introduce Stein's unbiased risk estimator, which is the unbiased estimator of the mean square error for optimizing the network without the need for label images. This modification not only tackles the overfitting problem inherent in the deep image prior approach but also operates within a purely unsupervised framework. In addition, instead of using phase images, we use real and imaginary images, which aligns with the theoretical model of the risk estimator. Our generative model needs neither the preparation of training datasets nor prior training procedure, and it maintains adaptability across various resolutions and signal-to-noise ratio levels. In testing. our method significantly diminished residual error remaining in phase maps, quantitatively as well as qualitatively, for both phantom and simulated brain data. Furthermore, it outperformed other denoising methods in reducing noise amplification and boundary error. When applied to healthy volunteer and patient data, the proposed method revealed reduced error in the reconstructed conductivity maps, with conductivity values aligning well with established literature values. To the best of our knowledge, this is the first blind approach using a purely unsupervised denoising framework that can implement a 2D phase-based MR-EPT reconstruction algorithm. The source code is available at https://github.com/Yonsei-MILab/Implicit-Regularization-forMREPT-with-SURE.

Supervise-Assisted Self-Supervised Deep-Learning Method for Hyperspectral Image Restoration.

Hyperspectral image (HSI) restoration is a challenging research area, covering a variety of inverse problems. Previous works have shown the great success of deep learning in HSI restoration. However, facing the problem of distribution gaps between training HSIs and target HSI, those data-driven methods falter in delivering satisfactory outcomes for the target HSIs. In addition, the degradation process of HSIs is usually disturbed by noise, which is not well taken into account in existing restoration methods. The existence of noise further exacerbates the dissimilarities within the data, rendering it challenging to attain desirable results without an appropriate learning approach. To track these issues, in this article, we propose a supervise-assisted self-supervised deep-learning method to restore noisy degraded HSIs. Initially, we facilitate the restoration network to acquire a generalized prior through supervised learning from extensive training datasets. Then, the self-supervised learning stage is employed and utilizes the specific prior of the target HSI. Particularly, to restore clean HSIs during the self-supervised learning stage from noisy degraded HSIs, we introduce a noise-adaptive loss function that leverages inner statistics of noisy degraded HSIs for restoration. The proposed noise-adaptive loss consists of Stein's unbiased risk estimator (SURE) and total variation (TV) regularizer and fine-tunes the network with the presence of noise. We demonstrate through experiments on different HSI tasks, including denoising, compressive sensing, super-resolution, and inpainting, that our method outperforms state-of-the-art methods on benchmarks under quantitative metrics and visual quality. The code is available at https://github.com/ying-fu/SSDL-HSI.

Self-supervised MRI denoising: leveraging Stein’s unbiased risk estimator and spatially resolved noise maps

Thermal noise caused by the imaged object is an intrinsic limitation in magnetic resonance imaging (MRI), resulting in an impaired clinical value of the acquisitions. Recently, deep learning (DL)-based denoising methods achieved promising results by extracting complex feature representations from large data sets. Most approaches are trained in a supervised manner by directly mapping noisy to noise-free ground-truth data and, therefore, require extensive paired data sets, which can be expensive or infeasible to obtain for medical imaging applications. In this work, a DL-based denoising approach is investigated which operates on complex-valued reconstructed magnetic resonance (MR) images without noise-free target data. An extension of Stein’s unbiased risk estimator (SURE) and spatially resolved noise maps quantifying the noise level with pixel accuracy were employed during the training process. Competitive denoising performance was achieved compared to supervised training with mean squared error (MSE) despite optimizing the model without noise-free target images. The proposed DL-based method can be applied for MR image enhancement without requiring noise-free target data for training. Integrating the noise maps as an additional input channel further enables the regulation of the desired level of denoising to adjust to the preference of the radiologist.

Robust signal dimension estimation via SURE

The estimation of signal dimension under heavy-tailed latent variable models is studied. As a primary contribution, robust extensions of an earlier estimator based on Gaussian Stein’s unbiased risk estimation are proposed. These novel extensions are based on the framework of elliptical distributions and robust scatter matrices. Extensive simulation studies are conducted in order to compare the novel methods with several well-known competitors in both estimation accuracy and computational speed. The novel methods are applied to a financial asset return data set.

A machine learning approach combined with wavelet analysis for automatic detection of Pc5 geomagnetic pulsations observed at geostationary orbits

Pc5 geomagnetic pulsations are ultra-low frequency (ULF) waves whose period lies within 150–600 s. Their detection is crucial for accurate space weather modelling, monitoring, and analysis. Identification of these pulsations using manual approaches is difficult and time-consuming because of the smaller magnitudes and the limited durations within which they occur. To overcome this challenge, we propose a robust Cascade Forward Neural Network (CFNN) model combined with the wavelet technique for automatically detecting Pc5 events from satellite datasets observed at geostationary orbits. The dataset used in this study is the magnetic field vector measurements retrieved from the Geostationary Operational Environmental Satellite-10 (GOES-10) from 2000 to 2009. Pc5 geomagnetic pulsations were extracted from the toroidal component of the field perturbation using a bandpass Butterworth filter. Continuous wavelet transform (CWT) analysis using the mother Morlet wavelet was utilized to validate the integrity of the extracted signal in the time–frequency domain. The extracted Pc5 signal was decomposed into details and approximations using Daubechies wavelet transform to separate the intelligible signal from the incoherent noise that left their trace on the magnetic field time series. The detail signal was subjected to denoising using the heuristic Stein Unbiased Risk Estimate (SURE) approach with soft thresholding to obtain the denoised Pc5 events. The denoised Pc5 events were utilized as the target in the machine learning whereas the inputs were obtained from 5-dimensional space transformation of the toroidal component of the time series. The developed algorithm performed well and demonstrated a detection accuracy of 80 % and a Root Mean Square Error (RMSE) of 0.13 nT indicating a high performance in detecting Pc5 events. For effective validation of the model, Pc5 events detected by our model were correlated with the Kp index and the amplitude of the Pc5 events observed at geostationary orbit. A good correlation was obtained in both cases, making our model a practical and preferred choice for Pc5 pulsation detection in contrast to conventional frequency analysis tools which take much time for signal processing on large data sets.

Hyperspectral Prediction Model of Nitrogen Content in Citrus Leaves Based on the CEEMDAN–SR Algorithm

Nitrogen content is one of the essential elements in citrus leaves (CL), and many studies have been conducted to determine the nutrient content in CL using hyperspectral technology. To address the key problem that the conventional spectral data-denoising algorithms directly discard high-frequency signals, resulting in missing effective signals, this study proposes a denoising preprocessing algorithm, complete ensemble empirical mode decomposition with adaptive noise joint sparse representation (CEEMDAN–SR), for CL hyperspectral data. For this purpose, 225 sets of fresh CL were collected at the Institute of Fruit Tree Research of the Guangdong Academy of Agricultural Sciences, to measure their elemental nitrogen content and the corresponding hyperspectral data. First, the spectral data were preprocessed using CEEMDAN–SR, Stein’s unbiased risk estimate and the linear expansion of thresholds (SURE–LET), sparse representation (SR), Savitzky–Golay (SG), and the first derivative (FD). Second, feature extraction was carried out using principal component analysis (PCA), uninformative variables elimination (UVE), and the competitive adaptive re-weighted sampling (CARS) algorithm. Finally, partial least squares regression (PLSR), support vector regression (SVR), random forest (RF), and Gaussian process regression (GPR) were used to construct a CL nitrogen prediction model. The results showed that most of the prediction models preprocessed using the CEEMDAN–SR algorithm had better accuracy and robustness. The prediction models based on CEEMDAN–SR preprocessing, PCA feature extraction, and GPR modeling had an R2 of 0.944, NRMSE of 0.057, and RPD of 4.219. The study showed that the CEEMDAN–SR algorithm can be effectively used to denoise CL hyperspectral data and reduce the loss of effective information. The prediction model using the CEEMDAN–SR+PCA+GPR algorithm could accurately obtain the nitrogen content of CL and provide a reference for the accurate fertilization of citrus trees.

Sparse additive models in high dimensions with wavelets

AbstractIn multiple regression, when covariates are numerous, it is often reasonable to assume that only a small number of them has predictive information. In some medical applications for instance, it is believed that only a few genes out of thousands are responsible for cancer. In that case, the aim is not only to propose a good fit, but also to select the relevant covariates (genes). We propose to perform model selection with additive models in high dimensions (sample size and number of covariates). Our approach is computationally efficient thanks to fast wavelet transforms, it does not rely on cross validation, and it solves a convex optimization problem for a prescribed penalty parameter, called the quantile universal threshold. We also propose a second rule based on Stein unbiased risk estimation geared toward prediction. We use Monte Carlo simulations and real data to compare various methods based on false discovery rate (FDR), true positive rate (TPR) and mean squared error. Our approach is the only one to handle high dimensions, and has a good FDR–TPR trade‐off.

Two new Bayesian-wavelet thresholds estimations of elliptical distribution parameters under non-linear exponential balanced loss

The estimation of mean vector parameters is very important in elliptical and spherically models. Among different methods, the Bayesian and shrinkage estimation are interesting. In this paper, the estimation of p-dimensional location parameter for p-variate elliptical and spherical distributions under an asymmetric loss function is investigated. We find generalized Bayes estimator of location parameters for elliptical and spherical distributions. Also we show the minimaxity and admissibility of generalized Bayes estimator in class of We introduce two new shrinkage soft-wavelet threshold estimators based on Huang shrinkage wavelet estimator (empirical) and Stein’s unbiased risk estimator (SURE) for elliptical and spherical distributions under non-linear exponential-balanced loss function. At the end, we present a simulation study to test the validity of the class of proposed estimators and physicochemical properties of the tertiary structure data set that is given to test the efficiency of this estimators in denoising.

Cross-Modal Retrieval with Partially Mismatched Pairs.

In this paper, we study a challenging but less-touched problem in cross-modal retrieval, i.e., partially mismatched pairs (PMPs). Specifically, in real-world scenarios, a huge number of multimedia data (e.g., the Conceptual Captions dataset) are collected from the Internet, and thus it is inevitable to wrongly treat some irrelevant cross-modal pairs as matched. Undoubtedly, such a PMP problem will remarkably degrade the cross-modal retrieval performance. To tackle this problem, we derive a unified theoretical Robust Cross-modal Learning framework (RCL) with an unbiased estimator of the cross-modal retrieval risk, which aims to endow the cross-modal retrieval methods with robustness against PMPs. In detail, our RCL adopts a novel complementary contrastive learning paradigm to address the following two challenges, i.e., the overfitting and underfitting issues. On the one hand, our method only utilizes the negative information which is much less likely false compared with the positive information, thus avoiding the overfitting issue to PMPs. However, these robust strategies could induce underfitting issues, thus making training models more difficult. On the other hand, to address the underfitting issue brought by weak supervision, we present to leverage of all available negative pairs to enhance the supervision contained in the negative information. Moreover, to further improve the performance, we propose to minimize the upper bounds of the risk to pay more attention to hard samples. To verify the effectiveness and robustness of the proposed method, we carry out comprehensive experiments on five widely-used benchmark datasets compared with nine state-of-the-art approaches w.r.t. the image-text and video-text retrieval tasks. The code is available at https://github.com/penghu-cs/RCL.

Positive Distribution Pollution: Rethinking Positive Unlabeled Learning from a Unified Perspective

Positive Unlabeled (PU) learning, which has a wide range of applications, is becoming increasingly prevalent. However, it suffers from problems such as data imbalance, selection bias, and prior agnostic in real scenarios. Existing studies focus on addressing part of these problems, which fail to provide a unified perspective to understand these problems. In this paper, we first rethink these problems by analyzing a typical PU scenario and come up with an insightful point of view that all these problems are inherently connected to one problem, i.e., positive distribution pollution, which refers to the inaccuracy in estimating positive data distribution under very little labeled data. Then, inspired by this insight, we devise a variational model named CoVPU, which addresses all three problems in a unified perspective by targeting the positive distribution pollution problem. CoVPU not only accurately separates the positive data from the unlabeled data based on discrete normalizing flows, but also effectively approximates the positive distribution based on our derived unbiased rebalanced risk estimator and supervises the approximation based on a novel prior-free variational loss. Rigorous theoretical analysis proves the convergence of CoVPU to an optimal Bayesian classifier. Extensive experiments demonstrate the superiority of CoVPU over the state-of-the-art PU learning methods under these problems.