Noisy Samples Research Articles

Error analysis for denoising smooth modulo signals on a graph

In many applications, we are given access to noisy modulo samples of a smooth function with the goal being to robustly unwrap the samples, i.e., to estimate the original samples of the function. In a recent work, Cucuringu and Tyagi [11] proposed denoising the modulo samples by first representing them on the unit complex circle and then solving a smoothness regularized least squares problem – the smoothness measured w.r.t. the Laplacian of a suitable proximity graph G – on the product manifold of unit circles. This problem is a quadratically constrained quadratic program (QCQP) which is nonconvex, hence they proposed solving its sphere-relaxation leading to a trust region subproblem (TRS). In terms of theoretical guarantees, ℓ 2 error bounds were derived for (TRS). These bounds are however weak in general and do not really demonstrate the denoising performed by (TRS). In this work, we analyse the (TRS) as well as an unconstrained relaxation of (QCQP). For both these estimators we provide a refined analysis in the setting of Gaussian noise and derive noise regimes where they provably denoise the modulo observations w.r.t. the ℓ 2 norm. The analysis is performed in a general setting where G is any connected graph.

An imbalanced learning method by combining SMOTE with Center Offset Factor

SMOTE is a well-known oversampling method for learning on imbalanced datasets. However, it has the risk of introducing noisy instances and overfitting problems. In order to improve its performance, this paper proposes an oversampling method called SMOTE-COF, which is an improvement of SMOTE based on center offset factor. The SMOTE-COF method first removes noisy samples, then computes center offset factor to select sparsely distributed minority class samples. Furthermore, these samples are used to generate new minority class samples with other minority class instances distributed in the same sub-cluster by SMOTE. Comparative experiments on one simulated dataset and fourteen UCI datasets provide evidence that the SMOTE-COF can effectively reduce noisy samples, generate better minority classes, and improve classification performance for imbalanced datasets.

Penalty based robust learning with noisy labels

In general, deep neural network is vulnerable to noisy labels also known as erroneous labels. As a main solution to mitigate this problem, sample selection techniques have been actively studied. However, if the labels are dominantly corrupted by some classes (these noisy samples are called dominant noisy labeled samples), the network also learns dominant noisy labeled samples rapidly via content-aware optimization. This can cause memorization (reduce generalization) in the deep neural network. In this study, we propose a compelling criteria to penalize dominant-noisy-labeled samples intensively through class-wise penalty labels. By averaging prediction confidences for the each observed label, we obtain suitable penalty labels that have high values if the labels are largely corrupted by some classes. Additionally, to enhance the accuracy of penalty labels, temporal ensembling and weight are exploited. Experiments were performed using benchmarks (CIFAR-10, CIFAR-100, Tiny-ImageNet) and real-world datasets (ANIMAL-10 N, Clothing1M) to evaluate the proposed criteria in various scenarios with different noise rates. Using the proposed sample selection, the learning process of the network becomes significantly robust to noisy labels compared to existing methods in several noise types. Moreover, the proposed criteria can be easily combined with the algorithms of loss correction and hybrid categories through a simple modification to improve learning performance.

Cooperative Refinement Learning for domain adaptive person Re-identification

Domain adaptive person re-identification (re-ID) targets at identifying the same persons’ images in unlabeled target domain. Some existing domain adaptive Person re-ID methods assigned pseudo labels by clustering algorithms on the target domain, which tends to generate noisy labels and neglect samples with low confidence as outliers. These may hinder the retraining process, thereby limiting the model’s generalization ability. In order to overcome these problems, we propose a Cooperative Refinement Learning (CooRL) framework, which resists noisy labels and takes advantage of the outliers by developing a multi-branches structure with refinement mechanism. Specifically, a mean-attention guided network is leveraged to learn more complementary features from pure and noisy samples generated by clustering, which includes a mean network and two sub-branches. Meanwhile, to better optimize the neural networks, CooRL jointly refines and aligns the pseudo labels of sub-branches by progressively adjusting the predicted logits through the mean network. Comprehensive experimental results have demonstrated that our proposed method can achieve excellent performances on benchmark datasets.

Ideal ratio mask estimation using supervised DNN approach for target speech signal enhancement

The most challenging process in recent Speech Enhancement (SE) systems is to exclude the non-stationary noises and additive white Gaussian noise in real-time applications. Several SE techniques suggested were not successful in real-time scenarios to eliminate noises in the speech signals due to the high utilization of resources. So, a Sliding Window Empirical Mode Decomposition including a Variant of Variational Model Decomposition and Hurst (SWEMD-VVMDH) technique was developed for minimizing the difficulty in real-time applications. But this is the statistical framework that takes a long time for computations. Hence in this article, this SWEMD-VVMDH technique is extended using Deep Neural Network (DNN) that learns the decomposed speech signals via SWEMD-VVMDH efficiently to achieve SE. At first, the noisy speech signals are decomposed into Intrinsic Mode Functions (IMFs) by the SWEMD Hurst (SWEMDH) technique. Then, the Time-Delay Estimation (TDE)-based VVMD was performed on the IMFs to elect the most relevant IMFs according to the Hurst exponent and lessen the low- as well as high-frequency noise elements in the speech signal. For each signal frame, the target features are chosen and fed to the DNN that learns these features to estimate the Ideal Ratio Mask (IRM) in a supervised manner. The abilities of DNN are enhanced for the categories of background noise, and the Signal-to-Noise Ratio (SNR) of the speech signals. Also, the noise category dimension and the SNR dimension are chosen for training and testing manifold DNNs since these are dimensions often taken into account for the SE systems. Further, the IRM in each frequency channel for all noisy signal samples is concatenated to reconstruct the noiseless speech signal. At last, the experimental outcomes exhibit considerable improvement in SE under different categories of noises.

An extension of Synthetic Minority Oversampling Technique based on Kalman filter for imbalanced datasets

More often than not, data collected in real-time tends to be imbalanced i.e., the samples belonging to a particular class are significantly more than the others. This degrades the performance of the predictor. One of the most notable algorithms to handle such an imbalance in the dataset by fabricating synthetic data, is the “Synthetic Minority Oversampling Technique (SMOTE)”. However, data imbalance is not solely responsible for the poor performance of the classifier. Certain research works have demonstrated that noisy samples can have a significant role in misclassifying the dataset. Also, handling large data is computationally expensive. Hence, data reduction is imperative. In this work, we put forth a novel extension of SMOTE by integrating it with the Kalman filter. The proposed method, Kalman-SMOTE (KSMOTE), filters out the noisy samples in the final dataset after SMOTE, which includes both the raw data and the synthetically generated samples, thereby reducing the size of the dataset. Our model is validated with a wide range of datasets. An experimental analysis of the results shows that our model outperforms the presently available techniques.

Unsupervised person re-identification with multi-label learning guided self-paced clustering

Although unsupervised person re-identification (Re-ID) has drawn increasing research attention recently, it remains challenging to learn discriminative features without annotations across disjoint camera views. In this paper, we address the unsupervised person Re-ID with a conceptually novel yet simple framework, termed as Multi-label Learning guided self-paced Clustering (MLC). MLC mainly learns discriminative features with three crucial modules, namely a multi-scale network, a multi-label learning module, and a self-paced clustering module. Specifically, the multi-scale network generates multi-granularity person features in both global and local views. The multi-label learning module leverages a memory feature bank and assigns each image with a multi-label vector based on the similarities between the image and feature bank. After multi-label training for several epochs, the self-paced clustering joins in training and assigns a pseudo label for each image. The benefits of our MLC come from three aspects: i) the multi-scale person features for better similarity measurement, ii) the multi-label assignment based on the whole dataset ensures that every image can be trained, and iii) the self-paced clustering removes some noisy samples for better feature learning. Extensive experiments on three popular large-scale Re-ID benchmarks demonstrate that our MLC outperforms previous state-of-the-art methods and significantly improves the performance of unsupervised person Re-ID.

An Unsupervised Noisy Sample Detection Method for Deep Learning-Based Health Status Prediction

Due to rough environment or data-acquisition equipment failure, the machine monitoring data may be contaminated with noise, which may lead to the generation of noisy samples. For the widely used deep learning-based prediction model, some noisy samples (with low-intensity noise) may improve model generalization while some noisy samples (with high-intensity noise) may hamper the prediction models. To detect the noisy samples that hamper the prediction models, an unsupervised noisy sample detection method is proposed, thereby promoting the accuracy of the deep learning-based prediction model. First, the transforming model with an attention mechanism is built to transform the multivariate input sequences of samples to attention weight vectors, which are related to the prediction task. Then, a clustering method without hyperparameters is proposed to cluster these attention weight vectors and the hierarchy of clusters is created. Afterward, the noisy samples that hamper the prediction models are detected and removed based on the clustering thresholds, and a prediction model with higher prediction accuracy can thus be obtained. The effectiveness of the proposed method is verified using a simulated turbofan engine degradation dataset and a real milling machine monitoring dataset. The results show that the prediction errors for the two datasets have been reduced by 13.44% and 12.97% on average after removing the noisy samples detected by the proposed method. The comparison results show that the proposed method outperforms other methods in terms of improving the prediction accuracy.

Attention-Based Deep Learning Model for Early Detection of Parkinson's Disease

Parkinson's disease (PD), classified under the category of a neurological syndrome, affects the brain of a person which leads to the motor and non-motor symptoms. Among motor symptoms, one of the major disabling symptom is Freezing of Gait (FoG) that affects the daily standard of living of PD patients. Available treatments target to improve the symptoms of PD. Detection of PD at the early stages is an arduous task due to being indistinguishable from a healthy individual. This work proposed a novel attention-based model for the detection of FoG events and PD, and measuring the intensity of PD on the United Parkinson's Disease Rating Scale. Two separate datasets, that is, UCF Daphnet dataset for detection of Freezing of Gait Events and PhysioNet Gait in PD Dataset were used for training and validating on their respective problems. The results show a definite rise in the various performance metrics when compared to landmark models on these problems using these datasets. These results strongly suggest that the proposed state of the art attention-based deep learning model provide a consistent as well as an efficient solution to the selected problem. High values were obtained for various performance metrics like accuracy of 98.74% for detection FoG, 98.72% for detection of PD and 98.05% for measuring the intensity of PD on UPDRS. The model was also analyzed for robustness against noisy samples, where also model exhibited consistent performance. These results strongly suggest that the proposed model provides a better classification method for selected problem.

TC-Net: Detecting Noisy Labels Via Transform Consistency

It is crucial to distinguish mislabeled samples for dealing with noisy labels. Previous methods such as “Co-teaching” and “JoCoR” introduce two different networks to select clean samples out of the noisy ones and only use these clean samples to train the deep models. Different from these methods which require to train two networks simultaneously, we propose a simple and effective method to identify clean samples only using one single network. We discover that the clean samples prefer to reach consistent predictions for the original images and the transformed images while noisy samples usually suffer from inconsistent predictions. Motivated by this observation, we propose a noisy label detection approach, named Transform Consistency Network (TC-Net), which constrains the transform consistency ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.,</i> category consistency and visual attention consistency) between the original images and the transformed images for network training. Then we can select small-loss samples to update the parameters of the network. Furthermore, in order to mitigate the negative influence of noisy labels, we design a classification loss by using the off-line hard labels and on-line soft labels to provide more reliable supervisions for training a robust model. We conduct comprehensive experiments on CIFAR-10, CIFAR-100 and Clothing1M datasets. Compared with the clean sample selection baselines, we achieve the state-of-the-art performance. Especially, in most cases, our proposed method outperforms the baselines by a large margin.

Deep Low-Rank Graph Convolutional Subspace Clustering for Hyperspectral Image

Deep subspace clustering (DSC) has achieved considerable success in the classification task of hyperspectral image (HSI) without background (defined as noisy samples) compared with traditional subspace clustering methods. Unfortunately, directly applying DSC to classify land-cover on HSI datasets with background may suffer from the degradation of classification performance. In this paper, we propose an effective deep low-rank graph convolutional subspace clustering (DLR-GCSC) framework for improving the performance of land-cover classification on HSI datasets with background. Specifically, we design a joint spatial-spectral network to extract band-level and patch-level features simultaneously by combining 1D and 2D auto-encoder. Moreover, we construct a low-rank constrained fully connected layer as self-expression layer in the network to make the joint features more discriminative. To reduce the influence of noisy samples and obtain an informative affinity matrix, we recast the joint features into a non-Euclidean domain by introducing graph convolution. Finally, spectral clustering is applied on the informative affinity matrix to obtain the classification results. Experiments on three benchmark HSI datasets show that our proposed method achieves competitive classification performance to the state-of-the-art methods on both HSI data with background and without background.

PoGaIN: Poisson-Gaussian Image Noise Modeling From Paired Samples

Image noise can often be accurately fitted to a Poisson-Gaussian distribution. However, estimating the distribution parameters from a noisy image only is a challenging task. Here, we study the case when paired noisy and noise-free samples are accessible. No method is currently available to exploit the noise-free information, which may help to achieve more accurate estimations. To fill this gap, we derive a novel, cumulant-based, approach for Poisson-Gaussian noise modeling from paired image samples. We show its improved performance over different baselines, with special emphasis on MSE, effect of outliers, image dependence, and bias. We additionally derive the log-likelihood function for further insights and discuss real-world applicability.

Unsupervised Speech Enhancement Using Dynamical Variational Autoencoders

Dynamical variational autoencoders (DVAEs) are a class of deep generative models with latent variables, dedicated to model time series of high-dimensional data. DVAEs can be considered as extensions of the variational autoencoder (VAE) that include temporal dependencies between successive observed and/or latent vectors. Previous work has shown the interest of using DVAEs over the VAE for speech spectrograms modeling. Independently, the VAE has been successfully applied to speech enhancement in noise, in an unsupervised noise-agnostic set-up that requires neither noise samples nor noisy speech samples at training time, but only requires clean speech signals. In this paper, we extend these works to DVAE-based single-channel unsupervised speech enhancement, hence exploiting both speech signals unsupervised representation learning and dynamics modeling. We propose an unsupervised speech enhancement algorithm that combines a DVAE speech prior pre-trained on clean speech signals with a noise model based on nonnegative matrix factorization, and we derive a variational expectation-maximization (VEM) algorithm to perform speech enhancement. The algorithm is presented with the most general DVAE formulation and is then applied with three specific DVAE models to illustrate the versatility of the framework. Experimental results show that the proposed DVAE-based approach outperforms its VAE-based counterpart, as well as several supervised and unsupervised noise-dependent baselines, especially when the noise type is unseen during training.

Signal Reconstruction from Noisy Multichannel Samples

We consider the signal reconstruction problem under the case of the signals sampled in the multichannel way and with the presence of noise. Observing that if the samples are inexact, the rigorous enforcement of multichannel interpolation is inappropriate. Thus the reasonable smoothing and regularized corrections are indispensable. In this paper, we propose several alternative methods for the signal reconstruction from the noisy multichannel samples under different smoothing and regularization principles. We compare these signal reconstruction methods theoretically and experimentally in the various situations. To demonstrate the effectiveness of the proposed methods, the probability interpretation and the error analysis for these methods are provided. Additionally, the numerical simulations as well as some guidelines to use the methods are also presented.

Robust Online Tracking With Meta-Updater.

In a sequence, the appearance of both the target and background often changes dramatically. Offline-trained models may not handle huge appearance variations well, causing tracking failures. Most discriminative trackers address this issue by introducing an online update scheme, making the model dynamically adapt the changes of the target and background. Although the online update scheme plays an important role in improving the tracker's accuracy, it inevitably pollutes the model with noisy observation samples. It is necessary to reduce the risk of the online update scheme for better tracking. In this work, we propose a novel offline-trained Meta-Updater to address an important but unsolved problem: Is the tracker ready for updating in the current frame? The proposed module can effectively integrate geometric, discriminative, and appearance cues in a sequential manner, and then mine the sequential information with a designed cascaded LSTM module. Moreover, we strengthen the effect of appearance information on the module, i.e., the additional local outlier factor is introduced to integrate into a newly designed network. We integrate our meta-updater into eight different types of online update trackers. Extensive experiments on four long-term and two short-term tracking benchmarks demonstrate that our meta-updater is effective and has strong generalization ability.

Unsupervised Cross-Domain Person Re-Identification Method Based on Attention Block and Refined Clustering

Most unsupervised cross-domain person re-identification methods based on clustering suffer from a lack of feature discrimination and clustering generates pseudo-labels noise, leading to a decrease in accuracy. To solve these problems, this paper proposes an unsupervised cross-domain person re-identification method based on attention block and refined clustering. Firstly, ResNet50 is selected as the backbone network, coordinate attention and triple attention are concatenated and embedded in ResNet50 to extract fine-grained features, perform feature aggregation, and mine fine-grained information. Secondly, a refined clustering strategy is proposed to achieve a coarse-to-fine clustering process by designing the measurement standards for clustering, determining its reliability, and eliminating noisy samples. Finally, the hybrid memory bank dynamically stores cluster centers and continues to update them with iterations, adapting to changes in clusters and performing invariant learning. The experimental results show that the new method designed in the paper improves the accuracy of rank-1 and mAP by 0.4% and 2.4%, respectively, on the target domain Market-1501 dataset, and improves the accuracy of rank-1 and mAP by 0.4% and 1.1%, respectively, on the target domain DukeMTMC-ReID dataset, compared with other typical methods.

Complementary Learning-Based Scene Classification of Remote Sensing Images With Noisy Labels

Recently, many deep convolutional neural network (DCNN)-based methods have been proposed for remote sensing (RS) image scene classification (SC). In general, DCNNs obtain good generalization capabilities under the condition of correct labels. Unfortunately, the given samples are sometimes mislabeled. In this letter, the classification of RS images with noisy labels is investigated. First, complementary learning (CL), which learns from complementary labels rather than the original labels, is introduced for RS image classification with noisy labels. CL can decrease the probability of learning from incorrect information, and therefore, it is robust to noisy labels. Then, soft CL, which randomly disturbs the complementary labels of the training samples, is proposed to prevent the overfitting issue in training a DCNN. Moreover, an RS image scene classification framework combining ordinary learning (OL) and CL (RS-COCL) is proposed, which uses CL to obtain a good model and OL to fine-tune the deep model. Additionally, noisy labels filtering is used in RS-COCL (RS-COCL-NLF) to detected and corrected noisy samples. At last, soft CL is used in RS-COCL-NLF to obtain better classification performance. The proposed methods are tested on two widely used datasets (i.e., Northwestern Polytechnical University (NWPU)-RESISC45 and PatternNet) and the obtained results show that the proposed methods provide competitive classification accuracy compared to the state-of-the-art methods.

Robust Learning From Noisy Web Images Via Data Purification for Fine-Grained Recognition

Manually labeling fine-grained datasetsis laborious and typically requires domain-specific expert knowledge. Conversely, a vast amount of web data is relatively easy to obtain with nearly no human effort. Therefore, learning from noisy web data for fine-grained tasks is attracting increasing attention in recent years. However, the presence of noise in web images is a huge obstacle for training robust fine-grained recognition models. To this end, we propose a novel approach to identify noisy images as well as specifically distinguish in- and out-of-distribution samples. It can purify the noisy web training set by discarding out-of-distribution noise and relabeling in-distribution noisy samples. Then we can train the model on the purified dataset to alleviate the harmful effects of noise and make the most of web images to achieve better performance. Extensive experiments on three commonly used fine-grained datasets demonstrate that our approach is far superior to current state-of-the-art web-supervised methods. The data and source code of this work have been made publicly available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/NUST-Machine-Intelligence-Laboratory/Dataset-Purification</uri> .

Seek Common Ground While Reserving Differences: A Model-Agnostic Module for Noisy Domain Adaptation

Noisy domain adaptation aims to solve the problem that the source dataset contains noisy labels in domain adaptation. Previous methods handle noisy labels by selecting the small-loss samples with inconsistent predictions between two models and discarding the consistent samples, resulting in many noises contained in the selected samples. By jointly considering the consistent and inconsistent samples, we propose a model-agnostic module, named Seek Common Ground While Reserving Differences (SCGWRD), to reduce the impact of noisy samples. The proposed SCGWRD module consists of Seek Common Ground (SCG) component and Reserve Differences (RD) component by utilizing the outputs of two symmetrical domain adaptation models. As the common samples with consistent predictions between two models are more likely to be clean samples, the SCG component applies the small-loss strategy to select the reliable samples with consistent predictions. Unlike SCG, the RD component maintains the divergences between two models with mutual learning and reduces the effect of noisy data using the samples with different predictions and small losses. Evaluations on three benchmarks demonstrate the effectiveness and robustness of the proposed SCGWRD module for noisy domain adaptation.

Change Detection From Synthetic Aperture Radar Images via Graph-Based Knowledge Supplement Network

Synthetic aperture radar (SAR) image change detection is a vital yet challenging task in the field of remote sensing image analysis. Most previous works adopt a self-supervised method which uses pseudolabeled samples to guide subsequent training and testing. However, deep networks commonly require many high-quality samples for parameter optimization. The noise in pseudolabels inevitably affects the final change detection performance. To solve the problem, we propose a graph-based knowledge supplement network (GKSNet). To be more specific, we extract discriminative information from the existing labeled dataset as additional knowledge, to suppress the adverse effects of noisy samples to some extent. Afterward, we design a graph transfer module to distill contextual information attentively from the labeled dataset to the target dataset, which bridges feature correlation between datasets. To validate the proposed method, we conducted extensive experiments on four SAR datasets, which demonstrated the superiority of the proposed GKSNet as compared to several state-of-the-art baselines. Our codes are available: at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/summitgao/SAR_CD_GKSNet</uri> .