Noisy Samples Research Articles

Robust Mechanical Fault Diagnosis With Noisy Label Based on Multistage True Label Distribution Learning

Fault diagnosis is an essential means to ensure the regular operation of mechanical systems. The existing data-driven algorithms are developed based on the assumption that the given label is entirely correct. However, mislabeling is common, which often occurs in industrial applications. These methods will overfit these mislabeled samples, resulting in inferior generalization. To this end, this article proposes a novel multistage true label distribution learning algorithm. Specifically, based on the training characteristics of data-driven algorithms on noisy datasets, a novel multistage adversarial loss function (MSA-Loss) is proposed. MSA-Loss can make the model construct the true label distribution from noisy datasets, prevent the model from overfitting the noisy samples, and finally keep the model with good generalization. The proposed method can be easily applied to any existing data-driven algorithm to improve its performance on noisy datasets. Our method is verified on high-speed aeronautical bearing and motor datasets, which prove that MSA-Loss has an excellent performance in noisy label scenarios. It can significantly improve the potential of existing diagnostic models in practical industrial applications.

Arrhythmia Detection Based on New Multi-Model Technique for ECG Inter-Patient Classification

This paper presents a novel model for arrhythmia detection based on a cascading technique that utilizes a combination of the One-Sided Selection (OSS) method, Support Vector Machine (SVM), and K-Nearest Neighbor (KNN) algorithms, this model denoted by (OWSK) model to classify four types of electrocardiogram (ECG) heartbeats following inter-patient scheme. The OWSK model consists of three stages. The first stage involves resampling using the One-Sided Selection (OSS) method to solve the imbalance problem and reduce data by removing noisy, borderline, and redundant samples. The second stage involves using Wavelet Transformation (WT) and Power Spectral Density (PSD) to extract the most relevant frequency domain features. The third stage involves a cascading process by constructing the classifier from SVM trained on the whole dataset to classify normal and abnormal beats. Then, KNN (K-Nearest Neighbors) is trained on only the three irregular minority classes to classify the three types of arrhythmias for the detection of ventricular ectopic beats, supraventricular ectopic beats, and fusion beats (V, S, and F). The performance of the proposed model is evaluated in terms of different metrics, including accuracy, recall, precision, and F1 score. The results show the superiority of the proposed model in medical diagnosis compared to the latest works, where it achieves 90%, 90%, 93%, and 91% for accuracy, recall, precision, and F1 score under the inter-patient paradigm and 98%, 98%, 98%, and 98% under the intra-patient paradigm.

Self-supervised denoising model based on deep audio prior using single noisy marine mammal sound sample

Self-supervised denoising model based on deep audio prior using single noisy marine mammal sound sample

An effective technique for detecting minority attacks in NIDS using deep learning and sampling approach

Anomaly-based intrusion detection system have been consistently used in business organizations and military to detect a breach in network by identifying any activity that deviates from the baseline pattern. In this paper, we propose an effective intrusion detection technique to identify and predict the minority attacks with three layers. Here, the first layer utilizes a Weighted Deep Neural Network (WDNN) for identifying the suspicious traffic samples in network and it is passed to the second layer. Layer 2 classifies the traffic samples as normal or majority and minority attacks using Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM). Any traffic sample classified as minority attack is sent to Layer 3 that utilizes XGBoost algorithm. Layer 3 classifies the samples into their respective minority attack classes. To boost the detection rate of minority attacks, system employs a One-Sided Selection under-sampling algorithm to remove noisy samples from the majority attack classes. An Adaptive Synthetic (ADASYN) oversampling algorithm generates synthetic samples of minority attack classes. To evaluate the system, the datasets namely NSL KDD, CICIDS-2017 and CIDDS 001 dataset are used. The system attained an overall accuracy of 97.94% on NSL KDD dataset, 98.3% on CICIDS-2017 dataset and 97.9% on CIDDS 001 dataset.

MoULDyS: Monitoring of autonomous systems in the presence of uncertainties

We introduce MoULDyS, that implements efficient offline and online monitoring algorithms of black-box cyber-physical systems w.r.t. safety properties. MoULDyS takes as input an uncertain log (with noisy and missing samples), as well as a bounding model in the form of an uncertain linear system; this latter model plays the role of an over-approximation so as to reduce the number of false alarms. MoULDyS is Python-based and available under the GNU General Public License v3.0 (gpl-3.0). We further provide easy-to-use scripts to recreate the results of two case studies introduced in an earlier work.

Diffusion models in medical imaging: A comprehensive survey.

Denoising diffusion models, a class of generative models, have garnered immense interest lately in various deep-learning problems. A diffusion probabilistic model defines a forward diffusion stage where the input data is gradually perturbed over several steps by adding Gaussian noise and then learns to reverse the diffusion process to retrieve the desired noise-free data from noisy data samples. Diffusion models are widely appreciated for their strong mode coverage and quality of the generated samples in spite of their known computational burdens. Capitalizing on the advances in computer vision, the field of medical imaging has also observed a growing interest in diffusion models. With the aim of helping the researcher navigate this profusion, this survey intends to provide a comprehensive overview of diffusion models in the discipline of medical imaging. Specifically, we start with an introduction to the solid theoretical foundation and fundamental concepts behind diffusion models and the three generic diffusion modeling frameworks, namely, diffusion probabilistic models, noise-conditioned score networks, and stochastic differential equations. Then, we provide a systematic taxonomy of diffusion models in the medical domain and propose a multi-perspective categorization based on their application, imaging modality, organ of interest, and algorithms. To this end, we cover extensive applications of diffusion models in the medical domain, including image-to-image translation, reconstruction, registration, classification, segmentation, denoising, 2/3D generation, anomaly detection, and other medically-related challenges. Furthermore, we emphasize the practical use case of some selected approaches, and then we discuss the limitations of the diffusion models in the medical domain and propose several directions to fulfill the demands of this field. Finally, we gather the overviewed studies with their available open-source implementations at our GitHub.1 We aim to update the relevant latest papers within it regularly.

Evaluating Classification Model Against Bayes Error Rate

For a classification task, we usually select an appropriate classifier via model selection. How to evaluate whether the chosen classifier is optimal? One can answer this question via Bayes error rate (BER). Unfortunately, estimating BER is a fundamental conundrum. Most existing BER estimators focus on giving the upper and lower bounds of the BER. However, evaluating whether the selected classifier is optimal based on these bounds is hard. In this paper, we aim to learn the exact BER instead of bounds on BER. The core of our method is to transform the BER calculation problem into a noise recognition problem. Specifically, we define a type of noise called Bayes noise and prove that the proportion of Bayes noisy samples in a data set is statistically consistent with the BER of the data set. To recognize the Bayes noisy samples, we present a method consisting of two parts: selecting reliable samples based on percolation theory and then employing a label propagation algorithm to recognize the Bayes noisy samples based on the selected reliable samples. The superiority of the proposed method compared to the existing BER estimators is verified on extensive synthetic, benchmark, and image data sets.

Error Guarantees for Least Squares Approximation with Noisy Samples in Domain Adaptation

Error Guarantees for Least Squares Approximation with Noisy Samples in Domain Adaptation

Fault Diagnosis of Electric Submersible Pumps Using a Three‐Stage Multiscale Feature Transformation Combined with CNN–SVM

A convolutional neural network support vector machine (CNN–SVM) method based on multichannel feature fusion is used for progressive fault diagnosis of offshore oil and gas wells. The excellent classification performance of CNN is attributed to its ability to extract feature representations from large amounts of easily distinguishable data. However, the capability of CNN is severely constrained by the noisy and small sample amount of the electric submersible pump fault data to be studied in this article. First, 12 representative statistical features are extracted from the raw data to reduce the noise. Then, the feature mapping model is designed based on CNN migration learning. Finally, SVM is used instead of softmax function to adopt representative features directly from the mapping model for fault classification. Comparative experimental results show that the accuracy of fault diagnosis using feature‐extracted samples is better than using the raw samples directly. The proposed CNN–SVM approach has the best classification results compared to SVM, BPNN, CNN, BPNN–SVM, CNN–Attention, CNN–LSTM, and CNN–LSTM–Attention, which implies that manual feature extraction is still an indispensable tool in the fault diagnosis process.

Fuzzy set-based Bernoulli Random Noise Weighted Loss for unsupervised person re-identification

Clustering-based methods have achieved impressive performance on unsupervised person ReID task. However, most of these models suffer from noisy labels. In this paper, we propose a novel method to evaluate and suppress noisy labels. Since the feature network is barely trained and the clustering results are unreliable consequently, noisy samples can be hardly screened out with a hard threshold in the early stage of training. To address this issue, we construct a Noisy Label Fuzzy Set (NLFS) whose membership function can be used to evaluate the noise level of each sample. Furthermore, we propose a new loss function, referred to as Bernoulli Random Noise Weighted Loss (BRNWL), with which some samples with higher noise membership will be recognized as noisy samples in higher probability and assigned with corresponding weights to limit their contributions to the training. Through above improvements, randomness is introduced into the alternating update between the feature network and the clustering results, which can effectively prevent the model from trapping into local minima or vicious circles. Extensive experiments demonstrate that our method achieves state-of-the-art performance on popular ReID datasets.

Self-Supervised Electroencephalogram Representation Learning for Automatic Sleep Staging: Model Development and Evaluation Study.

Deep learning models have shown great success in automating tasks in sleep medicine by learning from carefully annotated electroencephalogram (EEG) data. However, effectively using a large amount of raw EEG data remains a challenge. In this study, we aim to learn robust vector representations from massive unlabeled EEG signals, such that the learned vectorized features (1) are expressive enough to replace the raw signals in the sleep staging task, and (2) provide better predictive performance than supervised models in scenarios involving fewer labels and noisy samples. We propose a self-supervised model, Contrast with the World Representation (ContraWR), for EEG signal representation learning. Unlike previous models that use a set of negative samples, our model uses global statistics (ie, the average representation) from the data set to distinguish signals associated with different sleep stages. The ContraWR model is evaluated on 3 real-world EEG data sets that include both settings: at-home and in-laboratory EEG recording. ContraWR outperforms 4 recently reported self-supervised learning methods on the sleep staging task across 3 large EEG data sets. ContraWR also supersedes supervised learning when fewer training labels are available (eg, 4% accuracy improvement when less than 2% of data are labeled on the Sleep EDF data set). Moreover, the model provides informative, representative feature structures in 2D projection. We show that ContraWR is robust to noise and can provide high-quality EEG representations for downstream prediction tasks. The proposed model can be generalized to other unsupervised physiological signal learning tasks. Future directions include exploring task-specific data augmentations and combining self-supervised methods with supervised methods, building upon the initial success of self-supervised learning reported in this study.

Reactive buffering window trajectory segmentation: RBW-TS

Mobility data of a moving object, called trajectory data, are continuously generated by vessel navigation systems, wearable devices, and drones, to name a few. Trajectory data consist of samples that include temporal, spatial, and other descriptive features of object movements. One of the main challenges in trajectory data analysis is to divide trajectory data into meaningful segments based on certain criteria. Most of the available segmentation algorithms are limited to processing data offline, i.e., they cannot segment a stream of trajectory samples. In this work, we propose an approach called Reactive Buffering Window - Trajectory Segmentation (RBW-TS), which partitions trajectory data into segments while receiving a stream of trajectory samples. Another novelty compared to existing work is that the proposed algorithm is based on multidimensional features of trajectories, and it can incorporate as many relevant features of the underlying trajectory as needed. This makes RBW-TS general and applicable to numerous domains by simply selecting trajectory features relevant for segmentation purposes. The proposed online algorithm incurs lower computational and memory requirements. Furthermore, it is robust to noisy samples and outliers. We validate RBW-TS on three use cases: (a) segmenting human-movement trajectories in different modes of transportation, (b) segmenting trajectories generated by vessels in the maritime domain, and (c) segmenting human-movement trajectories in a commercial shopping center. The numerical results detailed in the paper demonstrate that (i) RBW-TS is capable of detecting the true breakpoints of segments in all three usecases while processing a stream of trajectory points; (ii) despite low memory and computational requirements, the performance in terms of the harmonic mean of purity and coverage is comparable to that of state-of-the-art batch and online algorithms; (iii) RBW-TS achieves different levels of accuracy depending on the various internal parameter estimation methods used; and (iv) RBW-TS can tackle real-world trajectory data for segmentation purposes.

Entrepreneurial borrowing overdue prediction based on stacking model transfer learning

PurposeBy introducing Stacking algorithm to solve the underfitting problem caused by insufficient data in traditional machine learning, this paper provides a new solution to the cold start problem of entrepreneurial borrowing risk control.Design/methodology/approachThe authors introduce semi-supervised learning and integrated learning into the field of migration learning, and innovatively propose the Stacking model migration learning, which can independently train models on entrepreneurial borrowing credit data, and then use the migration strategy itself as the learning object, and use the Stacking algorithm to combine the prediction results of the source domain model and the target domain model.FindingsThe effectiveness of the two migration learning models is evaluated with real data from an entrepreneurial borrowing. The algorithmic performance of the Stacking-based model migration learning is further improved compared to the benchmark model without migration learning techniques, with the model area under curve value rising to 0.8. Comparing the two migration learning models reveals that the model-based migration learning approach performs better. The reason for this is that the sample-based migration learning approach only eliminates the noisy samples that are relatively less similar to the entrepreneurial borrowing data. However, the calculation of similarity and the weighing of similarity are subjective, and there is no unified judgment standard and operation method, so there is no guarantee that the retained traditional credit samples have the same sample distribution and feature structure as the entrepreneurial borrowing data.Practical implicationsFrom a practical standpoint, on the one hand, it provides a new solution to the cold start problem of entrepreneurial borrowing risk control. The small number of labeled high-quality samples cannot support the learning and deployment of big data risk control models, which is the cold start problem of the entrepreneurial borrowing risk control system. By extending the training sample set with auxiliary domain data through suitable migration learning methods, the prediction performance of the model can be improved to a certain extent and more generalized laws can be learned.Originality/valueThis paper introduces the thought method of migration learning to the entrepreneurial borrowing scenario, provides a new solution to the cold start problem of the entrepreneurial borrowing risk control system and verifies the feasibility and effectiveness of the migration learning method applied in the risk control field through empirical data.

A hybrid cell image segmentation method based on the multilevel improvement of data

Over the years, several methods have been developed for the segmentation of cell images. Most of the related techniques operate directly on the raw data (noisy cell samples) of the medical image which leads to adverse effects on the structure of leucocytes because the medical images are affected by multiple distortions (varying illumination, deficient background light intensity, and non-uniform staining). To overcome these problems, we came up with an improved solution that performs the qualitative enhancement of cell images for the smooth extraction of cell-nucleus. Although various segmentation methods have adopted an image improvement operation in practice. These methods also amplify the magnitude of image noise which leads to over-sampling and under-sampling of data points. This mis-labelling of data points is minimized by the developed approach which adopts a collaborative fusion strategy (CNN and Nuclear-norm approach) for the qualitative improvement of cell images. The enhanced cell samples were forwarded to the U-net (deep learning model) model for the semantic segmentation of cell images. The performance evaluation of the model was performed on three biomedical cell imaging datasets, which include the ALL-IDB (99.89% accuracy, 99.51% recall, and 99.01% precision), CellaVision (99.68% accuracy, 98.75% precision, and 97.94% specificity) and JTSC (98.45% accuracy, 97.42% precision, and 97.21% specificity) dataset. The results were compared with the state-of-art methods in which the adopted hybrid approach has overpowered the related techniques in the quantitative and qualitative domains.

Unpaired sonar image denoising with simultaneous contrastive learning

Implementing underwater sonar image denoising algorithms through unpaired clean and noisy images is feasible and meaningful, since compliant paired real underwater sonar data is nearly impractical. Nevertheless, acquiring high-performance denoising algorithm is challenging without pairs of data as learning objects. In addition, the wide domain gap between noisy and clean samples as well as the noise contained in underwater sonar images is significantly more complex than that involved in normal images. Therefore, directly utilizing current unpaired optimization algorithms (e.g. generative adversarial learning and circular consistency learning) in underwater sonar image denoising tasks is not sufficient to explore the essential connection from noisy inputs to clean outputs. In this paper, we propose a novel framework for unpaired sonar image denoising which probes the same features of unpaired samples through a dual contrast optimization approach in latent space, named as NCD-GAN. The designed algorithm is composed of two basic parts: Simultaneous Transition Structure (STS) and Separable Contrast Unit (SCU). Separately, STS adopts a cyclic structure of multi-domain transformation to generate valuable surface information, and mines potential feature distributions between different domains via the addition of bidirectional mappings. Meanwhile, SCU introduces additional constraints in layer domains to encourage closer background layers between adjacent domains, while increasing the distance between noise layers so as to better promote the removal of noise and assist in image restoration. Sufficient experiments confirm that our algorithm provides better performance than existing unpaired denoising algorithms on multiple publicly available sonar image datasets and exhibits excellent robustness in the processing of multiple noises. Compared to the latest denoising algorithms, there is at most a 26.43% improvement in the evaluation metrics. Furthermore, it gains a 22.75% mAP boost on the subsequent advanced visual tasks.

EmoMatchSpanishDB: study of speech emotion recognition machine learning models in a new Spanish elicited database

In this paper we present a new speech emotion dataset on Spanish. The database is created using an elicited approach and is composed by fifty non-actors expressing the Ekman’s six basic emotions of anger, disgust, fear, happiness, sadness, and surprise, plus neutral tone. This article describes how this database has been created from the recording step to the performed crowdsourcing perception test step. The crowdsourcing has facilitated to statistically validate the emotion of each collected audio sample and also to filter noisy data samples. Hence we obtained two datasets EmoSpanishDB and EmoMatchSpanishDB. The first includes those recorded audios that had consensus during the crowdsourcing process. The second selects from EmoSpanishDB only those audios whose emotion also matches with the originally elicited. Last, we present a baseline comparative study between different state of the art machine learning techniques in terms of accuracy, precision, and recall for both datasets. The results obtained for EmoMatchSpanishDB improves the ones obtained for EmoSpanishDB and thereof, we recommend to follow the methodology that was used for the creation of emotional databases.

Learning Performance of Weighted Distributed Learning With Support Vector Machines.

The divide-and-conquer strategy is a very effective method of dealing with big data. Noisy samples in big data usually have a great impact on algorithmic performance. In this article, we introduce Markov sampling and different weights for distributed learning with the classical support vector machine (cSVM). We first estimate the generalization error of weighted distributed cSVM algorithm with uniformly ergodic Markov chain (u.e.M.c.) samples and obtain its optimal convergence rate. As applications, we obtain the generalization bounds of weighted distributed cSVM with strong mixing observations and independent and identically distributed (i.i.d.) samples, respectively. We also propose a novel weighted distributed cSVM based on Markov sampling (DM-cSVM). The numerical studies of benchmark datasets show that the DM-cSVM algorithm not only has better performance but also has less total time of sampling and training compared to other distributed algorithms.

Self-Training for Label-Efficient Information Extraction from Semi-Structured Web-Pages

Information Extraction (IE) from semi-structured web-pages is a long studied problem. Training a model for this extraction task requires a large number of human-labeled samples. Prior works have proposed transferable models to improve the label-efficiency of this training process. Extraction performance of transferable models however, depends on the size of their fine-tuning corpus. This holds true for large language models (LLM) such as GPT-3 as well. Generalist models like LLMs need to be fine-tuned on in-domain, human-labeled samples for competitive performance on this extraction task. Constructing a large-scale fine-tuning corpus with human-labeled samples, however, requires significant effort. In this paper, we develop a Label-Efficient Self-Training Algorithm (LEAST) to improve the label-efficiency of this fine-tuning process. Our contributions are two-fold. First , we develop a generative model that facilitates the construction of a large-scale fine-tuning corpus with minimal human-effort. Second , to ensure that the extraction performance does not suffer due to noisy training samples in our fine-tuning corpus, we develop an uncertainty-aware training strategy. Experiments on two publicly available datasets show that LEAST generalizes to multiple verticals and backbone models. Using LEAST, we can train models with less than ten human-labeled pages from each website, outperforming strong baselines while reducing the number of human-labeled training samples needed for comparable performance by up to 11 x.

An adaptive kernel dictionary learning method based on grey wolf optimizer for bearing intelligent fault diagnosis

In this study, an adaptive kernel dictionary learning method for intelligent fault diagnosis of bearings is proposed. Kernel KSVD (KKSVD) is an excellent dictionary learning method with the capacity to handle nonlinear signals. However, the choice of kernel parameters and sparse level is a key issue, since these parameters respectively determine the form of the high-dimensional kernel space and the capability of KKSVD to learn appropriate atomic information for representing the samples. As a result, it is difficult to achieve the maximum performance of KKSVD by pre-specifying the values of the parameters. To address this issue, an advanced meta-heuristic algorithm – that is, the grey wolf optimizer (GWO) is introduced into the KKSVD. Specifically, an objective function is first designed, in which the parameters to be optimized are involved in the learning process of KKSVD for the bearing train set and then applied to the testing of the bearing validation set to get the classification results. The classification accuracy is fed back to the GWO algorithm which will update the parameters iteratively and output the optimal parameters. Two case studies respectively corresponding to two common situations in bearing fault diagnosis – that is, strong noisy samples and unbalanced samples, are carried out. The analysis results demonstrate the effectiveness of the proposed method for adaptively obtaining the optimal parameters and improving the performance of KKSVD. Furthermore, the proposed method outperforms several state-of-art dictionary methods in terms of diagnosis accuracy and robustness.

Denoising Multi-Similarity Formulation: A Self-Paced Curriculum-Driven Approach for Robust Metric Learning

Deep Metric Learning (DML) is a group of techniques that aim to measure the similarity between objects through the neural network. Although the number of DML methods has rapidly increased in recent years, most previous studies cannot effectively handle noisy data, which commonly exists in practical applications and often leads to serious performance deterioration. To overcome this limitation, in this paper, we build a connection between noisy samples and hard samples in the framework of self-paced learning, and propose a Balanced Self-Paced Metric Learning (BSPML) algorithm with a denoising multi-similarity formulation, where noisy samples are treated as extremely hard samples and adaptively excluded from the model training by sample weighting. Especially, due to the pairwise relationship and a new balance regularization term, the sub-problem w.r.t. sample weights is a nonconvex quadratic function. To efficiently solve this nonconvex quadratic problem, we propose a doubly stochastic projection coordinate gradient algorithm. Importantly, we theoretically prove the convergence not only for the doubly stochastic projection coordinate gradient algorithm, but also for our BSPML algorithm. Experimental results on several standard data sets demonstrate that our BSPML algorithm has better generalization ability and robustness than the state-of-the-art robust DML approaches.