Noisy Data Samples Research Articles

On Combining Instance Selection and Discretisation: A Comparative Study of Two Combination Orders

Data discretisation focuses on converting continuous attribute values to discrete ones which are closer to a knowledge-level representation that is easier to understand, use, and explain than continuous values. On the other hand, instance selection aims at filtering out noisy or unrepresentative data samples from a given training dataset before constructing a learning model. In practice, some domain datasets may require processing with both discretisation and instance selection at the same time. In such cases, the order in which discretisation and instance selection are combined will result in differences in the processed datasets. For example, discretisation can be performed first based on the original dataset, after which the instance selection algorithm is used to evaluate the discrete type of data for selection, whereas the alternative is to perform instance selection first based on the continuous type of data, then using the discretiser to transfer the attribute type of values of a reduced dataset. However, this issue has not been investigated before. The aim of this paper is to compare the performance of a classifier trained and tested over datasets processed by these combination orders. Specifically, the minimum description length principle (MDLP) and ChiMerge are used for discretisation, and IB3, DROP3 and GA for instance selection. The experimental results obtained using ten different domain datasets show that executing instance selection first and discretisation second perform the best, which can be used as the guideline for the datasets that require performing both steps. In particular, combining DROP3 and MDLP can provide classification accuracy of 0.85 and AUC of 0.8, which can be regarded as the representative baseline for future related researches.

SCAE-Stacked Convolutional Autoencoder for Fault Diagnosis of a Hydraulic Piston Pump with Limited Data Samples.

Deep learning (DL) models require enormous amounts of data to produce reliable diagnosis results. The superiority of DL models over traditional machine learning (ML) methods in terms of feature extraction, feature dimension reduction, and diagnosis performance has been shown in various studies of fault diagnosis systems. However, data acquisition can sometimes be compromised by sensor issues, resulting in limited data samples. In this study, we propose a novel DL model based on a stacked convolutional autoencoder (SCAE) to address the challenge of limited data. The innovation of the SCAE model lies in its ability to enhance gradient information flow and extract richer hierarchical features, leading to superior diagnostic performance even with limited and noisy data samples. This article describes the development of a fault diagnosis method for a hydraulic piston pump using time-frequency visual pattern recognition. The proposed SCAE model has been evaluated on limited data samples of a hydraulic piston pump. The findings of the experiment demonstrate that the suggested approach can achieve excellent diagnostic performance with over 99.5% accuracy. Additionally, the SCAE model has outperformed traditional DL models such as deep neural networks (DNN), standard stacked sparse autoencoders (SSAE), and convolutional neural networks (CNN) in terms of diagnosis performance. Furthermore, the proposed model demonstrates robust performance under noisy data conditions, further highlighting its effectiveness and reliability.

Wi-CHAR: A WiFi Sensing Approach with Focus on Both Scenes and Restricted Data.

Significant strides have been made in the field of WiFi-based human activity recognition, yet recent wireless sensing methodologies still grapple with the reliance on copious amounts of data. When assessed in unfamiliar domains, the majority of models experience a decline in accuracy. To address this challenge, this study introduces Wi-CHAR, a novel few-shot learning-based cross-domain activity recognition system. Wi-CHAR is meticulously designed to tackle both the intricacies of specific sensing environments and pertinent data-related issues. Initially, Wi-CHAR employs a dynamic selection methodology for sensing devices, tailored to mitigate the diminished sensing capabilities observed in specific regions within a multi-WiFi sensor device ecosystem, thereby augmenting the fidelity of sensing data. Subsequent refinement involves the utilization of the MF-DBSCAN clustering algorithm iteratively, enabling the rectification of anomalies and enhancing the quality of subsequent behavior recognition processes. Furthermore, the Re-PN module is consistently engaged, dynamically adjusting feature prototype weights to facilitate cross-domain activity sensing in scenarios with limited sample data, effectively distinguishing between accurate and noisy data samples, thus streamlining the identification of new users and environments. The experimental results show that the average accuracy is more than 93% (five-shot) in various scenarios. Even in cases where the target domain has fewer data samples, better cross-domain results can be achieved. Notably, evaluation on publicly available datasets, WiAR and Widar 3.0, corroborates Wi-CHAR's robust performance, boasting accuracy rates of 89.7% and 92.5%, respectively. In summary, Wi-CHAR delivers recognition outcomes on par with state-of-the-art methodologies, meticulously tailored to accommodate specific sensing environments and data constraints.

Deep Nonnegative Matrix Factorization with Joint Global and Local Structure Preservation

Deep Non-Negative Matrix Factorization (DNMF) methods provide an efficient low-dimensional representation of given data through their layered architecture. A limitation of such methods is that they cannot effectively preserve the local and global geometric structures of the data in each layer. Consequently, a significant amount of the geometrical information within the data, present in each layer of the employed deep framework, can be overlooked by the model. This can lead to an information loss and a subsequent drop in performance. In this paper, we propose a novel deep non-negative matrix factorization method, Deep Non-Negative Matrix Factorization with Joint Global and Local Structure Preservation (dubbed Dn2MFGL), that ensures the preservation of both global and local structures within the data space. Dn2MFGL performs representation learning through a sequential embedding procedure which involves both the global data structure by accounting for the data variance, and the local data relationships by utilizing information from neighboring data points. Moreover, a regularization term that promotes sparsity by utilizing the concept of the inner product is applied to the matrices representing the lower dimensions. This aims to retain the fundamental data structure while discarding less crucial features. Simultaneously, the residual matrix of Dn2MFGL is subjected to the L2,1 norm, which ensures the robustness of the model against noisy data samples. An effective and multiplicative updating process also facilitates Dn2MFGL in solving the employed objective function. The clustering performance of the proposed deep NMF method is explored across various benchmarks of face datasets. The results point to Dn2MFGL outperforming several existing classical and state-of-the-art NMF methods. The source code is available at https://github.com/FaridSaberi/Dn2MFGO.git.

Robust Ranking Kernel Support Vector Machine via Manifold Regularized Matrix Factorization for Multi-Label Classification

Multi-label classification has been extensively researched and utilized for several decades. However, the performance of these methods is highly susceptible to the presence of noisy data samples, resulting in a significant decrease in accuracy when noise levels are high. To address this issue, we propose a robust ranking support vector machine (Rank-SVM) method that incorporates manifold regularized matrix factorization. Unlike traditional Rank-SVM methods, our approach integrates feature selection and multi-label learning into a unified framework. Within this framework, we employ matrix factorization to learn a low-rank robust subspace within the input space, thereby enhancing the robustness of data representation in high-noise conditions. Additionally, we incorporate manifold structure regularization into the framework to preserve manifold relationships among low-rank samples, which further improves the robustness of the low-rank representation. Leveraging on this robust low-rank representation, we extract a resilient low-rank features and employ them to construct a more effective classifier. Finally, the proposed framework is extended to derive a kernelized ranking approach, for the creation of nonlinear multi-label classifiers. To effectively solve this non-convex kernelized method, we employ the augmented Lagrangian multiplier (ALM) and alternating direction method of multipliers (ADMM) techniques to obtain the optimal solution. Experimental evaluations conducted on various datasets demonstrate that our framework achieves superior classification results and significantly enhances performance in high-noise scenarios.

Ground truth free denoising by optimal transport

<p style='text-indent:20px;'>This paper proposes a new training strategy for a denoiser removing (additive) independent noise, with only as readily available data as possible and no further assumptions on the data nor noise. While every real-world measurement contains some noise, it seems that this problem remains unsolved for settings where clean data samples are lacking. We propose a pushforward operator formulation of an ideal denoiser and a corresponding GAN setup for training a denoiser ground truth free. The GAN trains solely on samples of noisy data and noise. In a series of denoising experiments in 1D and 2D, we demonstrate our training strategy's performance, which significantly improves the state-of-the-art of unsupervised denoising. Moreover, for some non-Gaussian noise, the method compares favorably even to naive supervised denoising.</p>

Learning Energy-Based Models in High-Dimensional Spaces with Multiscale Denoising-Score Matching.

Energy-based models (EBMs) assign an unnormalized log probability to data samples. This functionality has a variety of applications, such as sample synthesis, data denoising, sample restoration, outlier detection, Bayesian reasoning and many more. But, the training of EBMs using standard maximum likelihood is extremely slow because it requires sampling from the model distribution. Score matching potentially alleviates this problem. In particular, denoising-score matching has been successfully used to train EBMs. Using noisy data samples with one fixed noise level, these models learn fast and yield good results in data denoising. However, demonstrations of such models in the high-quality sample synthesis of high-dimensional data were lacking. Recently, a paper showed that a generative model trained by denoising-score matching accomplishes excellent sample synthesis when trained with data samples corrupted with multiple levels of noise. Here we provide an analysis and empirical evidence showing that training with multiple noise levels is necessary when the data dimension is high. Leveraging this insight, we propose a novel EBM trained with multiscale denoising-score matching. Our model exhibits a data-generation performance comparable to state-of-the-art techniques such as GANs and sets a new baseline for EBMs. The proposed model also provides density information and performs well on an image-inpainting task.

Diffusion models in medical imaging: A comprehensive survey.

Denoising diffusion models, a class of generative models, have garnered immense interest lately in various deep-learning problems. A diffusion probabilistic model defines a forward diffusion stage where the input data is gradually perturbed over several steps by adding Gaussian noise and then learns to reverse the diffusion process to retrieve the desired noise-free data from noisy data samples. Diffusion models are widely appreciated for their strong mode coverage and quality of the generated samples in spite of their known computational burdens. Capitalizing on the advances in computer vision, the field of medical imaging has also observed a growing interest in diffusion models. With the aim of helping the researcher navigate this profusion, this survey intends to provide a comprehensive overview of diffusion models in the discipline of medical imaging. Specifically, we start with an introduction to the solid theoretical foundation and fundamental concepts behind diffusion models and the three generic diffusion modeling frameworks, namely, diffusion probabilistic models, noise-conditioned score networks, and stochastic differential equations. Then, we provide a systematic taxonomy of diffusion models in the medical domain and propose a multi-perspective categorization based on their application, imaging modality, organ of interest, and algorithms. To this end, we cover extensive applications of diffusion models in the medical domain, including image-to-image translation, reconstruction, registration, classification, segmentation, denoising, 2/3D generation, anomaly detection, and other medically-related challenges. Furthermore, we emphasize the practical use case of some selected approaches, and then we discuss the limitations of the diffusion models in the medical domain and propose several directions to fulfill the demands of this field. Finally, we gather the overviewed studies with their available open-source implementations at our GitHub.1 We aim to update the relevant latest papers within it regularly.

EmoMatchSpanishDB: study of speech emotion recognition machine learning models in a new Spanish elicited database

In this paper we present a new speech emotion dataset on Spanish. The database is created using an elicited approach and is composed by fifty non-actors expressing the Ekman’s six basic emotions of anger, disgust, fear, happiness, sadness, and surprise, plus neutral tone. This article describes how this database has been created from the recording step to the performed crowdsourcing perception test step. The crowdsourcing has facilitated to statistically validate the emotion of each collected audio sample and also to filter noisy data samples. Hence we obtained two datasets EmoSpanishDB and EmoMatchSpanishDB. The first includes those recorded audios that had consensus during the crowdsourcing process. The second selects from EmoSpanishDB only those audios whose emotion also matches with the originally elicited. Last, we present a baseline comparative study between different state of the art machine learning techniques in terms of accuracy, precision, and recall for both datasets. The results obtained for EmoMatchSpanishDB improves the ones obtained for EmoSpanishDB and thereof, we recommend to follow the methodology that was used for the creation of emotional databases.

Distributionally Robust Optimization With Noisy Data for Discrete Uncertainties Using Total Variation Distance

Stochastic programs, where uncertainty distribution must be inferred from noisy data samples, are considered. They are approximated with distributionally/robust optimizations that minimize the worst-case expected cost over ambiguity sets, i.e., sets of distributions that are sufficiently compatible with observed data. The ambiguity sets capture probability distributions whose convolution with the noise distribution is within a ball centered at the empirical noisy distribution of data samples parameterized by total variation distance. Using the prescribed ambiguity set, the solutions of the distributionally/robust optimizations converge to the solutions of the original stochastic programs when the number of the data samples grow to infinity. Therefore, the proposed distributionally/robust optimization problems are asymptotically consistent. The distributionally/robust optimization problems can be cast as tractable optimization problems.

Uncertainty Based Optimal Sample Selection for Big Data

In Machine learning and pattern recognition, building a better predictive model is one of the key problems in the presence of big or massive data; especially, if that data contains noisy and unrepresentative data samples. These types of samples adversely affect the learning model and may degrade its performance. To alleviate this problem, sometimes, it becomes necessary to sample the data after eliminating unnecessary instances by maintaining the underlying distribution intact. This process is called sampling or instance selection (IS). However, in this process, a substantial computational cost is involved. This paper discusses an uncertainty based optimal sample selection (UBOSS) method which can select a subset of optimal samples efficiently. Our proposed work comprises three main steps; initially, it uses an IS method to identify the patterns of representative and unrepresentative samples from the original data set; then, an uncertainty-based selector is designed to obtain fuzziness (i.e., a type of uncertainty) of those samples using a classifier whose output is a membership or fuzzy vector; this process further utilizes the divide-and-conquer strategy to obtain a subset of representative samples. Experiments are conducted on six datasets to evaluate the performance of the proposed IS method. Results show that our proposed methodology outperforms when compared with the selection performance (i.e., optimum samples) of the baseline methods (i.e., CNN, IB3, and DROP3).

Ensemble Capsule Network with an Attention Mechanism for the Fault Diagnosis of Bearings from Imbalanced Data Samples.

In order to solve the problem of imbalanced and noisy data samples for the fault diagnosis of rolling bearings, a novel ensemble capsule network (Capsnet) with a convolutional block attention module (CBAM) that is based on a weighted majority voting method is proposed in this study. Firstly, the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) method was used to decompose the raw vibration signal into different IMF signals, which are noise reduction signals. Secondly, the IMF signals were input into the Capsnet with CBAM in order to diagnose the fault category preliminarily. Finally, the weighted majority voting method was utilized so as to fuse all of the preliminary diagnosis results in order to obtain the final diagnostic decision. In order to verify the effectiveness of the proposed ensemble of Capsnet with CBAM, this method was applied to the fault diagnosis of rolling bearings with imbalanced and different SNR data samples. The diagnostic results show that the proposed diagnostic method can achieve higher levels of accuracy than other methods, such as single CNN, single Capsnet, ensemble CNN and an ensemble capsule network without CBAM and that it has stronger immunity to noise than an ensemble capsule network without CBAM.

Data-Driven Dissipativity Analysis: Application of the Matrix S-Lemma

The concept of dissipativity, as introduced by Jan Willems, is one of the cornerstones of systems and control theory. Typically, dissipativity properties are verified by resorting to a mathematical model of the system under consideration. This article aims to assess dissipativity using computing storage functions for linear systems directly from measured data. As our main contributions, we provide conditions under which dissipativity can be ascertained from a finite collection of noisy data samples. Three different noise models are considered that can capture a variety of situations, including the cases where the data samples are noise-free, the energy of the noise is bounded, or the individual noise samples are bounded. All conditions are phrased in terms of data-based linear matrix inequalities, which can be readily solved using existing software packages.

Robust Spectral Clustering via Low-Rank Sample Representation

Traditional clustering methods neglect the data quality and perform clustering directly on the original data. Therefore, their performance can easily deteriorate since real-world data would usually contain noisy data samples in high-dimensional space. In order to resolve the previously mentioned problem, a new method is proposed, which builds on the approach of low-rank representation. The proposed approach first learns a low-rank coefficient matrix from data by exploiting the data’s self-expressiveness property. Then, a regularization term is introduced to ensure that the representation coefficient of two samples, which are similar in original high-dimensional space, is close to maintaining the samples’ neighborhood structure in the low-dimensional space. As a result, the proposed method obtains a clustering structure directly through the low-rank coefficient matrix to guarantee optimal clustering performance. A wide range of experiments shows that the proposed method is superior to compared state-of-the-art methods.

Edge-Preserving Image Denoising Based on Lipschitz Estimation

The information transmitted in the form of signals or images is often corrupted with noise. These noise elements can occur due to the relative motion, noisy channels, error in measurements, and environmental conditions (rain, fog, change in illumination, etc.) and result in the degradation of images acquired by a camera. In this paper, we address these issues, focusing mainly on the edges that correspond to the abrupt changes in the signal or images. Preserving these important structures, such as edges or transitions and textures, has significant theoretical importance. These image structures are important, more specifically, for visual perception. The most significant information about the structure of the image or type of the signal is often hidden inside these transitions. Therefore it is necessary to preserve them. This paper introduces a method to reduce noise and to preserve edges while performing Non-Destructive Testing (NDT). The method computes Lipschitz exponents of transitions to identify the level of discontinuity. Continuous wavelet transform-based multi-scale analysis highlights the modulus maxima of the respective transitions. Lipschitz values estimated from these maxima are used as a measure to preserve edges in the presence of noise. Experimental results show that the noisy data sample and smoothness-based heuristic approach in the spatial domain restored noise-free images while preserving edges.

Instance selection in medical datasets: A divide-and-conquer framework

Instance selection is an important problem in medical data mining. It focuses on selecting representative data samples from a given training set, whereas unrepresentative (or noisy) data samples are filtered out. This reduces the size of the training set, which then requires less storage space. In addition, when the instance selection algorithm was carefully chosen, a reduction in the training set so that it contains less noisy data can usually make the classifiers perform better than the ones without considering instance selection. In the literature, many instance selection algorithms have been proposed. However, different algorithms tend to use different criteria to determine the noisy data, making it difficult to find the best algorithm for different domain datasets. In other words, some algorithms may perform better than the others for some specific domain datasets, but may perform worse than others over other domain datasets. Instead of developing a novel algorithm that performs better than most other algorithms, this paper introduces a divide-and-conquer based instance selection (DCIS) framework that aims to improve the performance of each specific instance selection algorithm per se. Two well-known algorithms, i.e., DROP3 and IB3, are used as the baseline, and various small and large scale medical datasets are used in the experiments. Our results show that when DROP3 and IB3 are used to perform instance selection based on the DCIS framework, there is an improvement in the performance of the k-NN and SVM classifiers over the ones by the DROP3 and IB3 baselines, respectively.

Performance analysis of data mining classification algorithms for early prediction of diabetes mellitus 2

Diabetes mellitus (DM) generally referred to as diabetes. It is a group of metabolic infection in which there are high blood sugar levels over a prolonged period. Data mining is used for predicting various diseases. From many methods of data mining, classification is one of the main techniques. The classification techniques are used to classify the hidden information in all areas including medical diagnostic field. In this research work, we compare the machine learning classifiers (naïve Bayes, J48 decision tree, OneR, AdaBoost, random forest, random tree and support vector machines) to classify the patients into diabetic and non-diabetic mellitus. These algorithms have been tested with data samples downloaded from UCI. The performances of the algorithms have been considered in both the cases, i.e., data samples with noisy data and data samples set without noisy data. Results are evaluated in terms of accuracy, sensitivity, and specificity. Experimental results suggested that, support vector machine (SVM) classifier is the best classifier for predicting diabetes mellitus 2.

A comparative dimensionality reduction study in telecom customer segmentation using deep learning and PCA

Telecom Companies logs customer’s actions which generate a huge amount of data that can bring important findings related to customer’s behavior and needs. The main characteristics of such data are the large number of features and the high sparsity that impose challenges to the analytics steps. This paper aims to explore dimensionality reduction on a real telecom dataset and evaluate customers’ clustering in reduced and latent space, compared to original space in order to achieve better quality clustering results. The original dataset contains 220 features that belonging to 100,000 customers. However, dimensionality reduction is an important data preprocessing step in the data mining process specially with the presence of curse of dimensionality. In particular, the aim of data reduction techniques is to filter out irrelevant features and noisy data samples. To reduce the high dimensional data, we projected it down to a subspace using well known Principal Component Analysis (PCA) decomposition and a novel approach based on Autoencoder Neural Network, performing in this way dimensionality reduction of original data. Then K-Means Clustering is applied on both-original and reduced data set. Different internal measures were performed to evaluate clustering for different numbers of dimensions and then we evaluated how the reduction method impacts the clustering task.

An adaptive algorithm for fast and reliable online saccade detection.

To investigate visual perception around the time of eye movements, vision scientists manipulate stimuli contingent upon the onset of a saccade. For these experimental paradigms, timing is especially crucial, because saccade offset imposes a deadline on the display change. Although efficient online saccade detection can greatly improve timing, most algorithms rely on spatial-boundary techniques or absolute-velocity thresholds, which both suffer from weaknesses: late detections and false alarms, respectively. We propose an adaptive, velocity-based algorithm for online saccade detection that surpasses both standard techniques in speed and accuracy and allows the user to freely define the detection criteria. Inspired by the Engbert-Kliegl algorithm for microsaccade detection, our algorithm computes two-dimensional velocity thresholds from variance in the preceding fixation samples, while compensating for noisy or missing data samples. An optional direction criterion limits detection to the instructed saccade direction, further increasing robustness. We validated the algorithm by simulating its performance on a large saccade dataset and found that high detection accuracy (false-alarm rates of < 1%) could be achieved with detection latencies of only 3 ms. High accuracy was maintained even under simulated high-noise conditions. To demonstrate that purely intrasaccadic presentations are technically feasible, we devised an experimental test in which a Gabor patch drifted at saccadic peak velocities. Whereas this stimulus was invisible when presented during fixation, observers reliably detected it during saccades. Photodiode measurements verified that-including all system delays-the stimuli were physically displayed on average 20 ms after saccade onset. Thus, the proposed algorithm provides a valuable tool for gaze-contingent paradigms.

Sparse large-margin nearest neighbor embedding via greedy dyad functional optimization

We consider the sparse subspace learning problem where the intrinsic subspace is assumed to be low-dimensional and formed by sparse basis vectors. Confined to a few sparse bases, projecting data to the learned subspace essentially has an effect of feature selection by taking a small number of the most salient features while suppressing the rest as noise. Unlike existing sparse dimensionality reduction methods, however, we exploit the class labels to impose maximal margin data separation in the subspace, which was previously shown to yield improved prediction accuracy often times in non-sparse models. We first formulate an optimization problem with constraints on the matrix rank and the sparseness of the basis vectors. Instead of computationally demanding gradient-based learning strategies used in previous large-margin embedding, we propose an efficient greedy functional optimization algorithm over the infinite set of the sparse dyadic products. Each iteration in the proposed algorithm, after some shifting operations, effectively reduces to the famous sparse eigenvalue problem, and can be solved quickly by the recent truncated power method. We demonstrate the improved prediction performance of the proposed approach on several image/text classification datasets, especially characterized by high-dimensional noisy data samples with small training sets.