Denoising Autoencoder Model Research Articles

Nonlinear Dynamic Process Monitoring Based on Discriminative Denoising Autoencoder and Canonical Variate Analysis

Modern industrial processes are characterized by increasing complexity, often exhibiting pronounced dynamic behaviors and significant nonlinearity. Addressing these dynamic and nonlinear characteristics is essential for effective process monitoring. However, many existing methods for process monitoring and fault diagnosis are insufficient in handling these challenges. In this article, we present a novel process monitoring approach, CVA-DisDAE, which integrates an improved Denoising Autoencoder (DAE) with Canonical Variate Analysis (CVA) to address the challenges posed by dynamic behaviors and nonlinear relationships in industrial processes. First, CVA is employed to reduce data dimensionality and minimize information redundancy by maximizing correlations between past and future observations, thereby effectively capturing process dynamics. Following this, we introduce a discriminative DAE model (DisDAE) designed to serve as a semi-supervised denoising autoencoder for precise feature extraction. This is achieved by incorporating both between-class separability and within-class variability into the traditional DAE framework. The key distinction between the proposed DisDAE and the conventional DAE lies in the integration of a linear discriminant analysis (LDA) penalty into the DAE’s loss function, resulting in extracted features that are more conducive to fault classification. Finally, we validate the effectiveness of the proposed semi-supervised dynamic process monitoring approach through its application to the Tennessee Eastman benchmark process, demonstrating its superior performance.

Deep Neural Networks Based Denoising of Regional Seismic Waveforms and Impact on Analysis of North Korean Nuclear Tests

AbstractWe test a deep learning based denoising autoencoder algorithm on regional and teleseismic seismological and hydroacoustic datasets, which we compile from the International Monitoring System of the Comprehensive Nuclear-Test-Ban Treaty Organisation. We focus on stations which can be relevant to investigate North Korean nuclear tests. Denoising of waveform records using autoencoder techniques potentially enables improved signal detection and processing due to lowered signal-to-noise ratios. We train and compare the performance of several different denoising autoencoder models, for short- and long waveform periods, trained on the complete station network as well as on individual stations. We investigate if the denoised waveform signals are useful for seismic source analysis and if they can still be reliably used in downstream analysis for further inferences on the seismic source type, i.e. seismic moment tensor analysis. The declared North Korean nuclear tests are a suitable benchmark test set, as they have extensively been researched and their source type and location might be assumed known. Verification of the source type is of particular interest for potential nuclear tests under international law. We find that care needs to be taken using the denoised waveform data, as a slight bias is introduced in the seismic moment tensor analysis. However we also find promising results hinting at possible future use of the technique for standard analyses, as it improves the investigation of smaller events. Autoencoder based denoising techniques could be employed in future routine frameworks to increase earthquake catalog completeness and possibly aid in detecting smaller potential treaty relevant events.

Ultrasound image denoising autoencoder model based on lightweight attention mechanism.

The presence of noise in medical ultrasound images significantly degrades image quality and affects the accuracy of disease diagnosis. The convolutional neural network-denoising autoencoder (CNN-DAE) model extracts feature information by stacking regularly sized kernels. This results in the loss of texture detail, the over-smoothing of the image, and a lack of generalizability for speckle noise. A lightweight attention denoise-convolutional neural network (LAD-CNN) is proposed in the present study. Two different lightweight attention blocks (i.e., the lightweight channel attention (LCA) block and the lightweight large-kernel attention (LLA) block are concatenated into the downsampling stage and the upsampling stage, respectively. A skip connection is included before the upsampling layer to alleviate the problem of gradient vanishing during backpropagation. The effectiveness of our model was evaluated using both subjective visual effects and objective evaluation metrics. With the highest peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) values at all noise levels, the proposed model outperformed the other models. In the test of brachial plexus ultrasound images, the average PSNR of our model was 0.15 higher at low noise levels and 0.33 higher at high noise levels than the suboptimal model. In the test of fetal ultrasound images, the average PSNR of our model was 0.23 higher at low noise levels and 0.20 higher at high noise levels than the suboptimal model. The statistical analysis showed that the p values were less than 0.05, which indicated a statistically significant difference between our model and the other models. The results of this study suggest that the proposed LAD-CNN model is more efficient in denoising and preserving image details than both conventional denoising algorithms and existing deep-learning algorithms.

Elimination of Random Mixed Noise in ECG using Convolutional Denoising Autoencoder with Transformer Encoder.

Electrocardiogram (ECG) signals frequently encounter diverse types of noise, such as baseline wander (BW), electrode motion (EM) artifacts, muscle artifact (MA), and others. These noises often occur in combination during the actual data acquisition process, resulting in erroneous or perplexing interpretations for cardiologists. To suppress random mixed noise (RMN) in ECG with less distortion, we propose a Transformer-based Convolutional Denoising AutoEncoder model (TCDAE) in this study. The encoder of TCDAE is composed of three stacked gated convolutional layers and a Transformer encoder block with a point-wise multi-head self-attention module. To obtain minimal distortion in both time and frequency domains, we also propose a frequency weighted Huber loss function in training phase to better approximate the original signals. The TCDAE model is trained and tested on the QT Database (QTDB) and MIT-BIH Noise Stress Test Database (NSTDB), with the training data and testing data coming from different records. All the metrics perform the most robust in overall noise and separate noise intervals for RMN removal compared with the baseline methods. We also conduct generalization tests on the Icentia11k database where the TCDAE outperforms the state-of-the-art models, with a 55% reduction of the false positives in R peak detection after denoising. The TCDAE model approximates the short-term and long-term characteristics of ECG signals and has higher stability even under extreme RMN corruption. The memory consumption and inference speed of TCDAE are also feasible for its deployment in clinical applications.

Noise Removal from the Image Using Convolutional Neural Networks-Based Denoising Auto Encoder

With the exponential growth in the volume of digital images captured daily, there is an escalating demand for elevating image quality to achieve both accuracy and visual appeal. Addressing this need, the development of techniques for reducing image noise while preserving crucial features, such as edges, corners, and sharp structures, has become imperative. This paper delves into the significance of image denoising and introduces a novel approach utilizing a denoising autoencoder based on convolutional neural networks (CNNs). The proposed method adopts a meticulous two-step process to effectively eliminate noise. Initially, input images are segregated into training and testing sets. Subsequently, a denoising autoencoder model is trained using the designated training data. This model is then further refined through training on a CNN, enhancing its noise reduction capabilities. The evaluation of the system's performance is conducted using testing data to gauge its effectiveness. The study employs the MATLAB programming language for implementation and evaluation. Results, measured through RMSE (Root Mean Square Error) and PSNR (Peak Signal-to-Noise Ratio) criteria on two distinct datasets—the Covid19-radiography-database and SIIM-medical-images—reveal that our proposed method outperforms existing approaches significantly. This approach is particularly promising for applications demanding enhanced image quality, such as the resolution enhancement of medical images. The study contributes to the ongoing efforts in noise reduction research, offering a robust solution for improving visual perception in diverse image processing applications.

Signal Separation Method for Radiation Sources Based on a Parallel Denoising Autoencoder

Radiation source signal sorting in complex environments is currently a hot issue in the field of electronic countermeasures. The pulse repetition interval (PRI) can provide stable and obvious parametric features in radiation source identification, which is an important parameter relying on the signal sorting problem. To solve the problem linked to the difficulties in sorting the PRI in complex environments using the traditional method, a signal sorting method based on a parallel denoising autoencoder is proposed. This method implements the binarized preprocessing of known time-of-arrival (TOA) sequences and then constructs multiple parallel denoising autoencoder models using fully connected layers to achieve the simultaneous sorting of multiple signal types in the overlapping signals. The experimental results show that this method maintains high precision in scenarios prone to large error and can efficiently filter out noise and highlight the original features of the signal. In addition, the present model maintains its performance and some robustness in the sorting of different signal types. Compared with the traditional algorithm, this method improves the precision of sorting. The algorithm presented in this study still maintains above 90% precision when the pulse loss rate reaches 50%.

The role of noise in denoising models for anomaly detection in medical images.

Pathological brain lesions exhibit diverse appearance in brain images, in terms of intensity, texture, shape, size, and location. Comprehensive sets of data and annotations are difficult to acquire. Therefore, unsupervised anomaly detection approaches have been proposed using only normal data for training, with the aim of detecting outlier anomalous voxels at test time. Denoising methods, for instance classical denoising autoencoders (DAEs) and more recently emerging diffusion models, are a promising approach, however naive application of pixelwise noise leads to poor anomaly detection performance. We show that optimization of the spatial resolution and magnitude of the noise improves the performance of different model training regimes, with similar noise parameter adjustments giving good performance for both DAEs and diffusion models. Visual inspection of the reconstructions suggests that the training noise influences the trade-off between the extent of the detail that is reconstructed and the extent of erasure of anomalies, both of which contribute to better anomaly detection performance. We validate our findings on two real-world datasets (tumor detection in brain MRI and hemorrhage/ischemia/tumor detection in brain CT), showing good detection on diverse anomaly appearances. Overall, we find that a DAE trained with coarse noise is a fast and simple method that gives state-of-the-art accuracy. Diffusion models applied to anomaly detection are as yet in their infancy and provide a promising avenue for further research. Code for our DAE model and coarse noise is provided at: https://github.com/AntanasKascenas/DenoisingAE.

Attention-Aided Autoencoder-Based Channel Prediction for Intelligent Reflecting Surface-Assisted Millimeter-Wave Communications

Sixth-generation (6G) wireless communication networks will provide larger coverage and capacity with lower energy consumption and hardware costs than 5G. Intelligent reflecting surface (IRS)-aided millimeter-wave massive MIMO OFDM communication is a new technology that intelligently manipulates electromagnetic waves. This has recently attracted much attention given its potential to manage the wireless propagation environment at low hardware costs and with minimal energy usage. However, channel prediction is complicated by the fact that IRS is rarely equipped with power amplifiers, various radio frequency chains, or a significant number of reflecting components. In this paper, we propose a convolutional denoising autoencoder model and investigate a joint attention mechanism for channel prediction. Then, we employ the attention mechanism to identify features of channel subcarrier interference to improve the channel prediction performance. Long-range dependent specificity is captured through the attention mechanism to generate useful features from the input signal. The encoder-decoder design of the autoencoder serves as a dimensionality reduction method that enables the autoencoder to predict the spatial and temporal distribution features of continuous signals by exploiting the extraction of sequence features from the model. Numerical results show that the proposed algorithm significantly improves the performance of IRS-aided millimeter-wave massive MIMO OFDM communication systems compared with previous methods.

RunDAE model: Running denoising autoencoder models for denoising ECG signals

Background:The denoising autoencoder (DAE) is commonly used to denoise bio-signals such as electrocardiogram (ECG) signals through dimensional reduction. Typically, the DAE model needs to be trained using correlated input segments such as QRS-aligned segments or long ECG segments. However, using long ECG segments as an input can result in a complex deep DAE model that requires many hidden layers to achieve a low-dimensional representation, which is a major drawback. Methods:This work proposes a novel DAE model, called running DAE (RunDAE), for denoising short ECG segments without relying on the R-peak detection algorithm for alignment. The proposed RunDAE model employs a sample-by-sample processing approach, considering the correlation between consecutive, overlapped ECG segments. The performance of both the classical DAE and RunDAE models with convolutional and dense layers, respectively, is evaluated using corrupted QRS-aligned and non-aligned ECG segments with physical noise such as motion artifacts, electrode movement, baseline wander, and simulated noise such as Gaussian white noise. Results:The simulation results indicate that 1. QRS-aligned segments are preferable to non-aligned segments, 2. the RunDAE model outperforms the classical DAE model in denoising ECG signals, especially when using dense layers and QRS-aligned segments, 3. training the RunDAE models with normal and arrhythmic ECG signals enhance model’s properties/capabilities, and 4. the RunDAE is a multistage, non-causal, nonlinear adaptive filter. Conclusion:A shallow learning model, which consists of a couple of hidden layers, could achieve outstanding denoising performance using only the correlation among neighboring samples.

Deep learning-based framework for real-time transient stability prediction under stealthy data integrity attacks

Cyber-attacks can degrade the performance of online transient stability prediction (TSP) and assessment functions offered for power systems. This paper proposes a cyber-resilient real-time TSP framework created by considering both perspectives, of the attacker, and of the smart grid operator. From the attacker’s viewpoint, to minimize the attack cost, a model-free data-driven algorithm for finding the measurements to which the TSP function is vulnerable is considered. The attack vector that maximizes the TSP classifier’s inaccuracy reduces the performance of TSP significantly. From the perspective of the system operator and to defend the system against data integrity attacks, two unsupervised learning algorithms which are principal component analysis and fuzzy c-means clustering, are combined and employed to detect the falsified phasor measurement units (PMUs) in the system. In the proposed framework, a denoising autoencoder model is also developed to eliminate the effect of stealthy cyber-attacks. When a cyber-attack is detected, the denoising autoencoder-based model is triggered to recover the damage of the cyber-attack, while a well-tuned long short-term memory model is implemented for the TSP application. The proposed framework is tested on two IEEE test systems, including 39-bus and 127-bus systems, with various attack scenarios. Results show that the developed system is more resilient to the attacks than the existing well-established TSP functions.

Automated atrial fibrillation classification based on denoising stacked autoencoder and optimized deep network

This study proposed an efficient ECG preprocessing technique, and the preprocessed ECG data was subsequently used to classify atrial fibrillation (AFib) using end-to-end deep neural networks. This is significant since early identification of AFib can help to prevent mortality. With this in mind, the two-fold study is proposed, in which three denoising autoencoders for ECG signal pre-processing were evaluated and compared. Denoising autoencoders (DAEs) utilising convolutional neural networks (CNNs) as backbones outperformed the other two DAE models. As a result, the preprocessed signal is then used to classify atrial fibrillation using the three classification models. The combination of CNN-based DAE and CNN-based classification model yielded the best results, with 99.20 percent accuracy, 99.50 percent specificity, 99.50 percent sensitivity, and 99.00 percent true positive rate. The average accuracy of the algorithms we investigated was 96.26 percent, and our technique was 3.2 percent more accurate than the other algorithms we compared. Furthermore, with a 24-h ECG signal processing time of only 1.3 s, the proposed model is computationally inexpensive for real-time applications. To determine its robustness, the proposed framework is tested on previously unseen datasets with varied proportions of arrhythmias, producing a 99.10 percent recall rate and a 98.50 percent accuracy.

A text style transfer system for reducing the physician–patient expertise gap: An analysis with automatic and human evaluations

Physicians and patients often come from different backgrounds and have varying levels of education, which can result in communication difficulties in the healthcare process. To address this expertise gap, we present a “Text Style Transfer” system. Our system uses Semantic Textual Similarity techniques based on Sentence Transformers models to create pseudo-parallel datasets from a large, non-parallel corpus of lay and expert texts. This approach allowed us to train a denoising autoencoder model (BART), overcoming the limitations of previous systems. Our extensive analysis, which includes both automatic metrics and human evaluations from both lay (patients) and expert (physicians) individuals, shows that our system outperforms state-of-the-art models and is comparable to human-provided gold references in some cases.

Adaptive Scheme of Denoising Autoencoder for Estimating Indoor Localization Based on RSSI Analytics in BLE Environment

In indoor environments, estimating localization using a received signal strength indicator (RSSI) is difficult because of the noise from signals reflected and refracted by walls and obstacles. In this study, we used a denoising autoencoder (DAE) to remove noise in the RSSI of Bluetooth Low Energy (BLE) signals to improve localization performance. In addition, it is known that the signal of an RSSI can be exponentially aggravated when the noise is increased proportionally to the square of the distance increment. Based on the problem, to effectively remove the noise by adapting this characteristic, we proposed adaptive noise generation schemes to train the DAE model to reflect the characteristics in which the signal-to-noise ratio (SNR) considerably increases as the distance between the terminal and beacon increases. We compared the model’s performance with that of Gaussian noise and other localization algorithms. The results showed an accuracy of 72.6%, a 10.2% improvement over the model with Gaussian noise. Furthermore, our model outperformed the Kalman filter in terms of denoising.

Deep learning-based prognostic model using non-enhanced cardiac cine MRI for outcome prediction in patients with heart failure.

To evaluate the performance of a deep learning-based multi-source model for survival prediction and risk stratification in patients with heart failure. Patients with heart failure with reduced ejection fraction (HFrEF) who underwent cardiac magnetic resonance between January 2015 and April 2020 were retrospectively included in this study. Baseline electronic health record data, including clinical demographic information, laboratory data, and electrocardiographic information, were collected. Short-axis non-contrast cine images of the whole heart were acquired to estimate the cardiac function parameters and the motion features of the left ventricle. Model accuracy was evaluated using the Harrell's concordance index. All patients were followed up for major adverse cardiac events (MACEs), and survival prediction was assessed using Kaplan-Meier curves. A total of 329 patients were evaluated (age 54 ± 14years; men, 254) in this study. During a median follow-up period of 1041days, 62 patients experienced MACEs and their median survival time was 495days. When compared with conventional Cox hazard prediction models, deep learning models showed better survival prediction performance. Multi-data denoising autoencoder (DAE) model reached the concordance index of 0.8546 (95% CI: 0.7902-0.8883). Furthermore, when divided into phenogroups, the multi-data DAE model could significantly discriminate between the survival outcomes of the high-risk and low-risk groups compared with other models (p < 0.001). The proposed deep learning (DL) model based on non-contrast cardiac cine magnetic resonance imaging could independently predict the outcome of patients with HFrEF and showed better prediction efficiency than conventional methods. The proposed multi-source deep learning model based on cardiac magnetic resonance enables survival prediction in patients with heart failure. • A multi-source deep learning model based on non-contrast cardiovascular magnetic resonance (CMR) cine images was built to make robust survival prediction in patients with heart failure. • The ground truth definition contains electronic health record data as well as DL-based motion data, and cardiac motion information is extracted by optical flow method from non-contrast CMR cine images. • The DL-based model exhibits better prognostic value and stratification performance when compared with conventional prediction models and could aid in the risk stratification in patients with HF.

Cluster-based denoising autoencoders for rate prediction recommender systems

Recommender system (RS) is a suitable tool for filtering out items and providing the most relevant and suitable items to each user, based on their individual preferences. Deep learning algorithms achieve great success in several fields including RS. The issue with deep learning-based RS models is that, they ignore the differences of users’ preferences, and they build a model based on all the users’ rates. This paper proposed an optimized clustering-based denoising autoencoder model (OCB-DAE) which trains multiple models instead of one, based on users’ preferences using k-means algorithm combined with a nature-inspired algorithm (NIA) such as artificial fish swarm algorithm to determine the optimal initial centroids to cluster the users based on their similar preferences, and each cluster trains its own denoising autoencoder (DAE) model. The results proved that combining NIA with k-means gives better clustering results comparing with using k-means alone. OCB-DAE was trained and tested with MovieLens 1M dataset where 80% of it is used for training and 20% for testing. Root mean squared error (RMSE) score was used to evaluate the performance of the proposed model which was 0.618. It outperformed the other models that use autoencoder and denoising autoencoder without clustering with 38.5% and 29.5% respectively.

RDERL: Reliable deep ensemble reinforcement learning-based recommender system

Recommender systems (RSs) have been employed for many real-world applications including search engines, social networks, and information retrieval systems as powerful intelligent techniques for creating personalized recommendations. RSs mainly utilize different data resources such as user-item ratings, tag data, and social relationships in their recommendation process to overcome the data sparsity issue. One of the main challenges is how to integrate the results obtained by these data resources. Moreover, most of the existing RSs focus on the recommendation accuracy, and do not consider the reliability of the recommendations. To address these challenges, in this study, a reliable recommendation method is developed, which employs deep neural networks and reinforcement learning. The proposed method firstly uses a stacked denoising autoencoder model to obtain the latent features from the rating, tag, and trust data. The similarity value between each pair of users is then calculated in three different aspects according to the extracted latent features. An ensemble approach is developed by integrating the three similarities using reinforcement learning to determine the final similarity value. The ensemble similarity values are used in the rating prediction process, and the reliability of the predicted ratings is evaluated using an effective measure. Finally, a recommendation strategy is developed based on the integration of the predicted ratings and their reliability values. We evaluated the effectiveness of the proposed method by conducting numerous experiments on three datasets where the results show that the proposed method is far superior to other recommendation models in making more accurate recommendations.

Parameter Tuned Machine Learning Based Emotion Recognition on Arabic Twitter Data

Arabic is one of the most spoken languages across the globe. However, there are fewer studies concerning Sentiment Analysis (SA) in Arabic. In recent years, the detected sentiments and emotions expressed in tweets have received significant interest. The substantial role played by the Arab region in international politics and the global economy has urged the need to examine the sentiments and emotions in the Arabic language. Two common models are available: Machine Learning and lexicon-based approaches to address emotion classification problems. With this motivation, the current research article develops a Teaching and Learning Optimization with Machine Learning Based Emotion Recognition and Classification (TLBOML-ERC) model for Sentiment Analysis on tweets made in the Arabic language. The presented TLBOML-ERC model focuses on recognising emotions and sentiments expressed in Arabic tweets. To attain this, the proposed TLBOML-ERC model initially carries out data pre-processing and a Continuous Bag Of Words (CBOW)-based word embedding process. In addition, Denoising Autoencoder (DAE) model is also exploited to categorise different emotions expressed in Arabic tweets. To improve the efficacy of the DAE model, the Teaching and Learning-based Optimization (TLBO) algorithm is utilized to optimize the parameters. The proposed TLBOML-ERC method was experimentally validated with the help of an Arabic tweets dataset. The obtained results show the promising performance of the proposed TLBOML-ERC model on Arabic emotion classification.

Semi-supervised health assessment of pipeline systems based on optical fiber monitoring

Health assessment of pipeline systems is of deep significance for improving pipeline reliability and integrity. Traditional health assessment methods may be difficult or costly to perform on pipeline systems due to the long distances and environmental constraints of pipelines. This paper incorporates the distributed optical fiber sensor (DOFS) technique and the semi-supervised learning algorithm into the pipeline health assessment framework. Three critical problems that limit the application of DOFS in pipeline health assessments are addressed. First, an applicable damage monitoring experiment of a pipeline system is designed, which is effective in obtaining the necessary base data for data-driven modeling. Second, the correspondence between the pipeline health status and the monitored strain features is established. The experimental data are shared for public research, which is expected to solve the problem of the lack of benchmark research data in related fields. Third, considering the scarcity of labeled degradation data in pipelines, a semi-supervised denoising autoencoder model is proposed specifically for pipeline health assessment. The proposed method is demonstrated and validated using a comparative experimental case study.

Visualized Stacked Denoising Auto-Encoder Model for Extracting and Evaluating the State Features of Rolling Bearings

Extracting intuitive operating state features from vibration signals without prior knowledge is a prospective requirement for health monitoring and fault diagnosis in bearings. In this paper, a visualized stacked denoising auto-encoder (VSDAE) model is proposed for the unsupervised extraction and quantitative evaluation of bearings’ state features. First, the stacked denoising auto-encoder (SDAE) was used to reconstruct vibration signals. The intermediate vector of the SDAE, which is a high-information-density representation of vibration signals, was regarded as the pending state feature. Then, the dimension of the intermediate vector was reduced by the t-distributed stochastic neighbor embedding (t-SNE) method to the two-dimensional visualization space. Finally, the silhouette coefficient of feature distribution was calculated to quantitatively evaluate the extracted features. The proposed model was evaluated using experimental bearing signals simulating various operating states. The results proved that the features, extracted and evaluated by the VSDAE, allowed the recognition of the operating states of the examined bearings.

Machine Learning for analysis of speckle dynamics: quantification and outlier detection

X-ray photon correlation spectroscopy (XPCS) provides an understanding of complex dynamics in materials that are tied to their synthesis, properties, and behaviors. Analysis of XPCS data for dynamics that are far from equilibrium is labor intense and often can impede the discovery process, especially in experiments with high collection rates. Moreover, binning and averaging, involved in the analysis for alleviating poor signal-to-noise ratio, leads to a loss of temporal resolution and the accumulation of systematic error for the parameters quantifying the dynamics. Here, we integrate a denoising autoencoder model into workflows for the analysis of nonequilibrium two-time intensity-intensity correlation functions. Noise reduction allows for extracting the parameters that characterize the sample's dynamics with the temporal resolution limited only by frame rates. Not only does it improve the quantitative usage of the data, but it also creates the potential for automating the analytical workflow, which is a key to high-throughput or autonomous XPCS experiments. Various approaches for the uncertainty quantification and extension of the model for anomalies' detection are discussed.