Convolutional Denoising Autoencoder Research Articles

Integrated denoising and extraction of both temperature and strain based on a single CNN framework for a BOTDA sensing system.

We have proposed and demonstrated a denoising and extraction convolutional neural network (DECNN) composed of 1D denoising convolutional autoencoder (DCAE) and 1D residual attention network (RANet) modules to extract temperature and strain simultaneously in a Brillouin optical time-domain analysis (BOTDA) system. With DCAE for high-fidelity denoising and RANet for accurate and robust information extraction, integrated denoising and extraction of both temperature and strain have been realized for the first time under a single CNN framework. Both simulation and experiment have been conducted to statistically analyze the performance of the proposed scheme and compare it with the conventional equation solving method (CESM), which show that DECNN has large noise tolerance and robustness over a wide range of temperature/strain and signal-to-noise ratio (SNR) conditions. The mean standard deviation (SD) and root mean square error (RMSE) of the temperature/strain extracted by DECNN over a wide range of SNRs are only 0.2°C/9.7µɛ and 2°C/32.3µɛ at the end of 19.38 km long sensing fiber, respectively. At a relatively low SNR of 8.8 dB, DECNN shows 196 times better temperature/strain uncertainty and 146 times faster processing speed when compared with CESM.

Optical Fiber Fault Detection and Localization in a Noisy OTDR Trace Based on Denoising Convolutional Autoencoder and Bidirectional Long Short-Term Memory

Optical time-domain reflectometry (OTDR) has been widely used for characterizing fiber optical links and for detecting and locating fiber faults. OTDR traces are prone to be distorted by different kinds of noise, causing blurring of the backscattered signals, and thereby leading to a misleading interpretation and a more cumbersome event detection task. To address this problem, a novel method combining a denoising convolutional autoencoder (DCAE) and a bidirectional long short-term memory (BiLSTM) is proposed, whereby the former is used for noise removal of OTDR signals and the latter for fault detection, localization, and diagnosis with the denoised signal as input. The proposed approach is applied to noisy OTDR signals of different levels of input SNR ranging from -5 dB to 15 dB. The experimental results demonstrate that: (i) the DCAE is efficient in denoising the OTDR traces and it outperforms other deep learning techniques and the conventional denoising methods; and (ii) the BiLSTM achieves a high detection and diagnostic accuracy of 96.7% with an improvement of 13.74% compared to the performance of the same model trained with noisy OTDR signals.

A hybrid deep-learning model for fault diagnosis of rolling bearings in strong noise environments

Strong noise in practical engineering environments interferes with the signal of a rolling bearing, which leads to the decline of the diagnosis accuracy of intelligent diagnosis models. This paper proposes a novel hybrid model (a convolutional denoising auto-encoder (CDAE)-BLCNN) to address this problem. First, the rolling bearing vibration signal containing noise was input into the CDAE, which denoises the signal through unsupervised learning and then outputs the reconstructed data. Secondly, a hybrid neural network (BLCNN), composed of a multi-scale wide convolution neural network and a bidirectional long-short-term memory network, was used to extract intrinsic fault features from the reconstructed signal and diagnose fault types. The analysis results demonstrate that the proposed hybrid deep-learning model achieves higher detection accuracy, even under different noise levels and various rotating speeds. Compared with other models, there is a high fault recognition rate, robustness, and generalization ability, which may be favorable to practical applications.

Fault diagnosis of rolling bearings based on generative adversarial network and convolutional denoising auto-encoder

The vibration signals of rolling bearings are susceptible to strong noise interference. In addition, the lacking of fault samples for rolling bearings increases the difficulty of fault diagnosis. A fault diagnosis model based on conditional generative adversarial network (CGAN) and convolutional denoising auto-encoder (CDAE) is proposed to solve these problems. CGAN is used to generate new samples with the same distribution as the real samples. In order to improve the anti-noise ability of the model, we use CDAE as the discriminator model of CGAN to extract more robust features and achieve more accurate discrimination and classification. The generator and the discriminator are optimized by the adversarial mechanism to improve the quality of sample generation and the accuracy of fault classification. The experimental results show that the CGAN-CDAE model has good anti-noise ability, and achieves good fault diagnosis performance of rolling bearings in the case of small samples and class imbalance.

Efficient deep-learning-based history matching for fluvial channel reservoirs

In history matching, the calibration of a prior reservoir model is computationally expensive because many forward reservoir simulation runs are required. Multiple posterior (or calibrated) reservoir models need to be sampled to consider high reservoir uncertainty, which increases the computational cost significantly. In this study, we propose a novel deep-learning-based history matching method that efficiently samples posterior reservoir models for fluvial channel reservoirs. Three convolution-based neural networks (NNs) are used in the proposed method to sample posterior models quickly without conventional calibration processes: convolutional autoencoder (CAE), convolutional neural network (CNN), and convolutional denoising autoencoder (CDAE). First, low-dimensional latent features are extracted from prior models using CAE because the dimensionality of static data is too high to find the relation between the prior models and corresponding simulated dynamic (production) data. Next, CNN is used to find the relation between the latent features of the prior models and the corresponding production data, which are the output and input data of CNN, respectively. The CNN output is refined using CDAE to improve the geological connectivity of the posterior models. The performance of the proposed method is compared with non-convolution-based methods that combine fully-connected NN structures (multi-layer perceptron (MLP)) and dimension-reduction techniques (principal component analysis (PCA) and stacked autoencoder (SAE)) in the benchmark egg model. The proposed method outperforms the other methods (MLP-PCA and MLP-SAE) in terms of geological constraints for fluvial channels and the computational cost of sampling posterior models. • CAE compresses heterogeneous rock properties efficiently. • Posterior models can be sampled quickly using CAE and CNN. • The geological connectivity of posterior models can be improved using CDAE.

Intelligent Process Monitoring of Laser-Induced Graphene Production With Deep Transfer Learning

Three-dimensional graphene has been increasingly used in many applications due to its superior properties. The laser-induced graphene (LIG) technique is an effective way to produce 3-D graphene by combining graphene preparation and patterning into a single step using direct laser writing. However, the variation in process parameters and environment could largely affect the formation and crystallization quality of 3-D graphene. This article develops a vision and deep transfer learning-based processing monitoring system for LIG production. To solve the problem of limited labeled data, novel convolutional de-noising auto-encoder (CDAE)-based unsupervised learning is developed to utilize the available unlabeled images. The learned weights from CDAE are then transferred to a Gaussian convolutional deep belief network (GCDBN) model for further fine-tuning with a very small amount of labeled images. The experimental results show that the proposed method can achieve the state-of-art performance of precise and robust monitoring for the quality of the LIG formation.

Chaotic Signal Denoising Based on Simplified Convolutional Denoising Auto-Encoder

Chaotic Signal Denoising Based on Simplified Convolutional Denoising Auto-Encoder

Unsupervised defect detection algorithm for printed fabrics using content-based image retrieval techniques

Automatic detection of printing defect technology is significant for improving printing fabrics’ appearance and quality. In this research, we proposed an unsupervised printing defect detection method by processing the difference map between the test image and the reference image. Aimed at this, we adopted a content-based image retrieval (CBIR) method to retrieve the reference image, which includes an image database, a convolutional denoising auto-encoder (CDAE) and a hash encoder (HE): the elements of image database are extracted from only one defect-free sample image of the test fabric; the CDAE prevents the system being affected by the texture of the fabric and provides a reliable feature description of the patterns; the HE indexes the feature vectors to binary code while maintaining their similarity; both CDAE and HE are trained in an unsupervised manner. With the retrieved reference image, the defect is determined by applying the Tsallis entropy thresholding and opening operation on the difference map. The method can be implemented without labeled and defective samples, and without consideration of the periodical primitive of patterns. Experimental results demonstrate the effectiveness and efficiency of the proposed method in defect detection for printed fabrics with complex patterns.

Deep Convolutional Denoising Autoencoders with Network Structure Optimization for the High-Fidelity Attenuation of Random GPR Noise

The high-fidelity attenuation of random ground penetrating radar (GPR) noise is important for enhancing the signal-noise ratio (SNR). In this paper, a novel network structure for convolutional denoising autoencoders (CDAEs) was proposed to effectively resolve various problems in the noise attenuation process, including overfitting, the size of the local receptive field, and representational bottlenecks and vanishing gradients in deep learning; this approach also significantly improves the noise attenuation performance. We described the noise attenuation process of conventional CDAEs, and then presented the output feature map of each convolutional layer to analyze the role of convolutional layers and their impacts on GPR data. Furthermore, we focused on the problems of overfitting, the local receptive field size, and the occurrence of representational bottlenecks and vanishing gradients in deep learning. Subsequently, a network structure optimization strategy, including a dropout regularization layer, an atrous convolution layer, and a residual-connection structure, was proposed, namely convolutional denoising autoencoders with network structure optimization (CDAEsNSO), comprising an intermediate version, called atrous-dropout CDAEs (AD-CDAEs), and a final version, called residual-connection CDAEs (ResCDAEs), all of which effectively improve the performance of conventional CDAEs. Finally, CDAEsNSO was applied to attenuate noise for the H-beam model, tunnel lining model, and field pipeline data, confirming that the algorithm adapts well to both synthetic and field data. The experiments verified that CDAEsNSO not only effectively attenuates strong Gaussian noise, Gaussian spike impulse noise, and mixed noise, but it also causes less damage to the original waveform data and maintains high-fidelity information.

CCpos: WiFi Fingerprint Indoor Positioning System Based on CDAE-CNN.

WiFi is widely used for indoor positioning because of its advantages such as long transmission distance and ease of use indoors. To improve the accuracy and robustness of indoor WiFi fingerprint localization technology, this paper proposes a positioning system CCPos (CADE-CNN Positioning), which is based on a convolutional denoising autoencoder (CDAE) and a convolutional neural network (CNN). In the offline stage, this system applies the K-means algorithm to extract the validation set from the all-training set. In the online stage, the RSSI is first denoised and key features are extracted by the CDAE. Then the location estimation is output by the CNN. In this paper, the Alcala Tutorial 2017 dataset and UJIIndoorLoc are adopted to verify the performance of the CCpos system. The experimental results show that our system has excellent noise immunity and generalization performance. The mean positioning errors on the Alcala Tutorial 2017 dataset and the UJIIndoorLoc are 1.05 m and 12.4 m, respectively.

Signal‐to‐noise ratio enhancement for Raman spectra based on optimized Raman spectrometer and convolutional denoising autoencoder

AbstractThe signal‐noise ratio plays a key role in acquiring plentiful chemical structural information in the Raman spectrometer. The miniature spectrometer is generally compact at the expense of performance. In this work, we proposed a compact, signal‐to‐noise ratio (SNR) enhancement of the Raman spectrometer by the optimization of optical structure and a noise reduction method. Concerning its optical structure, the Raman spectrometer is increasing the intensity by adding an off‐the‐shelf cylindrical lens. On the other side of the algorithm, a relevant automatic denoising method of convolutional denoising autoencoder (CDAE) is proposed to further advance the SNR in Raman spectra without manual intervention. The results indicate the performance of the compact Raman spectrometer could increase to a certain extent by testing with 785 nm laser and Ne/Ar source. Besides, by using CDAE to deal with contaminated Raman spectra, a higher SNR is obtained. The results demonstrate that the improvement of the hardware and algorithm is effective for removing the noisy Raman signal and achieving higher SNR. This result may be helpful in further improving the performance of integrated Raman spectrometers and research on miniaturized instruments.

Deep Spatio-Temporal Representation and Ensemble Classification for Attention Deficit/Hyperactivity Disorder.

Attention deficit/Hyperactivity disorder (ADHD) is a complex, universal and heterogeneous neurodevelopmental disease. The traditional diagnosis of ADHD relies on the long-term analysis of complex information such as clinical data (electroencephalogram, etc.), patients' behavior and psychological tests by professional doctors. In recent years, functional magnetic resonance imaging (fMRI) has been developing rapidly and is widely employed in the study of brain cognition due to its non-invasive and non-radiation characteristics. We propose an algorithm based on convolutional denoising autoencoder (CDAE) and adaptive boosting decision trees (AdaDT) to improve the results of ADHD classification. Firstly, combining the advantages of convolutional neural networks (CNNs) and the denoising autoencoder (DAE), we developed a convolutional denoising autoencoder to extract the spatial features of fMRI data and obtain spatial features sorted by time. Then, AdaDT was exploited to classify the features extracted by CDAE. Finally, we validate the algorithm on the ADHD-200 test dataset. The experimental results show that our method offers improved classification compared with state-of-the-art methods in terms of the average accuracy of each individual site and all sites, meanwhile, our algorithm can maintain a certain balance between specificity and sensitivity.

Reduced Biquaternion Stacked Denoising Convolutional AutoEncoder for RGB-D Image Classification

RGB-D image classification based on convolutional neural networks have been extensively explored recently. However, they suffer from problems of effective representation of RGB-D image, intra-class variances and inter-class similarities. To address these problems, this letter proposes a novel RGB-D image classification framework based on reduced biquaternion stacked denoising convolutional autoencoder (RQ-SDCAE). The proposed framework can encode and extract the depth feature effectively by using the reduced biquaternion. The stacked training method is utilized to train the proposed reduced biquaternion convolutional autoencoder. Extensive evaluations for RGB-D image classification demonstrate that RQ-SDCAE outperforms the state-of-the-art methods.

Convolutional Autoencoder-Based Flaw Detection for Steel Wire Ropes.

Visual perception-based methods are a promising means of capturing the surface damage state of wire ropes and hence provide a potential way to monitor the condition of wire ropes. Previous methods mainly concentrated on the handcrafted feature-based flaw representation, and a classifier was constructed to realize fault recognition. However, appearances of outdoor wire ropes are seriously affected by noises like lubricating oil, dust, and light. In addition, in real applications, it is difficult to prepare a sufficient amount of flaw data to train a fault classifier. In the context of these issues, this study proposes a new flaw detection method based on the convolutional denoising autoencoder (CDAE) and Isolation Forest (iForest). CDAE is first trained by using an image reconstruction loss. Then, it is finetuned to minimize a cost function that penalizes the iForest-based flaw score difference between normal data and flaw data. Real hauling rope images of mine cableways were used to test the effectiveness and advantages of the newly developed method. Comparisons of various methods showed the CDAE-iForest method performed better in discriminative feature learning and flaw isolation with a small amount of flaw training data.

Heterogeneous Multi-Task Learning for Multiple Pseudo-Measurement Estimation to Bridge GPS Outages

To enhance the performance of the inertial navigation system (INS)/global position system (GPS) integrated navigation system for the land vehicle during GPS outages is an extremely challenging task. Though existing researches have made reasonable progress in positioning accuracy, they largely ignore sophisticated vehicle stopping events, and the further improvement of positioning performance is urgently needed in complex urban environments. In this article, we propose a heterogeneous multi-task learning (MTL) structure with a shared de-noising process to conduct pseudo-GPS position prediction and zero-velocity detection. The raised model builds upon three vital parts: 1) a shared de-noising convolutional autoencoder (CAE), which can effectively filter the measurement noises in the original inputs and provide more clean data for subsequent calculations without the ground-truth sensor data; 2) a predictor that uses a deep temporal convolutional network (TCN) to predict pseudo-GPS position to bridge GPS gaps; and 3) a robust zero-velocity detector that utilizes a 1-D deep convolutional neural network to accurately detect the vehicle stationary pattern, allowing for timely correcting the velocity and heading. Our proposed MTL model is evaluated on extensive practical road data and achieves a root mean square error of 3.794 m for 120-s GPS outages under long-term vehicle stopping scenarios, which obviously outperforms the stand-alone long short-term memory, TCN, and TCN + CAE. Experimental results also demonstrate that our proposed MTL method yields a remarkable accuracy of over 99.0% for vehicle stationary detection.

Combining Denoising Autoencoders and Dynamic Programming for Acoustic Detection and Tracking of Underwater Moving Targets.

Accurate detection and tracking of moving targets in underwater environments pose significant challenges, because noise in acoustic measurements (e.g., SONAR) makes the signal highly stochastic. In continuous marine monitoring a further challenge is related to the computational complexity of the signal processing pipeline—due to energy constraints, in off-shore monitoring platforms algorithms should operate in real time with limited power consumption. In this paper, we present an innovative method that allows to accurately detect and track underwater moving targets from the reflections of an active acoustic emitter. Our system is based on a computationally- and energy-efficient pre-processing stage carried out using a deep convolutional denoising autoencoder (CDA), whose output is then fed to a probabilistic tracking method based on the Viterbi algorithm. The CDA is trained on a large database of more than 20,000 reflection patterns collected during 50 designated sea experiments. System performance is then evaluated on a controlled dataset, for which ground truth information is known, as well as on recordings collected during different sea experiments. Results show that, compared to the benchmark, our method achieves a favorable trade-off between detection and false alarm rate, as well as improved tracking accuracy.

Missing-Insensitive Short-Term Load Forecasting Leveraging Autoencoder and LSTM

In most deep learning-based load forecasting, an intact dataset is required. Since many real-world datasets contain missing values for various reasons, missing imputation using deep learning is actively studied. However, missing imputation and load forecasting have been considered independently so far. In this article, we provide a deep learning framework that jointly considers missing imputation and load forecasting. We consider a family of autoencoder/long short-term memory (LSTM) combined models for missing-insensitive load forecasting. Specifically, autoencoder (AE), denoising autoencoder (DAE), convolutional autoencoder (CAE), and denoising convolutional autoencoder (DCAE) are considered for extracting features, of which the encoded outputs are fed into the input of LSTM. Our experiments show that the proposed DCAE/LSTM combined model significantly improves forecasting accuracy no matter what missing rate or type (random missing, consecutive block missing) occurs compared to the baseline LSTM.

Separating the EoR signal with a convolutional denoising autoencoder: a deep-learning-based method

When applying the foreground removal methods to uncover the faint cosmological signal from the epoch of reionization (EoR), the foreground spectra are assumed to be smooth. However, this assumption can be seriously violated in practice since the unresolved or mis-subtracted foreground sources, which are further complicated by the frequency-dependent beam effects of interferometers, will generate significant fluctuations along the frequency dimension. To address this issue, we propose a novel deep-learning-based method that uses a 9-layer convolutional denoising autoencoder (CDAE) to separate the EoR signal. After being trained on the SKA images simulated with realistic beam effects, the CDAE achieves excellent performance as the mean correlation coefficient ($\bar{\rho}$) between the reconstructed and input EoR signals reaches $0.929 \pm 0.045$. In comparison, the two representative traditional methods, namely the polynomial fitting method and the continuous wavelet transform method, both have difficulties in modelling and removing the foreground emission complicated with the beam effects, yielding only $\bar{\rho}_{\text{poly}} = 0.296 \pm 0.121$ and $\bar{\rho}_{\text{cwt}} = 0.198 \pm 0.160$, respectively. We conclude that, by hierarchically learning sophisticated features through multiple convolutional layers, the CDAE is a powerful tool that can be used to overcome the complicated beam effects and accurately separate the EoR signal. Our results also exhibit the great potential of deep-learning-based methods in future EoR experiments.

Supervised and Unsupervised End-to-End Deep Learning for Gene Ontology Classification of Neural In Situ Hybridization Images.

In recent years, large datasets of high-resolution mammalian neural images have become available, which has prompted active research on the analysis of gene expression data. Traditional image processing methods are typically applied for learning functional representations of genes, based on their expressions in these brain images. In this paper, we describe a novel end-to-end deep learning-based method for generating compact representations of in situ hybridization (ISH) images, which are invariant-to-translation. In contrast to traditional image processing methods, our method relies, instead, on deep convolutional denoising autoencoders (CDAE) for processing raw pixel inputs, and generating the desired compact image representations. We provide an in-depth description of our deep learning-based approach, and present extensive experimental results, demonstrating that representations extracted by CDAE can help learn features of functional gene ontology categories for their classification in a highly accurate manner. Our methods improve the previous state-of-the-art classification rate (Liscovitch, et al.) from an average AUC of 0.92 to 0.997, i.e., it achieves 96% reduction in error rate. Furthermore, the representation vectors generated due to our method are more compact in comparison to previous state-of-the-art methods, allowing for a more efficient high-level representation of images. These results are obtained with significantly downsampled images in comparison to the original high-resolution ones, further underscoring the robustness of our proposed method.

Fault Diagnosis of Rotating Machinery under Noisy Environment Conditions Based on a 1-D Convolutional Autoencoder and 1-D Convolutional Neural Network.

Deep learning methods have been widely used in the field of intelligent fault diagnosis due to their powerful feature learning and classification capabilities. However, it is easy to overfit depth models because of the large number of parameters brought by the multilayer-structure. As a result, the methods with excellent performance under experimental conditions may severely degrade under noisy environment conditions, which are ubiquitous in practical industrial applications. In this paper, a novel method combining a one-dimensional (1-D) denoising convolutional autoencoder (DCAE) and a 1-D convolutional neural network (CNN) is proposed to address this problem, whereby the former is used for noise reduction of raw vibration signals and the latter for fault diagnosis using the de-noised signals. The DCAE model is trained with noisy input for denoising learning. In the CNN model, a global average pooling layer, instead of fully-connected layers, is applied as a classifier to reduce the number of parameters and the risk of overfitting. In addition, randomly corrupted signals are adopted as training samples to improve the anti-noise diagnosis ability. The proposed method is validated by bearing and gearbox datasets mixed with Gaussian noise. The experimental result shows that the proposed DCAE model is effective in denoising and almost causes no loss of input information, while the using of global average pooling and input-corrupt training improves the anti-noise ability of the CNN model. As a result, the method combined the DCAE model and the CNN model can realize high-accuracy diagnosis even under noisy environment.