Deep Autoencoder Network Research Articles

Multisensor Feature Fusion Based Rolling Bearing Fault Diagnosis Method

To fully utilize the fault information and improve the diagnosis accuracy of rolling bearings, a multisensor feature fusion method is proposed. The method contains two steps. First, the intrinsic mode function (IMF) of each sensor vibration signal is calculated by variational mode decomposition (VMD), and the redundant information such as noise is eliminated. Then, the time-domain, frequency-domain and multiscale entropy features are extracted based on the preferred IMF and fused into one multidomain feature dataset. In the second step, the deep autoencoder network (DAEN) is constructed and the multisensor fusion features of the first step are used as input of the DAEN, and the multisensor fusion features are further extracted and classified. The experimental results show that the proposed model has a higher classification accuracy compared with the existing methods.

Robust deep auto-encoding network for real-time anomaly detection at nuclear power plants

Detecting anomaly conditions in nuclear reactor is a critical issue in safety management of Nuclear Power Plants (NPPs). Conventionally, the operating status are monitored in transient data with pre-designed labels by human operators or basic diagnosis systems. Nowadays, continuous time series data from multi-sensors are increasingly collected and emerging unlabeled abnormal status are monitored during the operation, making it challenging to capture both spatial and temporal dependency at each time steps without supervised labels. In this paper, a robust unsupervised Multi-Variate Convolutional GRU Encoder-Dncoder (MVCGED) method is proposed to perform anomaly detection and fault diagnosis in multi-sensor operation time series data. Specifically, MVCGED first construct each time steps into signature matrices to maintain both spatial and temporal features via sliding windows with inner-correlation and forget mechanism. Subsequently, A CNN feature extraction network, CNN-based GRU encoding network and CNN decoding network are implemented successively to capture and reconstruct the hidden patterns of the signature matrices. Finally, the reconstruction loss are further utilized to detect anomalies and diagnose faults. Extensive empirical studies based on PCTRAN nuclear power plant operation data demonstrate that MVCGED outperforms commonly-used baseline methods.

DAE-TPGM: A deep autoencoder network based on a two-part-gamma model for analyzing single-cell RNA-seq data

Single-cell RNA sequencing (scRNA-seq) can reveal differences in genetic material at the single-cell level and is widely used in biomedical studies. However, the minute RNA content within individual cells often results in a high number of dropouts and introduces random noise of scRNA-seq data, concealing the original gene expression pattern. Therefore, data normalization is critical in the analysis pipeline to adjust for unexpected biological and technical effects, leading to a particular bimodal expression pattern exhibited in the semi-continuous normalized data. We further find the positive continuous expression presents a right-skewed distribution, which is still under-explored by mainstream dimensionality reduction and imputation methods. We introduced a deep autoencoder network based on a two-part-gamma model (DAE-TPGM) for joint dimensionality reduction and imputation of scRNA-seq data. DAE-TPGM uses a two-part-gamma model to capture the statistical characteristics of semi-continuous normalized data and adaptively explores the potential relationships between genes for promoting data imputation by deep autoencoder. Just as the classic application scenarios that use an autoencoder in dimensionality reduction, our personalized autoendoer can capture phenotypic information on the peripheral blood mononuclear cells (PBMC) better and clearly infer continuous phenotype information for hematopoiesis in mice. Compared with that of mainstream imputation methods such as MAGIC, SAVER, scImpute and DCA, the new model achieved substantial improvement on the recognition of cellular phenotypes in two real datasets, and the comprehensive analyses on synthetic “ground truth” data demonstrated that our method obtains competitive advantages over other imputation methods in discovering underlying gene expression patterns in time-course data.

Unsupervised clustering of Roman potsherds via Variational Autoencoders

In this paper we propose an artificial intelligence imaging solution to support archaeologists in the classification task of Roman commonware potsherds. Usually, each potsherd is represented by its sectional profile as a two dimensional black–white image and printed in archaeological books related to specific archaeological excavations. The partiality and handcrafted variance of the fragments make their matching a challenging problem: we propose to pair similar profiles via the unsupervised hierarchical clustering of non-linear features learned in the latent space of a deep convolutional Variational Autoencoder (VAE) network. Our contribution also include the creation of a ROman COmmonware POTtery (ROCOPOT) database, with more than 4000 potsherds profiles extracted from 25 Roman pottery corpora, and a MATLAB GUI software for the easy inspection of shape similarities. Results are commented both from a mathematical and archaeological perspective so as to unlock new research directions in both communities.

Analysis of the Effectiveness of Various Machine Learning, Artificial Neural Network and Deep Learning Methods in Detecting Fraudulent Credit Card Transactions

A credit card is an important financial tool that has emerged in parallel with the developments in technology from the past to the present and has become an indispensable part of human life. The credit card has many advantages that can be listed as facilitating online shopping, providing installments in purchases, and preventing cash dependence. This is why the rate of use of credit cards worldwide is increasing day by day. On the other hand, there are some risks of the credit cards highlighted by security concerns. The fraudsters who access the identity and credit card information of the consumers through different means use it to shop online without the consumer’s knowledge and gain an unfair advantage. Therefore, it is crucial to eliminate this security vulnerability that the fraudsters exploit and to develop an effective solution to the customer victimization experienced by e-commerce companies due to the fraudulent credit card transactions. With this motivation, the performance of the methods from different research fields was examined to explore the solution space in detail in terms of the problem at hand within the scope of this study. For this purpose, three machine learning algorithms (K-Nearest Neighbor, Naive Bayes, Support Vector Machine), two artificial neural network algorithms (Binary Classifier, Autoencoder), and two deep learning algorithms (Deep Autoencoder and Deep Neural Network Classifier) were implemented. The effectiveness of the algorithms in question was tested with a famous dataset widely used in the literature. Experimental results showed that the Deep Neural Network Classifier outperformed the other algorithms used in this study and the best study ever reported in the literature in detecting fraudulent credit card transactions when accuracy and AUROC performance criteria were taken into account.

Deep cross-view autoencoder network for multi-view learning

Deep cross-view autoencoder network for multi-view learning

Flight maneuver intelligent recognition based on deep variational autoencoder network

The selection and training of aircraft pilots has high standards, long training cycles, high resource consumption, high risk, and high elimination rate. It is the particularly urgent and important requirement for the current talent training strategy of national and military to increase efficiency and speed up all aspects of pilot training, reduce the training cycle and reduce the elimination rate. To this end, this paper uses deep variational auto-encoder network and adaptive dynamic time warping algorithms as support to explore the establishment of an integrated evaluation system for flight maneuver recognition and quality evaluation, solve the industry difficulty faced by current flight training data mining applications, and achieve accurate recognition and reliable quality evaluation of flight regimes under the background of high mobility. It will fully explore the benefits of existing airborne flight data for military trainee pilots, support the personalized and accurate training of flight talents, and reduce the rate of talent elimination.

Deep auto-encoder network in predictive design of Helmholtz resonator: On-demand prediction of sound absorption peak

In this paper, a large sample database is established by using a classical Helmholtz resonator, and the acoustic performance parameters and geometric structural parameters are trained by using the deep auto-encoder (DAE) model based on machine learning. Next, 20 groups of target peak points (αE, fE) are arbitrary selected inside and outside the sample database, and the corresponding geometric structural parameters (dn, dc, hp, hn) are obtained by training the DAE model, and the predicted peak points (αD, fD) are obtained. By comparing the target peak points (αE, fE) with the predicted peak points (αD, fD), the verification results show that the DAE model has very high prediction accuracy for samples from the database, while the prediction accuracy for samples out of the database is slightly lower. The proposed method represents a transformation of the design paradigm of functional acoustic devices, which can be further applied to the predictive design of more complex acoustic structures, and has great application potential in the field of vibration and noise reduction.

Make the Rocket Intelligent at IoT Edge: Stepwise GAN for Anomaly Detection of LRE With Multisource Fusion

Anomaly detection (AD) for liquid rocket engine (LRE) is essential to improve the reliability and safety of space launch missions. However, it is difficult for existing methods to implement fast and precise detection with single-source information and absent anomalous samples. The Internet of Things (IoT) enables intelligent LRE AD via edge computing using multisource data. This article proposed an edge-based detection algorithm named stepwise generative adversarial network (StepGAN) for AD of LRE with multisource fusion at the Edge of IoT. StepGAN incorporates deep autoencoder and relativistic generative adversarial network. Through multistage stepwise training with unlabeled normal data, an encoder-generator-discriminator network is obtained to identify LRE anomalies at the edge. In addition, by fusing multisource information at feature level and aggregating neighboring information at decision level during real-time detection, the performance of StepGAN is further improved. To demonstrate its ability for IoT edge-based AD, the proposed method is compared with typical methods, using real static ignition data of LRE as cloud and edge data, and achieves the best result under multiple evaluation indexes. To evaluate the proposed fusing approach, a channel-varying case is carried out and extensive experimental results indicate its effectiveness.

Autoencoders for semivisible jet detection

The production of dark matter particles from confining dark sectors may lead to many novel experimental signatures. Depending on the details of the theory, dark quark production in proton-proton collisions could result in semivisible jets of particles: collimated sprays of dark hadrons of which only some are detectable by particle collider experiments. The experimental signature is characterised by the presence of reconstructed missing momentum collinear with the visible components of the jets. This complex topology is sensitive to detector inefficiencies and mis-reconstruction that generate artificial missing momentum. With this work, we propose a signal-agnostic strategy to reject ordinary jets and identify semivisible jets via anomaly detection techniques. A deep neural autoencoder network with jet substructure variables as input proves highly useful for analyzing anomalous jets. The study focuses on the semivisible jet signature; however, the technique can apply to any new physics model that predicts signatures with anomalous jets from non-SM particles.

Machinery New Emerge Fault Diagnosis Based on Deep Convolution Variational Autoencoder and Adaptive Label Propagation

In the research field of mechanical equipment fault diagnosis, usually only the existing fault types are identified, and the new emerge class of the fault is ignored, however, the new emerge fault class may also occur actually. In order to solve the problem, a novel fault diagnosis model based on deep convolution variational autoencoder network and adaptive label propagation (DCVAN-ALP) is proposed. Firstly, the initial high dimensional features are constructed by using the double tree complex wavelet packet method as the input of the model. Secondly, the convolutional neural network architecture is applied to construct the variational autoencoder, and the local and non-local characteristics of samples are embedded into the loss function for training, which is considered to improve the identification of hidden layer features of the neural network. Finally, t-SNE and the improved label propagation algorithm are adopted to process the hidden features of the neural network, which can achieve the purpose of diagnosing the existing fault class and especially new emerge fault class. Experimental results show that the proposed model can effectively extract the fault characteristics of the vibration signal, and it also has a significantly higher recognition accuracy rate than other typical deep learning methods and traditional classifiers in diagnosing new emerge fault class.

Spectral-Spatial Deep Support Vector Data Description for Hyperspectral Anomaly Detection

Hyperspectral anomaly detection (HAD) aims to distinguish anomalies from background-by-background modeling. Deep learning has been applied to HAD and achieves promising detection results. However, there exist several issues that need to be addressed: 1) unrealistic Gaussian assumption on the latent representations may limit its application; 2) deep features are not well-suited to anomaly detection due to the separation between feature learning and anomaly detection; 3) lack of adequate exploitation of spectral-spatial features; 4) negative effect caused by spectral band redundancy. In this article, we propose an end-to-end trainable deep one-class classification network for HAD. Specifically, a minimal enclosing hypersphere is trained to involve the deep features of background samples. These background samples are selected by a density clustering-based method. In this way, feature learning and anomaly detection are incorporated into a unified framework. Meanwhile, there is no explicit Gaussian assumption on the background features. Moreover, due to the complementarity of spectral and spatial features, a novel feature fusion strategy is proposed to fuse spectral and spatial features extracted by a two-stream deep convolutional autoencoder network. Finally, a band attention module is used to automatically learn small weights for redundant bands and thus reduce the negative effect caused by redundant bands. Experimental results on five public datasets demonstrate the superiority of the proposed method compared to several state-of-the-art HAD methods in the detection performance.

Hyperspectral Unmixing for Additive Nonlinear Models With a 3-D-CNN Autoencoder Network

Spectral unmixing is an important task in hyperspectral image processing for separating the mixed spectral data pertaining to various materials observed aiming at analyzing the material components in observed pixels. Recently, nonlinear spectral unmixing has received particular attention in hyperspectral image processing, as there are many situations in which the linear mixture model may not be appropriate and could be advantageously replaced by a nonlinear one. Existing nonlinear unmixing approaches are often based on specific assumptions on the nonlinearity and can be less effective when used for scenes with unknown nonlinearity. This article presents an unsupervised nonlinear spectral unmixing method that addresses a general model that consists of a linear mixture part and an additive nonlinear mixture part. The structure of a deep autoencoder network, which has a clear physical interpretation, is specifically designed to achieve this purpose. Moreover, a convolutional neural network (CNN) is used to capture the spectral-spatial priors from hyperspectral data. Extensive experiments with synthetic and real data illustrate the generality and effectiveness of this scheme compared with state-of-the-art methods.

A Deep Autoencoder-Based Convolution Neural Network Framework for Bearing Fault Classification in Induction Motors.

Fault diagnosis and classification for machines are integral to condition monitoring in the industrial sector. However, in recent times, as sensor technology and artificial intelligence have developed, data-driven fault diagnosis and classification have been more widely investigated. The data-driven approach requires good-quality features to attain good fault classification accuracy, yet domain expertise and a fair amount of labeled data are important for better features. This paper proposes a deep auto-encoder (DAE) and convolutional neural network (CNN)-based bearing fault classification model using motor current signals of an induction motor (IM). Motor current signals can be easily and non-invasively collected from the motor. However, the current signal collected from industrial sources is highly contaminated with noise; feature calculation thus becomes very challenging. The DAE is utilized for estimating the nonlinear function of the system with the normal state data, and later, the residual signal is obtained. The subsequent CNN model then successfully classified the types of faults from the residual signals. Our proposed semi-supervised approach achieved very high classification accuracy (more than 99%). The inclusion of DAE was found to not only improve the accuracy significantly but also to be potentially useful when the amount of labeled data is small. The experimental outcomes are compared with some existing works on the same dataset, and the performance of this proposed combined approach is found to be comparable with them. In terms of the classification accuracy and other evaluation parameters, the overall method can be considered as an effective approach for bearing fault classification using the motor current signal.

Unsupervised deep hashing through learning soft pseudo label for remote sensing image retrieval

Unsupervised hashing is an important approach for large-scale content-based remote sensing (RS) image retrieval. Existing unsupervised hashing methods usually utilize data clustering to generate pseudo labels as supervised signals. In the RS domain, images of each class cannot be accurately grouped into individual clusters because image features are not well characterized by the pre-trained models learned on a nature image dataset. As a result, to preserve the similarity of images sharing at least one class, intra-cluster and inter-cluster similarities need to be carefully learned. However, existing pseudo-labels are hard pseudo-labels represented by scalar values, which cannot well reflect the semantic distance between inter-cluster images or the semantic distance between intra-cluster images. To address these problems, this paper proposes a soft-pseudo-label-based unsupervised deep hashing method for content-based RS image retrieval, called SPL-UDH. Soft pseudo-labels can accurately describe the global similarity between inter-cluster images by binarized vectors. Specifically, we design a deep auto-encoder network to learn soft pseudo-labels automatically and meanwhile to generate a local similarity matrix representing the proximity between intra-cluster images. Based on soft pseudo-labels and local similarity matrix, we propose a deep hashing network to simultaneously learn the inter-cluster similarity and the intra-cluster similarity between RS images. Moreover, we design a new objective function based on Bayesian theory so that the deep hashing network can be trained by jointly learning the soft pseudo-labels and the local similarity matrix. Extensive experiments on public RS image retrieval datasets demonstrate that SPL-UDH outperforms various state-of-the-art unsupervised hashing methods.

Towards latent space based manipulation of elastic rods using autoencoder models and robust centerline extractions

The automatic shape control of deformable objects is a challenging (and currently hot) manipulation problem due to their high-dimensional geometric features and complex physical properties. In this study, a new methodology to manipulate elastic rods automatically into 2D desired shapes is presented. An efficient vision-based controller that uses a deep autoencoder network is designed to compute a compact representation of the object's infinite-dimensional shape. An online algorithm that approximates the sensorimotor mapping between the robots configuration and the object's shape features is used to deal with the latters (typically unknown) mechanical properties. The proposed approach computes the rods centerline from raw visual data in real-time by introducing an adaptive algorithm on the basis of a self-organizing network. Its effectiveness is thoroughly validated with simulations and experiments.

Machine learning inversion design and application verification of a broadband acoustic filtering structure

Most previous studies on acoustic structure performances have used the geometrical parameters to establish a theoretical model and calculate the acoustic performance to realize the forward design of an acoustic structure. This process strongly depends on the complexity and stability of the theoretical model. The development of machine learning has provided a new idea for the inverse design of acoustic structures, which can significantly improve the efficiency and accuracy of the inverse design of acoustic structures. This paper used the Helmholtz resonator and porous materials as a basic unit and established a theoretical model for the forward calculation of the sound absorption coefficient of the unit. By keeping two geometrical parameters fixed while changing the other four geometrical parameters, a database consisting of 79,730 acoustic performance parameter groups was constructed, and it corresponded to different sound-absorbing units with an absorption coefficient peak α ranging from 0.1467 to 0.9999, and a frequency f ranging from 255 Hz to 1658 Hz. A deep neural network (DNN) model was used to realize the reverse design of acoustic units corresponding to the performance parameters inside and outside the database. Because this paper aimed to achieve the prediction of acoustic performance parameters, and proposed a deep auto-encoder network (DAE) model with higher prediction accuracy after optimization based on DNN model. Using these two models, the prediction of the acoustic units corresponding to ten groups of performance parameters inside and outside the database was performed. The forward calculation results show that the two proposed models can realize the inversion design of acoustic structures, and the prediction accuracy of the DAE model is 2.64 times higher than that of the DNN model. The DAE model is used to realize the reverse design of the acoustic structure with a broadband acoustic filtering function. The reliability of the predicted broadband acoustic filtering structure was verified by the experiments with the predicted center frequency deviation of only 9 Hz. The proposed DNN and DAE models have certain reference significance for realizing the reverse design of acoustic performance and functional acoustic equipment.

Visual attention-driven framework to incorporate spatial-spectral features for hyperspectral anomaly detection

ABSTRACT Hyperspectral anomaly detection (HAD) is of particular interest due to not requiring any previous knowledge on spectra of ground objects. However, developing a fast and accurate detection approach for extracting both sub-pixel and supper-pixel anomalies has been remained challenging in the HAD field. A local and global scene analysis for HAD is investigated in this manuscript based on analytical modelling of the visual attention concept. The proposed method articulated the intrinsic spatial irregularities of the ground objects employing self-information of frequency spike components and adaptive local steering kernels. Also, the detection map obtained using spatial feature analysis is employed as prior information for the learning process of a deep autoencoder network to gain the maximum profit of spectral deviation of anomalous pixels. The potency of the proposed visual attention-based method is investigated on psychological patterns and various hyperspectral data sets. The results confirm the proposed method’s potency in detection accuracy and reduce false alarm rates viewpoints compared to some state-of-the-art methods.

Knowledge discovery of geochemical patterns from a data-driven perspective

We have entered the fourth research paradigm with the overwhelming availability of vast amounts of data. The processing and mining these data for a better understanding of earth systems and predicting mineral resources is challenging. This study discusses a data-driven knowledge discovery of geochemical patterns and presents a case study of geochemical data processing from a data-driven perspective. We employed local indicators of spatial association (LISA), principal component analysis (PCA), and deep autoencoder network (DAN) procedures to explore spatial association of geochemical patterns, extract elemental associations, and detect geochemical anomalies related to AuSb mineralization in the Daqiao district, Gansu Province, China. The results indicate the following: (1) both Au and Sb, and Pb and Zn have a close spatial correlation, indicating genetic connections among them; (2) the elemental association of Au, Sb, As, Hg and Ag can be adopted as a geochemical signature for the discovery of AuSb polymetallic mineralization in the study area; and (3) the geochemical anomalies identified by DAN exhibit a strong spatial relationship with locations of known mineral deposits and can provide a significant clue for further mineral exploration in this district. These findings indicate that data-driven procedures can help in the knowledge discovery of geochemical patterns in mineral exploration. Additional efforts are required for data-driven knowledge discovery in both geochemical prospecting and mineral exploration.

High-order differentiable autoencoder for nonlinear model reduction

This paper provides a new avenue for exploiting deep neural networks to improve physics-based simulation. Specifically, we integrate the classic Lagrangian mechanics with a deep autoencoder to accelerate elastic simulation of deformable solids. Due to the inertia effect, the dynamic equilibrium cannot be established without evaluating the second-order derivatives of the deep autoencoder network. This is beyond the capability of off-the-shelf automatic differentiation packages and algorithms, which mainly focus on the gradient evaluation. Solving the nonlinear force equilibrium is even more challenging if the standard Newton's method is to be used. This is because we need to compute a third-order derivative of the network to obtain the variational Hessian. We attack those difficulties by exploiting complex-step finite difference, coupled with reverse automatic differentiation. This strategy allows us to enjoy the convenience and accuracy of complex-step finite difference and in the meantime, to deploy complex-value perturbations as collectively as possible to save excessive network passes. With a GPU-based implementation, we are able to wield deep autoencoders (e.g., $10+$ layers) with a relatively high-dimension latent space in real-time. Along this pipeline, we also design a sampling network and a weighting network to enable \emph{weight-varying} Cubature integration in order to incorporate nonlinearity in the model reduction. We believe this work will inspire and benefit future research efforts in nonlinearly reduced physical simulation problems.