Deep Autoencoder Research Articles

Stable 3D Deep Convolutional Autoencoder Method for Ultrasonic Testing of Defects in Polymer Composites.

Ultrasonic testing is widely used for defect detection in polymer composites owing to advantages such as fast processing speed, simple operation, high reliability, and real-time monitoring. However, defect information in ultrasound images is not easily detectable because of the influence of ultrasound echoes and noise. In this study, a stable three-dimensional deep convolutional autoencoder (3D-DCA) was developed to identify defects in polymer composites. Through 3D convolutional operations, it can synchronously learn the spatiotemporal properties of the data volume. Subsequently, the depth receptive field (RF) of the hidden layer in the autoencoder maps the defect information to the original depth location, thereby mitigating the effects of the defect surface and bottom echoes. In addition, a dual-layer encoder was designed to improve the hidden layer visualization results. Consequently, the size, shape, and depth of the defects can be accurately determined. The feasibility of the method was demonstrated through its application to defect detection in carbon-fiber-reinforced polymers.

RDCSAE-RKRVFLN: A unified deep learning framework for robust and accurate DOA estimation

This paper introduces an innovative unified deep learning (DL) model, “reduced deep convolutional stack autoencoder (RDCSAE)-robust kernel-based random vector functional link network (RKRVFLN)” (RDCSAE-RKRVFLN), for addressing practical challenges like phase and gain uncertainty, mutual coupling effects, and element position errors in direction of arrival (DOA) estimation. The RDCSAE utilizes deep convolutional neural network (DCNN) and stack autoencoder (SAE) for robust feature extraction. The RKRVFLN integrates a concise SAE output with kernel-based learning. Thus, the integration of RDCSAE (suitable for unsupervised feature extraction) and RKRVFLN (for supervised learning) in the developed model results in efficient use of limited training data. Performance metrics of this method include among others the accuracy (separation between sources), root mean square error (RMSE), signal-to-noise ratio (SNR), etc. The method performs better than existing data driven methods in terms of computational efficiency, faster learning speed, enhanced model generalization, higher accuracy, and shorter event regression time and empirical methods. In the presence of above mentioned four perturbations, this method successfully resolves multiple sources with a resolution of 0.3° with a RMSE of 0.00376. For real time application, the paper implements RDCSAE-RKRVFLN on a high-speed reconfigurable Xilinx Virtex-5 field-programmable gate array (FPGA) realizing a digital DOA estimator for one source. The hardware estimates the DOA with a maximum error of 0.01953125 which follows closely the simulation results indicating the efficiency and feasibility of the method.

A feature reconstruction and SAE model based diagnosis method for multiple mixed faults

Owing to the merits of automatic feature extraction and depth structure, intelligent fault diagnosis based on deep neural networks has become a great concern. However, the non-fault state monitoring data volume of actual industrial machinery is rich, whereas the fault state data volume is insufficient and weak. Furthermore, achieving multiple mixed-fault diagnoses using skewed data distributions is extremely difficult. A feature reconstruction and sparse auto-encoder (AE) model-based diagnosis method for multiple mixed faults is proposed in this study to bridge these gaps. Such a feature reconstruction algorithm is designed and employed to address the following issues: (1) expensive computing resulting from the long sequential features of vibration monitoring data and (2) the extraction problem caused by the submersion of scarce data features. Furthermore, an adaptive loss function was formulated, and a deep AE network was constructed to identify the health status and determine the fault level. Diagnoses of artificial and real faults verify the availability and superiority of the proposed scheme, demonstrating the adaptability and robustness of these hyperparameters.

Dependency-aware deep generative models for multitasking analysis of spatial omics data.

Spatially resolved transcriptomics (SRT) technologies have significantly advanced biomedical research, but their data analysis remains challenging due to the discrete nature of the data and the high levels of noise, compounded by complex spatial dependencies. Here, we propose spaVAE, a dependency-aware, deep generative spatial variational autoencoder model that probabilistically characterizes count data while capturing spatial correlations. spaVAE introduces a hybrid embedding combining a Gaussian process prior with a Gaussian prior to explicitly capture spatial correlations among spots. It then optimizes the parameters of deep neural networks to approximate the distributions underlying the SRT data. With the approximated distributions, spaVAE can contribute to several analytical tasks that are essential for SRT data analysis, including dimensionality reduction, visualization, clustering, batch integration, denoising, differential expression, spatial interpolation, resolution enhancement and identification of spatially variable genes. Moreover, we have extended spaVAE to spaPeakVAE and spaMultiVAE to characterize spatial ATAC-seq (assay for transposase-accessible chromatin using sequencing) data and spatial multi-omics data, respectively.

New Health Indicator Construction and Fault Detection Network for Rolling Bearings via Convolutional Auto-Encoder and Contrast Learning

As one of the most important components in rotating machinery, if bearings fail, serious disasters may occur. Therefore, the remaining useful life (RUL) prediction of bearings is of great significance. Health indicator (HI) construction and early fault detection play a crucial role in data-driven RUL prediction. Unfortunately, most existing HI construction methods require prior knowledge and preset trends, making it difficult to reflect the actual degradation trend of bearings. And the existing early fault detection methods rely on massive historical data, yet manual annotation is time-consuming and laborious. To address the above issues, a novel deep convolutional auto-encoder (CAE) based on envelope spectral feature extraction is developed in this work. A sliding value window is defined in the envelope spectrum to obtain initial health indicators, which are used as preliminary labels for model training. Subsequently, CAE is trained by minimizing the composite loss function. The proposed construction method can reflect the actual degradation trend of bearings. Afterwards, the autoencoder is pre-trained through contrast learning (CL) to improve its discriminative ability. The model that has undergone offline pre-training is more sensitive to early faults. Finally, the HI construction method is combined with the early fault detection method to obtain a comprehensive network for online health assessment and fault detection, thus laying a solid foundation for subsequent RUL prediction. The superiority of the proposed method has been verified through experiments.

Zero-shot knowledge transfer for seismic damage diagnosis through multi-channel 1D CNN integrated with autoencoder-based domain adaptation

Accurate and timely structural damage diagnosis is crucial to efficient disaster response and city renovation in post-earthquake events. The scarcity of labeled data hinders the powerful deep learning techniques from in-domain damage detection on target structures. Cross-domain transfer learning has emerged as a captivating strategy through digging knowledge from the abundant source domain to detect the damage in the target domain. However, the heterogeneity among multi-domain structures poses the challenge in seismic damage diagnosis. This study proposes a novel zero-shot knowledge transfer approach for seismic damage diagnosis through multi-channel one-dimensional convolutional neural networks (1D CNN) integrated with deep autoencoder (DAE)-based domain adaptation (DA). The framework consists of three modules, namely, data preprocessor adaptive to seismic vibration signals, DAE-based DA module, and damage diagnosis via multi-channel 1D CNN. The DA module is customized to seamlessly translate the unseen target-domain data to mimic latent representation via a DAE pretrained on the source data, thus realizing rigorous zero-shot learning. Imbalanced data distribution is also considered during the network training and testing. Two representative phases of knowledge transfer are performed to substantiate the knowledge transferability of the proposed method. The first phase involves multi-class damage quantification on two ASCE benchmark models from the simplified model to the refined one, and the second phase conducts binary damage detection on a three-story reinforced frame structure from the finite element numerical model to the laboratory-tested physical model. Both examples show that the proposed method exhibits high prediction accuracy and a lower false negative rate in achieving zero-shot knowledge transfer for cross-domain structural damage diagnosis. With a delicate network design for diverse data, the proposed knowledge transfer framework can be further extended from the present zero-shot approach to the few-shot learning paradigm, thus suggesting a feasible algorithm adaptability and promising engineering applicability.

Constrained Symmetric Non-Negative Matrix Factorization with Deep Autoencoders for Community Detection

Recently, community detection has emerged as a prominent research area in the analysis of complex network structures. Community detection models based on non-negative matrix factorization (NMF) are shallow and fail to fully discover the internal structure of complex networks. Thus, this article introduces a novel constrained symmetric non-negative matrix factorization with deep autoencoders (CSDNMF) as a solution to this issue. The model possesses the following advantages: (1) By integrating a deep autoencoder to discern the latent attributes bridging the original network and community assignments, it adeptly captures hierarchical information. (2) Introducing a graph regularizer facilitates a thorough comprehension of the community structure inherent within the target network. (3) By integrating a symmetry regularizer, the model’s capacity to learn undirected networks is augmented, thereby facilitating the precise detection of symmetry within the target network. The proposed CSDNMF model exhibits superior performance in community detection when compared to state-of-the-art models, as demonstrated by eight experimental results conducted on real-world networks.

The quantification of southern corn leaf blight disease using deep UV fluorescence spectroscopy and autoencoder anomaly detection techniques.

Southern leaf blight (SLB) is a foliar disease caused by the fungus Cochliobolus heterostrophus infecting maize plants in humid, warm weather conditions. SLB causes production losses to corn producers in different regions of the world such as Latin America, Europe, India, and Africa. In this paper, we demonstrate a non-destructive method to quantify the signs of fungal infection in SLB-infected corn plants using a deep UV (DUV) fluorescence spectrometer, with a 248.6 nm excitation wavelength, to acquire the emission spectra of healthy and SLB-infected corn leaves. Fluorescence emission spectra of healthy and diseased leaves were used to train an Autoencoder (AE) anomaly detection algorithm-an unsupervised machine learning model-to quantify the phenotype associated with SLB-infected leaves. For all samples, the signature of corn leaves consisted of two prominent peaks around 450 nm and 325 nm. However, SLB-infected leaves showed a higher response at 325 nm compared to healthy leaves, which was correlated to the presence of C. heterostrophus based on disease severity ratings from Visual Scores (VS). Specifically, we observed a linear inverse relationship between the AE error and the VS (R2 = 0.94 and RMSE = 0.935). With improved hardware, this method may enable improved quantification of SLB infection versus visual scoring based on e.g., fungal spore concentration per unit area and spatial localization.

Mapping the Geometry of Law Using Natural Language Processing

Judicial documents and judgments are a rich source of information about legal cases, litigants, and judicial decision-makers. Natural language processing (NLP) based approaches have recently received much attention for their ability to decipher implicit information from text. NLP researchers have successfully developed data-driven representations of text using dense vectors that encode the relations between those objects. In this study, we explore the application of the Doc2Vec model to legal language to understand judicial reasoning and identify implicit patterns in judgments and judges. In an application to federal appellate courts, we show that these vectors encode information that distinguishes courts in time and legal topics. We use Doc2Vec document embeddings to study the patterns and train a classifier model to predict cases with a high chance of being appealed at the Supreme Court of the United States (SCOTUS). There are no existing benchmarks, and we present the first results at this task at scale. Furthermore, we analyze generic writing/judgment patterns of prominent judges using deep learning-based autoencoder models. Overall, we observe that Doc2Vec document embeddings capture important legal information and are helpful in downstream tasks.

Seismic characterization of individual geologic factors with disentangled features

Seismic attributes are critical in understanding geologic factors, such as sand body configuration, lithology, and porosity. However, existing attributes typically reflect the combined response of multiple geologic factors. The interplay between these factors can obscure the features of the target factor, posing a challenge to its direct seismic characterization, particularly when the factor is subtle. To address this, we develop an innovative neural network designed to disentangle and characterize the individual geologic factors within seismic data. Our approach divides the geologic information in the seismic data into two categories: a single geologic factor of interest and an aggregate of all other information. A novel feature-swapping mechanism within our network facilitates the disentanglement of these two categories, providing an interpretable representation. We use a triplet loss function to differentiate data samples with similar waveforms but varying subtle geologic details, thus enhancing the extraction of distinct features. In addition, our network uses a cotraining strategy to integrate the synthetic and actual field data during the training process. This strategy helps mitigate the potential performance degradation arising from the discrepancies between simulated and actual field data. We apply our method to synthetic data experiments and field data from two geologically distinct areas. Current results indicate that our method surpasses traditional approaches, such as a deep autoencoder and a convolutional neural network classifier, in extracting seismic attributes with more explicit geophysical implications.

Unsupervised fault detection approach based on depth auto-encoder for photovoltaic power generation system

Fault detection for photovoltaic power generation system is a challenging problem in condition monitoring and troubleshooting, which aims to maintain the safe operation of equipment and improve the benefit of photovoltaic industry. Aiming at the problems of frequent failures of photovoltaic power generation system, large amount of operating data and difficult to obtain fault samples, we propose an unsupervised fault detection approach for photovoltaic power generation system via bidirectional long/short memory deep auto-encoder which combines the auto-encoder in deep learning with the Bi-directional Long Short-Term Memory (BiLSTM). Specifically, We first take the statistical feature enhanced as the input of an auto-encoder based on BiLSTM. Then, we build a simulation model of Grid-connected PV system. Finally, we use the operation results under normal conditions to train the fault detection model to obtain the reconstruction error and determine the fault detection threshold, so as to judge the anomalies of the photovoltaic power generation system. We simulate the shadow occlusion fault and verify the effectiveness of the proposed method, and the fault detection accuracy of 0.95 is achieved. Compare with other models, the results show that it could set up better dependence on multi-dimensional data in time sequences, effectively testing solar panel failures and solving insufficient data labels problems.

Single-Pixel Imaging Based on Deep Learning Enhanced Singular Value Decomposition.

We propose and demonstrate a single-pixel imaging method based on deep learning network enhanced singular value decomposition. The theoretical framework and the experimental implementation are elaborated and compared with the conventional methods based on Hadamard patterns or deep convolutional autoencoder network. Simulation and experimental results show that the proposed approach is capable of reconstructing images with better quality especially under a low sampling ratio down to 3.12%, or with fewer measurements or shorter acquisition time if the image quality is given. We further demonstrate that it has better anti-noise performance by introducing noises in the SPI systems, and we show that it has better generalizability by applying the systems to targets outside the training dataset. We expect that the developed method will find potential applications based on single-pixel imaging beyond the visible regime.

Research on heart and lung sound separation method based on DAE–NMF–VMD

Auscultation is the most effective method for diagnosing cardiovascular and respiratory diseases. However, stethoscopes typically capture mixed signals of heart and lung sounds, which can affect the auscultation effect of doctors. Therefore, the efficient separation of mixed heart and lung sound signals plays a crucial role in improving the diagnosis of cardiovascular and respiratory diseases. In this paper, we propose a blind source separation method for heart and lung sounds based on deep autoencoder (DAE), nonnegative matrix factorization (NMF) and variational mode decomposition (VMD). Firstly, DAE is employed to extract highly informative features from the heart and lung sound signals. Subsequently, NMF clustering is applied to group the heart and lung sounds based on their distinct periodicities, achieving the separation of the mixed heart and lung sounds. Finally, variational mode decomposition is used for denoising the separated signals. Experimental results demonstrate that the proposed method effectively separates heart and lung sound signals and exhibits significant advantages in terms of standardized evaluation metrics when compared to contrast methods.

231 Anomaly detection for breed purity analysis

Abstract Recent advancements in genotyping technologies have revolutionized our ability to estimate breed composition in pigs. The classic compositional regression used for this purpose requires a multibreed panel to detect crossbreeding and its application is limited to breed in the panel. In this study, we presented a protocol based on classic anomaly detection algorithms that use single breed panels to detect crossbreeding. By utilizing these methods, we aim to identify and assess breed outliers within large pig genotype panels. The dataset consisted of 44,616 SNP genotypes recorded in purebred Large White, Landrace, Duroc, Pietrain (n = 2,000 from each breed) and 1,010 crossbred pigs (50% Pietrain, 25% Large White, 25% Landrace). Large White pigs (n = 1,500) were randomly selected as reference set. The remaining 500 Large White pigs, other purebred pigs, crossbred pigs, and simulated crossbred animals of known proportion of Large White segments were test sets. We applied two anomaly detection methods: 1) Principal Component Analysis (PCA) and 2) deep learning autoencoders (AE). For PCA the top 600 eigen vectors capturing 85% cumulative proportion of variance in the reference set were selected. Test set genotypes were projected onto the principal components and then they were reconstructed to the full genotype set. For AE, we used a previously published sparse convolutional denoising autoencoder (SCDA) that consisted of two main components: an encoder and a decoder. The encoder compressed the genotypes to 128 components and the decoder reconstructed the set to the original 44,616 SNP. We utilized the model trained in the reference set to reconstruct test set genotypes of the same breed, of alternative breed and of crossbred. Accuracy of reconstruction of genotypes was computed using correlation and mean square error. The proportion of rejected genotypes was used to assess the ability of anomaly detection algorithms to detect crossbreeding. For each criterion, the threshold to reject genotypes was selected as the value at which 5% of Large White animals were rejected (specificity = 0.95). Results are summarized in (Table 1). PCA provided better properties to detect genomic outliers compared with AE. PCA anomaly detection was able to detect purebred individuals with probability 1.0. It also detected over 90% of crossbred individuals that were 75% Large Whites. Crossbred with a lower proportion of Large White genomic segments were detected with P < 0.5. Anomaly detection algorithms can be used to detect genomic outliers but with relatively low sensitivity. More work is needed to optimize these algorithms for breeding applications.

A Deep Auto-Optimized Collaborative Learning (DACL) model for disease prognosis using AI-IoMT systems

In modern healthcare, integrating Artificial Intelligence (AI) and Internet of Medical Things (IoMT) is highly beneficial and has made it possible to effectively control disease using networks of interconnected sensors worn by individuals. The purpose of this work is to develop an AI-IoMT framework for identifying several of chronic diseases form the patients’ medical record. For that, the Deep Auto-Optimized Collaborative Learning (DACL) Model, a brand-new AI-IoMT framework, has been developed for rapid diagnosis of chronic diseases like heart disease, diabetes, and stroke. Then, a Deep Auto-Encoder Model (DAEM) is used in the proposed framework to formulate the imputed and preprocessed data by determining the fields of characteristics or information that are lacking. To speed up classification training and testing, the Golden Flower Search (GFS) approach is then utilized to choose the best features from the imputed data. In addition, the cutting-edge Collaborative Bias Integrated GAN (ColBGaN) model has been created for precisely recognizing and classifying the types of chronic diseases from the medical records of patients. The loss function is optimally estimated during classification using the Water Drop Optimization (WDO) technique, reducing the classifier’s error rate. Using some of the well-known benchmarking datasets and performance measures, the proposed DACL’s effectiveness and efficiency in identifying diseases is evaluated and compared.

PSO-XnB: a proposed model for predicting hospital stay of CAD patients.

Coronary artery disease poses a significant challenge in decision-making when predicting the length of stay for a hospitalized patient. This study presents a predictive model-a Particle Swarm Optimized-Enhanced NeuroBoost-that combines the deep autoencoder with an eXtreme gradient boosting model optimized using particle swarm optimization. The model uses a fuzzy set of rules to categorize the length of stay into four distinct classes, followed by data preparation and preprocessing. In this study, the dimensionality of the data is reduced using deep neural autoencoders. The reconstructed data obtained from autoencoders is given as input to an eXtreme gradient boosting model. Finally, the model is tuned with particle swarm optimization to obtain optimal hyperparameters. With the proposed technique, the model achieved superior performance with an overall accuracy of 98.8% compared to traditional ensemble models and past research works. The model also scored highest in other metrics such as precision, recall, and particularly F1 scores for all categories of hospital stay. These scores validate the suitability of our proposed model in medical healthcare applications.

Learnable faster kernel-PCA for nonlinear fault detection: Deep autoencoder-based realization

Kernel principal component analysis (KPCA) is a well-recognized nonlinear dimensionality reduction method that has been widely used in nonlinear fault detection tasks. As a kernel trick-based method, KPCA inherits two major problems. First, the form and the parameters of the kernel function are usually selected blindly, depending seriously on trial-and-error. As a result, there may be serious performance degradation in case of inappropriate selections. Second, at the online monitoring stage, KPCA has much computational burden and poor real-time performance, because the kernel method requires to leverage all the offline training data. In this work, to deal with the two drawbacks, a learnable faster realization of the conventional KPCA is proposed. The core idea is to parameterize all feasible kernel functions using the novel nonlinear DAE-FE (deep autoencoder based feature extraction) framework and propose DAE-PCA (deep autoencoder based principal component analysis) approach in detail. The proposed DAE-PCA method is proved to be equivalent to KPCA but has more advantage in terms of automatic searching of the most suitable nonlinear high-dimensional space according to the inputs, which helps to improve the accuracy of fault detection. Furthermore, the online computational efficiency improves by many times compared with the conventional KPCA. Finally, the Tennessee Eastman (TE) process benchmark and wastewater treatment plant (WWTP) benchmark are employed to illustrate the effectiveness of the proposed method, where the average fault detection rates of DAE-PCA are at least 0.27% and 4.69% higher than those of other methods, and its online computational efficiency is faster 90.48% and 24.57% times than that of KPCA respectively.

Disruption detection for a cognitive digital supply chain twin using hybrid deep learning

Recent disruptive events, such as COVID-19 and Russia–Ukraine conflict, had a significant impact of global supply chains. Digital supply chain twins have been proposed in order to provide decision makers with an effective and efficient tool to mitigate disruption impact. This paper introduces a hybrid deep learning approach for disruption detection within a cognitive digital supply chain twin framework to enhance supply chain resilience. The proposed disruption detection module utilises a deep autoencoder neural network combined with a one-class support vector machine algorithm. In addition, long-short term memory neural network models are developed to identify the disrupted echelon and predict time-to-recovery from the disruption effect. The obtained information from the proposed approach will help decision-makers and supply chain practitioners make appropriate decisions aiming at minimizing negative impact of disruptive events based on real-time disruption detection data. The results demonstrate the trade-off between disruption detection model sensitivity, encountered delay in disruption detection, and false alarms. This approach has seldom been used in recent literature addressing this issue.

Transformer fault diagnosis based on adversarial generative networks and deep stacked autoencoder

Establishing a deep learning model for transformer fault diagnosis using transformer oil chromatogram data requires a large number of fault samples. The lack and imbalance of oil chromatogram data can lead to overfitting, lack of representativeness of the model, and unsatisfactory prediction results on test set data, making it difficult to accurately diagnose transformer faults. A conditional Wasserstein generative adversarial network with gradient penalty optimization (CWGAN-GP) is adopted in this paper, which based on gradient penalty optimization and expand the oil chromatography fault samples of 500 sets of transformer oil chromatography data with 5 types of faults. The proposed method is used to classify transformer faults using a deep autoencoder, and the sample quality of the neural network model proposed in this paper is compared with several other variants of generative adversarial neural network models. The research results show that after using the method proposed in this paper for sample expansion, the overall accuracy of fault diagnosis can reach 93.2 %, which is 4.98 % higher than the original imbalanced samples. Compared with other sample expansion methods, the accuracy of fault diagnosis of the algorithm in this paper is improved by 1.70 %–3.05 %.

KFDAE: CircRNA-Disease Associations Prediction Based on Kernel Fusion and Deep Auto-Encoder.

CircRNA has been proved to play an important role in the diseases diagnosis and treatment. Considering that the wet-lab is time-consuming and expensive, computational methods are viable alternative in these years. However, the number of circRNA-disease associations (CDAs) that can be verified is relatively few, and some methods do not take full advantage of dependencies between attributes. To solve these problems, this paper proposes a novel method based on Kernel Fusion and Deep Auto-encoder (KFDAE) to predict the potential associations between circRNAs and diseases. Firstly, KFDAE uses a non-linear method to fuse the circRNA similarity kernels and disease similarity kernels. Then the vectors are connected to make the positive and negative sample sets, and these data are send to deep auto-encoder to reduce dimension and extract features. Finally, three-layer deep feedforward neural network is used to learn features and gain the prediction score. The experimental results show that compared with existing methods, KFDAE achieves the best performance. In addition, the results of case studies prove the effectiveness and practical significance of KFDAE, which means KFDAE is able to capture more comprehensive information and generate credible candidate for subsequent wet-lab.