Deep Autoencoder Research Articles

Forecasting dominance of SARS-CoV-2 lineages by anomaly detection using deep AutoEncoders.

The COVID-19 pandemic is marked by the successive emergence of new SARS-CoV-2 variants, lineages, and sublineages that outcompete earlier strains, largely due to factors like increased transmissibility and immune escape. We propose DeepAutoCoV, an unsupervised deep learning anomaly detection system, to predict future dominant lineages (FDLs). We define FDLs as viral (sub)lineages that will constitute >10% of all the viral sequences added to the GISAID, a public database supporting viral genetic sequence sharing, in a given week. DeepAutoCoV is trained and validated by assembling global and country-specific data sets from over 16 million Spike protein sequences sampled over a period of ~4years. DeepAutoCoV successfully flags FDLs at very low frequencies (0.01%-3%), with median lead times of 4-17weeks, and predicts FDLs between ~5 and ~25 times better than a baseline approach. For example, the B.1.617.2 vaccine reference strain was flagged as FDL when its frequency was only 0.01%, more than a year before it was considered for an updated COVID-19 vaccine. Furthermore, DeepAutoCoV outputs interpretable results by pinpointing specific mutations potentially linked to increased fitness and may provide significant insights for the optimization of public health 'pre-emptive' intervention strategies.

Robust estimation of hydrogeological parameters from wireline logs usingsemi-supervised deep neural networks assisted with global optimization-based regression methods

Understanding the distribution of hydrogeological properties of the aquifers is crucial for sustainable groundwater resource development. This research explores the application of deep autoencoder neural networks (AE-NN), assisted with global optimization methods for estimating hydrogeological parameters in the Quaternary aquifer system in the Debrecen area, Hungary. Traditional methods for estimating aquifer parameters typically depend on field experiments and laboratory analyses, which are both costly and time-consuming, and often fail to account for the heterogeneity of groundwater formations. In this study, deep AE-NN models are trained to extract latent space (LS) representations that capture key features from the available well logs, including spontaneous potential (SP), natural gamma ray (NGR), shallow resistivity (RS), and deep resistivity (RD). The LS log is then correlated with shale volume and hydraulic conductivity, as determined by the Larionov and Csókás methods, respectively. Regression analysis revealed a Gaussian relationship between the LS log and shale volume and a negative nonlinear relationship with hydraulic conductivity. Global optimization methods, including simulated annealing (SA) and particle swarm optimization (PSO), were used to refine the regression parameters, enhancing the predictive capabilities of the models. The results demonstrated that AE-NN assisted with global optimization methods can be effectively used to estimate shale volume and hydraulic conductivity, proposing a novel and independent approach for estimating hydrogeological parameters critical to groundwater flow and contaminant transport modeling.

Elastic deep multi-view autoencoder with diversity embedding

Current research on multi-view clustering (MVC) is pushing the boundaries of knowledge, allowing the extraction of valuable insights from various points of view. Recently, many researchers in this field have turned to deep learning techniques, particularly autoencoders (AE). These methods are highly effective at recognizing nonlinear and complex structures, making them a popular choice in this area. However, while they effectively handle Gaussian noise through the use of the Frobenius norm, they exhibit sensitivity to outlier data and Laplacian noise. Additionally, existing multi-view methodologies tend to usually rely on shared information across multi-view data, this means they often miss out on the diverse and complementary information each view can provide, leading in an inadequate utilization of the information available. In this paper, we present an innovative solution to address these challenges. Our novel Elastic Deep Multi-view Autoencoder with Diversity Embedding (EDMVAE-DE) method incorporates an Elastic Deep AE to enhance its robustness. Furthermore, we integrate an exclusivity constraint term to enhance the diversity of specific representations across distinct views, increasing both modeling consistency and diversity within a unified framework. To address problems arising from incorrectly classified neighbors, we also incorporate a graph constraint term. Our experiments, performed on various multi-view datasets, clearly show that EDMVAE-DE achieves the best performance in its class1.

Deep autoencoder with gated convolutional neural networks for improving speech quality in secured communications

In this study, we introduce a speech enhancement method to improve the quality of decrypted speech signals from hand-talk devices, which are highly susceptible to security attacks. Ensuring high-quality decrypted speech is essential because traditional speech enhancement methods struggle with artifacts only present during speech due to the encryption process applied selectively. This situation limits the effectiveness of traditional methods, which assume distortion is constant and can be estimated during silent periods. Our solution involves a deep-learning approach that employs a gated convolutional neural network (GCNN). Unlike typical convolutional neural networks (CNNs) that excel in processing spatial data but falter with temporal changes, our GCNN integrates a gating mechanism to enhance handling of temporal dynamics in speech data. This method directly maps distorted speech to its clean counterpart, bypassing the need for explicit noise estimation. Our experiments indicate that this deep-learning method significantly outperforms traditional speech enhancement techniques and conventional CNNs in several key evaluation metrics, offering a promising advancement in decrypted speech quality enhancement.

Physical model learning based false data injection attack on power system state estimation

The cyber security of power system state estimation (PSSE) is crucial, and its robustness against evolving false data injection attacks (FDIA) is being rigorously assessed to develop advanced countermeasures. Existing FDIA methods have achieved satisfactory success rates but often fail to align with practical constraints such as the assumption of partial or complete knowledge of the power system network by the attacker, modifications in generator output measurements, and the sparsity of the attacks. This work proposes a near practical, stealthy approach using a deep generative adversarial network-long short-term memory autoencoder (GAN-LSTMAE) learning based sparse FDIA method against AC PSSE, leveraging only measurement data. To evade the bad data detection (BDD) mechanism effectively, an LSTMAE-based PSSE mimic is proposed, further optimizing the GAN-based attack generator to embed the physical laws of the system along with measurement residuals and temporal dependencies of states to the generated false data. The proposed modified training data preparation algorithm, coupled with the attack sub-graph method, defines the optimal attack region while keeping generator output measurements intact. The generated attack is validated extensively using IEEE 14 and 118 bus test benchmarks against various defense techniques, demonstrating high success rates.

Deep Clustering for Epileptic Seizure Detection.

The objective of this study is to address the challenges of EEG-based epileptic seizure detection by introducing a novel methodology, Deep Embedded Gaussian Mixture (DEGM). The DEGM method begins with a deep autoencoder (DAE) for embedding the input EEG data, followed by Singular Value Decomposition (SVD) to enhance the representational quality of the embedding while achieving dimensionality reduction. A Gaussian Mixture Model (GMM) is then employed for clustering purposes. Unlike conventional supervised machine learning and deep learning techniques, DEGM leverages deep clustering (DC) algorithms for more effective seizure detection. Empirical results from two real-world epileptic datasets demonstrate the notable performance of DEGM. The method's effectiveness is particularly remarkable given the substantial size of the datasets, showcasing its ability to handle large-scale EEG data efficiently. In conclusion, the DEGM methodology provides a novel and effective approach for EEG-based epileptic seizure detection, addressing key challenges such as data variability and artifact contamination. By combining deep autoencoders, SVD, and GMM, DEGM achieves superior clustering performance compared to existing methods, representing a significant advancement in biomedical research and clinical applications for epilepsy. Its robust performance on large datasets underscores its potential for improving seizure detection accuracy, ultimately contributing to better patient outcomes.

Deep autoencoder-based behavioral pattern recognition outperforms standard statistical methods in high-dimensional zebrafish studies.

Zebrafish have become an essential model organism in screening for developmental neurotoxic chemicals and their molecular targets. The success of zebrafish as a screening model is partially due to their physical characteristics including their relatively simple nervous system, rapid development, experimental tractability, and genetic diversity combined with technical advantages that allow for the generation of large amounts of high-dimensional behavioral data. These data are complex and require advanced machine learning and statistical techniques to comprehensively analyze and capture spatiotemporal responses. To accomplish this goal, we have trained semi-supervised deep autoencoders using behavior data from unexposed larval zebrafish to extract quintessential "normal" behavior. Following training, our network was evaluated using data from larvae shown to have significant changes in behavior (using a traditional statistical framework) following exposure to toxicants that include nanomaterials, aromatics, per- and polyfluoroalkyl substances (PFAS), and other environmental contaminants. Further, our model identified new chemicals (Perfluoro-n-octadecanoic acid, 8-Chloroperfluorooctylphosphonic acid, and Nonafluoropentanamide) as capable of inducing abnormal behavior at multiple chemical-concentrations pairs not captured using distance moved alone. Leveraging this deep learning model will allow for better characterization of the different exposure-induced behavioral phenotypes, facilitate improved genetic and neurobehavioral analysis in mechanistic determination studies and provide a robust framework for analyzing complex behaviors found in higher-order model systems.

Physics-Informed Machine Learning for Calibrating Macroscopic Traffic Flow Models

Well-calibrated traffic flow models are fundamental to understanding traffic phenomena and designing control strategies. Traditional calibration has been developed based on optimization methods. In this paper, we propose a novel physics-informed, learning-based calibration approach that achieves performances comparable to and even better than those of optimization-based methods. To this end, we combine the classical deep autoencoder, an unsupervised machine learning model consisting of one encoder and one decoder, with traffic flow models. Our approach informs the decoder of the physical traffic flow models and thus induces the encoder to yield reasonable traffic parameters given flow and speed measurements. We also introduce the denoising autoencoder into our method so that it can handle not only with normal data but also corrupted data with missing values. We verified our approach with a case study of Interstate 210 Eastbound in California. It turns out that our approach can achieve comparable performance to the-state-of-the-art calibration methods given normal data and outperform them given corrupted data with missing values. History: This paper has been accepted for the Transportation Science Special Issue on ISTTT25 Conference. Funding: This study was supported by the National Science Foundation [Grant CMMI-1949710] and the C2SMART Research Center, a Tier 1 University Transportation Center.

A generalizable normative deep autoencoder for brain morphological anomaly detection: application to the multi-site StratiBip dataset on bipolar disorder in an external validation framework.

The heterogeneity of psychiatric disorders makes researching disorder-specific neurobiological markers an ill-posed problem. Here, we face the need for disease stratification models by presenting a generalizable multivariate normative modelling framework for characterizing brain morphology, applied to bipolar disorder (BD). We employed deep autoencoders in an anomaly detection framework, combined with a confounder removal step integrating training and external validation. The model was trained with healthy control (HC) data from the human connectome project and applied to multi-site external data of HC and BD individuals. We found that brain deviating scores were greater, more heterogeneous, and with increased extreme values in the BD group, with volumes prominently from the basal ganglia, hippocampus and adjacent regions emerging as significantly deviating. Similarly, individual brain deviating maps based on modified z scores expressed higher abnormalities occurrences, but their overall spatial overlap was lower compared to HCs. Our generalizable framework enabled the identification of subject- and group-level brain normative-deviating patterns, a step forward towards the development of more effective and personalized clinical decision support systems and patient stratification in psychiatry.

MRDPDA: A multi-Laplacian regularized deepFM model for predicting piRNA-disease associations.

PIWI-interacting RNAs (piRNAs) are a typical class of small non-coding RNAs, which are essential for gene regulation, genome stability and so on. Accumulating studies have revealed that piRNAs have significant potential as biomarkers and therapeutic targets for a variety of diseases. However current computational methods face the challenge in effectively capturing piRNA-disease associations (PDAs) from limited data. In this study, we propose a novel method, MRDPDA, for predicting PDAs based on limited data from multiple sources. Specifically, MRDPDA integrates a deep factorization machine (deepFM) model with regularizations derived from multiple yet limited datasets, utilizing separate Laplacians instead of a simple average similarity network. Moreover, a unified objective function to combine embedding loss about similarities is proposed to ensure that the embedding is suitable for the prediction task. In addition, a balanced benchmark dataset based on piRPheno is constructed and a deep autoencoder is applied for creating reliable negative set from the unlabeled dataset. Compared with three latest methods, MRDPDA achieves the best performance on the pirpheno dataset in terms of the five-fold cross validation test and independent test set, and case studies further demonstrate the effectiveness of MRDPDA.

SiSRS: Signed social recommender system using deep neural network representation learning

Most approaches used in social recommender systems concentrate mostly on positive relationshipswithin the social network, frequently ignoring the insightful information that negative relationships can offer. A more thorough understanding of user characteristics and behaviors within a social network can result from integrating positive and negative links to construct a signed graph. However, there are two major challenges in developing signed social recommender systems. Considering negative links in these systems needs to be done carefully to make sure that these relationships are appropriately represented and used. Moreover, it is difficult to predict user interactions and preferences effectively when relationships represented by positive and negative links conflict, known as social inconsistency. Therefore, working with signed graphs requires the management of social inconsistency. To address these issues, in this paper, a signed social recommender system called SiSRS is presented using a deep architecture combined with network representation learning. The SiSRS is composed of three principal components. First, the application of signed graph attention networks is explored to collate and disseminate information across the network via graph motifs, leading to the creation of node embeddings. Second, a deep autoencoder model is employed to assimilate top-k semantic signed social data within the deep learning framework. Third, a novel loss function is formulated to reduce the impact of social inconsistency. The efficacy of the SiSRS model was evaluated through extensive experiments conducted on two datasets in terms of various evaluation metrics. The results indicated a superior performance of this approach compared to existing state-of-the-art methods.

On the latent dimension of deep autoencoders for reduced order modeling of PDEs parametrized by random fields

Deep Learning is having a remarkable impact on the design of Reduced Order Models (ROMs) for Partial Differential Equations (PDEs), where it is exploited as a powerful tool for tackling complex problems for which classical methods might fail. In this respect, deep autoencoders play a fundamental role, as they provide an extremely flexible tool for reducing the dimensionality of a given problem by leveraging on the nonlinear capabilities of neural networks. Indeed, starting from this paradigm, several successful approaches have already been developed, which are here referred to as Deep Learning-based ROMs (DL-ROMs). Nevertheless, when it comes to stochastic problems parameterized by random fields, the current understanding of DL-ROMs is mostly based on empirical evidence: in fact, their theoretical analysis is currently limited to the case of PDEs depending on a finite number of (deterministic) parameters. The purpose of this work is to extend the existing literature by providing some theoretical insights about the use of DL-ROMs in the presence of stochasticity generated by random fields. In particular, we derive explicit error bounds that can guide domain practitioners when choosing the latent dimension of deep autoencoders. We evaluate the practical usefulness of our theory by means of numerical experiments, showing how our analysis can significantly impact the performance of DL-ROMs.

A detection method of oil content for maize kernels based on CARS feature selection and deep sparse autoencoder feature extraction

To achieve accurate, rapid, and non-destructive oil content determination in maize, a neural fitting model that combines feature selection and feature extraction is proposed. Competitive adaptive re-weighted sampling (CARS) is used to choose features, while the deep sparse autoencoder (DSAE) network is utilized to extract features. The 160 spectral data from the near-infrared spectra public datasets of maize served as experimental samples. Several preprocessing approaches were tested, and the first-order derivatives fared the best. To capture the various degrees of abstract features in spectral data, SSAE networks with two or three hidden layers are developed. The number of nodes in the network's hidden layer (30, 50, 70, 100, 200, 300, respectively) and the network's sparsity parameter (0.001, 0.004, 0.007, 0.01, 0.04, 0.07, 0.1, 0.2, …, 1) are tuned to generate partial least squares regression (PLSR) models for testing the feature learning effect. The highest performance comes from a three-layer network with 200 hidden layer nodes and a sparsity parameter of 0.04. This network extracts three levels of features from the preprocessed spectral data: F1, F2, and F3. Full_cars, F1_cars, F2_cars, and F3_cars are obtained using the CARS algorithm for variable selection on the preprocessed spectral data, F1, F2, and F3, respectively. They are then integrated into a final feature set to build a neural-fitting network (FNN). After training and testing, the test set has a correlation coefficient of 0.964205 and an mean square error (MSE) of 0.00636809. Comparing this model to support vector machine regression (SVR), PLSR, and gaussian process regression (GPR), the findings demonstrate that the model described in this study outperforms the others in terms of fitting accuracy and generalization performance. To summarize, this paper introduces a new regression analysis method based on near-infrared spectra.

Structure-Based Drug Design with a Deep Hierarchical Generative Model.

Recently, the remarkable growth of available crystal structure data and libraries of commercially available or readily synthesizable molecules have unlocked previously inaccessible regions of chemical space for drug development. Paired with improvements in virtual ligand screening methods, these expanded libraries are having a notable impact on early drug design efforts. Yet screening-based methods still face scalability limits, due to computational constraints and the sheer scale of drug-like space. Machine learning approaches are overcoming these limitations by learning the fundamental intra- and intermolecular relationships in drug-target systems from existing data. Here, we introduce DrugHIVE, a deep hierarchical variational autoencoder that outperforms state-of-the-art autoregressive and diffusion-based methods in both speed and performance on common generative benchmarks. DrugHIVE's hierarchical design enables improved control over molecular generation. Its capabilities include dramatically increasing virtual screening efficiency and accelerating a wide range of common drug design tasks, including de novo generation, molecular optimization, scaffold hopping, linker design, and high-throughput pattern replacement. Our highly scalable method can even be applied to receptors with high-confidence AlphaFold-predicted structures, extending the ability to generate high-quality drug-like molecules to a majority of the unsolved human proteome.

Modality-Aware Discriminative Fusion Network for Integrated Analysis of Brain Imaging Genomics.

Mild cognitive impairment (MCI) represents an early stage of Alzheimer's disease (AD), characterized by subtle clinical symptoms that pose challenges for accurate diagnosis. The quest for the identification of MCI individuals has highlighted the importance of comprehending the underlying mechanisms of disease causation. Integrated analysis of brain imaging and genomics offers a promising avenue for predicting MCI risk before clinical symptom onset. However, most existing methods face challenges in: 1) mining the brain network-specific topological structure and addressing the single nucleotide polymorphisms (SNPs)-related noise contamination and 2) extracting the discriminative properties of brain imaging genomics, resulting in limited accuracy for MCI diagnosis. To this end, a modality-aware discriminative fusion network (MA-DFN) is proposed to integrate the complementary information from brain imaging genomics to diagnose MCI. Specifically, we first design two modality-specific feature extraction modules: the graph convolutional network with edge-augmented self-attention module (GCN-EASA) and the deep adversarial denoising autoencoder module (DAD-AE), to capture the topological structure of brain networks and the intrinsic distribution of SNPs. Subsequently, a discriminative-enhanced fusion network with correlation regularization module (DFN-CorrReg) is employed to enhance inter-modal consistency and between-class discrimination in brain imaging and genomics. Compared to other state-of-the-art approaches, MA-DFN not only exhibits superior performance in stratifying cognitive normal (CN) and MCI individuals but also identifies disease-related brain regions and risk SNPs locus, which hold potential as putative biomarkers for MCI diagnosis.

Clustering Molecules at a Large Scale: Integrating Spectral Geometry with Deep Learning.

This study conducts an in-depth analysis of clustering small molecules using spectral geometry and deep learning techniques. We applied a spectral geometric approach to convert molecular structures into triangulated meshes and used the Laplace-Beltrami operator to derive significant geometric features. By examining the eigenvectors of these operators, we captured the intrinsic geometric properties of the molecules, aiding their classification and clustering. The research utilized four deep learning methods: Deep Belief Network, Convolutional Autoencoder, Variational Autoencoder, and Adversarial Autoencoder, each paired with k-means clustering at different cluster sizes. Clustering quality was evaluated using the Calinski-Harabasz and Davies-Bouldin indices, Silhouette Score, and standard deviation. Nonparametric tests were used to assess the impact of topological descriptors on clustering outcomes. Our results show that the DBN + k-means combination is the most effective, particularly at lower cluster counts, demonstrating significant sensitivity to structural variations. This study highlights the potential of integrating spectral geometry with deep learning for precise and efficient molecular clustering.

A novel approach to coral species classification using deep learning and unsupervised feature extraction

ABSTRACT This study presents a novel methodology to enhance coral species classification in underwater images by integrating Tri Convolutional Deep Autoencoders (TRI-CDAE) and Sparse Deep Autoencoders (SPDAE). TRI-CDAE employs three parallel CDAEs trained on distinct color spaces (RGB, HSV, LUV) to capture diverse features. These extracted features are fused and refined using SPDAE, promoting sparsity and enhancing discriminative power. The refined features are then classified using a softmax classifier. Evaluation on four coral image datasets shows exceptional performance, with recall (96.5%), F1 score (97.4%), precision (97.2%), accuracy (98.5%), Cohen’s J (0.959), Jaccard Index (0.971), Cohen’s Kappa (0.961), and Matthews Correlation Coefficient (0.982).

A novel deep contrastive convolutional autoencoder based binning approach for taxonomic independent metagenomics data

A novel deep contrastive convolutional autoencoder based binning approach for taxonomic independent metagenomics data

Unsupervised Hybrid Models Integrating Deep Autoencoders and Process Controllers’ Models for Enhanced Process Monitoring and Fault Detection

Unsupervised Hybrid Models Integrating Deep Autoencoders and Process Controllers’ Models for Enhanced Process Monitoring and Fault Detection

Deep clustering hierarchical latent representation for anomaly-based cyber-attack detection

In the field of anomaly detection, well-known techniques and state-of-the-art models often face challenges when interpreting the latent space, which hinders their behavioral classification accuracy. Firstly, the sub-optimal distribution of data points within the latent space makes normal behavioral regions verbose and indistinguishable from abnormal regions. Secondly, within the latent space, it can be difficult to identify meaningful, separable, and indicative features. Finally, the processing time at the inference stage is still relatively slow.This paper aims to improve the accuracy of network anomaly detection mechanisms by proposing two novel deep hierarchical representation learning models: Deep Nested Clustering Auto-Encoder (DNCAE) and Deep Clustering Hierarchical Auto-Encoder (DCHAE). Both models adopt a nested branch structure, utilizing dual deep auto-encoders to establish hierarchical latent spaces; in each, clustering algorithms are used to spatially optimize and refine the data points. This approach results in improved separation between normal and abnormal data points, and easier identification of notable and/or indicative features.To ascertain the effectiveness of the approach and the quality of resulting features, both models were used in conjunction with ten different one-class anomaly detectors. Each of these ten anomaly detectors was evaluated on popular network intrusion datasets, notably: NSL-KDD, UNSW-NB15, CIC-IDS-2017, CSE-CIC-IDS-2018, and CTU13. Experimental results have confirmed that both of the proposed models produced higher levels of accuracy than existing baselines and current state-of-the-art models. Additionally, the processing time at the inference stage shows a significant reduction.