Adversarial Autoencoder Research Articles

An unsupervised adversarial autoencoder for cyber attack detection in power distribution grids

Detection of cyber attacks in smart power distribution grids with unbalanced configurations poses challenges due to the inherent nonlinear nature of these uncertain and stochastic systems. It originates from the intermittent characteristics of the distributed energy resources (DERs) generation and load variations. Moreover, the unknown behavior of cyber attacks, especially false data injection attacks (FDIAs) in the distribution grids with complex temporal correlations and the limited amount of labeled data increases the vulnerability of the grids and imposes a high risk in the secure and reliable operation of the grids. To address these challenges, this paper proposes an unsupervised adversarial autoencoder (AAE) model to detect FDIAs in unbalanced power distribution grids integrated with DERs, i.e., PV systems and wind generation. The proposed method utilizes long short-term memory (LSTM) in the structure of the autoencoder to capture the temporal dependencies in the time-series measurements and leverages the power of generative adversarial networks (GANs) for better reconstruction of the input data. The advantage of the proposed data-driven model is that it can detect anomalous points for the system operation without reliance on abstract models or mathematical representations and data labels. To evaluate the efficacy of the approach, it is tested on IEEE 13-bus and 123-bus systems with historical meteorological data (wind speed, ambient temperature, and solar irradiance) as well as historical real-world load data under three types of data falsification functions. The comparison of the detection results of the proposed model with other data-driven methods verifies its superior performance in detecting cyber attacks in unbalanced power distribution grids.

Hyperspectral target detection using self-supervised background learning

Hyperspectral target detection is challenging in scenarios where spectral variability is high due to noise, spectral redundancy, and mixing. In addition, this spectral variability also creates the need for target detection algorithms to be robust against variations in the detection threshold. To overcome these challenges, this paper proposes a novel two-stage process for improved target detection in hyperspectral data. In the first stage, coarse detection is performed using a detector with a high probability of detection to identify background samples. These background samples are then used for background learning using an adversarial autoencoder (AAE) network, having spectral angle mapper (SAM) and Huber loss functions to minimize the impact of target pixels’ contamination. In the second stage, an inference is made using the spectral difference between the hyperspectral data and the output of the learned background model, which helps in reducing the false alarm rate of the first stage. The proposed approach is compared with seven other target detection techniques using multiple datasets and evaluated through several metrics, such as the area under the curve (AUC) and signal-to-noise probability ratio (SNPR). Results reveal that the proposed technique outperforms other detectors in terms of SNPR, indicating improved target detectability, background suppressibility, and more tolerance to variations in the detection threshold.

Synergizing meat Science and AI: Enhancing long-chain saturated fatty acids prediction

In the field of the food industry, establishing a global predictive model for the content of long-chain saturated fatty acids (LC-SFAs) in red meat is of profound significance. However, this work requires the accumulation of a large number of diverse samples for model calibration. To address these formidable challenges, the Generative Inference Adversarial Autoencoder (GI-AAE) was adopted to enhance the model training process. Through multiple iterations, GI-AAE successfully generated simulated samples of higher quality than traditional generative adversarial networks and autoencoders. After incorporating these generated samples into the training dataset, multiple sets of models showed significant improvements. These results clearly demonstrated that our approach can effectively overcome the challenges posed by generative models in the regression domain, greatly enhancing the model's generalization ability. This advancement holds significant potential for applications in the food industry.

History matching for geological carbon storage using data-space inversion with spatio-temporal data parameterization

History matching based on monitoring data will enable uncertainty reduction, and thus improved aquifer management, in industrial-scale carbon storage operations. In traditional model-based data assimilation, geomodel parameters are modified to force agreement between flow simulation results and observations. In data-space inversion (DSI), history-matched quantities of interest, e.g., posterior pressure and saturation fields conditioned to observations, are inferred directly, without constructing posterior geomodels. This is accomplished efficiently using a set of O(1000) prior simulation results, data parameterization, and posterior sampling within a Bayesian setting. In this study, we develop and implement (in DSI) a deep-learning-based parameterization to represent spatio-temporal pressure and CO2 saturation fields at a set of time steps. The new parameterization uses an adversarial autoencoder (AAE) for dimension reduction and a convolutional long short-term memory (convLSTM) network to represent the spatial distribution and temporal evolution of the pressure and saturation fields. This parameterization is used with an ensemble smoother with multiple data assimilation (ESMDA) in the DSI framework to enable posterior predictions. A realistic 3D system characterized by prior geological realizations drawn from a range of geological scenarios is considered. A local grid refinement procedure is introduced to estimate the error covariance term that appears in the history matching formulation. Extensive history matching results are presented for various quantities, for multiple synthetic true models, using the new DSI framework. Substantial uncertainty reduction in posterior pressure and saturation fields is achieved in all cases. The framework is also applied to efficiently provide posterior predictions for a range of error covariance specifications. Such an assessment would be very expensive using a traditional model-based approach.

Bi-AAE: A binary adversarial autoencoder deep neural network model for anomaly detection in system-levels marine diesel engines

Marine diesel engines typically use multivariate time series for health condition monitoring, the anomaly detection of which is fundamental and critical for an entity's operation and management. However, detecting anomalies in multivariate time series remains a grand challenge due to the complex behavior of marine machinery. Therefore, this research proposes a binary adversarial autoencoder model to overcome this issue. The framework is built upon the principles of reconstruction models by fusing Generative Adversarial Networks and Adversarial Autoencoder approaches to analyze the degree of anomaly in multivariate time series. Additionally, it identifies anomalies through a threshold obtained by a statistical distribution-based and unsupervised setting method. To emphasize the model's performance, multivariate condition parameters from actual ship diesel engines are utilized for validation. Two membership functions are recognized as the model metrics to characterize its best performance, evidenced by scores of 0.937 for F1 scores and 0.910 for precision. The proposed strategy can be applied to different types of monitoring systems in the ship's engine room to realize system-level operation and maintenance.

HAVAE – An Advanced Approach for Malware Detection Using Deep Learning

Abstract: The dynamic situation of cybersecurity necessitates continuous adaptation to the evolving and sophisticated nature of malware. This study proposes an innovative approach to enhancing threat detection methodologies by combining Adversarial Autoencoders (AAEs) and Variational Autoencoders (VAEs) for unsupervised malware detection. AAEs, with their EncoderDecoder structure and adversarial techniques, are integrated with VAEs to discern latent representations which are crucial for discriminating between malware and harmless software. This model, referred to as Hybrid Adversarial-Variational Autoencoder (HAVAE), takes advantage of both of their strengths architectures, capturing nuanced features within a latent space through unsupervised learning. The HAVAE model employs the Reparameterization Technique, crucial for sampling latent variables, ensuring the generation of realistic samples while retaining discriminative attributes essential for accurate malware identification. Through comprehensive evaluations across diverse datasets, the efficiency of HAVAE is assessed using metrics encompassing precision, recall, and F1-score. The evaluation underscores the model's robust ability to detect malicious software effectively, emphasizing its potential as a versatile cybersecurity tool. This innovative approach represents a revolution in cybersecurity, utilizing the strength of unsupervised learning techniques, AAEs, and VAEs. The findings signify a significant advancement in adaptive and resilient malware detection systems, illuminating pathways for improved threat identification and mitigation in the ever-evolving cybersecurity landscape

Semantic prior guided fine-grained facial expression manipulation

AbstractFacial expression manipulation has gained wide attention and has been applied in various fields, such as film production, electronic games, and short videos. However, existing facial expression manipulation methods often overlook the details of local regions in images, resulting in the failure to preserve local structures and textures of images. To solve this problem, this paper proposes a local semantic segmentation mask-based GAN (LSGAN) to generate fine-grained facial expression images. LSGAN is composed of a semantic mask generator, an adversarial autoencoder, a transformative generator, and an AU-intensity discriminator. Our semantic mask generator generates eye, mouth, and cheek masks of face images. Then, our transformative generator integrates target expression labels and corresponding facial region features to generate a vivid target facial expression image. In this fashion, we can capture expressions from target face images explicitly. Furthermore, an AU-intensity discriminator is designed to capture facial expression variations and evaluate quality of generated images. Extensive experiments demonstrate that our method achieves authentic face images with accurate facial expressions and outperforms state-of-the-art methods qualitatively and quantitatively.

Early detection of dark-affected plant mechanical responses using enhanced electrical signals

BackgroundMechanical damage to plants triggers local and systemic electrical signals that are eventually decoded into plant defense responses. These responses are constantly affected by other environmental stimuli in nature, for instance, light fluctuation. In recent years, studies on decoding plant electrical signals powered by various machine learning models are increasing in a sense of early prediction or detection of different environmental stresses that threaten plant growth or crop yields. However, the main bottleneck is the low-throughput nature of plant electrical signals, making it challenging to obtain a substantial amount of training data. Consequently, training these models with small datasets often leads to unsatisfactory performance.ResultsIn the present work, we set out to decode wound-induced electrical signals (also termed slow wave potentials, SWPs) from plants that are deprived of light to different extents. Using non-invasive electrophysiology, we separately collected sets of local and distal SWPs from the treated plants. Then, we proposed a workflow based on few-shot learning to automatically identify SWPs. This workflow incorporates data preprocessing, feature extraction, data augmentation and classifier training. We established the integral and the first-order derivative as features for efficiently classifying SWPs. We then proposed an Adversarial Autoencoder (AAE) structure to augment the SWP samples. Combining them, the Random Forest classifier allowed remarkable classification accuracies of 0.99 for both local and systemic SWPs. In addition, in comparison to two other reported methods, our proposed AAE structure enabled better classification results using our tested features and classifiers.ConclusionsThe results of this study establish new features for efficiently classifying wound-induced electrical signals, which allow for distinguishing dark-affected local and systemic plant wound responses. We also propose a new data augmentation structure to generate virtual plant electrical signals. The methods proposed in this study could be further applied to build models for crop plants using electrical signals as inputs, and also to process other small-scale signals.

Risk-Conditioned Reinforcement Learning: A Generalized Approach for Adapting to Varying Risk Measures

In application domains requiring mission-critical decision making, such as finance and robotics, the optimal policy derived by reinforcement learning (RL) often hinges on a preference for risk management. Yet, the dynamic nature of risk measures poses considerable challenges to achieving generalization and adaptation of risk-sensitive policies in the context of RL. In this paper, we propose a risk-conditioned RL model that enables rapid policy adaptation to varying risk measures via a unified risk representation, the Weighted Value-at-Risk (WV@R). To sample risk measures that avoid undue optimism, we construct a risk proposal network employing a conditional adversarial auto-encoder and a normalizing flow. This network establishes coherent representations for risk measures, preserving the continuity in terms of the Wasserstein distance on the risk measures. The normalizing flow is used to support non-crossing quantile regression that obtains valid samples for risk measures, and it is also applied to the agent’s critic to ascertain the preservation of monotonicity in quantile estimations. Through experiments with locomotion, finance, and self-driving scenarios, we show that our model is capable of adapting to a range of risk measures, achieving comparable performance to the baseline models individually trained for each measure. Our model often outperforms the baselines, especially in the cases when exploration is required during training but risk-aversion is favored during evaluation.

Investigation the influence related to parameters configuration of Generative Adversarial Networks in face image generation

Due to the excellent performance of Generative Adversarial Networks (GAN) for age regression on face images, it is particularly important to explore the effect of different parameters on model training. In this study, the origin and development of Artificial Intelligence (AI) is first discussed, from which the concept and principles of GAN are derived. This is followed by a brief introduction of the UTKface dataset used in this research, and the Conditional Adversarial Autoencoder (CAAE) framework based on the GAN technique. The division of labor and roles of the encoder, generator, and the two discriminators in the model are described. The various learning rates as well as batch size combinations attempted in this study are then illustrated, and the training results of the model are shown in the form of graphs and plots of the loss value function. A situation where the model stops learning is highlighted in the results, which is similar to pattern descent in GAN, and is shown to be characterized by the inability of the discriminator to successfully recognize it. Ultimately, drawing from the acquired outcomes, it can be deduced that employing a larger batch size serves to enhance the pace of model training. It is advisable to concurrently elevate the learning rate by an equivalent factor when augmenting the batch size, thereby ensuring a consistent trajectory for model convergence.

Navigating legal pathways for accelerating urban energy transition: A comprehensive deep analysis of photovoltaic power prediction and policy instruments

This research proposes a navigation method that explores how to accelerate urban energy transformation through legal means from the perspective of energy law, in order to improve the accuracy of photovoltaic output power prediction in sustainable energy transformation. Firstly, a deep learning model based on adversarial autoencoder (AAE) architecture was introduced to predict photovoltaic output power according to clean environment policies. Secondly, in order to effectively fine tune the parameters of AAE, a heuristic optimization method based on the Imperial Competition Algorithm was proposed to optimize the AAE configuration to adapt to the unique characteristics of photovoltaic systems, thereby improving prediction accuracy. In addition, the study analyzed the challenges and opportunities faced by current urban energy transformation, and then proposed sustainable ways to accelerate urban energy transformation from the aspects of formulating transformation plans, promoting technological innovation, promoting green transportation, promoting building energy conservation, and developing circular economy. The synergistic effect of deep learning and heuristic optimization, combined with strict stability and sensitivity assessment, puts this research at the forefront of advancing photovoltaic output power prediction methods, which is of great significance for optimizing renewable energy utilization and grid integration.

Log Anomaly Detection by Adversarial Autoencoders With Graph Feature Fusion

Log Anomaly Detection by Adversarial Autoencoders With Graph Feature Fusion

Anomaly detection using deep learning based model with feature attention

<span lang="EN-US">Anomaly detection is a difficult problem with numerous industrial applications, such as analyzing the quality of objects using images. Anomaly detection is the process of identifying outliers in a given dataset. Recently, machine learning approaches to computer vision problems have outperformed classical state-of-the-art approaches. Anomaly detection problems can be solved using supervised approaches. However, labelled datasets are hard to obtain. Thus, many researchers have taken an unsupervised approach to solving the problem of anomaly detection. In this study, we use an adversarial auto encoder model as a base model and create a custom model to detect anomalies in images and videos. The model was trained exclusively on normal data. The modified national institute of standards and technology database (MNIST) dataset achieved an area under curve (AUC) score of 0.872 for anomaly detection, while the University of California San Diego (UCSD) anomaly dataset (Video dataset) achieved an AUC score of 0.74 for Ped1 and 0.87 for Ped2. To calculate the anomaly score, the concept of attention weights is combined with the reconstruction loss, and the proposed method outperformed other similar methods designed for the same problem. However, the usefulness of the proposed model was demonstrated through the detection of anomalies, and the model is still being improved for use in real-world situations.</span>

Deep learning integrated Bayesian health indicator for cross-machine health monitoring

Constructing a good health indicator (HI) plays a key role in machinery health monitoring. However, due to the training dataset dependency, most conventional deep-learning-based HIs exhibit inconsistent HI scales and limited cross-domain applicability. This study proposes a deep learning integrated Bayesian health indicator (DBHI) to quantify and evaluate system health states in cross-domain applications accurately. The construction of the DBHI comprises two steps: probability space mapping and probabilistic inference. In the first step, the degradation features are extracted from the data and mapped into a probability space by the proposed adversarial autoencoder model. In the second step, the DBHI is calculated by evaluating the relative risk of the current state based on Bayes’ theorem. The effectiveness of the proposed DBHI was validated on two run-to-failure degradation test datasets. In a cross-domain application, the proposed DBHI achieved good performance without retraining compared to other deep learning-based methods that showed invalid results. The proposed DBHI addresses the limitations of conventional deep-learning-based HIs and provides a more reliable and consistent measure of system health, even in a cross-domain application.

Hierarchical Deep Reinforcement Learning-Based Propofol Infusion Assistant Framework in Anesthesia.

This article aims to provide a hierarchical reinforcement learning (RL)-based solution to the automated drug infusion field. The learning policy is divided into the tasks of: 1) learning trajectory generative model and 2) planning policy model. The proposed deep infusion assistant policy gradient (DIAPG) model draws inspiration from adversarial autoencoders (AAEs) and learns latent representations of hypnotic depth trajectories. Given the trajectories drawn from the generative model, the planning policy infers a dose of propofol for stable sedation of a patient under total intravenous anesthesia (TIVA) using propofol and remifentanil. Through extensive evaluation, the DIAPG model can effectively stabilize bispectral index (BIS) and effect site concentration given a potentially time-varying target sequence. The proposed DIAPG shows an increased performance of 530% and 15% when a human expert and a standard reinforcement algorithm are used to infuse drugs, respectively.

Anomaly Detection With Memory-Augmented Adversarial Autoencoder Networks for Industry 5.0

Anomaly Detection With Memory-Augmented Adversarial Autoencoder Networks for Industry 5.0

A deep learning adversarial autoencoder with dynamic batching displays high performance in denoising and ordering scRNA-seq data

By providing high-resolution of cell-to-cell variation in gene expression, single-cell RNA sequencing (scRNA-seq) offers insights into cell heterogeneity, differentiating dynamics, and disease mechanisms. However, challenges such as low capture rates and dropout events can introduce noise in data analysis. Here, we propose a deep neural generative framework, the dynamic batching adversarial autoencoder (DB-AAE), which excels at denoising scRNA-seq datasets. DB-AAE directly captures optimal features from input data and enhances feature preservation, including cell type-specific gene expression patterns. Comprehensive evaluation on simulated and real datasets demonstrates that DB-AAE outperforms other methods in denoising accuracy and biological signal preservation. It also improves the accuracy of other algorithms in establishing pseudo-time inference. This study highlights DB-AAE's effectiveness and potential as a valuable tool for enhancing the quality and reliability of downstream analyses in scRNA-seq research.

Prediction of single-cell RNA expression profiles in live cells by Raman microscopy with Raman2RNA.

Single-cell RNA sequencing and other profiling assays have helped interrogate cells at unprecedented resolution and scale, but are inherently destructive. Raman microscopy reports on the vibrational energy levels of proteins and metabolites in a label-free and nondestructive manner at subcellular spatial resolution, but it lacks genetic and molecular interpretability. Here we present Raman2RNA (R2R), a method to infer single-cell expression profiles in live cells through label-free hyperspectral Raman microscopy images and domain translation. We predict single-cell RNA sequencing profiles nondestructively from Raman images using either anchor-based integration with single molecule fluorescence in situ hybridization, or anchor-free generation with adversarial autoencoders. R2R outperformed inference from brightfield images (cosine similarities: R2R >0.85 and brightfield <0.15). In reprogramming of mouse fibroblasts into induced pluripotent stem cells, R2R inferred the expression profiles of various cell states. With live-cell tracking of mouse embryonic stem cell differentiation, R2R traced the early emergence of lineage divergence and differentiation trajectories, overcoming discontinuities in expression space. R2R lays a foundation for future exploration of live genomic dynamics.

Adversarial random graph neural network for anomaly detection

Anomaly detection is distinguishing unusual objects from normal patterns. It is a complex task due to unpredictable nature of anomalies, which can appear in many forms or they can be hidden by mimicking normal behaviors in a graph structure. Such diversity makes this Deep learning approaches can solve these problems by extracting complex patterns from networks. However, addressing different forms of anomalous instances is essential for successfully implementing these approaches, as different anomaly types require further analysis. Additionally, it is challenging to interpret anomalies beforehand without focusing on every aspect of anomalies. Our objective is to propose an architecture capable of handling all types of anomalous entities by tackling challenges across various domains. In this paper, we introduce ARNAD, a novel framework that integrates three deep models to identify anomalies in graphs: graph neural network, autoencoder, and adversarial autoencoder. ARNAD approaches graph anomaly detection by utilizing the features of the deep parts, and four key elements stand out: (1) the autoencoder learns the overall graph structure and identifies highly deviated ones, (2) the graph neural network exploits graph structure to detect anomalies among the communities, (3) a fixed-size randomized neighborhood that prevents overfitting while reducing complexity (4) the adversarial autoencoder improves the robustness of the framework and discriminates anomalies. To detect anomalies, four receptive components assign risk scores to objects in the attributed network. We evaluated the framework with three synthetic datasets that simulate different behaviors of anomalies and six widely used real attributed networks. Our experimental results show that ARNAD performs competitively with other state-of-the-art models in detecting anomalous entities while minimizing false positives, demonstrating ARNAD's effectiveness in detecting graph anomalies.

Non-Intrusive Speech Quality Assessment Based on Deep Neural Networks for Speech Communication.

Traditionally, speech quality evaluation relies on subjective assessments or intrusive methods that require reference signals or additional equipment. However, over recent years, non-intrusive speech quality assessment has emerged as a promising alternative, capturing much attention from researchers and industry professionals. This article presents a deep learning-based method that exploits large-scale intrusive simulated data to improve the accuracy and generalization of non-intrusive methods. The major contributions of this article are as follows. First, it presents a data simulation method, which generates degraded speech signals and labels their speech quality with the perceptual objective listening quality assessment (POLQA). The generated data is proven to be useful for pretraining the deep learning models. Second, it proposes to apply an adversarial speaker classifier to reduce the impact of speaker-dependent information on speech quality evaluation. Third, an autoencoder-based deep learning scheme is proposed following the principle of representation learning and adversarial training (AT) methods, which is able to transfer the knowledge learned from a large amount of simulated speech data labeled by POLQA. With the help of discriminative representations extracted from the autoencoder, the prediction model can be trained well on a relatively small amount of speech data labeled through subjective listening tests. Fourth, an end-to-end speech quality evaluation neural network is developed, which takes magnitude and phase spectral features as its inputs. This phase-aware model is more accurate than the model using only the magnitude spectral features. A large number of experiments are carried out with three datasets: one simulated with labels obtained using POLQA and two recorded with labels obtained using subjective listening tests. The results show that the presented phase-aware method improves the performance of the baseline model and the proposed model with latent representations extracted from the adversarial autoencoder (AAE) outperforms the state-of-the-art objective quality assessment methods, reducing the root mean square error (RMSE) by 10.5% and 12.2% on the Beijing Institute of Technology (BIT) dataset and Tencent Corpus, respectively. The code and supplementary materials are available at https://github.com/liushenme/AAE-SQA.