Variational Autoencoder Model Research Articles

VAEAT: Variational AutoeEncoder with adversarial training for multivariate time series anomaly detection

High labor costs and the requirement for significant domain expertise often result in a lack of anomaly labels in most time series. Consequently, employing unsupervised methods becomes critical for practical industrial applications. However, prevailing reconstruction-based anomaly detection algorithms encounter challenges in capturing intricate underlying correlations and temporal dependencies in time series. This study introduces an unsupervised anomaly detection model called Variational AutoeEncoder with Adversarial Training for Multivariate Time Series Anomaly Detection (VAEAT). Its fundamental concept involves adopting a two-phase training strategy to improve anomaly detection precision through adversarial reconstruction of raw data. In the first phase, the model reconstructs raw data to extract its basic features by training two enhanced variational autoencoders (VAEs) that incorporate both the long short-term memory (LSTM) network and the attention mechanism in their common encoder. In the second phase, the model refines reconstructed data to optimize the reconstruction quality. In this manner, this two-phase VAE model effectively captures intricate underlying correlation and temporal dependencies. A large number of experiments are conducted to evaluate the performance on five publicly available datasets, and experimental results illustrate that VAEAT exhibits robust performance and effective anomaly detection capabilities. The source code of the proposed VAEAT can be available at https://github.com/Du-Team/VAEAT.

Revealing intracavitary catheter positions: artificial intelligence-enhanced localization with an electrogram-driven variational autoencoder model

Abstract Background The acquisition of electroanatomical (EA) maps in clinical electrophysiology procedures typically depends on locating intracardiac mapping catheters through magnetic triangulation with external patches placed on the patient's skin. However, this configuration is hampered by shifts in the reference system if the patient moves or if the setup changes during the procedure. Leveraging the feature extraction capabilities of artificial intelligence models, it becomes possible to assess catheter positions within the atria solely based on the information encoded in intracavitary electrograms (EGMs). Consequently, a neural network model could serve as an alternative or complementary system for locating EA mapping catheters, eliminating the need for external reference patches. Purpose We aimed to create a deep learning model utilizing a disentangled variational autoencoder (d-VAE) and unipolar EGM signals. The goal was to extract the precise position of a multi-electrode catheter within the left atrium and predict the specific atrial region in which it is situated, relying solely on the information derived from unipolar EGMs. Methods The model was trained using data from 51 patients with persistent atrial fibrillation who underwent pulmonary vein ablation. We exported 3D EA maps obtained with a 20-pole multi-electrode catheter, comprising a total of 40,000 unipolar EGMs with a duration of 30 seconds. Atrial geometries were segmented into 12 different areas (refer to the Figure). Subsequently, the d-VAE model was trained with unipolar EGMs to acquire encoded features, which were then input into a classifier model. The classifier was combined with atrial positions using one-hot encodings as the ground truth. The optimal architecture was selected through a grid search involving different layer numbers, sizes, and fine-tuning. For the encoding part, we assessed the negative evidence lower bound (ELBO) and Pearson’s correlation. Regarding the classifier, we evaluated accuracy, extended accuracy (indicating accuracy in predicting the region or an adjacent one), and the area under the curve (AUC). Results The best-performing autoencoder architecture presented a negative ELBO of -0.087 and a Pearson’s correlation of 0.971; and the classifier obtained an accuracy of 38.1%, an extended accuracy of 75.2%, and an AUC of 0.814. Conclusion We introduced an alternative method for localizing intracardiac catheters within the atrial chambers based on deep-learning techniques. In contrast to traditional hardware technologies, our proposed method eliminates the need for external magnetic or impedance reference patches. This approach is not limited to AF and can be extended to other supra-ventricular procedures. The results of our proposed method are promising, suggesting potential advancements in alternative systems for locating intracardiac catheters to complement existing classical hardware methods in the future.Variational autoencoder architecture

The manatee variational autoencoder model for predicting gene expression alterations caused by transcription factor perturbations

We present the Manatee variational autoencoder model to predict transcription factor (TF) perturbation-induced transcriptomes. We demonstrate that the Manatee in silico perturbation analysis recapitulates target transcriptomic phenotypes in diverse cellular lineage transitions. We further propose the Manatee in silico screening analysis for prioritizing TF combinations targeting desired transcriptomic phenotypes.

Dependency-aware deep generative models for multitasking analysis of spatial omics data.

Spatially resolved transcriptomics (SRT) technologies have significantly advanced biomedical research, but their data analysis remains challenging due to the discrete nature of the data and the high levels of noise, compounded by complex spatial dependencies. Here, we propose spaVAE, a dependency-aware, deep generative spatial variational autoencoder model that probabilistically characterizes count data while capturing spatial correlations. spaVAE introduces a hybrid embedding combining a Gaussian process prior with a Gaussian prior to explicitly capture spatial correlations among spots. It then optimizes the parameters of deep neural networks to approximate the distributions underlying the SRT data. With the approximated distributions, spaVAE can contribute to several analytical tasks that are essential for SRT data analysis, including dimensionality reduction, visualization, clustering, batch integration, denoising, differential expression, spatial interpolation, resolution enhancement and identification of spatially variable genes. Moreover, we have extended spaVAE to spaPeakVAE and spaMultiVAE to characterize spatial ATAC-seq (assay for transposase-accessible chromatin using sequencing) data and spatial multi-omics data, respectively.

Explaining deep learning-based representations of resting state functional connectivity data: focusing on interpreting nonlinear patterns in autism spectrum disorder.

Resting state Functional Magnetic Resonance Imaging fMRI (rs-fMRI) has been used extensively to study brain function in psychiatric disorders, yielding insights into brain organization. However, the high dimensionality of the rs-fMRI data presents significant challenges for data analysis. Variational autoencoders (VAEs), a type of neural network, have been instrumental in extracting low-dimensional latent representations of resting state functional connectivity (rsFC) patterns, thereby addressing the complex nonlinear structure of rs-fMRI data. Despite these advances, interpreting these latent representations remains a challenge. This paper aims to address this gap by developing explainable VAE models and testing their utility using rs-fMRI data in autism spectrum disorder (ASD). One-thousand one hundred and fifty participants (601 healthy controls [HC] and 549 patients with ASD) were included in the analysis. RsFC correlation matrices were extracted from the preprocessed rs-fMRI data using the Power atlas, which includes 264 regions of interest (ROIs). Then VAEs were trained in an unsupervised manner. Lastly, we introduce our latent contribution scores to explain the relationship between estimated representations and the original rs-fMRI brain measures. We quantified the latent contribution scores for both the ASD and HC groups at the network level. We found that both ASD and HC groups share the top network connectivitives contributing to all estimated latent components. For example, latent 0 was driven by rsFC within ventral attention network (VAN) in both the ASD and HC. However, we found significant differences in the latent contribution scores between the ASD and HC groups within the VAN for latent 0 and the sensory/somatomotor network for latent 2. This study introduced latent contribution scores to interpret nonlinear patterns identified by VAEs. These scores effectively capture changes in each observed rsFC feature as the estimated latent representation changes, enabling an explainable deep learning model that better understands the underlying neural mechanisms of ASD.

Adaptive Temporal-Spatial Pyramid Variational Autoencoder Model for Multirate Dynamic Chemical Process Soft Sensing Application.

Data-driven soft sensors play an important role in practical processes and have been widely applied. They provide real-time prediction of quality variables and then guide production and improve product quality. In practical chemical production processes, nonlinear dynamic multirate data is widespread and challenging to model. This paper innovatively proposes a temporal-spatial pyramid variational autoencoder (TS-PVAE) model for the nonlinear temporal-spatial feature pyramid extraction from multirate data. This structure not only selectively utilizes multirate data but also handles complex nonlinear time-series data. Based on this, integrated with just-in-time (JIT) learning, an adaptive TS-PVAE (ATS-PVAE) model is developed. In this model, historical data are used for real-time fine-tuning of the model, leading to the development of an adaptive model. Finally, the proposed models are validated by an industrial case of a methanation furnace, demonstrating a superior estimation performance.

Graph neural network-based topological relationships automatic identification of geological boundaries

Topological relationship identification of geological boundaries greatly affects 3D geological modeling efficiency. However, due to the complex spatial distribution and unstructured data architecture of geological boundaries, the current topological identification still mainly depends on the manual operation and interpretation, which is time-consuming and labor-intensive. And geologists' judgments are based on knowledge and experience that are challenging to replicate through conventional algorithmic approaches. To address these challenges, a graph neural network-based geological boundary topological relationships automatic identification method is proposed. Based on graph theory, we conceptualize geological boundaries and their topological relationships as a graph transforming the identification task into an edge prediction problem within this topological graph. We employ the variational graph auto-encoder (VGAE) model to predict unknown edges in the topological graph, enabling automatic identification of geological boundary topological relationships. A geological information transformation method is proposed to extract and encode the geological boundaries’ features as inputs for VGAE. The proposed method is verified in an engineering case study. The results reveal that this method can efficiently and accurately analyze the geological boundary topological relationships. Moreover, it can greatly reduce the manual workload, requiring only 30% of the manual effort to obtain the geological boundary topological relationships, while achieving an average prediction accuracy of approximately 95%.

Dimensionality reducing Gaussian mixture‐based reconstruction for fault detection in multimode processes

AbstractModern industrial processes increasingly prioritize demands for safety and reliability, spurring substantial research on process monitoring models. Among existing research subjects, concurrent multimode operating conditions are vital for effective process monitoring. This work proposes an efficient dimensionality‐reducing Gaussian mixture‐based reconstruction approach for multimode industrial process monitoring. The t‐SNE method is first employed to transform high‐dimensional data into a lower‐dimensional space that retains critical operational information. Using these reduced dimensions, a robust Gaussian mixture model is established to partition the operation data into different modes. Furthermore, the original data are assigned to the corresponding operating modes, and local variational autoencoder (VAE) reconstruction models are established, respectively. For each VAE model, two statistics are designed, termed and , to detect abnormalities. The proposed method is applied to a three‐phase flow facility, and the superiority over the comparison methods is proved.

A domain knowledge-informed design space exploration methodology for mechanical layout design

Layout designs have a large number of design variables and various physical constraints. Conventional design exploration approaches are time-consuming and may require human intervention. A unique feature about physical layout designs is the availability of domain knowledge, which can be utilised to speed up the design process. In this paper, we propose a generative deep learning-based design space exploration (DSE) methodology that is capable of learning design constraints in the layout design without explicit supervision. Moreover, it can incorporate domain knowledge in the generated layouts, thereby speeding up the design process. This is realised by constructing a layout generation variational autoencoder (LGVAE) model, which uses a latent space as an interface to generate the layouts. By training the LGVAE model, significantly lower-dimensional representations can be learned compared to the original dimensionality of the design space. Therefore, the number of design variables is greatly reduced. We showcase the performance of the proposed DSE approach by solving the heat source layout design problem encountered in thermal management of chips. Experiments demonstrate that the LGVAE model is capable of generating compressed latent representations that capture the characteristics of the input samples, which makes the DSE cost-effective.

심전도 이상 탐지를 위한 Variational Autoencoder 비교 연구

Electrocardiogram is widely used as a tool for detecting and diagnosing heart disease. Previous research revealed that deep learning models based on supervised learning have limited performance due to imbalance between normal and anomaly. To overcome this problem, anomaly detection models using variational autoencoder (VAE) have been proposed. After training VAE model using normal data, a threshold to distinguish normal and anomaly is set by using validation data, then test data is applied. VAE model has encoder, a latent variable layer, and decoder. and deep learning models are embedded in encoder and decoder. In this paper, VAE models embedding MLP, RNN, and LSTM are compared under various settings which are the type of data (original data or the first-order differential data) and the assumption for the variance of decoder. As results of analysis, VAE model based on LSTM is performed best when original data is used and a deep learning model for the variance of decoder is embedded. Additionally, the performance of VAE models using original data were better than models using first-order differential data.

An LSTM-based adversarial variational autoencoder framework for self-supervised neural decoding of behavioral choices.

Objective. This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: 1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and 2) extraction of subject-invariant features for the development of generalized neural decoding models.

Approach. For behavioral analysis and label generation, an unsupervised method, which employs an autoencoder to transform behavior data into a cluster-friendly feature space is presented. The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding neural activity, adversarial learning in combination with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) model is employed. By using an adversary network to constrain the latent representations, the model captures shared information among subjects' neural activity, making it proper for cross-subject transfer learning.

Main results. The proposed approach is evaluated using cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging during a motivational licking behavioral experiment. The results show that the proposed model achieves an accuracy of 89.7% in cross-subject neural decoding, outperforming other well-known autoencoder-based feature learning models. These findings suggest that incorporating an adversary network eliminates subject dependency in representations, leading to improved cross-subject transfer learning performance, while also demonstrating the effectiveness of LSTM-based models in capturing the temporal dependencies within neural data.

Significance. Results demonstrate the feasibility of the proposed framework in unsupervised clustering and label generation of behavioral data, as well as achieving high accuracy in cross-subject neural decoding, indicating its potentials for relating neural activity to behavior.

Abstract 2281: A tumor-intrinsic signature involving immunosuppression via MIF-CD74 signaling is associated with overall survival in ICT-treated lung adenocarcinoma

Abstract Introduction: Immune checkpoint therapies (ICT) have changed cancer care, yielding robust and durable responses in a subset of patients. Identifying patients who are likely to respond to ICT remains an ongoing challenge. In addition, only a portion of patients with clinical biomarkers respond to therapy. Signatures of RNA expression have been developed to predict response, the majority of which focus on T-cell and cytotoxicity markers, yet have been unable to substantially improve outcome predictions. Here, we present a RNA signature that instead describes tumor-intrinsic immune resistance and a potential mechanism of immunosuppression via tumor signaling on macrophages, derived from single-cell RNA-sequencing (scRNA-seq). Methods: We performed dimensionality reduction using a Variational Autoencoder (VAE) on 30 scRNA-seq samples from 15 lung adenocarcinoma (LUAD) patients, comprising a total of 183,873 cells from the Cell Ranger pipeline. The VAE model was trained on each sample for 250 iterations, yielding 20 signatures from each patient, for a total of 300 signatures. The relationship of each signature with real-world overall survival (rwOS) across 1,983 bulk RNA-sequenced LUAD patients treated with an FDA approved ICT was assessed via a Cox proportional hazards model with risk set adjustment, using TMB and line of therapy as covariates. The NATMI python package was used to identify putative active ligand-receptor interactions between tumor cells and the immune environment. Results: Among the VAE encodings, we identified an immune resistance signature significantly associated with decreased rwOS across the patient cohort (HR = 4.2 [2.8-6.3], adjusted p = 4.3e-10). This signature was derived from a small cluster of neoplastic cells (4.4% of cells) in a patient sample that was otherwise dominated by immune cells, including a substantial fraction of cytotoxic CD8 T cells (24.5%). The strongest predicted ligand-receptor interactions were found between the neoplastic cells and macrophages, via the MIF-CD74 interaction, an interaction we found upregulated in multiple single-cell tumor samples. Further, MIF RNA expression alone was significantly associated with rwOS across the 1,983 patient LUAD cohort (HR = 1.5 [1.1-2.1], p = 0.004). Discussion: MIF-CD74 is a known immunosuppressive interaction and MIF signaling from tumor cells on macrophages has been previously shown to have immunosuppressive effects in a mouse model of melanoma that were largely reversible via MIF-CD74 blockade. Taken together, these results identify a signature of tumor intrinsic immune suppression that can indicate patients likely to experience reduced benefit from ICT. In addition, this signature provides evidence to support blockade of the MIF-CD74 axis as a means to enhance anti-tumor immune responses in LUAD. Citation Format: Rossin Erbe, Michelle M. Stein, Tim A. Rand, Justin Guinney. A tumor-intrinsic signature involving immunosuppression via MIF-CD74 signaling is associated with overall survival in ICT-treated lung adenocarcinoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2281.

Abstract 4912: Unsupervised detection of stromal phenotypes with distinct fibrogenic and inflamed properties in NSCLC

Abstract Background: Understanding the composition of cancer-associated stroma (CAS) is vital, as the number and location of immune cells and fibroblasts, as well as the degree of extracellular matrix deposition, have implications for cancer progression and response to treatment, including in non-small cell lung cancer (NSCLC). Manual analysis of CAS does not fully describe the stromal milieu, especially from a spatial perspective, and is highly subjective. To this end, we have developed an unsupervised machine learning (ML) model to characterize the CAS in NSCLC from hematoxylin and eosin (H&E) stained whole slide images (WSI) at scale. Methods: PathExploreTM models were deployed to predict stromal tissue and cell types, while another ML model was used to detect collagen fibers from H&E stained WSIs from the TCGA LUAD (N=536) and LUSC (N=464) datasets. Stroma was divided into small regions (median = 0.02 mm2), and 88 features characterizing cell distribution, tissue composition and fiber density were extracted from each region. Graphs were generated connecting neighboring regions (nodes), and an unsupervised variational graph auto-encoder (VGAE) model was trained to learn 8 latent features through dimensionality reduction. Stromal phenotypes were then derived from the latent features using k-means clustering. The fraction of each phenotype in the stroma was correlated against immune- and stroma-related gene expression signatures (GES) and overall survival (OS). Results: Deployment of VGAE on LUAD and LUSC WSIs revealed three distinct stromal phenotypes - P0, P1 and P2. Fibroblast density was elevated in P0 and P1 regions (p<0.001), immune cell density was elevated in P2 regions (p<0.001), and collagen fiber intensity was highest in P1 regions (p<0.001). P2 enrichment was correlated with elevated expression of the T cell-inflamed gene expression profile (TGEP; Spearman ρ = 0.43 in LUAD; ρ = 0.27 in LUSC) and with improved OS (HR = 0.696; 95% CIs: 0.571-0.847 in LUSC). Conversely, P1 enrichment was positively associated with a transforming growth factor-β-induced cancer associated fibroblast GES (TGFβ-CAF: ρ = 0.19 in LUAD and ρ = 0.12 in LUSC) and poor OS (HR = 1.358; 95% CIs: 1.149-1.603 in LUSC). These phenotypes are consistent with fibroblast-enriched, collagen-depleted stroma (P0), collagen-rich, fibroblast-enriched tumor-promoting stroma (P1), and immune cell-enriched, tumor-suppressive stroma (P2). Conclusions: We describe an unsupervised, data-driven method of predicting stromal regions with discrete patterns of cell composition and collagen deposition in NSCLC. This approach identified three phenotypes of NSCLC stroma. These results highlight the ability of ML models to characterize and find meaningful patterns within the cell, tissue, and matrix components of a tumor. This work provides further evidence of the potential of ML to discover novel precision medicine biomarkers in NSCLC. Citation Format: Neel Patel, Nhat Le, Tan Nguyen, Fedaa Najdawi, Sandhya Srinivasan, Adam Stanford-Moore, Deeksha Kartik, Jun Zhang, Jacqueline Brosnan-Cashman, Robert Egger, Justin Lee, Matthew Bronnimann. Unsupervised detection of stromal phenotypes with distinct fibrogenic and inflamed properties in NSCLC [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4912.

A Novel Distributed Process Monitoring Framework of VAE-Enhanced with Deep Neural Network

Intelligent manufacturing process needs to adopt distributed monitoring scenario due to its massive, high-dimensional and complex data. Distributed process monitoring has been introduced into global monitoring and local monitoring to analyze the characteristic relationship between process data. However, the existing framework methods ignore or suppress the fault information and thus cannot effectively identify the local faults and the time sequence characteristics between units in the batch production system. This paper proposes a novel distributed process monitoring framework based on Girvan-Newman algorithm modular subunit partitioning and probabilistic learning model with deep neural networks. First, Girvan-Newman algorithm is used to divide the complex manufacturing system modularized to reduce the latitude of data processing. Second, variational autoencoder (VAE) is adopted to ensure the stability of local analysis, and long short-term memory is adopted to improve the VAE model to detect global multi-time scale anomalies. Finally, distributed process fault detection is carried out for each subunit in a separate and integrated manner, and the performance of the framework in distributed process monitoring is analyzed through two fault detection indicators T2 and SPE statistics. A case study of the Tennessee Eastman Process is used to demonstrate the performance and applicability of the proposed framework. Results show that the proposed VAE enhancement framework based on the DNN could accurately identify faults in distributed process monitoring and locate the specific sub-units where the fault occurs. Compared with VAE-DNN method and traditional process monitoring methods, the framework proposed in this paper has higher fault detection rate and lower false alarm rate, and the detection rate of some faults can reach 100%.

Comparison of VAE model and diffusion model in lung cancer images generation

In the rapidly evolving domain of medical imaging, there's an increasing interest in harnessing deep learning models for enhanced diagnosis and prognosis. Among these, the Variational Autoencoder (VAE) and the Diffusion model stand out for their potential in generating synthetic lung cancer images. This research article delves into a comparative analysis of both models, focusing on their application in lung cancer imaging. Drawing from the "Iraq-Oncology Teaching Hospital/National Center for Cancer Diseases (IQ-OTH/NCCD) lung cancer dataset," the study investigates the efficiency, accuracy, and fidelity of the images generated by each model. The findings suggest that while the VAE model offers faster image generation, its output is notably blurrier than its counterpart. Conversely, the Diffusion model, despite its relatively slower speed, is capable of producing highly detailed synthetic images even with limited epochs. This comprehensive comparison not only highlights the strengths and shortcomings of each model but also lays the groundwork for further refinements and potential clinical implementations. The broader objective is to catalyze advancements in lung cancer diagnosis, ultimately leading to better patient outcomes.

Recognition of Yuan blue and white porcelain produced in Jingdezhen based on graph anomaly detection combining portable X-ray fluorescence spectrometry

The blue and white porcelain produced in Jingdezhen during China’s Yuan Dynasty is an outstanding cultural heritage of ceramic art that has attracted wide attention for its identification. However, the traditional visual identification method is susceptible to misjudgment, thermoluminescence dating damages the samples, and the methods based on chemical analysis are limited by the accuracy and specificity of the elemental features. In this paper, we address the identification challenge by using machine learning techniques combined with portable X-ray Fluorescence Spectrometer (pXRF) analysis. We collect a large dataset of chemical compositions of Yuan blue and white porcelain from Jingdezhen using pXRF, and propose a graph anomaly detection method based on gradient attention map (GRAM) to identify the porcelain from different dynasties. We treat the porcelain produced in the Yuan dynasty as normal samples and those from other dynasties as abnormal samples. For GRAM, we merely train the variational graph autoencoder (VGAE) model with normal graphs and then use its encoder to extract graph features and compute the anomaly scores by utilizing the GRAM of the graph representations with respect to the node feature embeddings. Finally, we compare GRAM with state-of-the-art graph anomaly detection techniques and show that it achieves superior performance.

Advanced collision risk estimation in terminal manoeuvring areas using a disentangled variational autoencoder for uncertainty quantification

Air Traffic Management aims at ensuring safety during aircraft operations, particularly within Terminal Manoeuvring Areas where traffic density is high. The challenge lies in balancing safety and efficiency by closely managing the likelihood of mid-air collisions regarding the airport movements. Traditional models like the Reich and Anderson-Hsu models have been influential, but they fall short in representing the complex reality of Terminal Manoeuvring Areas. Data-driven approaches are emerging, with Monte Carlo simulations offering a more flexible methodology for collision risk estimation. This paper introduces a framework for assessing Mid-Air Collision likelihood resulting from Terminal Manoeuvring Area procedures by combining the field of Deep Generative Modelling using a Variational Autoencoder with the domain of rare event statistics through Subset Simulation. By incorporating disentanglement into the Variational Autoencoders model, we create a latent space that aligns dimensions with distinctive trajectory traits. Then, Subset Simulation is employed to gauge Mid-Air Collision probability, utilizing latent representations as input. Finally, sensitivity analysis reveals pivotal factors influencing collision risk, correlated with trajectory attributes via disentanglement. The methodology is applied to traffic around Zurich Airport: it evaluates the risk arising from go-around and take-off procedures using Automatic Dependent Surveillance-Broadcast data.

Robust errant beam prognostics with conditional modeling for particle accelerators

Particle accelerators are complex and comprise thousands of components, with many pieces of equipment running at their peak power. Consequently, they can fault and abort operations for numerous reasons, lowering efficiency and science output. To avoid these faults, we apply anomaly detection techniques to predict unusual behavior and perform preemptive actions to improve the total availability. Supervised machine learning (ML) techniques such as siamese neural network models can outperform the often-used unsupervised or semi-supervised approaches for anomaly detection by leveraging the label information. One of the challenges specific to anomaly detection for particle accelerators is the data’s variability due to accelerator configuration changes within a production run of several months. ML models fail at providing accurate predictions when data changes due to changes in the configuration. To address this challenge, we include the configuration settings into our models and training to improve the results. Beam configurations are used as a conditional input for the model to learn any cross-correlation between the data from different conditions and retain its performance. We employ conditional siamese neural network (CSNN) models and conditional variational auto encoder (CVAE) models to predict errant beam pulses at the spallation neutron source under different system configurations and compare their performance. We demonstrate that CSNNs outperform CVAEs in our application.

Wind power anomaly data detection based on unsupervised methods

During the actual operation of the wind turbine, a large number of abnormal data will be generated due to environmental or human factors, which will have a great impact on the condition assessment and output prediction. In order to make wind energy a reliable source of energy, it is very important to establish an efficient and accurate wind power detection model. Therefore, it becomes essential to identify abnormal data for a more precise evaluation of wind turbine performance. Based on the data mining clustering technology K-means algorithm, this paper introduces an unsupervised abnormal wind power detection algorithm combining the variational autoencoders (VAE) model. The focus of this abnormal wind power detection method is primarily on assessing the reconstruction error, which, in turn, generates an abnormality score for wind power data. This score is then used to determine the presence of abnormal wind power data. Finally, the wind power data in 2021 is tested.

Application of Variational Graph Autoencoder in Traction Control of Energy-Saving Driving for High-Speed Train

In a high-speed rail system, the driver repeatedly adjusts the train’s speed and traction while driving, causing a high level of energy consumption. This also leads to the instability of the train’s operation, affecting passengers’ experiences and the operational efficiency of the system. To solve this problem, we propose a variational graph auto-encoder (VGAE) model using a neural network to learn the posterior distribution. This model can effectively capture the correlation between the components of a high-speed rail system and simulate drivers’ operating state accurately. The specific traction control is divided into two parts. The first part employs an algorithm based on the K-Nearest Neighbors (KNN) algorithm and undersampling to address the negative impact of imbalanced quantities in the training dataset. The second part utilizes a variational graph autoencoder to derive the initial traction control of drivers, thereby predicting the energy performance of the drivers’ operation. An 83,786 m long high-speed train driving section is used as an example for verification. By using a confusion matrix for our comparative analysis, it was concluded that the energy consumption is approximately 18.78% less than that of manual traction control. This shows the potential and effect of the variational graph autoencoder model for optimizing energy consumption in high-speed rail systems.