Conditional Variational Autoencoder Research Articles

Identifying mechanisms and therapeutic targets in muscle using Bayesian parameter estimation with conditional variational autoencoders.

Machine learning techniques have potential to accelerate discoveries in biologically complex systems. However, they require large data sets and can be challenging in high dimensional systems such as cardiac muscle. In this study, we combined experimental measures of cardiac muscle twitch forces with mechanistic simulations and a newly developed mixture of Bayesian inference with neural networks (in autoencoders) to solve the inverse problem of determining the underlying kinetics for observed force generation by cardiac muscle. The autoencoders are trained on millions of simulations spanning parameter spaces that correspond to the mechanochemistry of cardiac sarcomeres. We apply the trained model to experimental data in order to infer parameters that can explain a diseased twitch and ways to recover it.

Conditional Variational Autoencoder and generative adversarial network-based approach for long-tailed fault diagnosis for the motor system

This study addresses the challenge of diagnosing motor faults in long-tailed data distributions, characterized by dominant healthy states and rare fault types. We propose the LT-CVAE-GAN model, which integrates a Conditional Variational Autoencoder (CVAE) with a Conditional Generative Adversarial Network (CGAN) to enhance long-tailed fault diagnosis. Initially, we train the CVAE-GAN model using traditional CVAE and CGAN losses such as Kullback–Leibler (KL) divergence loss, reconstruction loss, and adversarial loss. Additionally, we introduce mean feature matching loss and pairwise feature matching loss to mitigate mode collapse and improve model stability, thereby enhancing the generation ability of less frequent fault samples under long-tail conditions. Subsequently, the pre-trained Generator is used to produce infrequent fault mode data to rebalance the dataset. Classifier parameters are fine-tuned in this step to improve fault diagnosis accuracy. Experimental results demonstrate that our LT-CVAE-GAN surpasses state-of-the-art models in diverse long-tailed conditions.

Reverse design and application of phononic crystals based on deep learning

Abstract This paper reverse-design phononic crystals with band gaps within a targeted frequency band using the trained conditional variational autoencoder (CVAE) and further studies the vibro-acoustic characteristics of a composite sandwich plate with a phononic crystal panel as the core layer. Firstly, a matrix composed of 0 s and 1 s, representing scatterers and substrates, is randomly generated by MATLAB to represent two-dimensional phononic crystals. The three-dimensional phononic crystals are obtained by stretching the two-dimensional phononic crystals along the average direction, and COMSOL Multiphysics is used to calculate the band gap. In order to maximize the production of phononic crystals with a band gap distribution, the convolutional neural network is trained to predict whether the generated phononic crystals have band gaps. Finally, using data on the structures of phononic crystals and their band gap distributions, the CVAE is trained to achieve the reverse design of artificial periodic structures based on the target band gap. To verify the effectiveness of the structures obtained through the reverse design method on vibration and noise reduction, the submerged vibro-acoustic characteristics of a composite sandwich plate are studied. The plate consists of a phononic crystal panel and carbon fiber panels. The model of the composite sandwich plate is fabricated, and its submerged vibro-acoustic characteristics are tested and compared with numerical results. Finally, the submerged vibro-acoustic response levels of composite sandwich plates with phononic crystal panels and honeycomb panels as core layers are compared using numerical methods. This comparison assesses the phononic crystal panel’s vibration and noise reduction effects.

A balanced mineral prospectivity model of Canadian magmatic Ni (± Cu ± Co ± PGE) sulphide mineral systems using conditional variational autoencoders

A balanced mineral prospectivity model of Canadian magmatic Ni (± Cu ± Co ± PGE) sulphide mineral systems using conditional variational autoencoders

STGlow: A Flow-Based Generative Framework With Dual-Graphormer for Pedestrian Trajectory Prediction.

The pedestrian trajectory prediction task is an essential component of intelligent systems. Its applications include but are not limited to autonomous driving, robot navigation, and anomaly detection of monitoring systems. Due to the diversity of motion behaviors and the complex social interactions among pedestrians, accurately forecasting their future trajectory is challenging. Existing approaches commonly adopt generative adversarial networks (GANs) or conditional variational autoencoders (CVAEs) to generate diverse trajectories. However, GAN-based methods do not directly model data in a latent space, which may make them fail to have full support over the underlying data distribution. CVAE-based methods optimize a lower bound on the log-likelihood of observations, which may cause the learned distribution to deviate from the underlying distribution. The above limitations make existing approaches often generate highly biased or inaccurate trajectories. In this article, we propose a novel generative flow-based framework with a dual-graphormer for pedestrian trajectory prediction (STGlow). Different from previous approaches, our method can more precisely model the underlying data distribution by optimizing the exact log-likelihood of motion behaviors. Besides, our method has clear physical meanings for simulating the evolution of human motion behaviors. The forward process of the flow gradually degrades complex motion behavior into simple behavior, while its reverse process represents the evolution of simple behavior into complex motion behavior. Furthermore, we introduce a dual-graphormer combined with the graph structure to more adequately model the temporal dependencies and the mutual spatial interactions. Experimental results on several benchmarks demonstrate that our method achieves much better performance compared to previous state-of-the-art approaches.

Performance comparison on improved data-driven building energy prediction under data shortage scenarios in four perspectives: Data generation, incremental learning, transfer learning, and physics-informed

Accurate building energy predictions (BEPs) are crucial for maintaining a built environment's sustainability and energy systems. Many data-driven BEPs rely heavily on sufficient data. However, circumstances of practical data shortage restrict the application of these data-driven models to information-poor buildings. This study aims to address this research gap by enhancing BEP performance via the consideration of four perspectives: data generation (DG), incremental learning (IL), transfer learning (TL) and physics-informed (PI). Long short-term memory (LSTM) is selected as the baseline model for comparing performance under extreme (target building dataset: 1-week), heavy (4-weeks) and mild (13-weeks) data shortage scenarios with respect to prediction errors, time costs and the Taylor diagram, among others. The implementation algorithms of four LSTM improved models are conditional variational autoencoder (CVAE), crude incremental learning, domain-adversarial neural network and statistics-informed (SI) for DG, IL, TL and PI, respectively. The effectiveness of the four enhanced models was confirmed using the open-source dataset Building Data Genome Project 2. A comprehensive and fair performance comparison recommends CVAE for extreme and heavy data shortage scenarios with average mean absolute per cent error (MAPE) of 12.79 % and 8.63 % as well as performance improvement ratios (PIRs) of 0.45 and 0.57. SI should be recommended for mild data shortage scenarios with MAPE of 8.01 % and PIR of 0.66.

Airfoil Shape Generation and Feature Extraction Using the Conditional VAE-WGAN-gp

A machine learning method was applied to solve an inverse airfoil design problem. A conditional VAE-WGAN-gp model, which couples the conditional variational autoencoder (VAE) and Wasserstein generative adversarial network with gradient penalty (WGAN-gp), is proposed for an airfoil generation method, and then, it is compared with the WGAN-gp and VAE models. The VAEGAN model couples the VAE and GAN models, which enables feature extraction in the GAN models. In airfoil generation tasks, to generate airfoil shapes that satisfy lift coefficient requirements, it is known that VAE outperforms WGAN-gp with respect to the accuracy of the reproduction of the lift coefficient, whereas GAN outperforms VAE with respect to the smoothness and variations of generated shapes. In this study, VAE-WGAN-gp demonstrated a good performance in all three aspects. Latent distribution was also studied to compare the feature extraction ability of the proposed method.

Addressing high-performance data sparsity in metasurface inverse design using multi-objective optimization and diffusion probabilistic models

Recent advancements in deep learning, particularly generative networks capable of producing high-freedom structures, have significantly enhanced the precise generation of meta-atoms. However, these methodologies typically rely on an abundance of high-performance data, which remains scarce in many practical design scenarios. To bridge this gap, our study introduces what we believe to be a novel approach that synergistically combines multi-objective optimization algorithms with an enhanced diffusion model featuring an attention mechanism, termed MetaDiffusion-Att. Using the complex design task of dual-polarized, wide-angle incidence, and broadband low-emissivity electromagnetic glass as an application example, we demonstrate the effectiveness of our method through qualitative and quantitative experiments. The introduced multi-objective optimization method significantly captures more high-performance samples while ensuring high degrees of freedom, compared to currently widely used generic dataset construction methods. The MetaDiffusion-Att model, improved by the introduced attention mechanism, significantly outperforms conventional WGAN-GP and conditional VAE methods in generation accuracy and quality under small datasets. Furthermore, the proposed method exhibits extrapolation capabilities, generating new structures with performance surpassing that of the dataset, further enriching the design space. This framework provides a promising solution for the inverse design of metasurfaces in challenging scenarios with sparse high-performance samples.

Enhancing De Novo Drug Design across Multiple Therapeutic Targets with CVAE Generative Models.

Drug discovery is a costly and time-consuming process, necessitating innovative strategies to enhance efficiency across different stages, from initial hit identification to final market approval. Recent advancement in deep learning (DL), particularly in de novo drug design, show promise. Generative models, a subclass of DL algorithms, have significantly accelerated the de novo drug design process by exploring vast areas of chemical space. Here, we introduce a Conditional Variational Autoencoder (CVAE) generative model tailored for de novo molecular design tasks, utilizing both SMILES and SELFIES as molecular representations. Our computational framework successfully generates molecules with specific property profiles validated though metrics such as uniqueness, validity, novelty, quantitative estimate of drug-likeness (QED), and synthetic accessibility (SA). We evaluated our model's efficacy in generating novel molecules capable of binding to three therapeutic molecular targets: CDK2, PPARγ, and DPP-IV. Comparing with state-of-the-art frameworks demonstrated our model's ability to achieve higher structural diversity while maintaining the molecular properties ranges observed in the training set molecules. This proposed model stands as a valuable resource for advancing de novo molecular design capabilities.

Leveraging Deep Learning and Generative AI for Predicting Rheological Properties and Material Compositions of 3D Printed Polyacrylamide Hydrogels.

Artificial intelligence (AI) has the ability to predict rheological properties and constituent composition of 3D-printed materials with appropriately trained models. However, these models are not currently available for use. In this work, we trained deep learning (DL) models to (1) predict the rheological properties, such as the storage (G') and loss (G") moduli, of 3D-printed polyacrylamide (PAA) substrates, and (2) predict the composition of materials and associated 3D printing parameters for a desired pair of G' and G". We employed a multilayer perceptron (MLP) and successfully predicted G' and G" from seven gel constituent parameters in a multivariate regression process. We used a grid-search algorithm along with 10-fold cross validation to tune the hyperparameters of the MLP, and found the R2 value to be 0.89. Next, we adopted two generative DL models named variational autoencoder (VAE) and conditional variational autoencoder (CVAE) to learn data patterns and generate constituent compositions. With these generative models, we produced synthetic data with the same statistical distribution as the real data of actual hydrogel fabrication, which was then validated using Student's t-test and an autoencoder (AE) anomaly detector. We found that none of the seven generated gel constituents were significantly different from the real data. Our trained DL models were successful in mapping the input-output relationship for the 3D-printed hydrogel substrates, which can predict multiple variables from a handful of input variables and vice versa.

Enhancing Planning for Autonomous Driving via an Iterative Optimization Framework Incorporating Safety-Critical Trajectory Generation

Ensuring the safety of autonomous vehicles (AVs) in complex and high-risk traffic scenarios remains a critical unresolved challenge. Current AV planning methods exhibit limitations in generating robust driving trajectories that effectively avoid collisions, highlighting the urgent need for improved planning strategies to address these issues. This paper introduces a novel iterative optimization framework that incorporates safety-critical trajectory generation to enhance AV planning. The use of the HighD dataset, which is collected using the wide-area remote sensing capabilities of unmanned aerial vehicles (UAVs), is fundamental to the framework. Remote sensing enables large-scale real-time observation of traffic conditions, providing precise data on vehicle dynamics, road structures, and surrounding environments. To generate safety-critical trajectories, the decoder within the conditional variational auto-encoder (CVAE) is innovatively designed through a data-mechanism integration method, ensuring that these trajectories strictly adhere to vehicle kinematic constraints. Furthermore, two parallel CVAEs (Dual-CVAE) are trained collaboratively by a shared objective function to implicitly model the multi-vehicle interactions. Inspired by the concept of “learning to collide”, adversarial optimization is integrated into the Dual-CVAE (Adv. Dual-CVAE), facilitating efficient generation from normal to safety-critical trajectories. Building upon this, these generated trajectories are then incorporated into an iterative optimization framework, significantly enhancing the AV’s planning ability to avoid collisions. This framework decomposes the optimization process into stages, initially addressing normal trajectories and progressively tackling more safety-critical and collision trajectories. Finally, comparative case studies of enhancing AV planning are conducted and the simulation results demonstrate that the proposed method can efficiently enhance AV planning by generating safety-critical trajectories.

Deep Learning with Crested Porcupine Optimizer for Detection and Classification of Paddy Leaf Diseases for Sustainable Agriculture

India has a vast number of inhabitants and the main food source distribution is from agriculture. Agricultural lands are being demolished generally owing to plant and crop illnesses. The detection of plant diseases by using image processing models can aid agriculturalists in defending the farming area from damaging or affecting it. Paddy is the main harvest worldwide. Early recognition of the paddy diseases at dissimilar phases of development is very vital in paddy production. However, the present manual technique in identifying and classifying paddy diseases needs a very educated farmer and is time-consuming. Deep learning (DL) is an effectual research area in the classification of agriculture patterns where it can efficiently solve the problems of diseases identification. Therefore, the articles focus on the design and expansion of Deep Learning based Crested Porcupine Optimizer for the Detection and Classification of Paddy Leaf Diseases (DLCPO-DCPLD) method for Sustainable Agriculture. The main aim of the DLCPO-DCPLD method use DL method for the recognition and identification of rice plant leaf diseases. To accomplish this, the DLCPO-DCPLD technique performs the image pre-processing using Median Filtering (MF) to recover the excellence of the input frames. Next, the ConvNeXt-L method is applied for extraction of feature vectors from the pre-processed images. Also, the Conditional Variational Autoencoder (CVAE) model is utilized for the automated classification of Paddy Leaf diseases. Eventually, the hyperparameter tuning of the CVAE technique is accomplished by implementing the Crested Porcupine Optimizer (CPO) technique. To safeguard the enhanced predictive results of the DLCPO-DCPLD method, a sequence of experimentations is implemented on the benchmark dataset. The experimental validation of the DLCPO-DCPLD method portrayed a superior accuracy value of 99.12% over existing approaches.

Generating comprehensive lithium battery charging data with generative AI

In optimizing the performance and extending the lifespan of lithium batteries, accurate state prediction is crucial. Traditional regression and classification methods have achieved some success in predicting battery states. However, the effectiveness of these data-driven approaches largely depends on the availability and quality of public datasets. Additionally, generating electrochemical data primarily through battery experiments is a lengthy and costly process, making it extremely difficult to obtain high-quality data. This difficulty, combined with data incompleteness, significantly affects prediction accuracy. To address these challenges, this study introduces End of Life (EOL) and Equivalent Cycle Life (ECL) as supervised conditions for generative AI models. By integrating an embedding layer into the CVAE model, we developed the Refined Conditional Variational Autoencoder (RCVAE). Through preprocessing data into a quasi-video format, coupled with customized training and inference algorithms, the RCVAE generates the required charging data in real time based on the battery’s ECL and EOL. This avoids storing irrelevant information, saving space and resources. RCVAE uses a lightweight architecture, enabling fast generation of necessary voltage, current, temperature, and charging capacity data. This approach provides users with a comprehensive electrochemical dataset, pioneering a new research domain for the artificial synthesis of lithium battery data. Furthermore, based on the detailed synthetic data, various battery state indicators can be calculated, offering new perspectives and possibilities for lithium battery performance prediction.

An Emotional Dialogue System Using Conditional Generative Adversarial Networks with a Sequence-to-Sequence Transformer Encoder

Understanding the expression of emotion and generating appropriate responses are key steps toward constructing emotional, conversational agents. In this paper, we propose a framework for single-turn emotional conversation generation, and there are three main components in our model, namely, a sequence-to-sequence model with stacked encoders, a conditional variational autoencoder, and conditional generative adversarial networks. For the sequence-to-sequence model with stacked encoders, we designed a two-layer encoder by combining Transformer with gated recurrent units-based neural networks. Because of the flexibility of the sequence-to-sequence model, we adopted a conditional variational autoencoder in our framework, which uses latent variables to learn a distribution over potential responses and generates diverse responses. Furthermore, we regard a conditional variational autoencoder-based, sequence-to-sequence model as the generative model, and the training of the generative model is assisted by both a content discriminator and an emotion classifier, which assists our model in promoting content information and emotion expression. We use automated evaluation and human evaluation to evaluate our model and baselines on the NTCIR short text conversation task (STC-3) Chinese emotional conversation generation (CECG) Subtask dataset [44], and the experimental results demonstrate that our proposed framework can generate semantically reasonable and emotionally appropriate responses.

Text2shape: Intelligent computational design of car outer contour shapes based on improved conditional Wasserstein generative adversarial network

To provide technical support for the initial design of products, we propose an innovative text2shape-based technology for intelligent computational design, which can map engineering semantics to functional/structural/Kansei feature spaces to generate product shapes. New energy vehicles were selected as the application object of this technology, as there are many creative ideas for the outer contour design of new energy vehicles. Firstly, a dataset with 2900 + samples was built based on feature engineering (FE) and Kansei engineering (KE). Each sample contains the car’s outer contour shape’s functional, structural, and Kansei features. Secondly, we proposed an improved conditional Wasserstein generative adversarial network (CWGAN) model suitable to the dataset. The generator’s loss in the model is designed to evaluate the authenticity of the generated results, while the discriminator’s loss assesses the conditional matching of these results. Finally, in case studies, the trained CWGAN was compared with the conditional variational auto-encoder (C-VAE), diffusion, Wasserstein generative adversarial network with gradient penalty (WGAN-GP) and style generative adversarial network (StyleGAN) models, demonstrating superior performance.

Development of a conditional variational autoencoder for handwritten digit recognition.

This study examines the performance of Conditional Variational Autoencoder (CVAE) in handwritten digit recognition. Using the MNIST dataset, two variants of the CVAE models — convolutional and multilevel architecture — were developed and compared. The research methodology includes comprehensive data preprocessing, architecture design, training, and thorough evaluation processes. The obtained data highlight the better performance of the convolutional model-based CVAE in achieving recognition accuracy compared to its multilayer counterpart. Evaluation metrics include analysis of original and reconstructed images, comparison of hidden layer vector distribution patterns, and visualization of loss function dynamics. In addition, the study highlights the practical implications of CVAEs in various fields, highlighting their performance in digit recognition tasks due to their inherent robustness and extraordinary generalizability.

Reconstructing and analyzing the invariances of low-dose CT image denoising networks.

Deep learning-based methods led to significant advancements in many areas of medical imaging, most of which are concerned with the reduction of artifacts caused by motion, scatter, or noise. However, with most neural networks being black boxes, they remain notoriously difficult to interpret, hindering their clinical implementation. In particular, it has been shown that networks exhibit invariances w.r.t. input features, that is, they learn to ignore certain information in the inputdata. To improve the interpretability of deep learning-based low-dose CT image denoisingnetworks. We learn a complete data representation of low-dose input images using a conditional variational autoencoder (cVAE). In this representation, invariances of any given denoising network are then disentangled from the information it is not invariant to using a conditional invertible neural network (cINN). At test time, image-space invariances are generated by applying the inverse of the cINN and subsequent decoding using the cVAE. We propose two methods to analyze sampled invariances and to find those that correspond to alterations of anatomicalstructures. The proposed method is applied to four popular deep learning-based low-dose CT image denoising networks. We find that the networks are not only invariant to noise amplitude and realizations, but also to anatomicalstructures. The proposed method is capable of reconstructing and analyzing invariances of deep learning-based low-dose CT image denoising networks. This is an important step toward interpreting deep learning-based methods for medical imaging, which is essential for their clinical implementation.

A Bearing Fault Diagnosis Method in Scenarios of Imbalanced Samples and Insufficient Labeled Samples

In practical working environments, rolling bearings are one of the components that are prone to failure. Their vibration signal samples are faced with challenges, mainly including the imbalance between normal and fault samples as well as an insufficient number of labeled samples. This study proposes a sample-expansion method based on generative adversarial networks (GANs) and a fault diagnosis method based on a transformer to solve the above issues. First, selective kernel networks (SKNets) and a genetic algorithm (GA) were introduced to construct a conditional variational autoencoder–evolutionary generative adversarial network with a selective kernel (CVAE-SKEGAN) to achieve a balance between the proportion of normal and faulty samples. Then, a semi-supervised learning–variational convolutional Swin transformer (SSL-VCST) network was built for the fault classification, specifically introducing variational attention and semi-supervised mechanisms to reduce the overfitting risk of the model and solve the problem of a shortage of labeled samples. Three typical operating conditions were designed for the multi-case applicability verification. The results show that the method proposed in this study had good application effects when solving both sample imbalances and labeled-sample deficiencies and improved the accuracy of fault diagnosis in the above scenarios.

Novel multi-omics deconfounding variational autoencoders can obtain meaningful disease subtyping.

Unsupervised learning, particularly clustering, plays a pivotal role in disease subtyping and patient stratification, especially with the abundance of large-scale multi-omics data. Deep learning models, such as variational autoencoders (VAEs), can enhance clustering algorithms by leveraging inter-individual heterogeneity. However, the impact of confounders-external factors unrelated to the condition, e.g. batch effect or age-on clustering is often overlooked, introducing bias and spurious biological conclusions. In this work, we introduce four novel VAE-based deconfounding frameworks tailored for clustering multi-omics data. These frameworks effectively mitigate confounding effects while preserving genuine biological patterns. The deconfounding strategies employed include (i) removal of latent features correlated with confounders, (ii) a conditional VAE, (iii) adversarial training, and (iv) adding a regularization term to the loss function. Using real-life multi-omics data from The Cancer Genome Atlas, we simulated various confounding effects (linear, nonlinear, categorical, mixed) and assessed model performance across 50 repetitions based on reconstruction error, clustering stability, and deconfounding efficacy. Our results demonstrate that our novel models, particularly the conditional multi-omics VAE (cXVAE), successfully handle simulated confounding effects and recover biologically driven clustering structures. cXVAE accurately identifies patient labels and unveils meaningful pathological associations among cancer types, validating deconfounded representations. Furthermore, our study suggests that some of the proposed strategies, such as adversarial training, prove insufficient in confounder removal. In summary, our study contributes by proposing innovative frameworks for simultaneous multi-omics data integration, dimensionality reduction, and deconfounding in clustering. Benchmarking on open-access data offers guidance to end-users, facilitating meaningful patient stratification for optimized precision medicine.

DeepResBat: Deep residual batch harmonization accounting for covariate distribution differences

Pooling MRI data from multiple datasets requires harmonization to reduce undesired inter-site variabilities, while preserving effects of biological variables (or covariates). The popular harmonization approach ComBat uses a mixed effect regression framework that explicitly accounts for covariate distribution differences across datasets. There is also significant interest in developing harmonization approaches based on deep neural networks (DNNs), such as conditional variational autoencoder (cVAE). However, current DNN approaches do not explicitly account for covariate distribution differences across datasets. Here, we provide mathematical results, suggesting that not accounting for covariates can lead to suboptimal harmonization. We propose two DNN-based covariate-aware harmonization approaches: covariate VAE (coVAE) and DeepResBat. The coVAE approach is a natural extension of cVAE by concatenating covariates and site information with site- and covariate-invariant latent representations. DeepResBat adopts a residual framework inspired by ComBat. DeepResBat first removes the effects of covariates with nonlinear regression trees, followed by eliminating site differences with cVAE. Finally, covariate effects are added back to the harmonized residuals. Using three datasets from three continents with a total of 2787 participants and 10,085 anatomical T1 scans, we find that DeepResBat and coVAE outperformed ComBat, CovBat and cVAE in terms of removing dataset differences, while enhancing biological effects of interest. However, coVAE hallucinates spurious associations between anatomical MRI and covariates even when no association exists. Future studies proposing DNN-based harmonization approaches should be aware of this false positive pitfall. Overall, our results suggest that DeepResBat is an effective deep learning alternative to ComBat. Code for DeepResBat can be found here: https://github.com/ThomasYeoLab/CBIG/tree/master/stable_projects/harmonization/An2024_DeepResBat.