Deep Generative Models Research Articles

SDA-FC: Bridging federated clustering and deep generative model

Federated clustering (FC) is an extension of centralized clustering in federated settings. The key here is how to construct a global similarity measure without sharing private data, since the local similarity may be insufficient to group local data correctly, and the similarity of samples across clients cannot be directly measured due to privacy constraints. Obviously, the most straightforward way to analyze FC is to employ methods extended from centralized ones, such as K-means (KM) and fuzzy c-means (FCM). However, they are vulnerable to non independent-and-identically-distributed (non-IID) data among clients. To handle this, we propose a pretty simple and effective federated clustering framework instantiated with generative adversarial network (GAN), named synthetic data aided federated clustering (SDA-FC). It trains generative adversarial network locally in each client and uploads the generated synthetic data to the server, where KM or FCM is performed on the synthetic data. The synthetic data can make the model immune to the non-IID problem and enable us to capture the global similarity characteristics more effectively without sharing private data. Comprehensive experiments reveal the advantages of SDA-FC, including superior performance in addressing the non-IID problem and the device failures. The code is available at https://github.com/Jarvisyan/SDA-FC.

GPianoroll: a Deep Learning System with Human Feedback for Music Generation

We present a human-feedback-based evaluation pipeline for deep learning generative models, tested on music composition. Leveraging Bayesian Optimization and asking users to grade the generated songs, we efficiently find the point of a given model’s latent space that yields each volunteer’s favorite without retraining.

ScSemiProfiler: Advancing large-scale single-cell studies through semi-profiling with deep generative models and active learning

Single-cell sequencing is a crucial tool for dissecting the cellular intricacies of complex diseases. Its prohibitive cost, however, hampers its application in expansive biomedical studies. Traditional cellular deconvolution approaches can infer cell type proportions from more affordable bulk sequencing data, yet they fall short in providing the detailed resolution required for single-cell-level analyses. To overcome this challenge, we introduce “scSemiProfiler”, an innovative computational framework that marries deep generative models with active learning strategies. This method adeptly infers single-cell profiles across large cohorts by fusing bulk sequencing data with targeted single-cell sequencing from a few rigorously chosen representatives. Extensive validation across heterogeneous datasets verifies the precision of our semi-profiling approach, aligning closely with true single-cell profiling data and empowering refined cellular analyses. Originally developed for extensive disease cohorts, “scSemiProfiler” is adaptable for broad applications. It provides a scalable, cost-effective solution for single-cell profiling, facilitating in-depth cellular investigation in various biological domains.

A comprehensive study of auto-encoders for anomaly detection: Efficiency and trade-offs

Unsupervised anomaly detection (UAD) is a diverse research area explored across various application domains. Over time, numerous anomaly detection techniques, including clustering, generative, and variational inference-based methods, are developed to address specific drawbacks and advance state-of-the-art techniques. Deep learning and generative models recently played a significant role in identifying unique challenges and devising advanced approaches. Auto-encoders (AEs) represent one such powerful technique that combines generative and probabilistic variational modeling with deep architecture. Auto-Encoder aims to learn the underlying data distribution to generate consequential sample data. This concept of data generation and the adoption of generative modeling have emerged in extensive research and variations in Auto-Encoder design, particularly in unsupervised representation learning. This study systematically reviews 11 Auto-Encoder architectures categorized into three groups, aiming to differentiate their reconstruction ability, sample generation, latent space visualization, and accuracy in classifying anomalous data using the Fashion-MNIST (FMNIST) and MNIST datasets. Additionally, we closely observed the reproducibility scope under different training parameters. We conducted reproducibility experiments utilizing similar model setups and hyperparameters and attempted to generate comparative results to address the scope of improvements for each Auto-Encoder. We conclude this study by analyzing the experimental results, which guide us in identifying the efficiency and trade-offs among auto-encoders, providing valuable insights into their performance and applicability in unsupervised anomaly detection techniques.

A survey of vision-based condition monitoring methods using deep learning: A synthetic fiber rope perspective

Computer vision technology has attracted significant interest in the condition monitoring (CM) community due to its potential to automate visual inspection and analysis of structures and components. By facilitating the processing and interpretation of visual information, including images and video data, computer vision holds promise for CM applications. However, it is essential to distinguish computer vision from non-contact CM techniques regarding their underlying principles and methods. While computer vision enables non-contact, remote monitoring, and condition assessment with minimal disruption to daily operations, it is distinct from non-contact CM techniques, which utilize various sensors to assess the condition of assets without physical contact or interference. Building upon the potential of computer vision technology, this survey paper presents a comprehensive overview of the current state-of-the-art CM methods based on computer vision and deep learning (DL) techniques, focusing on their application in monitoring synthetic fiber ropes (SFRs). SFRs are a viable alternative to steel wire ropes for underwater equipment and cranes that handle heavy loads. This is due to their high resistance to frictional wear, high tensile strength, lightweight, and flexibility. New materials, technologies, and processes for CM are being developed to meet the growing demand for SFRs. The paper explores ongoing research in applications that monitor the wear and aging of materials, as well as estimate their remaining useful life. The survey briefly discusses the traditional non-destructive testing and machine learning (ML) methods for CM applications. More importantly, DL-based methods, including supervised, unsupervised, semi-supervised, and self-supervised methods, are discussed in detail, together with the use of deep generative models and the recently developed diffusion models in the generation of synthetic datasets. Furthermore, the paper addresses the difficulties present in DL-based CM applications, including the scarcity of labeled data and the complexity and variety of the models used. The article ends by discussing the benefits of employing DL-based visual methods to understand SFR degradation processes, particularly in monitoring and maintenance.

Control over self and others’ face: exploitation and exploration

The face serves as a crucial cue for self-identification, while the sense of agency plays a significant role in determining our influence through actions in the environment. The current study investigates how self-identification through facial recognition may influence the perception of control via motion. We propose that self-identification might engender a belief in having control over one’s own face, leading to a more acute detection and greater emphasis on discrepancies between their actions and the sensory feedback in control judgments. We refer to the condition governed by the belief in having control as the exploitation mode. Conversely, when manipulating another individual’s face, the belief in personal control is absent. In such cases, individuals are likely to rely on the regularity between actions and sensory input for control judgments, exhibiting behaviors that are exploratory in nature to glean such information. This condition is termed the explorative mode. The study utilized a face-motion mixing paradigm, employing a deep generative model to enable participants to interact with either their own or another person’s face through facial and head movements. During the experiment, participants observed either their own face or someone else’s face (self-face vs. other-face) on the screen. The motion of the face was driven either purely by their own facial and head motion or by an average of the participant’s and the experimenter’s motion (full control vs. partial control). The results showed that participants reported a higher sense of agency over the other-face than the self-face, while their self-identification rating was significantly higher for the self-face. More importantly, controlling someone else’s face resulted in more movement diversity than controlling one’s own face. These findings support our exploration–exploitation theory: When participants had a strong belief in control triggered by the self-face, they became highly sensitive to any sensorimotor prediction errors, leading to a lower sense of agency. In contrast, when the belief of control was absent, the exploration mode triggered more explorative behaviors, allowing participants to efficiently gather information to establish a sense of agency.

Turbulence scaling from deep learning diffusion generative models

Complex spatial and temporal structures are inherent characteristics of turbulent fluid flows and comprehending them poses a major challenge. This comprehension necessitates an understanding of the space of turbulent fluid flow configurations. We employ a diffusion-based generative model to learn the distribution of turbulent vorticity profiles and generate snapshots of turbulent solutions to the incompressible Navier-Stokes equations. We consider the inverse cascade in two spatial dimensions and generate diverse turbulent solutions that differ from those in the training dataset. We analyze the statistical scaling properties of the new turbulent profiles, calculate their structure functions, energy power spectrum, velocity probability distribution function and moments of local energy dissipation. All the learnt scaling exponents are consistent with the expected Kolmogorov scaling This agreement with established turbulence characteristics provides strong evidence of the model's capability to capture essential features of real-world turbulence.

Field-of-view extension for brain diffusion MRI via deep generative models.

In brain diffusion magnetic resonance imaging (dMRI), the volumetric and bundle analyses of whole-brain tissue microstructure and connectivity can be severely impeded by an incomplete field of view (FOV). We aim to develop a method for imputing the missing slices directly from existing dMRI scans with an incomplete FOV. We hypothesize that the imputed image with a complete FOV can improve whole-brain tractography for corrupted data with an incomplete FOV. Therefore, our approach provides a desirable alternative to discarding the valuable brain dMRI data, enabling subsequent tractography analyses that would otherwise be challenging or unattainable with corrupted data. We propose a framework based on a deep generative model that estimates the absent brain regions in dMRI scans with an incomplete FOV. The model is capable of learning both the diffusion characteristics in diffusion-weighted images (DWIs) and the anatomical features evident in the corresponding structural images for efficiently imputing missing slices of DWIs in the incomplete part of the FOV. For evaluating the imputed slices, on the Wisconsin Registry for Alzheimer's Prevention (WRAP) dataset, the proposed framework achieved , , , and ; on the National Alzheimer's Coordinating Center (NACC) dataset, it achieved , , , and . The proposed framework improved the tractography accuracy, as demonstrated by an increased average Dice score for 72 tracts ( ) on both the WRAP and NACC datasets. Results suggest that the proposed framework achieved sufficient imputation performance in brain dMRI data with an incomplete FOV for improving whole-brain tractography, thereby repairing the corrupted data. Our approach achieved more accurate whole-brain tractography results with an extended and complete FOV and reduced the uncertainty when analyzing bundles associated with Alzheimer's disease.

Efficient high‐fidelity deep convolutional generative adversarial network model for received signal strength reconstruction in indoor environments

AbstractWith the rapid development of wireless communication systems, particularly in the era of 5G and the Internet of Things, deploying wireless communication networks in indoor environments has become crucial. Indoor infrastructure deployment necessitates innovative approaches for efficiently and accurately obtaining received signal strength (RSS) maps. However, traditional methods for acquiring RSS maps, such as empirical and deterministic models, are limited by significant inaccuracies and high computational demands. Empirical models often fail to capture the complex dynamics of indoor environments, resulting in deviations from actual signal behaviours. On the other hand, deterministic models, while more accurate, are computationally intensive due to their reliance on detailed physical modelling of wave propagation. This study introduces a machine learning approach based on deep convolutional generative adversarial networks (DCGAN) aimed at reconstructing indoor RSS maps with minimal RSS measurements. By leveraging DCGAN's generative and adversarial training capabilities, the method not only surpasses traditional interpolation methods in efficiency and precision but also offers new possibilities for the rapid deployment and optimization of wireless communication systems in indoor environments.

Data-driven topology optimization using a multitask conditional variational autoencoder with persistent homology

Topology optimization is crucial for the mechanical design of vehicles and aircraft, allowing changes in the shape of structures and the placement of features. Recent advances have integrated deep generative models, particularly convolutional neural networks, to streamline this process.to streamline this process. However, these models struggle to preserve subtle structural features. To overcome these limitations, this study introduced a generative model adept at identifying the topological features inherent in real shapes, such as connectivity and holes, to enhance the effectiveness of topology optimization. A conditional variational autoencoder (CVAE) was employed to predict both the shape and compliance simultaneously. This model, CVAE with persistent homology, generates optimal material distributions by considering topological properties. The learning process introduced a term that minimizes the difference in topological features between true and reconstructed shapes. The proposed model can generate optimal material distributions by considering topological properties, eliminating the need for iterative calculations. This approach was validated using two numerical examples. The accuracy of the generated material distributions was compared with conventional methods using the mean-squared error. An average improvement in accuracy of approximately 36.85% was observed across the two results. This confirms that shapes considering compliance and connectivity can be accurately predicted.

Deep generative model of the distal tibial classic metaphyseal lesion in infants: assessment of synthetic images.

The classic metaphyseal lesion (CML) is a distinctive fracture highly specific to infant abuse. To increase the size and diversity of the training CML database for automated deep-learning detection of this fracture, we developed a mask conditional diffusion model (MaC-DM) to generate synthetic images with and without CMLs. To objectively and subjectively assess the synthetic radiographic images with and without CMLs generated by MaC-DM. For retrospective testing, we randomly chose 100 real images (50 normals and 50 with CMLs; 39 infants, male = 22, female = 17; mean age = 4.1 months; SD = 3.1 months) from an existing distal tibia dataset (177 normal, 73 with CMLs), and generated 100 synthetic distal tibia images via MaC-DM (50 normals and 50 with CMLs). These test images were shown to 3 blinded radiologists. In the first session, radiologists determined if the images were normal or had CMLs. In the second session, they determined if the images were real or synthetic. We analyzed the radiologists' interpretations and employed t-distributed stochastic neighbor embedding technique to analyze the data distribution of the test images. When presented with the 200 images (100 synthetic, 100 with CMLs), radiologists reliably and accurately diagnosed CMLs (kappa = 0.90, 95% CI = [0.88-0.92]; accuracy = 92%, 95% CI = [89-97]). However, they were inaccurate in differentiating between real and synthetic images (kappa = 0.05, 95% CI = [0.03-0.07]; accuracy = 53%, 95% CI = [49-59]). The t-distributed stochastic neighbor embedding analysis showed substantial differences in the data distribution between normal images and those with CMLs (area under the curve = 0.996, 95% CI = [0.992-1.000], P < .01), but minor differences between real and synthetic images (area under the curve = 0.566, 95% CI = [0.486-0.647], P = .11). Radiologists accurately diagnosed images with distal tibial CMLs but were unable to distinguish real from synthetically generated ones, indicating that our generative model could synthesize realistic images. Thus, MaC-DM holds promise as an effective strategy for data augmentation in training machine-learning models for diagnosis of distal tibial CMLs.

Research Project Topic Recommender System Using Generative Language Model

Education has become a driver of a person's continuous innovation to improve their quality. Currently, the use of artificial intelligence determines progress in education. In this research, artificial intelligence technology was applied to develop a web-based recommendation system to help students at the Faculty of Computer Science, Klabat University, choose appropriate research topics for their final assignments. To provide personalized and contextually relevant suggestions, the recommendation system leverages deep learning and generative language models, specifically GPT-3. The Rapid Application Development process model is employed to develop the system. Its key components include semantic search, rapid engineering, and an advanced vector database for effective data management and retrieval. The functions provided by the system include user account registration, login, input of major subject grades and research preferences, and personalized recommendation results. Some additional features such as profile management, previous recommendation history, and password reset options are also provided. All these functions have been tested using the black box method.

In silico generation and augmentation of regulatory variants from massively parallel reporter assay using conditional variational autoencoder.

Predicting the functional consequences of genetic variants in non-coding regions is a challenging problem. Massively parallel reporter assays (MPRAs), which are an in vitro high-throughput method, can simultaneously test thousands of variants by evaluating the existence of allele specific regulatory activity. Nevertheless, the identified labelled variants by MPRAs, which shows differential allelic regulatory effects on the gene expression are usually limited to the scale of hundreds, limiting their potential to be used as the training set for achieving a robust genome-wide prediction. To address the limitation, we propose a deep generative model, MpraVAE, to in silico generate and augment the training sample size of labelled variants. By benchmarking on several MPRA datasets, we demonstrate that MpraVAE significantly improves the prediction performance for MPRA regulatory variants compared to the baseline method, conventional data augmentation approaches as well as existing variant scoring methods. Taking autoimmune diseases as one example, we apply MpraVAE to perform a genome-wide prediction of regulatory variants and find that predicted regulatory variants are more enriched than background variants in enhancers, active histone marks, open chromatin regions in immune-related cell types, and chromatin states associated with promoter, enhancer activity and binding sites of cMyC and Pol II that regulate gene expression. Importantly, predicted regulatory variants are found to link immune-related genes by leveraging chromatin loop and accessible chromatin, demonstrating the importance of MpraVAE in genetic and gene discovery for complex traits.

Interpretable deep generative model in myocardial perfusion SPECT images based on the creation of patient-matched normal maps

Interpretable deep generative model in myocardial perfusion SPECT images based on the creation of patient-matched normal maps

GPT-based normative models of brain sMRI correlate with dimensional psychopathology

Abstract Generative Pre-trained Transformer (GPT) models have been widely used for language tasks with surprising results. Furthermore, neuroimaging studies using deep generative normative modeling show promise in detecting brain abnormalities from brain structural MRI (sMRI). Meanwhile, psychiatric disorders are typically diagnosed through clinical assessment, which is particularly challenging in children and adolescents who present early symptoms or are in the early stages of the disease. Brain biomarkers research may contribute to the complex task of disentangling typical neurodevelopment from emergent psychiatric disorders. Here, we investigate whether a GPT-based normative architecture can detect psychiatric symptoms and disorders from brain sMRI of youths. The studied datasets contain measures of dimensional psychopathology: Brazilian High-Risk Cohort Study (BHRCS, n = 737) and Adolescent Brain Cognitive Development (ABCD, n = 11,031), and scores and diagnostic of psychiatric disorders: Attention Deficit Hyperactivity Disorder (ADHD-200, n = 922) and Autism Brain Imaging Data Exchange II (ABIDE-II, n = 580). We examined the associations of all brain regions with: the Child Behavior Checklist (CBCL) symptom groups, ADHD scores, and Autism Spectrum Disorder (ASD) diagnosis. Results showed the whole-brain typicality likelihood as correlated with social problems (ABCD test set) and ASD diagnosis (ABIDE-II dataset). Analysis by brain regions linked different areas to several CBCL scales, ADHD scores, and ASD diagnostic. This is the first successful study assessing all dimensional groups of CBCL symptoms, from all brain regions, based exclusively on sMRI. The normative models based on GPT are promising to investigate the gap between the phenotypes of psychiatric conditions and their neurobiological substrates.

AC-ModNet: Molecular Reverse Design Network Based on Attribute Classification.

Deep generative models are becoming a tool of choice for exploring the molecular space. One important application area of deep generative models is the reverse design of drug compounds for given attributes (solubility, ease of synthesis, etc.). Although there are many generative models, these models cannot generate specific intervals of attributes. This paper proposes a AC-ModNet model that effectively combines VAE with AC-GAN to generate molecular structures in specific attribute intervals. The AC-ModNet is trained and evaluated using the open 250K ZINC dataset. In comparison with related models, our method performs best in the FCD and Frag model evaluation indicators. Moreover, we prove the AC-ModNet created molecules have potential application value in drug design by comparing and analyzing them with medical records in the PubChem database. The results of this paper will provide a new method for machine learning drug reverse design.

AI nutrition recommendation using a deep generative model and ChatGPT

In recent years, major advances in artificial intelligence (AI) have led to the development of powerful AI systems for use in the field of nutrition in order to enhance personalized dietary recommendations and improve overall health and well-being. However, the lack of guidelines from nutritional experts has raised questions on the accuracy and trustworthiness of the nutritional advice provided by such AI systems. This paper aims to address this issue by introducing a novel AI-based nutrition recommendation method that leverages the speed and explainability of a deep generative network and the use of novel sophisticated loss functions to align the network with established nutritional guidelines. The use of a variational autoencoder to robustly model the anthropometric measurements and medical condition of users in a descriptive latent space, as well as the use of an optimizer to adjust meal quantities based on users’ energy requirements enable the proposed method to generate highly accurate, nutritious and personalized weekly meal plans. Coupled with the ability of ChatGPT to provide an unparalleled pool of meals from various cuisines, the proposed method can achieve increased meal variety, accuracy and generalization capabilities. Extensive experiments on 3000 virtual user profiles and 84000 daily meal plans, as well as 1000 real profiles and 7000 daily meal plans, demonstrate the exceptional accuracy of the proposed diet recommendation method in generating weekly meal plans that are appropriate for the users in terms of energy intake and nutritional requirements, as well as the easiness with which it can be integrated into future diet recommendation systems.

Exploring VAE-driven implicit parametric unit cells for multiscale topology optimization

The simulation and optimization of metamaterial, known for their engineered properties in various applications, encounter intricate challenges due to complex microstructure interactions, extensive design spaces, and substantial computational requirements. To address these challenges, our research introduces a novel data-driven framework utilizing deep generative modeling to enhance the design process of metamaterials.Focusing on composite metamaterial with double negative coefficients of thermal expansion and Poisson's ratio, we apply an Alternative Active Phase and Objective functions (AAPO) method for deciding the initial metamaterial dataset. To enhance dataset diversity, a distortion filter is applied, broadening the range of design possibilities. Subsequently, we utilize a Variational Autoencoder (VAE), integrated with a regressor, to train on this diversified database. This training effectively maps complex unit cell geometries to a coherent latent space, simultaneously correlating them with continuous material properties.Our approach demonstrates robustness in multi-phase and multi-physics optimization as well as efficiency in generating specialized databases of unit cells. This framework is pivotal in systematically designing unit cells and multiscale systems, specifically aiming for distinct thermo-mechanical behavior targets.To mitigate the computational demands encountered during multiple design meta-materials via gradient-based topology optimization, we have integrated high-performance methods and automatic differentiation. This integration marks a significant advancement in the data-driven design of metamaterials, offering substantial practical and theoretical benefits in the field.

Deep representation learning of chemical-induced transcriptional profile for phenotype-based drug discovery

Artificial intelligence transforms drug discovery, with phenotype-based approaches emerging as a promising alternative to target-based methods, overcoming limitations like lack of well-defined targets. While chemical-induced transcriptional profiles offer a comprehensive view of drug mechanisms, inherent noise often obscures the true signal, hindering their potential for meaningful insights. Here, we highlight the development of TranSiGen, a deep generative model employing self-supervised representation learning. TranSiGen analyzes basal cell gene expression and molecular structures to reconstruct chemical-induced transcriptional profiles with high accuracy. By capturing both cellular and compound information, TranSiGen-derived representations demonstrate efficacy in diverse downstream tasks like ligand-based virtual screening, drug response prediction, and phenotype-based drug repurposing. Notably, in vitro validation of TranSiGen’s application in pancreatic cancer drug discovery highlights its potential for identifying effective compounds. We envisage that integrating TranSiGen into the drug discovery and mechanism research holds significant promise for advancing biomedicine.

Exploring master scenarios for autonomous driving tests from police-reported historical crash data using an adaptive search sampling framework

Crash scenario-based testing is crucial for assessing autonomous driving safety. However, existing studies on scenario generation tend to prioritize concrete scenarios for direct testing, neglecting the construction of fundamentally functional scenarios with a broader range. Police-reported historical crash data is a valuable supplement, yet detecting all potential crash scenarios is laborious. In order to address this issue, this study proposes an adaptive search sampling framework based on deep generative model and surrogate model (SM) to extract master scenario samples from police-reported historical crash data. The framework starts with selecting representative samples from the full crash dataset as initial master scenario samples using various sampling techniques. Evaluation indexes are then constructed, and derived scenario samples are synthesized using the deep generative model. To enhance efficiency, an SM is established to replace the generative model’s training and data generation process. Based on the SM, an adaptive search sampling method is developed, which iteratively adjusts the sampling strategy using the Similarity Score to achieve comprehensive sampling. Experimental results demonstrate the notable advantage of the adaptive search sampling method over other sampling methods. Furthermore, statistical analysis and visualization assessments confirm the effectiveness and accuracy of the proposed method.