Latent Space Representation Research Articles

Disentangling brain atrophy heterogeneity in Alzheimer's disease: A deep self-supervised approach with interpretable latent space

Alzheimer's disease (AD) is heterogeneous, but existing methods for capturing this heterogeneity through dimensionality reduction and unsupervised clustering have limitations when it comes to extracting intricate atrophy patterns. In this study, we propose a deep learning based self-supervised framework that characterizes complex atrophy features using latent space representation. It integrates feature engineering, classification, and clustering to synergistically disentangle heterogeneity in Alzheimer's disease. Through this representation learning, we trained a clustered latent space with distinct atrophy patterns and clinical characteristics in AD, and replicated the findings in prodromal Alzheimer's disease. Moreover, we discovered that these clusters are not solely attributed to subtypes but also reflect disease progression in the latent space, representing the core dimensions of heterogeneity, namely progression and subtypes. Furthermore, longitudinal latent space analysis revealed two distinct disease progression pathways: medial temporal and parietotemporal pathways. The proposed approach enables effective latent representations that can be integrated with individual-level cognitive profiles, thereby facilitating a comprehensive understanding of AD heterogeneity.

Multi-domain awareness for compressed deepfake videos detection over social networks guided by common mechanisms between artifacts

The viral spread of massive deepfake videos over social networks has caused serious security problems. Despite the remarkable advancements achieved by existing deepfake detection algorithms, deepfake videos over social networks are inevitably influenced by compression factors. This causes deepfake detection performance to be limited by the following challenging issues: (a) interfering with compression artifacts, (b) loss of feature information, and (c) aliasing of feature distributions. In this paper, we analyze the common mechanism between compression artifacts and deepfake artifacts, revealing the structural similarity between them and providing a reliable theoretical basis for enhancing the robustness of deepfake detection models against compression. Firstly, based on the common mechanism between artifacts, we design a frequency domain adaptive notch filter to eliminate the interference of compression artifacts on specific frequency bands. Secondly, to reduce the sensitivity of deepfake detection models to unknown noise, we propose a spatial residual denoising strategy. Thirdly, to exploit the intrinsic correlation between feature vectors in the frequency domain branch and the spatial domain branch, we enhance deepfake features using an attention-based feature fusion method. Finally, we adopt a multi-task decision approach to enhance the discriminative power of the latent space representation of deepfakes, achieving deepfake detection with robustness against compression. Extensive experiments show that compared with the baseline methods, the detection performance of the proposed algorithm on compressed deepfake videos has been significantly improved. In particular, our model is resistant to various types of noise disturbances and can be easily combined with baseline detection models to improve their robustness.

Privacy Preserving Human Mobility Generation Using Grid-Based Data and Graph Autoencoders

This paper proposes a one-to-one trajectory synthetization method with stable long-term individual mobility behavior based on a generalizable area embedding. Previous methods concentrate on producing highly detailed data on short-term and restricted areas for, e.g., autonomous driving scenarios. Another possibility consists of city-wide and beyond scales that can be used to predict general traffic flows. The now-presented approach takes the tracked mobility behavior of individuals and creates coherent synthetic mobility data. These generated data reflect the person’s long-term mobility behavior, guaranteeing location persistency and sound embedding within the point-of-interest structure of the observed area. After an analysis and clustering step of the original data, the area is distributed into a geospatial grid structure (H3 is used here). The neighborhood relationships between the grids are interpreted as a graph. A feed-forward autoencoder and a graph encoding–decoding network generate a latent space representation of the area. The original clustered data are associated with their respective H3 grids. With a greedy algorithm approach and concerning privacy strategies, new combinations of grids are generated as top-level patterns for individual mobility behavior. Based on the original data, concrete locations within the new grids are found and connected to ways. The goal is to generate a dataset that shows equivalence in aggregated characteristics and distances in comparison with the original data. The described method is applied to a sample of 120 from a study with 1000 participants whose mobility data were generated in the city of Munich in Germany. The results show the applicability of the approach in generating synthetic data, enabling further research on individual mobility behavior and patterns. The result comprises a sharable dataset on the same abstraction level as the input data, which can be beneficial for different applications, particularly for machine learning.

Multiscale simulation of spatially correlated microstructure via a latent space representation

When deformation gradients act on the scale of the microstructure of a part due to geometry and loading, spatial correlations and finite-size effects in simulation cells cannot be neglected. We propose a multiscale method that accounts for these effects using a variational autoencoder to encode the structure–property map of the stochastic volume elements making up the statistical description of the part. In this paradigm the autoencoder can be used to directly encode the microstructure or, alternatively, its latent space can be sampled to provide likely realizations. We demonstrate the method on three examples using the common additively manufactured material AlSi10Mg in: (a) a comparison with direct numerical simulation of the part microstructure, (b) a push forward of microstructural uncertainty to performance quantities of interest, and (c) a simulation of functional gradation of a part with stochastic microstructure.

Automatic fault detection in grid-connected photovoltaic systems via variational autoencoder-based monitoring

Anomaly detection is indispensable for ensuring the reliable operation of grid-connected photovoltaic (PV) systems. This study introduces a semi-supervised deep learning approach for fault detection in such systems. The method leverages a variational autoencoder (VAE) to extract features and identify anomalies. By training the VAE on normal operation data, a compact latent space representation is created. Abnormal observations, indicating faults, exhibit distinct feature vectors in this latent space. Multiple anomaly detection algorithms, including Isolation Forest, Epileptic Envelope, Local Outlier Factor, and One-Class SVM, are employed to discern normal and abnormal observations. This semi-supervised approach only requires fault-free data for training, without labeled faults, making it attractive in practice. A publicly available dataset, the Grid-connected PV System Faults (GPVS-Faults) dataset, which includes data from a PV plant operating in both maximum power point tracking (MPPT) and intermediate power point tracking (IPPT) switching modes, is used for evaluation. The proposed approach is assessed across various fault scenarios, such as partial shading, inverter faults, and MPPT/IPPT controller faults in boost converters. The outcomes underscore the effectiveness of VAE-based techniques in accurately identifying these faults, with accuracy rates reaching up to 92.90% for MPPT mode and 92.99% for IPPT mode, thus contributing to the robustness of fault detection in grid-connected PV systems.

Generative AI unlocks PET insights: brain amyloid dynamics and quantification.

Studying the spatiotemporal patterns of amyloid accumulation in the brain over time is crucial in understanding Alzheimer's disease (AD). Positron Emission Tomography (PET) imaging plays a pivotal role because it allows for the visualization and quantification of abnormal amyloid beta (Aβ) load in the living brain, providing a powerful tool for tracking disease progression and evaluating the efficacy of anti-amyloid therapies. Generative artificial intelligence (AI) can learn complex data distributions and generate realistic synthetic images. In this study, we demonstrate for the first time the potential of Generative Adversarial Networks (GANs) to build a low-dimensional representation space that effectively describes brain amyloid load and its dynamics. Using a cohort of 1,259 subjects with AV45 PET images from the Alzheimer's Disease Neuroimaging Initiative (ADNI), we develop a 3D GAN model to project images into a latent representation space and generate back synthetic images. Then, we build a progression model on the representation space based on non-parametric ordinary differential equations to study brain amyloid evolution. We found that global SUVR can be accurately predicted with a linear regression model only from the latent representation space (RMSE = 0.08 ± 0.01). We generated synthetic PET trajectories and illustrated predicted Aβ change in four years compared with actual progression. Generative AI can generate rich representations for statistical prediction and progression modeling and simulate evolution in synthetic patients, providing an invaluable tool for understanding AD, assisting in diagnosis, and designing clinical trials. The aim of this study was to illustrate the huge potential that generative AI has in brain amyloid imaging and to encourage its advancement by providing use cases and ideas for future research tracks.

Comprehensive consensus representation learning for incomplete multiview subspace clustering

Incomplete multiview clustering (IMVC) aims to address clustering problems caused by missing views. While some progress has been made, the current approaches still have some problems. First, the existing IMVC methods excessively rely on the original high-dimensional multiview data, leading to suboptimal results when the data are heavily contaminated. Second, the utilization of high-dimensional data often overlooks the complementarity of and consistency across multiple views. Third, the cluster structure of the input data is not adequately considered, which is detrimental to the resulting clustering performance. To address these problems, we propose a novel method: comprehensive consensus representation learning for incomplete multiview subspace clustering (CLR-IMVC). Specifically, CLR-IMVC assumes that all views are generated from a latent representation space. We model the original incomplete multiple views and the latent space to explore their correlations. Subsequently, we use the latent space to generate complete views and construct a subspace self-representation matrix for each view. Moreover, we innovatively treat the latent representation itself as new a view and likewise extract its self-representation matrix. Furthermore, we construct a third-order tensor by using these self-representation matrices and impose low-rank constraints on the tensor to explore the high-order correlations among the matrices. In addition, a consensus representation learning term is added to explore the consistency information between the views and the latent space, ultimately generating a robust clustering structure. Finally, the proposed method can be effectively solved by the augmented Lagrangian multiplier (ALM). Extensive experiments conducted on diverse datasets demonstrate the effectiveness of CLR-IMVC.

Comparing receptor binding properties of SARS-CoV-2 and of SARS-CoV virus by using unsupervised machine learning models

This work continues our recent molecular dynamics investigation of the three systems of the human ACE2 receptor interacting with the viral RBDs of SARS-CoV virus and two variants of SARS-CoV-2 viruses. The simulations are extended and analyzed using unsupervised machine learning models to give complementary descriptions of hidden features of the viral binding mechanism. Specifically, the principal component analysis (PCA) and the variational autoencoder (VAE) models are employed, both are classified as dimensionality reduction approaches with different focuses. The results support the molecular dynamics results that the two variants of SARS-CoV-2 bind stronger and more stable to the human ACE2 receptor than SARS-CoV virus does. Moreover, stronger bindings also affect the structure of the human receptor, making it fluctuate more, a sensitive feature which is hard to detect using standard analyses. Unexpectedly, it is found that the VAE model can learn and arrange randomly shuffled protein structures obtained from molecular dynamics in time order in the latent space representation. This result potentially has promising application in computational biomolecules. One could use this VAE model to jump forward in time during a molecular dynamics simulation, and to enhance the sampling of protein configuration space.

Radio Galaxy Zoo: Leveraging latent space representations from variational autoencoder

We propose to learn latent space representations of radio galaxies, and train a very deep variational autoencoder (VDVAE) on RGZ DR1, an unlabeled dataset, to this end. We show that the encoded features can be leveraged for downstream tasks such as classifying galaxies in labeled datasets, and similarity search. Results show that the model is able to reconstruct its given inputs, capturing the salient features of the latter. We use the latent codes of galaxy images, from MiraBest Confident and FR-DEEP NVSS datasets, to train various non-neural network classifiers. It is found that the latter can differentiate FRI from FRII galaxies achieving accuracy ≥ 76%, roc-auc ≥ 0.86, specificity ≥ 0.73 and recall ≥ 0.78 on MiraBest Confident dataset, comparable to results obtained in previous studies. The performance of simple classifiers trained on FR-DEEP NVSS data representations is on par with that of a deep learning classifier (CNN based) trained on images in previous work, highlighting how powerful the compressed information is. We successfully exploit the learned representations to search for galaxies in a dataset that are semantically similar to a query image belonging to a different dataset. Although generating new galaxy images (e.g. for data augmentation) is not our primary objective, we find that the VDVAE model is a relatively good emulator. Finally, as a step toward detecting anomaly/novelty, a density estimator — Masked Autoregressive Flow (MAF) — is trained on the latent codes, such that the log-likelihood of data can be estimated. The downstream tasks conducted in this work demonstrate the meaningfulness of the latent codes.

Multimodal Image Classification by Multiview Latent Pattern Extraction, Selection, and Correlation.

The large amount of data available in the modern big data era opens new opportunities to expand our knowledge by integrating information from heterogeneous sources. Multiview learning has recently achieved tremendous success in deriving complementary information from multiple data modalities. This article proposes a framework called multiview latent space projection (MVLSP) to integrate features extracted from multiple sources in a discriminative way to facilitate binary and multiclass classifications. Our approach is associated with three innovations. First, most existing multiview learning algorithms promote pairwise consistency between two views and do not have a natural extension to applications with more than two views. MVLSP finds optimum mappings from a common latent space to match the feature space in each of the views. As the matching is performed on a view-by-view basis, the framework can be readily extended to multiview applications. Second, feature selection in the common latent space can be readily achieved by adding a class view, which matches the latent space representations of training samples with their corresponding labels. Then, high-order view correlations are extracted by considering feature-label correlations. Third, a technique is proposed to optimize the integration of different latent patterns based on their correlations. The experimental results on the prostate image dataset demonstrate the effectiveness of the proposed method.

Unsupervised Learning for Improved Gamma-Ray Spectrometry in Pixelated Cadmium Zinc Telluride (CZT) Detectors

Machine learning has been found to be ubiquitously useful across many industries, presenting an opportunity to improve radiation detection performance using data-driven algorithms. Improved detector resolution can aid in the detection, identification, and quantification of radionuclides. In this work, a novel, data-driven, unsupervised learning approach is developed to improve detector spectral characteristics by learning, and subsequently rejecting, poorly performing regions of the pixelated detector. Feature engineering is used to fit individual characteristic photo peaks to a Doniach lineshape with a linear background model. Then, principal component analysis is used to learn a lower-dimension latent space representation of each photo peak where the pixels are clustered, and subsequently ranked, based on the cluster mean distance to an optimal point. Pixels within the worst cluster(s) are rejected to improve the full-width at half-maximum (FWHM) by 10% to 15% (relative to the bulk detector) at 50% net efficiency when applied to training data obtained from measurements of a 100 μCi 154Eu source using a H3D M400i pixelated cadmium zinc telluride detector. These results compare well with, but do not outperform, a greedy algorithm that accumulates pixels in order of FWHM from lowest to highest used as a benchmark. In the future, this approach can be extended to include the detector energy and angular response. Finally, the model is applied to newly seen natural and enriched uranium spectra relevant for nuclear safeguards applications.

Dynamic spatial‐temporal network for traffic forecasting based on joint latent space representation

AbstractIn the era of data‐driven transportation development, traffic forecasting is crucial. Established studies either ignore the inherent spatial structure of the traffic network or ignore the global spatial correlation and may not capture the spatial relationships adequately. In this work, a Dynamic Spatial‐Temporal Network (DSTN) based on Joint Latent Space Representation (JLSR) is proposed for traffic forecasting. Specifically, in the spatial dimension, a JLSR network is developed by integrating graph convolution and spatial attention operations to model complex spatial dependencies. Since it can adaptively fuse the representation information of local topological space and global dynamic space, a more comprehensive spatial dependency can be captured. In the temporal dimension, a Stacked Bidirectional Unidirectional Gated Recurrent Unit (SBUGRU) network is developed, which captures long‐term temporal dependencies through both forward and backward computations and superimposed recurrent layers. On these bases, DSTN is developed in an encoder‐decoder framework and periodicity is flexibly modeled by embedding branches. The performance of DSTN is validated on two types of real‐world traffic flow datasets, and it improves over baselines.

Real‐time topology optimization via learnable mappings

AbstractIn traditional topology optimization, the computing time required to iteratively update the material distribution within a design domain strongly depends on the complexity or size of the problem, limiting its application in real engineering contexts. This work proposes a multi‐stage machine learning strategy that aims to predict an optimal topology and the related stress fields of interest, either in 2D or 3D, without resorting to any iterative analysis and design process. The overall topology optimization is treated as regression task in a low‐dimensional latent space, that encodes the variability of the target designs. First, a fully‐connected model is employed to surrogate the functional link between the parametric input space characterizing the design problem and the latent space representation of the corresponding optimal topology. The decoder branch of an autoencoder is then exploited to reconstruct the desired optimal topology from its latent representation. The deep learning models are trained on a dataset generated through a standard method of topology optimization implementing the solid isotropic material with penalization, for varying boundary and loading conditions. The underlying hypothesis behind the proposed strategy is that optimal topologies share enough common patterns to be compressed into small latent space representations without significant information loss. Results relevant to a 2D Messerschmitt‐Bölkow‐Blohm beam and a 3D bridge case demonstrate the capabilities of the proposed framework to provide accurate optimal topology predictions in a fraction of a second.

Deep Survival Analysis With Latent Clustering and Contrastive Learning.

Survival analysis is employed to analyze the time before the event of interest occurs, which is broadly applied in many fields. The existence of censored data with incomplete supervision information about survival outcomes is one key challenge in survival analysis tasks. Although some progress has been made on this issue recently, the present methods generally treat the instances as separate ones while ignoring their potential correlations, thus rendering unsatisfactory performance. In this study, we propose a novel Deep Survival Analysis model with latent Clustering and Contrastive learning (DSACC). Specifically, we jointly optimize representation learning, latent clustering and survival prediction in a unified framework. In this way, the clusters distribution structure in latent representation space is revealed, and meanwhile the structure of the clusters is well incorporated to improve the ability of survival prediction. Besides, by virtue of the learned clusters, we further propose a contrastive loss function, where the uncensored data in each cluster are set as anchors, and the censored data are treated as positive/negative sample pairs according to whether they belong to the same cluster or not. This design enables the censored data to make full use of the supervision information of the uncensored samples. Through extensive experiments on four popular clinical datasets, we demonstrate that our proposed DSACC achieves advanced performance in terms of both C-index (0.6722, 0.6793, 0.6350, and 0.7943) and Integrated Brier Score (IBS) (0.1616, 0.1826, 0.2028, and 0.1120).

Electromagnetic Source Imaging via a Data-Synthesis-Based Convolutional Encoder-Decoder Network.

Electromagnetic source imaging (ESI) requires solving a highly ill-posed inverse problem. To seek a unique solution, traditional ESI methods impose various forms of priors that may not accurately reflect the actual source properties, which may hinder their broad applications. To overcome this limitation, in this article, a novel data-synthesized spatiotemporally convolutional encoder-decoder network (DST-CedNet) method is proposed for ESI. The DST-CedNet recasts ESI as a machine learning problem, where discriminative learning and latent-space representations are integrated in a CedNet to learn a robust mapping from the measured electroencephalography/magnetoencephalography (E/MEG) signals to the brain activity. In particular, by incorporating prior knowledge regarding dynamical brain activities, a novel data synthesis strategy is devised to generate large-scale samples for effectively training CedNet. This stands in contrast to traditional ESI methods where the prior information is often enforced via constraints primarily aimed for mathematical convenience. Extensive numerical experiments as well as analysis of a real MEG and epilepsy EEG dataset demonstrate that the DST-CedNet outperforms several state-of-the-art ESI methods in robustly estimating source signals under a variety of source configurations.

Multimodal Moore-Penrose Inverse-Based Recomputation Framework for Big Data Analysis.

Most multilayer Moore-Penrose inverse (MPI)-based neural networks, such as deep random vector functional link (RVFL), are structured with two separate stages: unsupervised feature encoding and supervised pattern classification. Once the unsupervised learning is finished, the latent encoding is fixed without supervised fine-tuning. However, in complex tasks such as handling the ImageNet dataset, there are often many more clues that can be directly encoded, while unsupervised learning, by definition, cannot know exactly what is useful for a certain task. There is a need to retrain the latent space representations in the supervised pattern classification stage to learn some clues that unsupervised learning has not yet been learned. In particular, the residual error in the output layer is pulled back to each hidden layer, and the parameters of the hidden layers are recalculated with MPI for more robust representations. In this article, a recomputation-based multilayer network using Moore-Penrose inverse (RML-MP) is developed. A sparse RML-MP (SRML-MP) model to boost the performance of RML-MP is then proposed. The experimental results with varying training samples (from 3k to 1.8 million) show that the proposed models provide higher Top-1 testing accuracy than most representation learning algorithms. For reproducibility, the source codes are available at https://github.com/W1AE/Retraining.

Fast Direct Optimal Control for Humanoids Based on Dynamics Representation in FPC Latent Space

Fast Direct Optimal Control for Humanoids Based on Dynamics Representation in FPC Latent Space

Enhancing Generalizability in Protein-Ligand Binding Affinity Prediction with Multimodal Contrastive Learning.

Improving the generalization ability of scoring functions remains a major challenge in protein-ligand binding affinity prediction. Many machine learning methods are limited by their reliance on single-modal representations, hindering a comprehensive understanding of protein-ligand interactions. We introduce a graph-neural-network-based scoring function that utilizes a triplet contrastive learning loss to improve protein-ligand representations. In this model, three-dimensional complex representations and the fusion of two-dimensional ligand and coarse-grained pocket representations converge while distancing from decoy representations in latent space. After rigorous validation on multiple external data sets, our model exhibits commendable generalization capabilities compared to those of other deep learning-based scoring functions, marking it as a promising tool in the realm of drug discovery. In the future, our training framework can be extended to other biophysical- and biochemical-related problems such as protein-protein interaction and protein mutation prediction.

Latent space representation method for structural acoustic assessments

When targeting structural acoustic objectives, engineering practitioners face epistemic uncertainties in the selection of optimal geometries and material distributions, particularly during early stages of the design process. Models built for simulating acoustic phenomena generally produce vector-valued output quantities of interest, such as autospectral density and frequency response functions. Given finite compute resources and time we seek computationally parsimonious ways to distill meaningful design information into actionable results from a limited set of model runs, and thus aim to use machine learning to perform model order reduction. Unlike time series data for which recurrent neural networks can learn from prior time steps to inform subsequent steps, frequency-dependent data demands a different machine learning paradigm. We thus evaluate the utility of autoencoders to represent structural acoustic results with a low dimensional latent space to enable such objectives as surrogate modeling for design optimization. We demonstrate the accuracy of autoencoder based methods of constructing a manifold representation for frequency dependent functions of varying modal density and damping, and discuss the predictive capability thereof.

Image‐based characterization of flocculation processes through PLS inspired representation learning in convolutional neural networks

AbstractMonitoring of flocculation processes such as those used in downstream processing of a fermentation broth is essential for process control. One approach is to apply microscopic imaging combined with image analysis for characterizing the state of the process. In this work, we investigate and compare the use of supervised feedforward convolutional neural network (CNN) architectures to predict the process states from the image information and compare the results with the traditional alternative of characterizing flocs based on manually engineered image features guided by human expertise. From a well‐defined image data set representing six process states, the objective is to establish end‐to‐end classification models which are accurate but at the same time learn meaningful latent variable space representations. Specifically, we evaluate three different CNN architectures with varying degrees of regularization and compare results with logistic regression models based on inputs from two different traditional feature engineering methods. By applying global average pooling as a structural regularizer to the CNN architecture, we significantly improve the generalization performance in comparison with the classification accuracies of the traditional feature engineered models. Furthermore, we show that by imposing a projection to latent structures (PLS) like regularization framework onto the CNN, it can also learn a latent variable representation that mimics the features selected by human expertise.