Latent Embedding Research Articles

Zero-shot-capable identification of phage-host relationships with whole-genome sequence representation by contrastive learning.

Accurately identifying phage-host relationships from their genome sequences is still challenging, especially for those phages and hosts with less homologous sequences. In this work, focusing on identifying the phage-host relationships at the species and genus level, we propose a contrastive learning based approach to learn whole-genome sequence embeddings that can take account of phage-host interactions (PHIs). Contrastive learning is used to make phages infecting the same hosts close to each other in the new representation space. Specifically, we rephrase whole-genome sequences with frequency chaos game representation (FCGR) and learn latent embeddings that 'encapsulate' phages and host relationships through contrastive learning. The contrastive learning method works well on the imbalanced dataset. Based on the learned embeddings, a proposed pipeline named CL4PHI can predict known hosts and unseen hosts in training. We compare our method with two recently proposed state-of-the-art learning-based methods on their benchmark datasets. The experiment results demonstrate that the proposed method using contrastive learning improves the prediction accuracy on known hosts and demonstrates a zero-shot prediction capability on unseen hosts. In terms of potential applications, the rapid pace of genome sequencing across different species has resulted in a vast amount of whole-genome sequencing data that require efficient computational methods for identifying phage-host interactions. The proposed approach is expected to address this need by efficiently processing whole-genome sequences of phages and prokaryotic hosts and capturing features related to phage-host relationships for genome sequence representation. This approach can be used to accelerate the discovery of phage-host interactions and aid in the development of phage-based therapies for infectious diseases.

Alignment and stability of embeddings: Measurement and inference improvement

Representation learning (RL) methods learn objects’ latent embeddings where information is preserved by distance. Since certain distance functions are invariant to certain linear transformations, one may obtain different embeddings while preserving the same information. In dynamic systems, a temporal difference in embeddings may be explained by the stability of the system or by the misalignment of embeddings due to arbitrary transformations. This study focuses on the embedding alignment problem to distinguish structural changes inherent to a system from arbitrary changes caused by representation learning methods, and quantify their magnitudes. In order to avoid any confusion due to the naming conventions in the literature, it should be noted that embedding alignment problems are different from graph matching/network alignment problems. In the representation learning literature, although the embedding alignment issue has been acknowledged, its measurement and empirical analysis have not received sufficient interest. In this work, we explore the embedding alignment and its parts, propose novel metrics to measure alignment and stability, and show their suitability through synthetic experiments. Real-world experiments show that both static and dynamic RL methods are prone to produce misaligned embeddings and such misalignment worsens the performance of dynamic network inference tasks. By ensuring alignment, the prediction accuracy raises by up to 90% in static and up to 40% in dynamic RL methods.

Cold-Start Next-Item Recommendation by User-Item Matching and Auto-Encoders

Recommendation systems provide personalized service to users and aim at suggesting to them items that they may prefer. There is an increasing requirement of next-item recommendation systems to infer a user's next favor item based on his/her historical selection of items. In this paper, we study the next-item recommendation under the cold-start situation, where the users in the system share no interaction with the new items. Specifically, we seek to address the problem from the perspective of zero-shot learning (ZSL), which classifies samples whose classes are unseen during training. To this end, we crystallize the relationship and setting from ZSL to cold-start next-item recommendation, and further propose a novel model called User-Item Matching and Auto-encoders (UIMA) which learns the latent embeddings for both users and items by exploiting user historical preferences and item attributes. Concretely, UIMA consists of three components, i.e., two auto-encoders for learning user and item embeddings and a matching network to explore the relationship between the learned user and item embeddings. We perform experiments on several cold-start next-item recommendation datasets, including movies, music, and bookmarks. Promising results demonstrate the effectiveness of the proposed method for cold-start next-item recommendation.

DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Video

For few-shot learning, it is still a critical challenge to realize photo-realistic face visually dubbing on high-resolution videos. Previous works fail to generate high-fidelity dubbing results. To address the above problem, this paper proposes a Deformation Inpainting Network (DINet) for high-resolution face visually dubbing. Different from previous works relying on multiple up-sample layers to directly generate pixels from latent embeddings, DINet performs spatial deformation on feature maps of reference images to better preserve high-frequency textural details. Specifically, DINet consists of one deformation part and one inpainting part. In the first part, five reference facial images adaptively perform spatial deformation to create deformed feature maps encoding mouth shapes at each frame, in order to align with input driving audio and also the head poses of input source images. In the second part, to produce face visually dubbing, a feature decoder is responsible for adaptively incorporating mouth movements from the deformed feature maps and other attributes (i.e., head pose and upper facial expression) from the source feature maps together. Finally, DINet achieves face visually dubbing with rich textural details. We conduct qualitative and quantitative comparisons to validate our DINet on high-resolution videos. The experimental results show that our method outperforms state-of-the-art works.

Unsupervised Deep Embedded Fusion Representation of Single-Cell Transcriptomics

Cell clustering is a critical step in analyzing single-cell RNA sequencing (scRNA-seq) data, which allows us to characterize the cellular heterogeneity of transcriptional profiling at the single-cell level. Single-cell deep embedded representation models have recently become popular since they can learn feature representation and clustering simultaneously. However, the model still suffers from a variety of significant challenges, including the massive amount of data, pervasive dropout events, and complicated noise patterns in transcriptional profiling. Here, we propose a Single-Cell Deep Embedding Fusion Representation (scDEFR) model, which develop a deep embedded fusion representation to learn fused heterogeneous latent embedding that contains both the transcriptome gene-level information and the cell topology information. We first fuse them layer by layer to obtain compressed representations of intercellular relationships and transcriptome information. After that, the zero-inflated negative binomial model (ZINB)-based decoder is proposed to capture the global probabilistic structure of the data and reconstruct the final gene expression information and cell graph. Finally, by simultaneously integrating the clustering loss, crossentropy loss, ZINB loss, and the cell graph reconstruction loss, scDEFR can optimize clustering performance and learn the latent representation in fused information under a joint mutual supervised strategy. We conducted extensive and comprehensive experiments on 15 single-cell RNA-seq datasets from different sequencing platforms to demonstrate the superiority of scDEFR over a variety of state-of-the-art methods.

SelfCF: A Simple Framework for Self-supervised Collaborative Filtering

Collaborative filtering (CF) is widely used to learn informative latent representations of users and items from observed interactions. Existing CF-based methods commonly adopt negative sampling to discriminate different items. That is, observed user-item pairs are treated as positive instances; unobserved pairs are considered as negative instances and are sampled under a defined distribution for training. Training with negative sampling on large datasets is computationally expensive. Further, negative items should be carefully sampled under the defined distribution, in order to avoid selecting an observed positive item in the training dataset. Unavoidably, some negative items sampled from the training dataset could be positive in the test set. Recently, self-supervised learning (SSL) , has emerged as a powerful tool to learn a model without negative samples. In this paper, we propose a self-supervised collaborative filtering framework (SelfCF) , that is specially designed for recommender scenario with implicit feedback. The proposed SelfCF framework simplifies Siamese networks and can be easily applied to existing deep-learning based CF models, which we refer to as backbone networks. The main idea of SelfCF is to augment the latent embeddings generated by backbone networks instead of the raw input of user/item ids. We propose and study three embedding perturbation techniques that can be applied to different types of backbone networks including both traditional CF models and graph-based models. The framework enables learning informative representations of users and items without negative samples, and is agnostic to the encapsulated backbones. We conduct experimental comparisons on four datasets, one self-supervised framework, and eight baselines to show that our framework may achieve even better recommendation accuracy than the encapsulated supervised counterpart with a 2×–4× faster training speed. The results also demonstrate that SelfCF can boost up the accuracy of a self-supervised framework BUIR by 17.79% on average and shows competitive performance with baselines.

MRGCN: cancer subtyping with multi-reconstruction graph convolutional network using full and partial multi-omics dataset.

Cancer is a molecular complex and heterogeneous disease. Each type of cancer is usually composed of several subtypes with different treatment responses and clinical outcomes. Therefore, subtyping is a crucial step in cancer diagnosis and therapy. The rapid advances in high-throughput sequencing technologies provide an increasing amount of multi-omics data, which benefits our understanding of cancer genetic architecture, and yet poses new challenges in multi-omics data integration. We propose a graph convolutional network model, called MRGCN for multi-omics data integrative representation. MRGCN simultaneously encodes and reconstructs multiple omics expression and similarity relationships into a shared latent embedding space. In addition, MRGCN adopts an indicator matrix to denote the situation of missing values in partial omics, so that the full and partial multi-omics processing procedures are combined in a unified framework. Experimental results on 11 multi-omics datasets show that cancer subtypes obtained by MRGCN with superior enriched clinical parameters and log-rank test P-values in survival analysis over many typical integrative methods. https://github.com/Polytech-bioinf/MRGCN.git https://figshare.com/articles/software/MRGCN/23058503.

Joint variational autoencoders for multimodal imputation and embedding.

Single-cell multimodal datasets have measured various characteristics of individual cells, enabling a deep understanding of cellular and molecular mechanisms. However, multimodal data generation remains costly and challenging, and missing modalities happen frequently. Recently, machine learning approaches have been developed for data imputation but typically require fully matched multimodalities to learn common latent embeddings that potentially lack modality specificity. To address these issues, we developed an open-source machine learning model, Joint Variational Autoencoders for multimodal Imputation and Embedding (JAMIE). JAMIE takes single-cell multimodal data that can have partially matched samples across modalities. Variational autoencoders learn the latent embeddings of each modality. Then, embeddings from matched samples across modalities are aggregated to identify joint cross-modal latent embeddings before reconstruction. To perform cross-modal imputation, the latent embeddings of one modality can be used with the decoder of the other modality. For interpretability, Shapley values are used to prioritize input features for cross-modal imputation and known sample labels. We applied JAMIE to both simulation data and emerging single-cell multimodal data including gene expression, chromatin accessibility, and electrophysiology in human and mouse brains. JAMIE significantly outperforms existing state-of-the-art methods in general and prioritized multimodal features for imputation, providing potentially novel mechanistic insights at cellular resolution.

Adversarial learning based residual variational graph normalized autoencoder for network representation

With the success of Graph Neural Networks (GNNs) on non-Euclidean data, some GNN-based approaches to network representation learning have emerged in recent years. Graph autoencoder (GAE) and variational GAE (VGAE) are unsupervised learning frameworks widely used to learn latent embedding representations based on graph-structured data. However, it performs poorly in link prediction, node clustering, and graph visualization task when isolated nodes are involved or when the raw data representation is weak. Based on the perspective of improving the capability of the latent embedding representation and the robustness of the model, we propose an Adversarial Learning based Residual Variational Graph Normalized Autoencoder (ARVGNA). We introduce a residual network to augment the raw data representation, use L2-normalization to derive better node embeddings, and add adversarial learning to promote the activation of the node representation and enhance the generalization of the model. The experiments on three benchmark datasets show that the proposed ARVGNA greatly outperforms the baselines in the tasks of link prediction, node clustering, and graph visualization, and exhibits strong competitive power and high robustness in network representation.

Learnable latent embeddings for joint behavioural and neural analysis

Mapping behavioural actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioural data increases, there is growing interest in modelling neural dynamics during adaptive behaviours to probe neural representations1–3. In particular, although neural latent embeddings can reveal underlying correlates of behaviour, we lack nonlinear techniques that can explicitly and flexibly leverage joint behaviour and neural data to uncover neural dynamics3–5. Here, we fill this gap with a new encoding method, CEBRA, that jointly uses behavioural and neural data in a (supervised) hypothesis- or (self-supervised) discovery-driven manner to produce both consistent and high-performance latent spaces. We show that consistency can be used as a metric for uncovering meaningful differences, and the inferred latents can be used for decoding. We validate its accuracy and demonstrate our tool’s utility for both calcium and electrophysiology datasets, across sensory and motor tasks and in simple or complex behaviours across species. It allows leverage of single- and multi-session datasets for hypothesis testing or can be used label free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, for the production of consistent latent spaces across two-photon and Neuropixels data, and can provide rapid, high-accuracy decoding of natural videos from visual cortex.

Graph deep learning enabled spatial domains identification for spatial transcriptomics

Advancing spatially resolved transcriptomics (ST) technologies help biologists comprehensively understand organ function and tissue microenvironment. Accurate spatial domain identification is the foundation for delineating genome heterogeneity and cellular interaction. Motivated by this perspective, a graph deep learning (GDL) based spatial clustering approach is constructed in this paper. First, the deep graph infomax module embedded with residual gated graph convolutional neural network is leveraged to address the gene expression profiles and spatial positions in ST. Then, the Bayesian Gaussian mixture model is applied to handle the latent embeddings to generate spatial domains. Designed experiments certify that the presented method is superior to other state-of-the-art GDL-enabled techniques on multiple ST datasets. The codes and dataset used in this manuscript are summarized at https://github.com/narutoten520/SCGDL.

Multi-auxiliary Augmented Collaborative Variational Auto-encoder for Tag Recommendation

Recommending appropriate tags to items can facilitate content organization, retrieval, consumption, and other applications, where hybrid tag recommender systems have been utilized to integrate collaborative information and content information for better recommendations. In this article, we propose a multi-auxiliary augmented collaborative variational auto-encoder (MA-CVAE) for tag recommendation, which couples item collaborative information and item multi-auxiliary information, i.e., content and social graph, by defining a generative process. Specifically, the model learns deep latent embeddings from different item auxiliary information using variational auto-encoders (VAE), which could form a generative distribution over each auxiliary information by introducing a latent variable parameterized by deep neural network. Moreover, to recommend tags for new items, item multi-auxiliary latent embeddings are utilized as a surrogate through the item decoder for predicting recommendation probabilities of each tag, where reconstruction losses are added in the training phase to constrain the generation for feedback predictions via different auxiliary embeddings. In addition, an inductive variational graph auto-encoder is designed to infer latent embeddings of new items in the test phase, such that item social information could be exploited for new items. Extensive experiments on MovieLens and citeulike datasets demonstrate the effectiveness of our method.

MARSY: a multitask deep-learning framework for prediction of drug combination synergy scores.

Combination therapies have emerged as a treatment strategy for cancers to reduce the probability of drug resistance and to improve outcomes. Large databases curating the results of many drug screening studies on preclinical cancer cell lines have been developed, capturing the synergistic and antagonistic effects of combination of drugs in different cell lines. However, due to the high cost of drug screening experiments and the sheer size of possible drug combinations, these databases are quite sparse. This necessitates the development of transductive computational models to accurately impute these missing values. Here, we developed MARSY, a deep-learning multitask model that incorporates information on the gene expression profile of cancer cell lines, as well as the differential expression signature induced by each drug to predict drug-pair synergy scores. By utilizing two encoders to capture the interplay between the drug pairs, as well as the drug pairs and cell lines, and by adding auxiliary tasks in the predictor, MARSY learns latent embeddings that improve the prediction performance compared to state-of-the-art and traditional machine-learning models. Using MARSY, we then predicted the synergy scores of 133722 new drug-pair cell line combinations, which we have made available to the community as part of this study. Moreover, we validated various insights obtained from these novel predictions using independent studies, confirming the ability of MARSY in making accurate novel predictions. An implementation of the algorithms in Python and cleaned input datasets are provided in https://github.com/Emad-COMBINE-lab/MARSY.

A Knowledge Concept Recommendation Model Based on Tensor Decomposition and Transformer Reordering

To help students choose the knowledge concepts that meet their needs so that they can learn courses in a more personalized way, thus improving the effectiveness of online learning, this paper proposes a knowledge concept recommendation model based on tensor decomposition and transformer reordering. Firstly, the student tensor, knowledge concept tensor, and interaction tensor are created based on the heterogeneous data of the online learning platform are fused and simplified as an integrated tensor; secondly, we perform multi-dimensional comprehensive analysis on the integrated tensor with tensor-based high-order singular value decomposition to obtain the student personalized feature matrix and the initial recommendation sequence of knowledge concepts, and then obtain the latent embedding matrix of knowledge concepts via Transformer that combine initial recommendation sequence of knowledge concepts and knowledge concept learning sequential information; finally, the final Top-N knowledge concept recommendation list is generated by fusing the latent embedding matrix of knowledge concepts and the students’ personalized feature matrix. Experiments on two real datasets show that the model recommendation performance of this paper is better compared to the baseline model.

Learning latent embedding via weighted projection matrix alignment for incomplete multi-view clustering

Multi-view data are generated from multiple perspectives or diverse domains. Although many multi-view clustering methods have achieved a lot of successes based on the assumption that the data is integrity, in fact, these data may exist the case of missing instances in real applications, resulting in incomplete multi-view data. Thus, incomplete multi-view clustering has been proposed to handle this issue, which has gained considerable attention. However, for the most of existing approaches, there still have the following limitations: (i) the latent common representation information within views is not well exploited. (ii) the potential information hidden in missing views are ignored to some extent. Therefore, we proposed a novel Learning Latent Embedding via weighted projection matrix alignment for Incomplete Multi-view Clustering, termed as LLE-IMC. Specifically, a view completion model is introduced in latent embedding learning to infer the missing information. To further explore the consistency information of different views, different projection matrices are enforced to align to cluster centers by ℓ2,1 norm regularization. Furthermore, an efficient optimization algorithm is presented to resolve the proposed model with convergence guarantee. On several incomplete multi-view datasets, experimental results show that our proposed LLE-IMC performs better in comparison to the state-of-the-art methods, in terms of many metrics.

Distance transform learning for structural and functional analysis of coronary artery from dual-view angiography

The structural and functional analysis of coronary arteries in X-ray angiography (XRA) images has great clinical importance for the intraoperative treatment of coronary artery disease. However, due to the complexity of image surroundings, the non-intuition of functional information, and the complex relationships between structure and function, the structural and functional analysis in XRA images is a challenging task. To address these problems, we are proposing an efficient framework, termed AngioSFA. It can effectively associate structural analysis with functional analysis. The AngioSFA mainly consists of a distance transform segmentation network (DTSN) for structural analysis and a dual-view sequential learning network (DSLN) for functional analysis. Specifically, DTSN exploits distance transform in the encoder–decoder architecture to predict the distance map of the artery structure. It considers the geometric property of coronary artery structure to preserve a proper shape prototype. DSLN exploits a dual-view temporal convolutional network to learn the non-intuitive physiological knowledge related to the functional significance from the radius sequences extracted from dual-view XRA images. Furthermore, a commonality sharing module is designed to map the latent embeddings of radius sequences into a shared latent space through orthogonal mapping. Moreover, we introduce a dual-view interaction module to explore the intra-view attentions and the inter-view correlations among the anatomical features of coronary arteries. Extensive experiments on two clinical datasets show that AngioSFA achieves an overall dice of 91.06% for structural analysis and an overall precision of 89.96% for functional analysis. The experimental results and the discussions demonstrate that AngioSFA has a great potential for the efficient intraoperative treatment of coronary artery disease.

Multi-Dimensional Information Alignment in Different Modalities for Generalized Zero-Shot and Few-Shot Learning

Generalized zero-shot learning (GZSL) aims to solve the category recognition tasks for unseen categories under the setting that training samples only contain seen classes while unseen classes are not available. This research is vital as there are always existing new categories and large amounts of unlabeled data in realistic scenarios. Previous work for GZSL usually maps the visual information of the visible classes and the semantic description of the invisible classes into the identical embedding space to bridge the gap between the disjointed visible and invisible classes, while ignoring the intrinsic features of visual images, which are sufficiently discriminative to classify themselves. To better use discriminative information from visual classes for GZSL, we propose the n-CADA-VAE. In our approach, we map the visual feature of seen classes to a high-dimensional distribution while mapping the semantic description of unseen classes to a low-dimensional distribution under the same latent embedding space, thus projecting information of different modalities to corresponding space positions more accurately. We conducted extensive experiments on four benchmark datasets (CUB, SUN, AWA1, and AWA2). The results show our model’s superior performance in generalized zero-shot as well as few-shot learning.

Autoencoding slow representations for semi-supervised data-efficient regression

The slowness principle is a concept inspired by the visual cortex of the brain. It postulates that the underlying generative factors of a quickly varying sensory signal change on a different, slower time scale. By applying this principle to state-of-the-art unsupervised representation learning methods one can learn a latent embedding to perform supervised downstream regression tasks more data efficient. In this paper, we compare different approaches to unsupervised slow representation learning such as L_p norm based slowness regularization and the SlowVAE, and propose a new term based on Brownian motion used in our method, the S-VAE. We empirically evaluate these slowness regularization terms with respect to their downstream task performance and data efficiency in state estimation and behavioral cloning tasks. We find that slow representations show great performance improvements in settings where only sparse labeled training data is available. Furthermore, we present a theoretical and empirical comparison of the discussed slowness regularization terms. Finally, we discuss how the Fréchet Inception Distance (FID), commonly used to determine the generative capabilities of GANs, can predict the performance of trained models in supervised downstream tasks.

Multi-Modality Sensing and Data Fusion for Multi-Vehicle Detection

With the recent surge in autonomous driving vehicles, the need for accurate vehicle detection and tracking is critical now more than ever. Detecting vehicles from visual sensors fails in non-line of sight (NLOS) settings. This can be compensated by the inclusion of other modalities in an multi-domain sensing environment. In this paper, we propose different deep learning based frameworks for fusing different modalities (image, radar, acoustic, seismic) through the exploitation of complementary latent embeddings among different modalities, and incorporating different state-of-the-art fusion strategies. Our proposed fusion frameworks perform considerably better than unimodal detection. Moreover, fusion between image and non-image modalities helps to track vehicles, as well as detect them, when direct line of sight (LOS) is not available. We validate our models on the real-world multimodal ESCAPE dataset, showing 33.16% improvement in vehicle detection performance by fusion (over visual inference alone) while 30-42% of the test scenarios have vehicles in NLOS. To showcase the generalization aspect of the proposed frameworks over multiple datasets, we also validate our models on multimodal NuScene dataset, showing ~22% improvement than the competing methods.

Semi-Supervised Bolt Anomaly Detection Based on Local Feature Reconstruction

During the process of high-speed railway online maintenance, anomaly detection of targets in grease injection facility is extremely challenging. Environmental uncertainty (noise disturbance, various illumination conditions, changeable camera perspective, etc.) limits the performance of existing unsupervised anomaly detection methods, which tend to struggle in such complex backgrounds without prior knowledge of anomalies. Therefore, we propose a novel semi-supervised anomaly detection algorithm that only learns from anomaly-free samples and annotations while can recognize both anomaly-free and anomaly targets (key-areas) and detecting anomalous targets simultaneously. We firstly propose a key-area recognition module (KRM) to segment anomaly-free targets and predict coordinates of both anomaly-free and anomalous key-areas’ center point (key-points). By using the predicted coordinates, we generated a circular mask to erase features within the key-areas. Based on the erased features, a local feature reconstruction and segmentation module (LFRSM) is proposed to restore the latent embeddings within the key-areas and segment both anomaly-free and anomalous key-areas. The final anomaly results are obtained by analyzing the original segmentation results of KRM and the reconstructed segmentation results of LFRSM. To validate the effectiveness of the proposed algorithm, we develop a dataset, called BOLT-DET, for bolt anomaly detection in complex scenes. The proposed algorithm outperforms other existing state-of-the-art (SOTA) anomaly detection algorithms (SPADE, DFR, CFLOW, Student-Teacher, PatchCore, and DRAEM) with mIoU of 0.66, AUROC of 0.92 and F1-Measure of 0.69 on BOLT-DET. It can accurately detect anomalies and meet the requirement of practical industrial applications.