Embedding Learning Research Articles

DMR: disentangled and denoised learning for multi-behavior recommendation

In recommender systems, leveraging auxiliary behaviors (e.g. view, cart) to enhance the recommendation in the target behavior (e.g. purchase) is crucial for mitigating the sparsity issue inherent in single-behavior recommendation. This has given rise to the multi-behavior recommendation (MBR). Existing MBR task faces two primary challenges. First, the irrelevant auxiliary behaviors that do not align with the target behavior, can negatively impact the prediction accuracy for user preference in the target behavior. Second, these methods typically learn coarse-grained user preferences, failing to model the consistency and distinctiveness among multiple behaviors at a fine-grained level. To address these issues, we propose a disentangled and denoised model for multi-behavior recommendation (DMR), which employs user preferences reflected in the target behavior to guide the learning of user and item embeddings in auxiliary behaviors. Specifically, we first design a disentangled graph convolutional network, modeling the fine-grained user preference under multiple behaviors in view of item attribute domains. We also propose a denoised contrastive learning strategy, where we align the user preferences in multiple behaviors by reducing the influence of noisy data existing in auxiliary behaviors. Experimental results on two real-world datasets show the proposal can improve the performance of MBR models effectively, which achieves on average 3.12% on the Retailrocket dataset and 3.28% on the Beibei dataset over the performance of state-of-the-art baselines. Extensive experiments also demonstrate our model’s competitive performance for fine-grained preference learning and denoised learning.

PREmbed: Balancing Conditional Generative Models with Embedding Pretraining and Regularization

Recent advancements in conditional generative models, e.g., Conditional Variational AutoEncoder (CVAE) and Conditional Denoising Diffusion Probabilistic Model (CDDPM), which utilize class-specific embeddings for generating images in specific classes, have demonstrated exceptional capabilities in producing high-quality images on balanced datasets. However, their performance decreases with imbalanced real-world data, affecting the fidelity and diversity of the generated images. This paper discusses the reasons and solutions for the imbalance issue in CVAE and CDDPM. By selectively reweighting the gradients of the embedding layer and main network, we identify the embedding layer’s bias as a critical bottleneck in these models. Motivated by this, we propose Embedding Pretraining and Regularization (PREmbed), a training methodology for addressing biased learning in the embedding layer of these models. PREmbed consists of two key components: (i) Masked Autoencoder pretraining on the original imbalanced dataset to obtain a robust initial embedding, and (ii) an embedding regularization loss to preserve class-level distances during the training phase to improve imbalanced embedding learning. Implementing PREmbed leads to the enhanced convergence of a robust embedding layer. Our experiments on imbalanced CIFAR-10 and CUB-200 datasets demonstrate that PREmbed enhances the performance of CVAE and CDDPM; it outperforms the baseline CVAE by 22.3% (FID 18.63) and the CDDPM by 30.3% (FID 12.70) in imbalanced CIFAR-10; 33.7% (FID 24.08) and 26.5% (FID 13.90) in CUB-200; and 16.4% (FID 49.41) and 15.2% (FID 33.05) in ImageNet-LT (which is where it achieves the state-of-the-art performance in imbalanced conditional image generation task).

Centering justice in the codesign of mindfulness and compassion-based college curricula.

Colleges and universities are increasingly common contexts in which young people navigate the transition to adulthood. Research suggests that mindfulness and compassion may support undergraduates as they navigate this developmental transition. Embedding learning about mindfulness, compassion, and flourishing into college curricula demonstrates promise in supporting undergraduate wellness and academic outcomes. However, there is a need to generate curricula that are relevant to the lived realities of undergraduates and attentive to relational dimensions of wellness, including social justice and systemic determinants of health. Codesign holds promise as a method to generate such curricula. This study used qualitative methods to examine the codesign of an accredited college-level course that teaches about the interrelationship between mindfulness, compassion, human flourishing, and social justice. Qualitative data that emerged during the codesign process were analyzed to answer the following research questions: (1) How did mindfulness and compassion practice support the codesign process? (2) What design tensions emerged during the adaptation and collaborative design of a social justice-oriented mindfulness and compassion-based course? We found that weaving shared mindfulness and compassion practice into the codesign process supported study participants in working with their emotions, connecting with others, and balancing power. In turn, these skills allowed codesign team to effectively grapple with complex design tensions that arose on the levels of vision, approach, and project tensions as the team sought to fulfill its commitments to individual and collective transformation. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Link prediction of heterogeneous complex networks based on an improved embedding learning algorithm.

Link prediction in heterogeneous networks is an active research topic in the field of complex network science. Recognizing the limitations of existing methods, which often overlook the varying contributions of different local structures within these networks, this study introduces a novel algorithm named SW-Metapath2vec. This algorithm enhances the embedding learning process by assigning weights to meta-path traces generated through random walks and translates the potential connections between nodes into the cosine similarity of embedded vectors. The study was conducted using multiple real-world and synthetic datasets to validate the proposed algorithm's performance. The results indicate that SW-Metapath2vec significantly outperforms benchmark algorithms. Notably, the algorithm maintains high predictive performance even when a substantial proportion of network nodes are removed, demonstrating its resilience and potential for practical application in analyzing large-scale heterogeneous networks. These findings contribute to the advancement of link prediction techniques and offer valuable insights and tools for related research areas.

Accurate multi-behavior sequence-aware recommendation via graph convolution networks.

How can we recommend items to users utilizing multiple types of user behavior data? Multi-behavior recommender systems leverage various types of user behavior data to enhance recommendation performance for the target behavior. These systems aim to provide personalized recommendations, thereby improving user experience, engagement, and satisfaction across different applications such as e-commerce platforms, streaming services, news websites, and content platforms. While previous approaches in multi-behavior recommendation have focused on incorporating behavioral order and dependencies into embedding learning, they often overlook the nuanced importance of individual behaviors in shaping user preferences during model training. We propose MBA (Multi-Behavior sequence-Aware recommendation via graph convolution networks), an accurate framework for multi-behavior recommendations. MBA adopts a novel approach by learning embeddings that capture both the dependencies between behaviors and their relative importance in influencing user preferences. Additionally, MBA employs sophisticated sampling strategies that consider the sequential nature of behaviors during model training, ensuring that the model effectively learns from the entire behavioral sequence. Through extensive experiments on real-world datasets, we demonstrate the superior performance of MBA compared to existing methods. MBA outperforms the best competitor, achieving improvements of up to 11.2% and 11.4% in terms of HR@10 and nDCG@10, respectively. These findings underscore the effectiveness of MBA in providing accurate and personalized recommendations tailored to individual user preferences.

Integrating single-cell multi-omics data through self-supervised clustering

Advances in single-cell sequencing techniques enable individual cells to be sequenced across multiple modalities simultaneously, such as transcriptomics, epigenomics, and proteomics. Integrating multi-omics single-cell data provides a deeper and more comprehensive vision of genomic mechanisms. Due to the huge distribution shift between modalities, current integration methods mostly align the modality through domain adaption or similar strategies. They achieved limited performance likely because the modalities are over-divergent. Here, we propose a novel single-cell multimodal fusion method, scFPN, to improve the learned embedding by a clustering strategy. Specifically, scFPN first embeds each modality data through a feature pyramid network with a modality-specific variational auto-encoder. The learned hierarchical embeddings are then fused and input into a dual self-supervision optimizing module for attracting similar cells and separating dissimilar cells. We conducted comprehensive experiments on recently produced six datasets from different sequencing platforms and demonstrated the superiority of scFPN over a variety of state-of-the-art methods. More importantly, scFPN showed bio-interpretability by marker enrichment analysis from de-noising and imputing raw profiles.

Susceptibility modeling of hydro-morphological processes considered river topology

ABSTRACT Hydro-Morphological Processes (HMP, any natural phenomenon contained within the spectrum defined between debris flows and flash floods) are most likely to occur in small catchments, especially buffer zones along or near rivers. Rivers transfer matter and energy between hydrographic units, thus potentially affecting the occurrence of HMPs in nearby catchments. To date, previous HMP susceptibility studies based on data-driven modeling lacked taking into account these interactions between catchments. In this work, we fully considered the role played by river topology and developed a Topology-based HMP susceptibility model (Topo-HMPSM) to emulate the interactions between catchments and predict the susceptibility of HMPs for the Yangtze River Basin during 1985–2015. Results confirmed that our proposed model outperforms four selected baseline models with the best F1-score (mean = 0.744, best = 0.756) and relatively lower uncertainties. A graph-based deep neural network improves the predictive and interpretability of HMP susceptibility modeling using embedding learning techniques. This work attempts to set a standard for incorporating river topology into deep learning models. Our findings highlight the importance of river topology in predicting HMP and support better informed hazard mitigation strategies.

Domain-weighted transfer learning and discriminative embeddings for low-resource speaker verification

Transfer learning has been shown to be effective in enhancing speaker verification performance in low-resource conditions. However, the inclusion of additional datasets may cause domain mismatch. Additionally, mismatched data volume and model complexity during fine-tuning can degrade speaker verification performance. In this paper, we propose a domain-weighted allocation fine-tuning strategy that employs the Kernel Mean Matching (KMM) algorithm to adjust the distribution differences between the in-domain and out-of-domain datasets. It assigns weights to each sample in the source datasets and utilizes the maximum mean discrepancy (MMD) distance to measure the effectiveness of distribution adaptation. The domain-weighted allocation fine-tuning strategy (DWA-FT) effectively mitigates the issue of domain mismatch during model training. We also propose two backend canonical correlation analysis (CCA) embedding transformation methods, the CCA embedding fusion and the CCA embedding constraint. These methods aim to enhance the quality of speaker embeddings. The experimental results demonstrate that the proposed methods effectively enhance the performance of the speaker verification system in low-resource scenarios. Compared to the baseline, our methods achieve relative improvements of 51.03% in PLDA scoring and 46.02% in cosine similarity scoring on the Himia dataset.

Contrastive Unsupervised Graph Neural Network in Financial Industry

This paper explores the application of the Contrastive Unsupervised Graph Neural Network (CuGNN) framework in financial domains, leveraging its heterophily-based adaptive convolution to address critical tasks like fraud detection, risk propagation, and portfolio optimization. CuGNN's ability to identify and utilize heterophilic patterns in financial transaction graphs enables robust representation learning even in unsupervised settings, where labeled data is scarce. Specifically, we adapt CuGNN to model risk spread across trading and investment networks by dynamically capturing high-frequency and low-frequency signals through its adaptive convolution mechanism. This approach allows us to differentiate between correlated and inverse-correlated asset behaviors, providing deeper insights into systemic risks and diversification strategies. Furthermore, CuGNN’s feature-distribution embedding and latent-space contrastive learning strategies are utilized to detect anomalous interactions in high-frequency trading networks and to identify heterophilic relationships in credit scoring and supply chain finance. By applying the CuGNN framework across diverse financial datasets, this study demonstrates its potential to uncover hidden structures and optimize decision-making in critical financial applications, addressing the growing need for explainable and adaptive graph-based methods in the sector.

PMHR: Path-Based Multi-Hop Reasoning Incorporating Rule-Enhanced Reinforcement Learning and KG Embeddings

Multi-hop reasoning provides a means for inferring indirect relationships and missing information from knowledge graphs (KGs). Reinforcement learning (RL) was recently employed for multi-hop reasoning. Although RL-based methods provide explainability, they face challenges such as sparse rewards, spurious paths, large action spaces, and long training and running times. In this study, we present a novel approach that combines KG embeddings and RL strategies for multi-hop reasoning called path-based multi-hop reasoning (PMHR). We address the issues of sparse rewards and spurious paths by incorporating a well-designed reward function that combines soft rewards with rule-based rewards. The rewards are adjusted based on the target entity and the path to it. Furthermore, we perform action filtering and utilize the vectors of entities and relations acquired through KG embeddings to initialize the environment, thereby significantly reducing the runtime. Experiments involving a comprehensive performance evaluation, efficiency analysis, ablation studies, and a case study were performed. The experimental results on benchmark datasets demonstrate the effectiveness of PMHR in improving KG reasoning accuracy while preserving interpretability. Compared to existing state-of-the-art models, PMHR achieved Hit@1 improvements of 0.63%, 2.02%, and 3.17% on the UMLS, Kinship, and NELL-995 datasets, respectively. PMHR provides not only improved reasoning accuracy and explainability but also optimized computational efficiency, thereby offering a robust solution for multi-hop reasoning.

Using deep learning and word embeddings for predicting human agreeableness behavior

The latest advancements of deep learning have resulted in a new era of natural language processing. The machines now possess an unparallel ability to interpret and engage with various tasks such as text classification, content generation and natural language understanding. This development extended to the analysis of human behavior, where deep learning models are used to decode human personality. Due to the rise of social media, generating huge amounts of textual data that reshaped communication patterns. Understanding personality traits is a challenging topic which helps us to explore the patterns of thoughts, feelings and behaviors which are helpful for recruitment, career counselling and consumers’ behavior for marketing, etc. In this research study, the main aim is to predict the human personality trait of agreeableness showing whether a person is emotional who feels a lot or thinker who is logical and has rational thinking. This behavior leads to analyzing them as cooperative, friendly and respecting difference of views. For comprehensive empirical analysis, shallow machine learning models, ensemble models, and deep learning technique including state of the art transformer-based models are applied on real-world dataset of MBTI. For feature engineering, textual features of TF-IDF and POS tagging and word embeddings such as word2vec, glove and sentence embeddings are explored. The results analysis shows the highest performance 91.57% with sentence embeddings utilizing Bi-LSTM algorithm that highlights the power of this study as compared to existing studies in the relevant literature.

IGCN: A Provably Informative GCN Embedding for Semi-Supervised Learning With Extremely Limited Labels.

Graph Neural Networks (GNNs) have gained much more attention in the representation learning for the graph-structured data. However, the labels are always limited in the graph, which easily leads to the overfitting problem and causes the poor performance. To solve this problem, we propose a new framework called IGCN, short for Informative Graph Convolutional Network, where the objective of IGCN is designed to obtain the informative embeddings via discarding the task-irrelevant information of the graph data based on the mutual information. As the mutual information for irregular data is intractable to compute, our framework is optimized via a surrogate objective, where two terms are derived to approximate the original objective. For the former term, it demonstrates that the mutual information between the learned embeddings and the ground truth should be high, where we utilize the semi-supervised classification loss and the prototype based supervised contrastive learning loss for optimizing it. For the latter term, it requires that the mutual information between the learned node embeddings and the initial embeddings should be high and we propose to minimize the reconstruction loss between them to achieve the goal of maximizing the latter term from the feature level and the layer level, which contains the graph encoder-decoder module and a novel architecture GCN Info. Moreover, we provably show that the designed GCN Info can better alleviate the information loss and preserve as much useful information of the initial embeddings as possible. Experimental results show that the IGCN outperforms the state-of-the-art methods on 7 popular datasets.

Bangla hate speech detection by embedding and hybrid machine learning algorithms

The worrisome increase in hate speech in Bangladesh, driven by the rapidly expanding social media user base, has adversely affected cybersecurity and online safety. Most hate speech detection technologies overlook less commonly spoken languages, such as Bangla, in favour of more widely used ones. This project aims to address the gap by identifying hate speech in Bangla through the utilisation of sophisticated word embedding techniques (Word2Vec, GloVe, and FastText) and machine learning algorithms (Multinomial Naive Bayes, Random Forest, K-Nearest Neighbours, and Extreme Gradient Boosting). We used a dataset of 30,000 annotated messages, encompassing both positive and hate speech, to train and test the models. FastText combined with hybrid RF and SVM yielded the highest performance among the models, with an accuracy of 96.03%. The system's generalisability and usefulness were evident when it identified hate speech not included in the training data. Enhancing the identification of harmful content in Bangla is essential for the burgeoning digital ecosystem in Bangladesh, hence contributing to cybersecurity. This endeavour establishes a foundation for future study by efficiently employing advanced word embedding and machine learning techniques to identify hate speech in Bangla. It could significantly influence the advancement of more secure digital environments and the overall condition of internet security.

Monitoring scene-level land use changes with similarity-assisted change detection network

ABSTRACT Scene-level change detection (SLCD), as opposed to pixel-level or object-level detection, offers semantic-level change information, which is vital for monitoring land use changes. The existing SLCD methods often rely on post-classification strategies that require separate classifiers for each temporal image, leading to a more complex process and decreased accuracy when classification errors occur. To address these issues, this study redefines SLCD as similarity-assisted category acquisition and introduces SACDNet (similarity-assisted change detection network), a novel network that integrates semantic information from single-temporal scenes with similarity information between bi-temporal scenes to enhance feature embedding learning. The effectiveness of SACDNet was evaluated on two public datasets, yielding impressive results. On the MtS-WH dataset, OA and F1 scores for binary change detection were 0.9701 and 0.9807, while for semantic change detection, both OA and F1 were 0.9042. A similar superior performance was also observed on the LUSCD dataset. Our experimental findings demonstrate that SACDNet significantly outperforms existing advanced SLCD methodologies, indicating its potential for improved change detection accuracy in various applications.

Wi-Fi CSI fingerprinting-based indoor positioning using deep learning and vector embedding for temporal stability

Fingerprinting systems based on channel state information (CSI) often rely on updated databases to achieve indoor positioning with high accuracy and resolution of centimeter-level. However, regularly maintaining a large fingerprint database is labor-intensive and computationally expensive. In this paper, we explore the use of deep learning for recognizing long-term temporal CSI data, wherein the site survey was completed weeks before the online testing phase. Compared to other positioning algorithms such as time-reversal resonating strength (TRRS), support vector machines (SVM), and Gaussian classifiers, our deep neural network (DNN) model shows a performance improvement of up to 10% for multi-position classification with centimeter-level resolution. We also exploit vector embeddings, such as i-vectors and d-vectors, which are traditionally employed in speech processing. With d-vectors as the compact representation of CSI, storage and processing requirements can be reduced without affecting performance, facilitating deployments on resource-constrained devices in IoT networks. By injecting i-vectors into a hidden layer, the DNN model originally for multi-position localization can be transformed to location-specific DNN to detect whether the device is static or has moved, resulting in a performance boost from 75.47% to 80.62%. This model adaptation requires a smaller number of recently collected fingerprints as opposed to a full database.

MCoGCN-motif high-order feature-guided embedding learning framework for social link prediction

Traditional social link prediction models primarily concentrate on the adjacency features of the network, overlooking the rich high-order structural information within. Therefore, the study of effective extraction and encoding of these high-order features, and their integration into prediction models, holds significant theoretical and practical value. To address this challenge, we propose a novel embedding learning framework guided by motif high-order features for social link prediction tasks. Firstly, we utilize the motif adjacency matrix to capture complex patterns in social networks. Through a propagation process, node embeddings can carry the structural information of the network. Subsequently, we design a simplified attention mechanism, allowing embeddings carrying motif high-order features to guide the representation of embeddings based on adjacency features. We then employ a feed-forward neural network to optimize node embeddings. Specifically, this framework addresses the issue of weakly correlated nodes in the network, which struggle to learn effective embeddings due to a lack of direct information. By guiding with high-order motif features, the framework enhances the similarity and predictive power of these node embeddings. Finally, we conducted a detailed evaluation of the predictive performance of our model on four social networks. The experimental results indicate that our model exhibits high accuracy and advantages in predicting social links.

Ion channel classification through machine learning and protein language model embeddings.

Ion channels are critical membrane proteins that regulate ion flux across cellular membranes, influencing numerous biological functions. The resource-intensive nature of traditional wet lab experiments for ion channel identification has led to an increasing emphasis on computational techniques. This study extends our previous work on protein language models for ion channel prediction, significantly advancing the methodology and performance. We employ a comprehensive array of machine learning algorithms, including k-Nearest Neighbors, Random Forest, Support Vector Machines, and Feed-Forward Neural Networks, alongside a novel Convolutional Neural Network (CNN) approach. These methods leverage fine-tuned embeddings from ProtBERT, ProtBERT-BFD, and MembraneBERT to differentiate ion channels from non-ion channels. Our empirical findings demonstrate that TooT-BERT-CNN-C, which combines features from ProtBERT-BFD and a CNN, substantially surpasses existing benchmarks. On our original dataset, it achieves a Matthews Correlation Coefficient (MCC) of 0.8584 and an accuracy of 98.35 %. More impressively, on a newly curated, larger dataset (DS-Cv2), it attains an MCC of 0.9492 and an ROC AUC of 0.9968 on the independent test set. These results not only highlight the power of integrating protein language models with deep learning for ion channel classification but also underscore the importance of using up-to-date, comprehensive datasets in bioinformatics tasks. Our approach represents a significant advancement in computational methods for ion channel identification, with potential implications for accelerating research in ion channel biology and aiding drug discovery efforts.

Deepfake Detection on Social Media: Leveraging Deep Learning and Fast Text Embeddings for Identifying Machine-Generated Tweets

Recent improvements in natural language generation offer a new means to influence public opinion on social media. Moreover, developments in language modelling have considerably augmented the generative capacities of deep neural models, equipping them with improved competencies for content creation. As a result, text-generative models have gained significant potency, enabling adversaries to leverage these capabilities to enhance social bots, facilitating the creation of authentic deepfake postings and swaying public conversation. In order to address this issue, trustworthy and efficient techniques for detecting social media deepfakes are essential. Automated text on Twitter has been identified in recent studies. To determine if tweets are generated by humans or bots, this study employs the publicly available Tweepfake dataset and a basic deep learning model that makes use of word embeddings. Using a CNN architecture, FastText word embeddings can detect deepfake tweets. This study utilised many machine learning models as baseline approaches to demonstrate the higher performance of the proposed approach. Some of the fundamental techniques used here were Term Frequency, Fast Text subword embeddings, FastText, and Term Frequency-Inverse Document Frequency. Additional comparisons with other DL models, such as CNN-LSTM and LSTM, demonstrate the effectiveness and benefits of the suggested approach in precisely resolving the issue. The experimental results demonstrate that Twitter data may be efficiently and accurately classified with a 93% accuracy rate using CNN architecture and FastText embeddings.

Generative 3D Cardiac Shape Modelling for in-silico Trials.

We propose a deep learning method to model and generate synthetic aortic shapes based on representing shapes as the zero-level set of a neural signed distance field, conditioned by a family of trainable embedding vectors with encode the geometric features of each shape. The network is trained on a dataset of aortic root meshes reconstructed from CT images by making the neural field vanish on sampled surface points and enforcing its spatial gradient to have unit norm. Empirical results show that our model can represent aortic shapes with high fidelity. Moreover, by sampling from the learned embedding vectors, we can generate novel shapes that resemble real patient anatomies, which can be used for in-silico trials.

Topological organization for hybrid rice growth stages Phenotype based on Contrastive clustering

Hybrid-rice seed production technology is a significant contributor to the high yield. The way to improve seed production’s yield and quality is purification. With the functional requirements of the intelligent Undesired-rice removal process, the environmental elements of field operation and the morphological parameters of Desired-rice and Undesired-rice are collected and measured. This study presents a comprehensive analysis of the topological organization of hybrid rice growth stages using a deep clustering algorithm, t-SimCLR. The algorithm was rigorously tested and compared with existing methods, demonstrating superior performance in clustering accuracy and the ability to capture semantic relationships within the dataset. It elucidated expert prior knowledge of abnormal mature varieties for hybrid rice and explored the growth stages of hybrid rice and panicles. Expert knowledge on identifying abnormal mature varieties within hybrid rice was incorporated to ensure accuracy in variety purification. The study utilized three different datasets to reveal underlying structures and patterns, providing insights into the growth dynamics of hybrid rice. The t-SimCLR algorithm, which employs Contrastive Learning and neighbor embeddings, outperformed traditional methods such as UMAP, t-SNE, and SimCLR, particularly in maintaining the local structure of data and offering more meaningful two-dimensional embeddings. The parametric mapping is trained from the high-dimensional pixel space into two dimensions and achieves classification accuracy by the t-SimCLR algorithm. This study contributes to the advancement of hybrid rice purification techniques and sets the stage for developing more sophisticated intelligent systems for agricultural applications.