Embedding-based Method Research Articles

Embedding-based entity alignment between multi-source temporal knowledge graphs

The goal of entity alignment is to identify entities in two multi-source knowledge graphs (KGs) that represent the same real-world object. Recent researches on multi-source entity alignment mainly concentrate on static KGs. In fact, temporal KGs have become valuable resources for numerous artificial intelligence applications, and entity alignment between multi-source temporal KGs is becoming more and more important. Current entity alignment models cannot support temporal tasks and fail to deal with the attributes with low literal similarity that share the same semantics through attribute embedding. Therefore, we propose a RDF (Resource Description Framework)-based model for representing temporal KGs, and an embedding-based entity alignment method for multi-source temporal KGs. This method computes for the similarity of temporal information and generates aligned attribute pairs in the predicate alignment module. We design an interactive module to make matched attributes and the matched entities help to find each other based on aligned attribute pairs. This module can calculate the similarity of attributes with low literal similarity. After getting the structure similarity of the structure embedding module, the final entity alignment result of temporal KGs is produced by the calculation of a binary linear regression function. Experimental results demonstrate that our proposed model outperforms existing approaches significantly.

Test

Semantic search is the process of retrieving relevant information from a large corpus of texts based on the meaning and context of the query. This paper is introduced in order to explore the use of large language models for semantic search of Quranic texts. The Quran, which is the central religious text of Islam, contains rich and complex linguistic and semantic features that pose challenges for traditional keyword-based search methods. This study investigates a semantic search approach utilizing. Large Language Models (LLM) embedding and assess the performance of LLM embedding in comparison to a baseline embedding-based search method using a set of queries that represent different semantic search levels. In addition, this study will also discuss the limitations and implications of using large language models for semantic search of Quranic texts and suggest directions for future research. A significant finding in this study is the consistent effectiveness of the LLM embedding across varying semantic complexities. This suggests that embedding using LLMs can capture deep semantic connections effectively. On the other hand, as a second finding, the state-of-the-art transformer, AraT5, outperforms LLM embeddings in low-level semantic searches, indicating potential for further LLM fine-tuning on Arabic text corpora

Embedding Search for Quranic Texts based on Large Language Models

Semantic search is the process of retrieving relevant information from a large corpus of texts based on the meaning and context of the query. This paper is introduced in order to explore the use of large language models for semantic search of Quranic texts. The Quran, which is the central religious text of Islam, contains rich and complex linguistic and semantic features that pose challenges for traditional keyword-based search methods. This study investigates a semantic search approach utilizing. Large Language Models (LLM) embedding and assess the performance of LLM embedding in comparison to a baseline embedding-based search method using a set of queries that represent different semantic search levels. In addition, this study will also discuss the limitations and implications of using large language models for semantic search of Quranic texts and suggest directions for future research. A significant finding in this study is the consistent effectiveness of the LLM embedding across varying semantic complexities. This suggests that embedding using LLMs can capture deep semantic connections effectively. On the other hand, as a second finding, the state-of-the-art transformer, AraT5, outperforms LLM embeddings in low-level semantic searches, indicating potential for further LLM fine-tuning on Arabic text corpora.

SimCAS: an embedding-based method for simulating single-cell chromatin accessibility sequencing data.

Single-cell chromatin accessibility sequencing (scCAS) technology provides an epigenomic perspective to characterize gene regulatory mechanisms at single-cell resolution. With an increasing number of computational methods proposed for analyzing scCAS data, a powerful simulation framework is desirable for evaluation and validation of these methods. However, existing simulators generate synthetic data by sampling reads from real data or mimicking existing cell states, which is inadequate to provide credible ground-truth labels for method evaluation. We present simCAS, an embedding-based simulator, for generating high-fidelity scCAS data from both cell- and peak-wise embeddings. We demonstrate simCAS outperforms existing simulators in resembling real data and show that simCAS can generate cells of different states with user-defined cell populations and differentiation trajectories. Additionally, simCAS can simulate data from different batches and encode user-specified interactions of chromatin regions in the synthetic data, which provides ground-truth labels more than cell states. We systematically demonstrate that simCAS facilitates the benchmarking of four core tasks in downstream analysis: cell clustering, trajectory inference, data integration, and cis-regulatory interaction inference. We anticipate simCAS will be a reliable and flexible simulator for evaluating the ongoing computational methods applied on scCAS data. simCAS is freely available at https://github.com/Chen-Li-17/simCAS.

Zero-Shot Learning with Joint Generative Adversarial Networks

Zero-shot learning (ZSL) is implemented by transferring knowledge from seen classes to unseen classes through embedding space or feature generation. However, the embedding-based method has a hubness problem, and the generation-based method may contain considerable bias. To solve these problems, a joint model with multiple generative adversarial networks (JG-ZSL) is proposed in this paper. Firstly, we combined the generation-based model and the embedding-based model to build a hybrid ZSL framework by mapping the real samples and the synthetic samples into the embedding space for classification, which alleviates the problem of data imbalance effectively. Secondly, based on the original generation-method model, a coupled GAN is introduced to generate semantic embeddings, which can generate semantic vectors for unseen classes in embedded space to alleviate the bias of mapping results. Finally, semantic-relevant self-adaptive margin center loss was used, which can explicitly encourage intra-class compactness and inter-class separability, and it can also guide coupled GAN to generate discriminative and representative semantic features. All the experiments on the four standard datasets (CUB, AWA1, AWA2, SUN) show that the proposed method is effective.

Generalized Zero-Shot Activity Recognition with Embedding-Based Method

Sensor-based human activity recognition aims to recognize the activities performed by people with the sensor readings. Most of existing works in this area rely on supervised classification algorithms, and can only recognize activities covered by the training data. Whereas, in many practical applications, while performing activity recognition, not only the activities covered by the training data, but also some previously unseen activities need to be recognized. In this paper, we study the problem of generalized zero-shot activity recognition. In this problem, the activities that need to be recognized contain both the activities covered by the training data and the previously unseen activities. We firstly give a formulation of this problem, and then propose an embedding-based method to address it. In this method, an embedding-compatibility model is learned. When performing activity recognition, the learned model and the calibrated stacking mechanism are employed. Extensive experiments on publicly available datasets demonstrate the effectiveness of our method.

Embedding-based Method for the Supervised Link Prediction in Social Networks

In recent years, social network analysis has received a lot of interest. Link prediction is an important area of research in this field that uses information from current networks to predict the likely links that will emerge in the future. It has attracted considerable attention from interdisciplinary research communities due to its ubiquitous applications in biological networks, computer science, transportation networks, bioinformatics, telecommunication networks, and so on. Currently, supervised machine learning is one of the critical techniques in the link prediction task. Several algorithms have been developed by many authors to predict the future link in the network, but there is still scope to improve the previous approaches. In the supervised link prediction process, feature selection is a crucial step. Most existing algorithms use one type of similarity-based feature to represent data, which is not well described due to the large scale and heterogeneity of social networks. One of the newest techniques for link prediction is embedding methods, which are used to preparing the feature vector for each the nonexisting links in the network. In this paper, we introduce a novel approach to supervised link prediction based on feature embedding methods in order to achieve better performance. Our contribution considers a set of embedding methods as the feature vector for training the machine learning classifiers. The main focus of this work is to investigate the relevance of different feature embedding methods to improve the performance of the supervised link prediction models. The experimental results on some real-world temporal networks revealed satisfactory results, which encourage us for further analysis. Moreover, the use of feature embedding methods will provide better performance in this regard.

Interlayer Link Prediction in Multiplex Social Networks Based on Multiple Types of Consistency Between Embedding Vectors.

Online users are typically active on multiple social media networks (SMNs), which constitute a multiplex social network. With improvements in cybersecurity awareness, users increasingly choose different usernames and provide different profiles on different SMNs. Thus, it is becoming increasingly challenging to determine whether given accounts on different SMNs belong to the same user; this can be expressed as an interlayer link prediction problem in a multiplex network. To address the challenge of predicting interlayer links, feature or structure information is leveraged. Existing methods that use network embedding techniques to address this problem focus on learning a mapping function to unify all nodes into a common latent representation space for prediction; positional relationships between unmatched nodes and their common matched neighbors (CMNs) are not utilized. Furthermore, the layers are often modeled as unweighted graphs, ignoring the strengths of the relationships between nodes. To address these limitations, we propose a framework based on multiple types of consistency between embedding vectors (MulCEVs). In MulCEV, the traditional embedding-based method is applied to obtain the degree of consistency between the vectors representing the unmatched nodes, and a proposed distance consistency index based on the positions of nodes in each latent space provides additional clues for prediction. By associating these two types of consistency, the effective information in the latent spaces is fully utilized. In addition, MulCEV models the layers as weighted graphs to obtain representation. In this way, the higher the strength of the relationship between nodes, the more similar their embedding vectors in the latent representation space will be. The results of our experiments on several real-world and synthetic datasets demonstrate that the proposed MulCEV framework markedly outperforms current embedding-based methods, especially when the number of training iterations is small.

ELMo4m6A: A Contextual Language Embedding-Based Predictor for Detecting RNA N6-Methyladenosine Sites.

N6-methyladenosine (m6A) is a universal post-transcriptional modification of RNAs, and it is widely involved in various biological processes. Identifying m6A modification sites accurately is indispensable to further investigate m6A-mediated biological functions. How to better represent RNA sequences is crucial for building effective computational methods for detecting m6A modification sites. However, traditional encoding methods require complex biological prior knowledge and are time-consuming. Furthermore, most of the existing m6A sites prediction methods are limited to single species, and few methods are able to predict m6A sites across different species and tissues. Thus, it is necessary to design a more efficient computational method to predict m6A sites across multiple species and tissues. In this paper, we proposed ELMo4m6A, a contextual language embedding-based method for predicting m6A sites from RNA sequences without any prior knowledge. ELMo4m6A first learns embeddings of RNA sequences using a language model ELMo, then uses a hybrid convolutional neural network (CNN) and long short-term memory (LSTM) to identify m6A sites. The results of 5-fold cross-validation and independent testing demonstrate that ELMo4m6A is superior to state-of-the-art methods. Moreover, we applied integrated gradients to find potential sequence patterns contributing to m6A sites.

Finding reinforced structural hole spanners in social networks via node embedding

Identifying structural hole spanners that benefit from acting as bridges between communities is a core study in social network analysis. Existing methods for identification mainly focus on measuring the ability of users to control information propagation by bridging holes, while ignoring the impact of reinforcement of the holes themselves on the benefits of bridging spanners. A recent sociological study shows that the more reinforced a hole is, the more likely it is to bring high benefits to its spanners. In this paper, we propose a node embedding-based method ReHSe for identifying reinforced structural hole spanners in social networks. Specifically, an integrated embedding method is devised to extract features encoding reinforcement properties of nodes into a low-dimensional space. Further, to improve the robustness and accuracy of identification, an incremental learning strategy based on a reserved set is employed to train a scoring network in this subspace, to find top-k reinforced hole spanners. Extensive experimental results show that the performance of hole spanners identified by the proposed method outperforms several existing methods.

MultiScale-CNN-4mCPred: a multi-scale CNN and adaptive embedding-based method for mouse genome DNA N4-methylcytosine prediction

N4-methylcytosine (4mC) is an important epigenetic mechanism, which regulates many cellular processes such as cell differentiation and gene expression. The knowledge about the 4mC sites is a key foundation to exploring its roles. Due to the limitation of techniques, precise detection of 4mC is still a challenging task. In this paper, we presented a multi-scale convolution neural network (CNN) and adaptive embedding-based computational method for predicting 4mC sites in mouse genome, which was referred to as MultiScale-CNN-4mCPred. The MultiScale-CNN-4mCPred used adaptive embedding to encode nucleotides, and then utilized multi-scale CNNs as well as long short-term memory to extract more in-depth local properties and contextual semantics in the sequences. The MultiScale-CNN-4mCPred is an end-to-end learning method, which requires no sophisticated feature design. The MultiScale-CNN-4mCPred reached an accuracy of 81.66% in the 10-fold cross-validation, and an accuracy of 84.69% in the independent test, outperforming state-of-the-art methods. We implemented the proposed method into a user-friendly web application which is freely available at: http://www.biolscience.cn/MultiScale-CNN-4mCPred/.

FLONE: fully Lorentz network embedding for inferring novel drug targets.

To predict drug targets, graph-based machine-learning methods have been widely used to capture the relationships between drug, target and disease entities in drug-disease-target (DDT) networks. However, many methods cannot explicitly consider disease types at inference time and so will predict the same target for a given drug under any disease condition. Meanwhile, DDT networks are usually organized hierarchically carrying interactive relationships between involved entities, but these methods, especially those based on Euclidean embedding cannot fully utilize such topological information, which might lead to sub-optimal results. We hypothesized that, by importing hyperbolic embedding specifically for modeling hierarchical DDT networks, graph-based algorithms could better capture relationships between aforementioned entities, which ultimately improves target prediction performance. We formulated the target prediction problem as a knowledge graph completion task explicitly considering disease types. We proposed FLONE, a hyperbolic embedding-based method based on capturing hierarchical topological information in DDT networks. The experimental results on two DDT networks showed that by introducing hyperbolic space, FLONE generates more accurate target predictions than its Euclidean counterparts, which supports our hypothesis. We also devised hyperbolic encoders to fuse external domain knowledge, to make FLONE enable handling samples corresponding to previously unseen drugs and targets for more practical scenarios. Source code and dataset information are at: https://github.com/arantir123/DDT_triple_prediction. Supplementary data are available at Bioinformatics Advances online.

Examine the Effectiveness of Patent Embedding-Based Company Comparison Method

A company’s benchmarking strategy is significantly determined by how it measures technological similarity. Researchers have measured the technological similarity between companies using a vector composed of the classification codes of patents that each company owns. However, patent classification code-based company comparison methods do not consider the text in patents and thus may not find similar companies accurately. To solve this problem, this study suggests a patent embedding-based company comparison method. The suggested method uses a text embedding model to vectorize the text in patents and calculates technological similarity based on the embedding vector. We examine the effectiveness of the suggested method by comparing it with the conventional patent classification code-based method. From the validation results for 11,227 Korean companies listed in the Korea Data Analysis, Retrieval, and Transfer system (DART), we find that the suggested method effectively retrieves technologically similar companies.

Multi-Task Learning Model with Data Augmentation for Arabic Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) is a fine-grained process. Its fundamental subtasks are aspect term extraction (ATE) and aspect polarity classification (APC), and these subtasks are dependent and closely related. However, most existing works on Arabic ABSA content separately address them, assume that aspect terms are preidentified, or use a pipeline model. Pipeline solutions design different models for each task, and the output from the ATE model is used as the input to the APC model, which may result in error propagation among different steps because APC is affected by ATE error. These methods are impractical for real-world scenarios where the ATE task is the base task for APC, and its result impacts the accuracy of APC. Thus, in this study, we focused on a multi-task learning model for Arabic ATE and APC in which the model is jointly trained on two subtasks simultaneously in a single model. This paper integrates the multi-task model, namely Local Cotext Foucse-Aspect Term Extraction and Polarity classification (LCF-ATEPC) and Arabic Bidirectional Encoder Representation from Transformers (AraBERT) as a shred layer for Arabic contextual text representation. The LCF-ATEPC model is based on a multi-head self-attention and local context focus mechanism (LCF) to capture the interactive information between an aspect and its context. Moreover, data augmentation techniques are proposed based on state-of-the-art augmentation techniques (word embedding substitution with constraints and contextual embedding (AraBERT)) to increase the diversity of the training dataset. This paper examined the effect of data augmentation on the multi-task model for Arabic ABSA. Extensive experiments were conducted on the original and combined datasets (merging the original and augmented datasets). Experimental results demonstrate that the proposed Multi-task model outperformed existing APC techniques. Superior results were obtained by AraBERT and LCF-ATEPC with fusion layer (AR-LCF-ATEPC-Fusion) and the proposed data augmentation word embedding-based method (FastText) on the combined dataset.

DFinder: A novel end-to-end graph embedding-based method to identify drug-food interactions.

Drug-food interactions (DFIs) occur when some constituents of food affect the bioaccessibility or efficacy of the drug by involving in drug pharmacodynamic (PD) and/or pharmacokinetic (PK) processes. Many computational methods have achieved remarkable results in link prediction tasks between biological entities which show the potential of computational methods in discovering novel DFIs. However, there are few computational approaches that pay attention to DFI identification. This is mainly due to the lack of drug-food interaction data. In addition, food is generally made up of a variety of chemical substances. The complexity of food makes it difficult to generate accurate feature representations for food. Therefore, it is urgent to develop effective computational approaches for learning the food feature representation and predicting drug-food interactions. In this paper, we first collect drug-food interaction data from DrugBank and PubMed respectively to construct two datasets, named DrugBank-DFI and PubMed-DFI. Based on these two datasets, two DFI networks are constructed. Then, we propose a novel end-to-end graph embedding-based method named DFinder to identify DFIs. DFinder combines node attribute features and topological structure features to learn the representations of drugs and food constituents. In topology space, we adopt a simplified graph convolution network-based method to learn the topological structure features. In feature space, we use a deep neural network to extract attribute features from the original node attributes. The evaluation results indicate that DFinder performs better than other baseline methods. The source code is available at https://github.com/23AIBox/23AIBox-DFinder. Supplementary data are available at Bioinformatics online.

MedGraph: A semantic biomedical information retrieval framework using knowledge graph embedding for PubMed.

Here we study the semantic search and retrieval problem in biomedical digital libraries. First, we introduce MedGraph, a knowledge graph embedding-based method that provides semantic relevance retrieval and ranking for the biomedical literature indexed in PubMed. Second, we evaluate our approach using PubMed's Best Match algorithm. Moreover, we compare our method MedGraph to a traditional TF-IDF-based algorithm. Third, we use a dataset extracted from PubMed, including 30 million articles' metadata such as abstracts, author information, citation information, and extracted biological entity mentions. We pull a subset of the dataset to evaluate MedGraph using predefined queries with ground truth ranked results. To our knowledge, this technique has not been explored before in biomedical information retrieval. In addition, our results provide some evidence that semantic approaches to search and relevance in biomedical digital libraries that rely on knowledge graph modeling offer better search relevance results when compared with traditional methods in terms of objective metrics.

EmbedSeg: Embedding-based Instance Segmentation for Biomedical Microscopy Data.

Automatic detection and segmentation of biological objects in 2D and 3D image data is central for countless biomedical research questions to be answered. While many existing computational methods are used to reduce manual labeling time, there is still a huge demand for further quality improvements of automated solutions. In the natural image domain, spatial embedding-based instance segmentation methods are known to yield high-quality results, but their utility to biomedical data is largely unexplored. Here we introduce EmbedSeg, an embedding-based instance segmentation method designed to segment instances of desired objects visible in 2D or 3D biomedical image data. We apply our method to four 2D and seven 3D benchmark datasets, showing that we either match or outperform existing state-of-the-art methods. While the 2D datasets and three of the 3D datasets are well known, we have created the required training data for four new 3D datasets, which we make publicly available online. Next to performance, also usability is important for a method to be useful. Hence, EmbedSeg is fully open source (https://github.com/juglab/EmbedSeg), offering (i)tutorial notebooks to train EmbedSeg models and use them to segment object instances in new data, and (ii)a napari plugin that can also be used for training and segmentation without requiring any programming experience. We believe that this renders EmbedSeg accessible to virtually everyone who requires high-quality instance segmentations in 2D or 3D biomedical image data.

HyperQuaternionE: A hyperbolic embedding model for qualitative spatial and temporal reasoning

Qualitative spatial/temporal reasoning (QSR/QTR) plays a key role in research on human cognition, e.g., as it relates to navigation, as well as in work on robotics and artificial intelligence. Although previous work has mainly focused on various spatial and temporal calculi, more recently representation learning techniques such as embedding have been applied to reasoning and inference tasks such as query answering and knowledge base completion. These subsymbolic and learnable representations are well suited for handling noise and efficiency problems that plagued prior work. However, applying embedding techniques to spatial and temporal reasoning has received little attention to date. In this paper, we explore two research questions: (1) How do embedding-based methods perform empirically compared to traditional reasoning methods on QSR/QTR problems? (2) If the embedding-based methods are better, what causes this superiority? In order to answer these questions, we first propose a hyperbolic embedding model, called HyperQuaternionE, to capture varying properties of relations (such as symmetry and anti-symmetry), to learn inversion relations and relation compositions (i.e., composition tables), and to model hierarchical structures over entities induced by transitive relations. We conduct various experiments on two synthetic datasets to demonstrate the advantages of our proposed embedding-based method against existing embedding models as well as traditional reasoners with respect to entity inference and relation inference. Additionally, our qualitative analysis reveals that our method is able to learn conceptual neighborhoods implicitly. We conclude that the success of our method is attributed to its ability to model composition tables and learn conceptual neighbors, which are among the core building blocks of QSR/QTR.

Knowledge Graph Entity Alignment Using Relation Structural Similarity

Embedding-based entity alignment, which represents knowledge graphs as low-dimensional embeddings and finds entities in different knowledge graphs that semantically represent the same real-world entity by measuring the similarities between entity embeddings, has achieved promising results. However, existing methods are still challenged by the error accumulation of embeddings along multi-step paths and the semantic information loss. This paper proposes a novel embedding-based entity alignment method that iteratively aligns both entities and relations with high similarities as training data. Newly-aligned entities and relations are used to calibrate the corresponding embeddings in the unified embedding space, which reduces the error accumulation. To reduce the negative impact of semantic information loss, the authors propose to use relation structural similarity instead of embedding similarity to align relations. Experimental results on five widely used real-world datasets show that the proposed method significantly outperforms several state-of-the-art methods for entity alignment.

BNEMDI: A Novel MicroRNA-Drug Interaction Prediction Model Based on Multi-Source Information With a Large-Scale Biological Network.

As a novel target in pharmacy, microRNA (miRNA) can regulate gene expression under specific disease conditions to produce specific proteins. To date, many researchers leveraged miRNA to reveal drug efficacy and pathogenesis at the molecular level. As we all know that conventional wet experiments suffer from many problems, including time-consuming, labor-intensity, and high cost. Thus, there is an urgent need to develop a novel computational model to facilitate the identification of miRNA–drug interactions (MDIs). In this work, we propose a novel bipartite network embedding-based method called BNEMDI to predict MDIs. First, the Bipartite Network Embedding (BiNE) algorithm is employed to learn the topological features from the network. Then, the inherent attributes of drugs and miRNAs are expressed as attribute features by MACCS fingerprints and k-mers. Finally, we feed these features into deep neural network (DNN) for training the prediction model. To validate the prediction ability of the BNEMDI model, we apply it to five different benchmark datasets under five-fold cross-validation, and the proposed model obtained excellent AUC values of 0.9568, 0.9420, 0.8489, 0.8774, and 0.9005 in ncDR, RNAInter, SM2miR1, SM2miR2, and SM2miR MDI datasets, respectively. To further verify the prediction performance of the BNEMDI model, we compare it with some existing powerful methods. We also compare the BiNE algorithm with several different network embedding methods. Furthermore, we carry out a case study on a common drug named 5-fluorouracil. Among the top 50 miRNAs predicted by the proposed model, there were 38 verified by the experimental literature. The comprehensive experiment results demonstrated that our method is effective and robust for predicting MDIs. In the future work, we hope that the BNEMDI model can be a reliable supplement method for the development of pharmacology and miRNA therapeutics.