Entity Embeddings Research Articles

An Expressway ETC Missing Data Restoration Model Considering Multi-Attribute Features

Electronic toll collection (ETC) data mining has become one of the hotspots in the research of intelligent expressway extension applications. Ensuring the integrity of ETC data stands as a critical measure in upholding data quality. ETC data are typical structured data, and although deep learning holds great potential in the ETC data restoration field, its applications in structured data are still in the early stages. To address these issues, we propose an expressway ETC missing transaction data restoration model considering multi-attribute features (MAF). Initially, we employ an entity embedding neural network (EENN) to automatically learn the representation of categorical features in multi-dimensional space, subsequently obtaining embedding vectors from networks that have been adequately trained. Then, we use long short-term memory (LSTM) neural networks to extract the changing patterns of vehicle speeds across several continuous sections. Ultimately, we merge the processed features with other features as input, using a three-layer multilayer perceptron (MLP) to complete the ETC data restoration. To validate the effectiveness of the proposed method, we conducted extensive tests using real ETC datasets and compared it with methods commonly used for structured data restoration. The experimental results demonstrate that the proposed method significantly outperforms others in restoration accuracy on two different datasets. Specifically, our sample data size reached around 400,000 entries. Compared to the currently best method, our method improved the restoration accuracy by 19.06% on non-holiday ETC datasets. The MAE and RMSE values reached optimal levels of 12.394 and 23.815, respectively. The fitting degree of the model to the dataset also reached its peak (R2 = 0.993). Meanwhile, the restoration stability of our method on holiday datasets increased by 5.82%. An ablation experiment showed that the EENN and LSTM modules contributed 7.60% and 9% to the restoration accuracy, as well as 4.68% and 7.29% to the restoration stability. This study indicates that the proposed method not only significantly improves the quality of ETC data but also meets the timeliness requirements of big data mining analysis.

Entity Embeddings in Remote Sensing: Application to Deformation Monitoring for Infrastructure

There is a critical need for a global monitoring capability for Tailings Storage Facilities (TSFs), to help protect the surrounding communities and the environment. Satellite Synthetic Aperture Radar Interferometry (InSAR) shows much promise towards this ambition. However, extracting meaningful information and interpreting the deformation patterns from InSAR data can be a challenging task. One approach to address this challenge is through the use of data science techniques. In this study, the representation of InSAR metadata as Entity Embeddings within a Deep Learning framework (EE-DL) is investigated for modelling the spatio-temporal deformation response. Entity embeddings are commonly used in natural-language-processing tasks. They represent discrete objects, such as words, as continuous, low-dimensional vectors that can be manipulated mathematically. We demonstrate that EE-DL can be used to predict anomalous patterns in the InSAR time series. To evaluate the performance of the EE-DL approach in SAR interferometry, we conducted experiments over a mining test site (Cadia, Australia), which has been subject to a TSF failure. This study demonstrated that EE-DL can detect and predict the fine spatial movement patterns that eventually resulted in the failure. We also compared the results with deformation predictions from common baseline models, the Random Forest model and Gaussian Process Regression (GPR). Both EE-DL and GPR greatly outperform Random Forest. While GPR is also able to predict displacement patterns with millimetric accuracy, it detects a significantly lower number of anomalies compared to EE-DL. Overall, our study showed that EE-DL is a promising approach for building early-warning systems for critical infrastructures that use InSAR to predict ground deformations.

Dynamic Bayesian Contrastive Predictive Coding Model for Personalized Product Search

In this article, we study the problem of dynamic personalized product search. Due to the data-sparsity problem in the real world, existing methods suffer from the challenge of data inefficiency. We address the challenge by proposing a Dynamic Bayesian Contrastive Predictive Coding model (DBCPC), which aims to capture the rich structured information behind search records to improve data efficiency. Our proposed DBCPC utilizes contrastive predictive learning to jointly learn dynamic embeddings with structure information of entities (i.e., users, products, and words). Specifically, our DBCPC employs structured prediction to tackle the intractability caused by non-linear output space and utilizes the time embedding technique to avoid designing different encoders each time in the Dynamic Bayesian models. In this way, our model jointly learns the underlying embeddings of entities (i.e., users, products, and words) via prediction tasks, which enables the embeddings to focus more on their general attributes and capture the general information during the preference evolution with time. For inferring the dynamic embeddings, we propose an inference algorithm combining the variational objective and the contrastive objectives. Experiments were conducted on an Amazon dataset and the experimental results show that our proposed DBCPC can learn the higher-quality embeddings and outperforms the state-of-the-art non-dynamic and dynamic models for product search.

Semi-Supervised Entity Alignment With Global Alignment and Local Information Aggregation

Entity alignment is a vital task in knowledge fusion, which aims to align entities from different knowledge graphs and merge them into one single graph. Existing entity alignment models focus on local features and try to minimize the distance between pairs of pre-aligned entities. Despite their success, these models heavily rely on the number of existing pre-aligned entity pairs and the topology information from the rest large set of unaligned entities is still largely unexplored. To overcome the limitation of existing models, we propose a model, termed Global Alignment and Local Information Aggregation, or GALA. GALA constructs global features for the knowledge graphs to be aligned using entity embeddings. It aligns the entities in the graphs by forcing their global features to match with each other and progressively updating the entity embeddings by aggregating local information from the other network. Empirical studies on commonly-used KG alignment data sets confirm the effectiveness of the proposed model.

MSEN: A Multi-Scale Evolutionary Network for Modeling the Evolution of Temporal Knowledge Graphs

Temporal knowledge graphs play an increasingly prominent role in scenarios such as social networks, finance, and smart cities. As such, research on temporal knowledge graphs continues to deepen. In particular, research on temporal knowledge graph reasoning holds great significance, as it can provide abundant knowledge for downstream tasks such as question answering and recommendation systems. Current reasoning research focuses primarily on interpolation and extrapolation. Extrapolation research aims to predict the likelihood of events occurring in future timestamps. Historical events are crucial for predicting future events. However, existing models struggle to fully capture the evolutionary characteristics of historical knowledge graphs. This paper proposes a multi-scale evolutionary network (MSEN) model that leverages Hierarchical Transfer aware Graph Neural Network (HT-GNN) in a local memory encoder to aggregate rich structural semantics from each timestamp’s knowledge graph. It also utilizes Time Related Graph Neural Network (TR-GNN) in a global memory encoder to model temporal-semantic dependencies of entities across the global knowledge graph, mining global evolutionary patterns. The model integrates information from both encoders to generate entity embeddings for predicting future events. The proposed MSEN model demonstrates strong performance compared to several baselines on typical benchmark datasets. Results show MSEN achieves the highest prediction accuracy.

Constructing negative samples via entity prediction for multi-task knowledge representation learning

Knowledge representation learning (KRL) aims at embedding the entities and relations in knowledge graphs (KGs) into vectors by learning. In recent years, several multi-task learning (MTL) frameworks were proposed to KRL and achieved promising results. Nevertheless, most of these models only share the initial embeddings of entities and relations among sub-tasks. We construct hard negative samples for relation prediction (RP) task and triple classification (TC) task by using the similarity matrix from the entity prediction (EP) task. We also share the features in hidden layers of EP task to TC. The proposed framework is applicable to CNN based models for KRL. Experimental results on WN18RR and FB15k-237 datasets show that the proposed framework promotes the performance of EP, RP and TC tasks evidently comparing with the single task models and obtains better results in RP and TC than the state-of-the-art MTL-based models/frameworks. Code of NS-KRL is available online.11https://github.com/fofilix/NS_KRL.

Incorporating entity-level knowledge in pretrained language model for biomedical dense retrieval

In recent years, pre-trained language models (PLMs) have dominated natural language processing (NLP) and achieved outstanding performance in various NLP tasks, including dense retrieval based on PLMs. However, in the biomedical domain, the effectiveness of dense retrieval models based on PLMs still needs to be improved due to the diversity and ambiguity of entity expressions caused by the enrichment of biomedical entities. To alleviate the semantic gap, in this paper, we propose a method that incorporates external knowledge at the entity level into a dense retrieval model to enrich the dense representations of queries and documents. Specifically, we first add additional self-attention and information interaction modules in the Transformer layer of the BERT architecture to perform fusion and interaction between query/document text and entity embeddings from knowledge graphs. We then propose an entity similarity loss to constrain the model to better learn external knowledge from entity embeddings, and further propose a weighted entity concatenation mechanism to balance the impact of entity representations when matching queries and documents. Experiments on two publicly available biomedical retrieval datasets show that our proposed method outperforms state-of-the-art dense retrieval methods. In term of NDCG metrics, the proposed method (called ELK) improves the ranking performance of coCondenser by at least 5% on both two datasets, and also obtains further performance gain over state-of-the-art EVA methods. Though having a more sophisticated architecture, the average query latency of ELK is still within the same order of magnitude as that of other efficient methods.

Recommendation Method of Power Knowledge Retrieval Based on Graph Neural Network

With the development of the digital and intelligent transformation of the power grid, the structure and operation and maintenance technology of the power grid are constantly updated, which leads to problems such as difficulties in information acquisition and screening. Therefore, we propose a recommendation method for power knowledge retrieval based on a graph neural network (RPKR-GNN). The method first uses a graph neural network to learn the network structure information of the power fault knowledge graph and realize the deep semantic embedding of power entities and relations. After this, it fuses the power knowledge graph paths to mine the potential power entity relationships and completes the power fault knowledge graph through knowledge inference. At the same time, we combine the user retrieval behavior features for knowledge aggregation to form a personal subgraph, and we analyze the user retrieval subgraph by matching the similarity of retrieval keyword features. Finally, we form a fusion subgraph based on the subgraph topology and reorder the entities of the subgraph to generate a recommendation list for the target users for the prediction of user retrieval intention. Through experimental comparison with various classical models, the results show that the models have a certain generalization ability in knowledge inference. The method performs well in terms of the MR and Hit@10 indexes on each dataset, and the F1 value can reach 87.3 in the retrieval recommendation effect, which effectively enhances the automated operation and maintenance capability of the power system.

HMCDA: a novel method based on the heterogeneous graph neural network and metapath for circRNA-disease associations prediction

Circular RNA (CircRNA) is a type of non-coding RNAs in which both ends are covalently linked. Researchers have demonstrated that many circRNAs can act as biomarkers of diseases. However, traditional experimental methods for circRNA-disease associations identification are labor-intensive. In this work, we propose a novel method based on the heterogeneous graph neural network and metapaths for circRNA-disease associations prediction termed as HMCDA. First, a heterogeneous graph consisting of circRNA-disease associations, circRNA-miRNA associations, miRNA-disease associations and disease-disease associations are constructed. Then, six metapaths are defined and generated according to the biomedical pathways. Afterwards, the entity content transformation, intra-metapath and inter-metapath aggregation are implemented to learn the embeddings of circRNA and disease entities. Finally, the learned embeddings are used to predict novel circRNA-disase associations. In particular, the result of extensive experiments demonstrates that HMCDA outperforms four state-of-the-art models in fivefold cross validation. In addition, our case study indicates that HMCDA has the ability to identify novel circRNA-disease associations.

Contrastive Multi-Modal Knowledge Graph Representation Learning

Representation learning of knowledge graphs (KGs) aims to embed both entities and relations as vectors in a continuous low-dimensional space, which has facilitated various applications such as link prediction and entity retrieval. Most existing KG embedding methods focus on modeling the structured fact triples independently and ignore the multi-type relations among triples as well as the variety of data types (e.g., texts and images) associated with entities in KGs, and thus fail to capture the complex and multi-modal information that is inherently inside the entity-relation triples. In this paper, we propose a novel approach for knowledge graph embedding named Contrastive Multi-modal Graph Neural Network (CMGNN), which can encapsulate comprehensive features from multi-modal content descriptions of entities and high-order connectivity structures. Specifically, CMGNN first learns entity embeddings from multi-modal content and then contrasts encodings from multi-relational local neighbors and high-order connectivities to obtain latent representations of entities and relations simultaneously. Experimental results demonstrate that CMGNN can effectively model the multi-modalities and multi-type structures in KGs, and significantly outperforms existing state-of-the-art methods on benchmark datasets for the tasks of link prediction and entity classification.

Infer the missing facts of D3FEND using knowledge graph representation learning

PurposeMITRE and the National Security Agency cooperatively developed and maintained a D3FEND knowledge graph (KG). It provides concepts as an entity from the cybersecurity countermeasure domain, such as dynamic, emulated and file analysis. Those entities are linked by applying relationships such as analyze, may_contains and encrypt. A fundamental challenge for collaborative designers is to encode knowledge and efficiently interrelate the cyber-domain facts generated daily. However, the designers manually update the graph contents with new or missing facts to enrich the knowledge. This paper aims to propose an automated approach to predict the missing facts using the link prediction task, leveraging embedding as representation learning.Design/methodology/approachD3FEND is available in the resource description framework (RDF) format. In the preprocessing step, the facts in RDF format converted to subject–predicate–object triplet format contain 5,967 entities and 98 relationship types. Progressive distance-based, bilinear and convolutional embedding models are applied to learn the embeddings of entities and relations. This study presents a link prediction task to infer missing facts using learned embeddings.FindingsExperimental results show that the translational model performs well on high-rank results, whereas the bilinear model is superior in capturing the latent semantics of complex relationship types. However, the convolutional model outperforms 44% of the true facts and achieves a 3% improvement in results compared to other models.Research limitations/implicationsDespite the success of embedding models to enrich D3FEND using link prediction under the supervised learning setup, it has some limitations, such as not capturing diversity and hierarchies of relations. The average node degree of D3FEND KG is 16.85, with 12% of entities having a node degree less than 2, especially there are many entities or relations with few or no observed links. This results in sparsity and data imbalance, which affect the model performance even after increasing the embedding vector size. Moreover, KG embedding models consider existing entities and relations and may not incorporate external or contextual information such as textual descriptions, temporal dynamics or domain knowledge, which can enhance the link prediction performance.Practical implicationsLink prediction in the D3FEND KG can benefit cybersecurity countermeasure strategies in several ways, such as it can help to identify gaps or weaknesses in the existing defensive methods and suggest possible ways to improve or augment them; it can help to compare and contrast different defensive methods and understand their trade-offs and synergies; it can help to discover novel or emerging defensive methods by inferring new relations from existing data or external sources; and it can help to generate recommendations or guidance for selecting or deploying appropriate defensive methods based on the characteristics and objectives of the system or network.Originality/valueThe representation learning approach helps to reduce incompleteness using a link prediction that infers possible missing facts by using the existing entities and relations of D3FEND.

A Domain-Oriented Entity Alignment Approach Based on Filtering Multi-Type Graph Neural Networks

Owing to the heterogeneity and incomplete information present in various domain knowledge graphs, the alignment of distinct source entities that represent an identical real-world entity becomes imperative. Existing methods focus on cross-lingual knowledge graph alignment, and assume that the entities of knowledge graphs in the same language are unique. However, due to the ambiguity of language, heterogeneous knowledge graphs in the same language are often duplicated, and relationship triples are far less than those of cross-lingual knowledge graphs. Moreover, existing methods rarely exclude noisy entities in the process of alignment. These make it impossible for existing methods to deal effectively with the entity alignment of domain knowledge graphs. In order to address these issues, we propose a novel entity alignment approach based on domain-oriented embedded representation (DomainEA). Firstly, a filtering mechanism employs the language model to extract the semantic features of entities and to exclude noisy entities for each entity. Secondly, a Structural Aggregator (SA) incorporates multiple hidden layers to generate high-order neighborhood-aware embeddings of entities that have few relationship connections. An Attribute Aggregator (AA) introduces self-attention to dynamically calculate weights that represent the importance of the attribute values of the entities. Finally, the approach calculates a transformation matrix to map the embeddings of distinct domain knowledge graphs onto a unified space, and matches entities via the joint embeddings of the SA and AA. Compared to six state-of-the-art methods, our experimental results on multiple food datasets show the following: (i) Our approach achieves an average improvement of 6.9% on MRR. (ii) The size of the dataset has a subtle influence on our approach; there is a positive correlation between the expansion of the dataset size and an improvement in most of the metrics. (iii) We can achieve a significant improvement in the level of recall by employing a filtering mechanism that is limited to the top-100 nearest entities as the candidate pairs.

Meta-Learning Based Dynamic Adaptive Relation Learning for Few-Shot Knowledge Graph Completion

As artificial intelligence gradually steps into cognitive intelligence stage, knowledge graphs (KGs) play an increasingly important role in many natural language processing tasks. Due to the prevalence of long-tail relations in KGs, few-shot knowledge graph completion (KGC) for link prediction of long-tail relations has gradually become a hot research topic. Current few-shot KGC methods mainly focus on the static representation of surrounding entities to explore the potential semantic features of entities, while ignoring the dynamic properties among entities and the special influence of the long-tail relation on link prediction. In this paper, a new meta-learning based dynamic adaptive relation learning model (DARL) is proposed for few-shot KGC. For obtaining better semantic information of the meta knowledge, the proposed DARL model applies a dynamic neighbor encoder to incorporate neighbor relations into entity embedding. In addition, DARL builds attention mechanism based fusion strategy for different attributes of the same relation to further enhance the relation-meta learning ability. We evaluate our DARL model on two public benchmark datasets NELL-One and WIKI-One for link prediction. Extensive experimental results indicate that our DARL outperforms the state-of-the-art models with an average relative improvement about 23.37%, 32.46% in MRR and [email protected] on NELL-One, respectively.

A Multi-View Filter for Relation-Free Knowledge Graph Completion

As knowledge graphs are often incomplete, knowledge graph completion methods have been widely proposed to infer missing facts by predicting the missing element of a triple given the other two elements. However, the assumption that the two elements have to be correlated is strong. Thus in this paper, we investigate relation-free knowledge graph completion to predict relation-tail(r-t) pairs given a head entity. Considering the large scale of candidate relation-tail pairs, previous work proposed to filter r-t pairs before ranking them relying on entity types, which fails when entity types are missing or insufficient. To tackle the limitation, we propose a relation-free knowledge graph completion method that can cope with knowledge graphs without additional ontological information, such as entity types. Specifically, we propose a multi-view filter, including two intra-view modules and an inter-view module, to filter r-t pairs. For the intra-view modules, we construct head-relation and tail-relation graphs based on triples. Two graph neural networks are respectively trained on these two graphs to capture the correlations between the head entities and the relations, as well as the tail entities and the relations. The inter-view module is learned to bridge the embeddings of entities that appeared in the two graphs. In terms of ranking, existing knowledge graph embedding models are applied to score and rank the filtered candidate r-t pairs. Experimental results show the efficiency of our method in preserving higher-quality candidate r-t pairs for knowledge graphs and resulting in better relation-free knowledge graph completion.

Bi-Neighborhood Graph Neural Network for cross-lingual entity alignment

The objective of the cross-lingual entity alignment (EA) task is to identify equivalent entities in knowledge graphs (KGs) across different languages. Most existing works rely on relation triples to learn entity representations. In addition, some methods introduce attribute information to improve the performance of EA, but they model relation triples and attribute triples separately in different networks, which prevents the alignment information in different triples from spreading across the networks and limits their mutual interaction and enhancement. In this paper, we propose integrating entities and attributes into a unified entity-attribute graph, where the neighbors of an entity include not only other entities but also its attributes. To model the entity-attribute graph, we further design a novel Bi-Neighborhood Graph Neural Network (BNGNN). Specifically, BNGNN applies attribute-aware self-attention to neighborhood aggregation and generates two types of neighborhood features in each layer, i.e., entity neighborhood features and attribute neighborhood features. Then, the two features are combined through a highway gate to generate more representative entity embeddings. Finally, the outputs of multiple layers are integrated to obtain global-aware entity representations with high-level structural information. Furthermore, a two-stage training strategy is adopted to jointly perform entity alignment and attribute alignment in a semi-supervised framework. Extensive experiments conducted on two real-world datasets (regular and sparse) demonstrate that our BNGNN model consistently outperforms existing EA methods.

Hic-KGQA: Improving multi-hop question answering over knowledge graph via hypergraph and inference chain

Question answering over knowledge graph (KGQA) aims at answering natural language questions posed over knowledge graphs (KGs). Moreover, multi-hop KGQA requires reasoning across multiple triplets in KGs to get to the answer. Unfortunately, KGs often lack complete information and contain many missing links, which poses huge challenges for multi-hop KGQA. To address this, recent several approaches have introduced KG embedding techniques, which have shown good performance on the multi-hop KGQA task. However, these methods ignore the semantic correlations between paths and questions, and the reasoning process is not easily explained. Furthermore, traditional KG embedding methods consider only low-order pairwise relations and ignore the higher-order relations among entities, leading to a sub-optimal embedding result. To address these problems, we propose Hic-KGQA, a novel hypergraph and inference chain-based model for multi-hop KGQA. Specifically, Hic-KGQA first generates pre-trained entity embeddings with multiple semantics via a hypergraph-based KGC module (HKM). Then, an inference chain modeling module (ICMM) is designed to learn the importance of different inference chains for the question and encode the highest-ranked inference chain into an embedding representation. Finally, two scoring networks are used to evaluate the correlation between the candidate answers and the questions from the perspective of both the triplet facts and the reasoning process to obtain more accurate answers. Furthermore, the mutual information maximization (MIM) is innovatively implemented to capture richer common path features from similar cases to alleviate the missing path problem caused by KG incompleteness. Experiments show that Hic-KGQA significantly outperforms existing state-of-the-art methods and is explainable.

Social world knowledge: Modeling and applications.

Social world knowledge is a key ingredient in effective communication and information processing by humans and machines alike. As of today, there exist many knowledge bases that represent factual world knowledge. Yet, there is no resource that is designed to capture social aspects of world knowledge. We believe that this work makes an important step towards the formulation and construction of such a resource. We introduce SocialVec, a general framework for eliciting low-dimensional entity embeddings from the social contexts in which they occur in social networks. In this framework, entities correspond to highly popular accounts which invoke general interest. We assume that entities that individual users tend to co-follow are socially related, and use this definition of social context to learn the entity embeddings. Similar to word embeddings which facilitate tasks that involve text semantics, we expect the learned social entity embeddings to benefit multiple tasks of social flavor. In this work, we elicited the social embeddings of roughly 200K entities from a sample of 1.3M Twitter users and the accounts that they follow. We employ and gauge the resulting embeddings on two tasks of social importance. First, we assess the political bias of news sources in terms of entity similarity in the social embedding space. Second, we predict the personal traits of individual Twitter users based on the social embeddings of entities that they follow. In both cases, we show advantageous or competitive performance using our approach compared with task-specific baselines. We further show that existing entity embedding schemes, which are fact-based, fail to capture social aspects of knowledge. We make the learned social entity embeddings available to the research community to support further exploration of social world knowledge and its applications.

DA-CBGRU-Seq2Seq Based Soft Sensor for Mechanical Properties of Hot Rolling Process

High-precision prediction of material properties is an advanced direction of hot rolled product technology in manufacturing processes. However, complex disturbance usually produces wrong or missing data in the acquisition and entry stages. The long process and the diverse operating conditions bring great challenges to constructing soft sensors for the hot rolling process. To address this problem, a new time-series soft sensor model framework is proposed for mechanical properties prediction. Firstly, in order to process the raw dataset, the isolation forest algorithm is used to remove the outliers, and the random forest algorithm is used to fill in missing values. Secondly, the auxiliary variables are extracted by the combination of the XGBoost algorithm and prior mechanistic knowledge. Then, a DA-CBGRU-Seq2Seq time-series model based on a two-stage attention mechanism is designed for soft sensor modeling. In the encoder stage, external factors are introduced by an entity embedding layer. A combination structure of bidirectional gated recurrent units and convolutional layers is designed for extracting and learning the input features adaptively with the input attention mechanism. In the decoder stage, the long-term time dependence of the time series is learned by the temporal attention mechanism. Finally, the reliability of the proposed sensor model is compared and verified by the hot rolling process. The results indicate that the proposed method has an accuracy of over 95% in predicting the mechanical properties of a real hot-rolling process dataset, and has better performance compared to other comparative soft sensor prediction methods.

Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding

We propose an entity-agnostic representation learning method for handling the problem of inefficient parameter storage costs brought by embedding knowledge graphs. Conventional knowledge graph embedding methods map elements in a knowledge graph, including entities and relations, into continuous vector spaces by assigning them one or multiple specific embeddings (i.e., vector representations). Thus the number of embedding parameters increases linearly as the growth of knowledge graphs. In our proposed model, Entity-Agnostic Representation Learning (EARL), we only learn the embeddings for a small set of entities and refer to them as reserved entities. To obtain the embeddings for the full set of entities, we encode their distinguishable information from their connected relations, k-nearest reserved entities, and multi-hop neighbors. We learn universal and entity-agnostic encoders for transforming distinguishable information into entity embeddings. This approach allows our proposed EARL to have a static, efficient, and lower parameter count than conventional knowledge graph embedding methods. Experimental results show that EARL uses fewer parameters and performs better on link prediction tasks than baselines, reflecting its parameter efficiency.

Lifelong Embedding Learning and Transfer for Growing Knowledge Graphs

Existing knowledge graph (KG) embedding models have primarily focused on static KGs. However, real-world KGs do not remain static, but rather evolve and grow in tandem with the development of KG applications. Consequently, new facts and previously unseen entities and relations continually emerge, necessitating an embedding model that can quickly learn and transfer new knowledge through growth. Motivated by this, we delve into an expanding field of KG embedding in this paper, i.e., lifelong KG embedding. We consider knowledge transfer and retention of the learning on growing snapshots of a KG without having to learn embeddings from scratch. The proposed model includes a masked KG autoencoder for embedding learning and update, with an embedding transfer strategy to inject the learned knowledge into the new entity and relation embeddings, and an embedding regularization method to avoid catastrophic forgetting. To investigate the impacts of different aspects of KG growth, we construct four datasets to evaluate the performance of lifelong KG embedding. Experimental results show that the proposed model outperforms the state-of-the-art inductive and lifelong embedding baselines.