Few-shot Relation Research Articles

Few-Shot Relation Extraction With Dual Graph Neural Network Interaction.

Recent advances in relation extraction with deep neural architectures have achieved excellent performance. However, current models still suffer from two main drawbacks: 1) they require enormous volumes of training data to avoid model overfitting and 2) there is a sharp decrease in performance when the data distribution during training and testing shift from one domain to the other. It is thus vital to reduce the data requirement in training and explicitly model the distribution difference when transferring knowledge from one domain to another. In this work, we concentrate on few-shot relation extraction under domain adaptation settings. Specifically, we propose DUAL GRAPH, a novel graph neural network (GNN) based approach for few-shot relation extraction. DUAL GRAPH leverages an edge-labeling dual graph (i.e., an instance graph and a distribution graph) to explicitly model the intraclass similarity and interclass dissimilarity in each individual graph, as well as the instance-level and distribution-level relations across graphs. A dual graph interaction mechanism is proposed to adequately fuse the information between the two graphs in a cyclic flow manner. We extensively evaluate DUAL GRAPH on FewRel1.0 and FewRel2.0 benchmarks under four few-shot configurations. The experimental results demonstrate that DUAL GRAPH can match or outperform previously published approaches. We also perform experiments to further investigate the parameter settings and architectural choices, and we offer a qualitative analysis.

Sample Feature Enhancement Model Based on Heterogeneous Graph Representation Learning for Few-shot Relation Classification

Few-Shot Relation Classification (FSRC) aims to predict novel relationships by learning from limited samples. Graph Neural Network (GNN) approaches for FSRC constructs data as graphs, effectively capturing sample features through graph representation learning. However, they often face several challenges: 1) They tend to neglect the interactions between samples from different support sets and overlook the implicit noise in labels, leading to sub-optimal sample feature generation. 2) They struggle to deeply mine the diverse semantic information present in FSRC data. 3) Over-smoothing and overfitting limit the model's depth and adversely affect overall performance. To address these issues, we propose a Sample Representation Enhancement model based on Heterogeneous Graph Neural Network (SRE-HGNN) for FSRC. This method leverages inter-sample and inter-class associations (i.e., label mutual attention) to effectively fuse features and generate more expressive sample representations. Edge-heterogeneous GNNs are employed to enhance sample features by capturing heterogeneous information of varying depths through different edge attentions. Additionally, we introduce an attention-based neighbor node culling method, enabling the model to stack higher levels and extract deeper inter-sample associations, thereby improving performance. Finally, experiments are conducted for the FSRC task, and SRE-HGNN achieves an average accuracy improvement of 1.84% and 1.02% across two public datasets.

CoFF-CHP: coarse-to-fine filters with concept heuristic prompt for few-shot relation classification

CoFF-CHP: coarse-to-fine filters with concept heuristic prompt for few-shot relation classification

Few-shot biomedical relation extraction using data augmentation and domain information

Relation extraction (RE) plays a pivotal role in biomedical information extraction. However, traditional approaches are often limited by high data annotation costs and extensive time investments. To address this challenge, this study proposes an innovative few-shot learning approach for biomedical RE tasks that utilizes data augmentation and domain information. Specifically, this method enhances the diversity of training data through a synthesized data augmentation strategy. At the same time, it combines domain information with prompt learning techniques. By incorporating domain information, the model’s generalization capability for rare data instances is improved. Furthermore, the prompt learning approach, by integrating prompt templates into the model input, guides the pre-trained language model to more accurately adapt to the RE task. Our model demonstrated exceptional performance in three different biomedical RE tasks, particularly on the I2B2-2010 RE dataset. In the 1-shot, 10-shot, and 50-shot settings, the F1 scores increased by 14.38%, 9.50%, and 6.05%, respectively.

DiffFSRE: Diffusion-Enhanced Prototypical Network for Few-Shot Relation Extraction.

Supervised learning methods excel in traditional relation extraction tasks. However, the quality and scale of the training data heavily influence their performance. Few-shot relation extraction is gradually becoming a research hotspot whose objective is to learn and extract semantic relationships between entities with only a limited number of annotated samples. In recent years, numerous studies have employed prototypical networks for few-shot relation extraction. However, these methods often suffer from overfitting of the relation classes, making it challenging to generalize effectively to new relationships. Therefore, this paper seeks to utilize a diffusion model for data augmentation to address the overfitting issue of prototypical networks. We propose a diffusion model-enhanced prototypical network framework. Specifically, we design and train a controllable conditional relation generation diffusion model on the relation extraction dataset, which can generate the corresponding instance representation according to the relation description. Building upon the trained diffusion model, we further present a pseudo-sample-enhanced prototypical network, which is able to provide more accurate representations for prototype classes, thereby alleviating overfitting and better generalizing to unseen relation classes. Additionally, we introduce a pseudo-sample-aware attention mechanism to enhance the model's adaptability to pseudo-sample data through a cross-entropy loss, further improving the model's performance. A series of experiments are conducted to prove our method's effectiveness. The results indicate that our proposed approach significantly outperforms existing methods, particularly in low-resource one-shot environments. Further ablation analyses underscore the necessity of each module in the model. As far as we know, this is the first research to employ a diffusion model for enhancing the prototypical network through data augmentation in few-shot relation extraction.

Synergistic Anchored Contrastive Pre-training for Few-Shot Relation Extraction

Few-shot Relation Extraction (FSRE) aims to extract relational facts from a sparse set of labeled corpora. Recent studies have shown promising results in FSRE by employing Pre-trained Language Models (PLMs) within the framework of supervised contrastive learning, which considers both instances and label facts. However, how to effectively harness massive instance-label pairs to encompass the learned representation with semantic richness in this learning paradigm is not fully explored. To address this gap, we introduce a novel synergistic anchored contrastive pre-training framework. This framework is motivated by the insight that the diverse viewpoints conveyed through instance-label pairs capture incomplete yet complementary intrinsic textual semantics. Specifically, our framework involves a symmetrical contrastive objective that encompasses both sentence-anchored and label-anchored contrastive losses. By combining these two losses, the model establishes a robust and uniform representation space. This space effectively captures the reciprocal alignment of feature distributions among instances and relational facts, simultaneously enhancing the maximization of mutual information across diverse perspectives within the same relation. Experimental results demonstrate that our framework achieves significant performance enhancements compared to baseline models in downstream FSRE tasks. Furthermore, our approach exhibits superior adaptability to handle the challenges of domain shift and zero-shot relation extraction. Our code is available online at https://github.com/AONE-NLP/FSRE-SaCon.

Joint data augmentation and knowledge distillation for few-shot continual relation extraction

Joint data augmentation and knowledge distillation for few-shot continual relation extraction

Relation-oriented few-shot knowledge graph prototype networks

Knowledge graph (KG) embedding has been widely researched, but it suffers from some problems in the few-shot scenarios. The topological and semantic connection among entities cannot satisfy the assumption of samples’ independent identically distribution. Additionally, these relations between entity pairs are various and complex. But most of the previous methods are node-oriented, which is weak in modeling complex relations. In order to solve the above problems, we propose a few-shot relation prototype network (FRPN). In our method, each relation is regarded as a learning object, instead of just entity pairs’ concatenation. A relation-oriented two-channel embedding mechanism is designed to achieve multi-scale information aggregation at the entity level and the relation level respectively. For the entity level, it aggregates information at different scales according to the relation type and the neighbors under each relation. For the relation level, our model obtains each kind of relation’s prototype from the ontology and the entity layer. The multi-scale aggregation contributes to KG embedding in the few-shot scenario. Compared with existing models, our model has significantly improved the performance on both the link prediction and triple classification tasks in two few-shot datasets.

Few-Shot Relation Extraction Through Prompt With Relation Information and Multi-Level Contrastive Learning

Few-Shot Relation Extraction Through Prompt With Relation Information and Multi-Level Contrastive Learning

Few-Shot Relation Extraction With Automatically Generated Prompts.

Relation extraction (RE) tends to struggle when the supervised training data is few and difficult to be collected. In this article, we elicit relational and factual knowledge from large pretrained language models (PLMs) for few-shot RE (FSRE) with prompting techniques. Concretely, we automatically generate a diverse set of natural language templates and modulate PLM's behavior through these prompts for FSRE. To mitigate the template bias which leads to unstableness of few-shot learning, we propose a simple yet effective template regularization network (TRN) to prevent deep networks from over-fitting uncertain templates and thus stabilize the FSRE models. TRN alleviates the template bias with three mechanisms: 1) an attention mechanism over mini-batch to weight each template; 2) a ranking regularization mechanism to regularize the attention weights and constrain the importance of uncertain templates; and 3) a template calibration module with two calibrating techniques to modify the uncertain templates in the lowest-ranked group. Experimental results on two benchmark datasets (i.e., FewRel and NYT) show that our model has robust superiority over strong competitors. For reproducibility, we will release our code and data upon the publication of this article.

SARF: Aliasing Relation-Assisted Self-Supervised Learning for Few-Shot Relation Reasoning.

Few-shot relation reasoning on knowledge graphs (FS-KGR) is an important and practical problem that aims to infer long-tail relations and has drawn increasing attention these years. Among all the proposed methods, self-supervised learning (SSL) methods, which effectively extract the hidden essential inductive patterns relying only on the support sets, have achieved promising performance. However, the existing SSL methods simply cut down connections between high-frequency and long-tail relations, which ignores the fact, i.e., the two kinds of information could be highly related to each other. Specifically, we observe that relations with similar contextual meanings, called aliasing relations (ARs), may have similar attributes. In other words, the ARs of the target long-tail relation could be in high-frequency, and leveraging such attributes can largely improve the reasoning performance. Based on the interesting observation above, we proposed a novel Self-supervised learning model by leveraging Aliasing Relations to assist FS-KGR, termed . Specifically, we propose a graph neural network (GNN)-based AR-assist module to encode the ARs. Besides, we further provide two fusion strategies, i.e., simple summation and learnable fusion, to fuse the generated representations, which contain extra abundant information underlying the ARs, into the self-supervised reasoning backbone for performance enhancement. Extensive experiments on three few-shot benchmarks demonstrate that achieves state-of-the-art (SOTA) performance compared with other methods in most cases.

Multi-Head Self-Attention-Enhanced Prototype Network with Contrastive–Center Loss for Few-Shot Relation Extraction

Few-shot relation extraction (FSRE) constitutes a critical task in natural language processing (NLP), involving learning relationship characteristics from limited instances to enable the accurate classification of new relations. The existing research primarily concentrates on using prototype networks for FSRE and enhancing their performance by incorporating external knowledge. However, these methods disregard the potential interactions among different prototype networks, and each prototype network can only learn and infer from its limited instances, which may limit the robustness and reliability of the prototype representations. To tackle the concerns outlined above, this paper introduces a novel prototype network called SACT (multi-head self-attention and contrastive-center loss), aimed at obtaining more comprehensive and precise interaction information from other prototype networks to bolster the reliability of the prototype network. Firstly, SACT employs a multi-head self-attention mechanism for capturing interaction information among different prototypes from traditional prototype networks, reducing the noise introduced by unknown categories with a small sample through information aggregation. Furthermore, SACT introduces a new loss function, the contrastive–center loss function, aimed at tightly clustering samples from a similar relationship category in the center of the feature space while dispersing samples from different relationship categories. Through extensive experiments on FSRE datasets, this paper demonstrates the outstanding performance of SACT, providing strong evidence for the effectiveness and practicality of SACT.

Relational concept enhanced prototypical network for incremental few-shot relation classification

Compared with conventional close-domain relation classification, incremental few-shot relation classification requires incrementally to learn novel relations through very few samples without forgetting base relations, which is more fitting to the needs of real scenarios. Existing incremental few-shot learning methods commonly leverage memory to freeze knowledge of the learned base classes, and then incrementally learn new classes from very few labeled samples. However, there are still two challenges to these existing approaches: overfitting novel relations and the catastrophic forgetting of base relations. In this paper, we propose a relational concept enhanced prototypical network (RC-EProto) to address these problems. First, we build a concept alignment module to eliminate the difference between relational concepts and samples, and we incorporate relational concepts for sample representation by this module to construct a more appropriate prototype. Second, we develop a contrastive approach on the base relations to address the forgetting of old knowledge. The samples of the base relation are pulled tightly with its prototype in the embedding space, while the samples of the other relations are pulled far from this base prototype. Extensive experiments on FewRel 1.0 and 2.0 datasets demonstrate the superiority of our method compared with the state-of-the-art methods. Notably, RC-EProto maintains an excellent incremental few-shot domain adaptation learning ability. Compared to existing methods, our model is able to achieve near-optimal results for the base relations while improving the classification accuracy of the novel relations by 13.48% and 9.63% for the 1-increment and 5-increment settings on FewRel 2.0, respectively.

Improving few-shot relation extraction through semantics-guided learning

Few-shot relation extraction (few-shot RE) aims to recognize relations between the entity pair in a given text by utilizing very few annotated instances. As a simple yet efficient approach, prototype network-based methods often directly incorporate relation information to enhance prototype representation or leverage contrastive learning to mitigate prediction confusion. Despite achieving good results, the above methods are still susceptible to false judgments of outlier samples and confusion of similar classes. To address these issues, we propose a novel Semantics-Guided Learning (SemGL) method that more effectively utilizes relation information to enhance both the representations of instances and prototypes for improving the performance of few-shot RE. First, SemGL employs the prompt encoder to encode various prompt templates of instances and relation information and obtains more accurate semantic representations of instances, instance prototypes, and concept prototypes via the prompt enhancement from large language models. Then, SemGL introduces a novel technique called relation graph learning, which leverages concept prototypes to cluster homogeneous instances together, emphasizing relation-specific features of concrete instances. Simultaneously, SemGL employs instance-level contrastive learning between instance prototypes and support instances to distinguish between intra-class instances and inter-class instances to promote shared features among intra-class instances. Additionally, prototype-level contrastive learning leverages concept prototypes to pull closer relation-specific features of the concept prototype and shared features of the instance prototype from the same relation. Finally, SemGL utilizes new relation prototypes that integrate interpretable features of concept prototypes and shared features of instance prototypes for prediction. Experimental results on two publicly available few-shot RE datasets demonstrate the effectiveness and efficiency of SemGL in introducing relation information, with particularly promising results for the domain adaptation challenge task.

A lightweight approach based on prompt for few-shot relation extraction

Few-shot relation extraction (FSRE) aims to predict the relation between two entities in a sentence using a few annotated samples. Many works solve the FSRE problem by training complex models with a huge number of parameters, which results in longer processing times to obtain results. Some recent works focus on introducing relation information into Prototype Networks in various ways. However, most of these methods obtain entity and relation representations by fine-tuning large pre-trained language models. This implies that a copy of the complete pre-trained model needs to be saved after fine-tuning for each specific task, leading to a shortage of computing and space resources. To address this problem, in this paper, we introduce a light approach that utilizes prompt-learning to assist in fine-tuning model by adjusting fewer parameters. To obtain a better prototype of relation, we design a new enhanced fusion module to fuse relation information and original prototype. We conduct extensive experiments on the common FSRE datasets FewRel 1.0 and FewRel 2.0 to varify the advantages of our method, the results show that our model achieves state-of-the-art performance.

Adaptive class augmented prototype network for few-shot relation extraction

Relation extraction is one of the most essential tasks of knowledge construction, but it depends on a large amount of annotated data corpus. Few-shot relation extraction is proposed as a new paradigm, which is designed to learn new relationships between entities with merely a small number of annotated instances, effectively mitigating the cost of large-scale annotation and long-tail problems. To generalize to novel classes not included in the training set, existing approaches mainly focus on tuning pre-trained language models with relation instructions and developing class prototypes based on metric learning to extract relations. However, the learned representations are extremely sensitive to discrepancies in intra-class and inter-class relationships and hard to adaptively classify the relations due to biased class features and spurious correlations, such as similar relation classes having closer inter-class prototype representation. In this paper, we introduce an adaptive class augmented prototype network with instance-level and representation-level augmented mechanisms to strengthen the representation space. Specifically, we design the adaptive class augmentation mechanism to expand the representation of classes in instance-level augmentation, and class augmented representation learning with Bernoulli perturbation context attention to enhance the representation of class features in representation-level augmentation and explore adaptive debiased contrastive learning to train the model. Experimental results have been demonstrated on FewRel and NYT-25 under various few-shot settings, and the proposed model has improved accuracy and generalization, especially for cross-domain and different hard tasks.

Multi-view semantic enhancement model for few-shot knowledge graph completion

In recent years, few-shot knowledge graph completion (FKGC) has gained popularity as a solution to the long-tail distribution problem of real-world knowledge graphs (KGs). The previous knowledge graph completion (KGC) models obtain triple representation merely relying on the structure view, ignoring the valuable semantic knowledge. In this paper, we propose a multi-view framework for few-shot relation learning to address the issue. Specifically, based the structural information of the graph obtained using the structure view, we add the text view and the commonsense view. The text view employs text descriptions of entities and relations to obtain a richer semantic representation. The commonsense view performs high-quality negative sampling based on complex relations. Moreover, commonsense semantic constraints are invoked to suppress the overfitting caused by the complexity of the relation matrix. Extensive experiments show that our model outperforms state-of-the-art FKGC methods on the frequently-used benchmark datasets FB15k237-One and NELL-One.

Hyperplane projection network for few-shot relation classification

Recently meta-learning-based few-shot learning methods have been widely used for relation classification. Previous work reveals that meta-learning performs poorly in scenarios where the edge probability distribution of the target domain dataset appears to be significantly different from the source domain. In this paper, we enhance the meta-learning framework with high-dimensional semantic feature extraction and hyperplane projection metrics for meta-tasks. First, we enhance the focus of BERT on entity words by adding entity markers and vector pooling. After that, the high-dimensional semantic features of the support set are extracted and transformed into hyperplanes. Finally, we obtain the classification results by calculating the projection distance between the query sample and the hyperplane. In addition, we design a auxiliary function with a plane correction factor, which can better amplify the plane spacing and reduce the degree of category confusion, which is important for solving the problem of metric spatial loss. Experiments on two real-world few-shot datasets show that our model HPN is more effective in classifying few-shot relations in the same domain and domain-adapted scenarios. And HPN is more stable on NOTA tasks.

Few-Shot Semantic Relation Prediction Across Heterogeneous Graphs

Semantic relation prediction aims to mine the implicit relationships between objects in heterogeneous graphs, which consist of different types of objects and different types of links. In real-world scenarios, new semantic relations constantly emerge and they typically appear with only a few labeled data. Since a variety of semantic relations exist in multiple heterogeneous graphs, the transferable knowledge can be mined from some existing semantic relations to help predict the new semantic relations with few labeled data. This inspires a novel problem of few-shot semantic relation prediction across heterogeneous graphs. However, the existing methods cannot solve this problem because they not only require a large number of labeled samples as input, but also focus on a single graph with a fixed heterogeneity. Targeting this novel and challenging problem, in this paper, we propose a Meta-learning based Graph neural network for Semantic relation prediction, named MetaGS. Firstly, MetaGS decomposes the graph structure between objects into multiple normalized subgraphs, then adopts a two-view graph neural network to capture local heterogeneous information and global structure information of these subgraphs. Secondly, MetaGS aggregates the information of these subgraphs with a hyper-prototypical network, which can learn from existing semantic relations and adapt to new semantic relations. Thirdly, using the well-initialized two-view graph neural network and hyper-prototypical network, MetaGS can effectively learn new semantic relations from different graphs while overcoming the limitation of few labeled data. Extensive experiments on three real-world datasets have demonstrated the superior performance of MetaGS over the state-of-the-art methods.

Few-Shot Relation Prediction of Knowledge Graph via Convolutional Neural Network with Self-Attention

Knowledge graph (KG) has become the vital resource for various applications like question answering and recommendation system. However, several relations in KG only have few observed triples, which makes it necessary to develop the method for few-shot relation prediction. In this paper, we propose the Convolutional Neural Network with Self-Attention Relation Prediction (CARP) model to predict new facts with few observed triples. First, to learn the relation property features, we build a feature encoder by using the convolutional neural network with self-attention from the few observed triples rather than background knowledge. Then, by incorporating the learned features, we give an embedding network to learn the representation of incomplete triples. Finally, we give the loss function and training algorithm of our CARP model. Experimental results on three real-world datasets show that our proposed method improves Hits@10 by 48% on average over the state-of-the-art competitors.