Relative Triplet Research Articles

TR-Net: Token Relation Inspired Table Filling Network for Joint Entity and Relation Extraction

Recently, table filling models have achieved promising performance in jointly extracting relation triplets from complex sentences, leveraging their inherent structural advantage of delineating entities and relations as table cells. Nonetheless, these models predominantly concentrate on the cells corresponding to entity pairs within the predicted tables, neglecting the interrelations among other token pairs. This oversight can potentially lead to the exclusion of essential token information. To address these challenges, we introduce the Token Relation-Inspired Network (TR-Net), a novel framework for the joint extraction of entities and relations. It encompasses a token relation generator that adaptively constructs a token relation table, concentrating on the prominent token cells. Moreover, it also uses a structure-enhanced encoder that integrates the structural and sequential data of sentences via a highway gate mechanism. Our experimental analysis demonstrates that TR-Net delivers considerable enhancements and achieves state-of-the-art performance on four public datasets.

Research on a Joint Extraction Method of Track Circuit Entities and Relations Integrating Global Pointer and Tensor Learning.

To address the issue of efficiently reusing the massive amount of unstructured knowledge generated during the handling of track circuit equipment faults and to automate the construction of knowledge graphs in the railway maintenance domain, it is crucial to leverage knowledge extraction techniques to efficiently extract relational triplets from fault maintenance text data. Given the current lag in joint extraction technology within the railway domain and the inefficiency in resource utilization, this paper proposes a joint extraction model for track circuit entities and relations, integrating Global Pointer and tensor learning. Taking into account the associative characteristics of semantic relations, the nesting of domain-specific terms in the railway sector, and semantic diversity, this research views the relation extraction task as a tensor learning process and the entity recognition task as a span-based Global Pointer search process. First, a multi-layer dilate gated convolutional neural network with residual connections is used to extract key features and fuse the weighted information from the 12 different semantic layers of the RoBERTa-wwm-ext model, fully exploiting the performance of each encoding layer. Next, the Tucker decomposition method is utilized to capture the semantic correlations between relations, and an Efficient Global Pointer is employed to globally predict the start and end positions of subject and object entities, incorporating relative position information through rotary position embedding (RoPE). Finally, comparative experiments with existing mainstream joint extraction models were conducted, and the proposed model's excellent performance was validated on the English public datasets NYT and WebNLG, the Chinese public dataset DuIE, and a private track circuit dataset. The F1 scores on the NYT, WebNLG, and DuIE public datasets reached 92.1%, 92.7%, and 78.2%, respectively.

Decomposed Prototype Learning for Few-Shot Scene Graph Generation

Today's scene graph generation (SGG) models typically require abundant manual annotations to learn new predicate types. Therefore, it is difficult to apply them to real-world applications with massive uncommon predicate categories whose annotations are hard to collect. In this paper, we focus on Few-Shot SGG (FSSGG) , which encourages SGG models to be able to quickly transfer previous knowledge and recognize unseen predicates well with only a few examples. However, current methods for FSSGG are hindered by the high intra-class variance of predicate categories in SGG: On one hand, each predicate category commonly has multiple semantic meanings under different contexts. On the other hand, the visual appearance of relation triplets with the same predicate differs greatly under different subject-object compositions. Such great variance of inputs makes it hard to learn generalizable representation for each predicate category with current few-shot learning (FSL) methods. However, we found that this intra-class variance of predicates is highly related to the composed subjects and objects. To model the intra-class variance of predicates with subject-object context, we propose a novel Decomposed Prototype Learning (DPL) model for FSSGG. Specifically, we first construct a decomposable prototype space to capture diverse semantics and visual patterns of subjects and objects for predicates by decomposing them into multiple prototypes. Afterwards, we integrate these prototypes with different weights to generate query-adaptive predicate representation with more reliable semantics for each query sample. We conduct extensive experiments and compare with various baseline methods to show the effectiveness of our method.

DeepCKID: A Multi-Head Attention-Based Deep Neural Network Model Leveraging Classwise Knowledge to Handle Imbalanced Textual Data

This paper presents DeepCKID, a Multi-Head Attention (MHA)-based deep learning model that exploits statistical and semantic knowledge corresponding to documents across different classes in the datasets to improve the model’s ability to detect minority class instances in imbalanced text classification. In this process, corresponding to each document, DeepCKID extracts — (i) word-level statistical and semantic knowledge, namely, class correlation and class similarity corresponding to each word, based on its association with different classes in the dataset, and (ii) class-level knowledge from the document using n-grams and relation triplets corresponding to classwise keywords present, identified using cosine similarity utilizing Transformers-based Pre-trained Language Models (PLMs). DeepCKID encodes the word-level and class-level features using deep convolutional networks, which can learn meaningful patterns from them. At first, DeepCKID combines the semantically meaningful Sentence-BERT document embeddings and word-level feature matrix to give the final document representation, which it further fuses to the different classwise encoded representations to strengthen feature propagation. DeepCKID then passes the encoded document representation and its different classwise representations through an MHA layer to identify the important features at different positions of the feature subspaces, resulting in a latent dense vector accentuating its association with a particular class. Finally, DeepCKID passes the latent vector to the softmax layer to learn the corresponding class label. We evaluate DeepCKID over six publicly available Amazon reviews datasets using four Transformers-based PLMs. We compare DeepCKID with three approaches and four ablation-like baselines. Our study suggests that in most cases, DeepCKID outperforms all the comparison approaches, including baselines.

Alzheimer's Disease Knowledge Graph Enhances Knowledge Discovery and Disease Prediction.

Alzheimer's disease (AD), a progressive neurodegenerative disorder, continues to increase in prevalence without any effective treatments to date. In this context, knowledge graphs (KGs) have emerged as a pivotal tool in biomedical research, offering new perspectives on drug repurposing and biomarker discovery by analyzing intricate network structures. Our study seeks to build an AD-specific knowledge graph, highlighting interactions among AD, genes, variants, chemicals, drugs, and other diseases. The goal is to shed light on existing treatments, potential targets, and diagnostic methods for AD, thereby aiding in drug repurposing and the identification of biomarkers. We annotated 800 PubMed abstracts and leveraged GPT-4 for text augmentation to enrich our training data for named entity recognition (NER) and relation classification. A comprehensive data mining model, integrating NER and relationship classification, was trained on the annotated corpus. This model was subsequently applied to extract relation triplets from unannotated abstracts. To enhance entity linking, we utilized a suite of reference biomedical databases and refine the linking accuracy through abbreviation resolution. As a result, we successfully identified 3,199,276 entity mentions and 633,733 triplets, elucidating connections between 5,000 unique entities. These connections were pivotal in constructing a comprehensive Alzheimer's Disease Knowledge Graph (ADKG). We also integrated the ADKG constructed after entity linking with other biomedical databases. The ADKG served as a training ground for Knowledge Graph Embedding models with the high-ranking predicted triplets supported by evidence, underscoring the utility of ADKG in generating testable scientific hypotheses. Further application of ADKG in predictive modeling using the UK Biobank data revealed models based on ADKG outperforming others, as evidenced by higher values in the areas under the receiver operating characteristic (ROC) curves. The ADKG is a valuable resource for generating hypotheses and enhancing predictive models, highlighting its potential to advance AD's disease research and treatment strategies.

TGIN: Translation-Based Graph Inference Network for Few-Shot Relational Triplet Extraction.

Extracting relational triplets aims at detecting entity pairs and their semantic relations. Compared with pipeline models, joint models can reduce error propagation and achieve better performance. However, all of these models require large amounts of training data, therefore performing poorly on many long-tail relations in reality with insufficient data. In this article, we propose a novel end-to-end model, called TGIN, for few-shot triplet extraction. The core of TGIN is a multilayer heterogeneous graph with two types of nodes (entity node and relation node) and three types of edges (relation-entity edge, entity-entity edge, and relation-relation edge). On the one hand, this heterogeneous graph with entities and relations as nodes can intuitively extract relational triplets jointly, thereby reducing error propagation. On the other hand, it enables the triplet information of limited labeled data to interact better, thus maximizing the advantage of this information for few-shot triplet extraction. Moreover, we devise a graph aggregation and update method that utilizes translation algebraic operations to mine semantic features while retaining structure features between entities and relations, thereby improving the robustness of the TGIN in a few-shot setting. After updating the node and edge features through layers, TGIN propagates the label information from a few labeled examples to unlabeled examples, thus inferring triplets from these unlabeled examples. Extensive experiments on three reconstructed datasets demonstrate that TGIN can significantly improve the accuracy of triplet extraction by 2.34%~10.74% compared with the state-of-the-art baselines. To the best of our knowledge, we are the first to introduce a heterogeneous graph for few-shot relational triplet extraction.

Complex Relation Embedding for Scene Graph Generation.

Given an input image, scene graph generation (SGG) aims to generate comprehensive visual relationships between objects in the form of graphs. Recently, more attention to the design of complex networks and complicated strategies has been paid to the long tail issue caused by the imbalanced class distribution. However, most existing methods adopt the concatenated features of two objects in real space as the final relation representation for a given triplet. We mainly argue that such a simple concatenation may neglect the importance of complex interactions between objects, which results in the diversity of visual relations. In addition, the representation learning in real space is also inadequate to express this property. To alleviate these issues, we seamlessly incorporate Hermitian inner product into existing models to facilitate the generation of scene graphs by learning Relation Embedding in Complex space (CoRE). More specifically, we first introduce the concept of complex-valued representations for entities and then formulate the relation triplets with Hermitian inner product in complex space. Finally, we investigate the effect of utilizing only real component or both of Hermitian inner product on inferring more reasonable interaction between objects for scene graphs. Comprehensive experiments on two widely used benchmark datasets, Visual Genome (VG) and Open Image, demonstrate our effectiveness, superiority, and generalization on various metrics for biased or unbiased inference.

KBPT: knowledge-based prompt tuning for zero-shot relation triplet extraction

Knowledge representation is increasingly recognized as an effective method for information extraction. Nevertheless, numerous studies have disregarded its potential applications in the zero-shot setting. In this article, a novel framework, called knowledge-based prompt tuning for zero-shot relation triplet extraction (KBPT), was developed, founded on external ontology knowledge. This framework serves as a catalyst for exploring relation triplet extraction (RTE) methods within low-resource scenarios, warranting further scrutiny. Zero-shot setting RTE aims to extract multiple triplets that consist of head entities, tail entities, and relation labels from an input sentence, where the extracted relation labels are those that do not exist in the training set. To address the data scarcity problem in zero-shot RTE, a technique was introduced to synthesize training samples by prompting language models to generate structured texts. Specifically, this involves integrating language model prompts with structured text methodologies to create a structured prompt template. This template draws upon relation labels and ontology knowledge to generate synthetic training examples. The incorporation of external ontological knowledge enriches the semantic representation within the prompt template, enhancing its effectiveness. Further, a multiple triplets decoding (MTD) algorithm was developed to overcome the challenge of extracting multiple relation triplets from a sentence. To bridge the gap between knowledge and text, a collective training method was established to jointly optimize embedding representations. The proposed model is model-agnostic and can be applied to various PLMs. Exhaustive experiments on four public datasets with zero-shot settings were conducted to demonstrate the effectiveness of the proposed method. Compared to the baseline models, KBPT demonstrated enhancements of up to 14.65% and 24.19% in F1 score on the Wiki-ZSL and TACRED-Revisit datasets, respectively. Moreover, the proposed model achieved better performance compared with the current state-of-the-art (SOTA) model in terms of F1 score, precision-recall (P–R) curves and AUC. The code is available at https://Github.com/Phevos75/KBPT.

Granular3D: Delving into multi-granularity 3D scene graph prediction

This paper addresses the significant challenges in 3D Semantic Scene Graph (3DSSG) prediction, essential for understanding complex 3D environments. Traditional approaches, primarily using PointNet and Graph Convolutional Networks, struggle with effectively extracting multi-grained features from intricate 3D scenes, largely due to a focus on global scene processing and single-scale feature extraction. To overcome these limitations, we introduce Granular3D, a novel approach that shifts the focus towards multi-granularity analysis by predicting relation triplets from specific sub-scenes. One key is the Adaptive Instance Enveloping Method (AIEM), which establishes an approximate envelope structure around irregular instances, providing shape-adaptive local point cloud sampling, thereby comprehensively covering the contextual environments of instances. Moreover, Granular3D incorporates a Hierarchical Dual-Stage Network (HDSN), which differentiates and processes features of instances and their pairs at varying scales, leading to a targeted prediction of instance categories and their relationships. To advance the perception of sub-scene in HDSN, we design a Gather Point Transformer structure (GaPT) that enables the combinatorial interaction of local information from multiple point cloud sets, achieving a more comprehensive local contextual feature extraction. Extensive evaluations on the challenging 3DSSG benchmark demonstrate that our methods provide substantial improvements, establishing a new state-of-the-art in 3DSSG prediction, boosting the top-50 triplet accuracy by ＋2.8%.

A Span-based Multivariate Information-aware Embedding Network for joint relational triplet extraction of threat intelligence

The extraction of relational triplets in threat intelligence is a critical aspect of constructing a knowledge graph. However, the field encounters challenges like high semantic similarity among entities, limited relevance of entities, and a heavy reliance on experts, leading to low extraction efficiency. Currently, there is a lack of research on extracting threat intelligence relational triplets, which necessitates the development of an efficient extraction model. This study proposes a Span-based Multivariate Information-aware Embedding Network (SMIEN) for the joint extraction of threat intelligence relation triplets. SMIEN introduces aware embedding modules to capture fine-grained features of multivariate information, including semantic, temporal order, dependency, spans, span pairs, entity labels, and relational labels, and to enhance their interaction. Designed a Type-Aware Graph Convolutional Network (TA-GCN) to enhance the representation of key information between less relevant tokens in threat intelligence text sentences within the Dependency-Aware Embedding layer. The Entity-Aware Embedding module is designed to enhance fine-grained interaction between span information and other multivariate information, while the Relationship-Aware Embedding module is designed to determine correlations between span pairs and enhance interaction with relationship labels. Experimental results on the HACKER, RE-DNRTI, and RE-IVTI datasets demonstrate micro F1 scores of 44.73%, 81.74%, and 44.15%, respectively, highlighting the method’s effectiveness in extracting cyber threat intelligence and Internet of Vehicle (IoV) threat intelligence information. In addition, we have constructed a threat intelligence ontology (CVTIO) on which we have built a preliminary threat intelligence knowledge graph (CVTIKG) with 23,636 relational triplets. The CVTIKG can be found at https://github.com/wangxtz/CVTIKG.

A Novel Tensor Learning Model for Joint Relational Triplet Extraction.

The relational triplet is a format to represent relational facts in the real world, which consists of two entities and a semantic relation between these two entities. Since the relational triplet is the essential component in a knowledge graph (KG), extracting relational triplets from unstructured texts is vital for KG construction and has attached increasing research interest in recent years. In this work, we find that relation correlation is common in real life and could be beneficial for the relational triplet extraction task. However, existing relational triplet extraction works neglect to explore the relation correlation that bottlenecks the model performance. Therefore, to better explore and take advantage of the correlation among semantic relations, we innovatively utilize a three-dimension word relation tensor to describe relations between words in a sentence. Then, we treat the relation extraction task as a tensor learning problem and propose an end-to-end tensor learning model based on Tucker decomposition. Compared with directly capturing correlation among relations in a sentence, learning the correlation of elements in a three-dimension word relation tensor is more feasible and could be addressed through tensor learning methods. To verify the effectiveness of the proposed model, extensive experiments are also conducted on two widely used benchmark datasets, that is, NYT and WebNLG. Results show that our model outperforms the state-of-the-art by a large margin of F1 scores, such as the developed model has an improvement of 3.2% on the NYT dataset compared to the state-of-the-art. Source codes and data can be found at https://github.com/Sirius11311/TLRel.git.

SGTR+: End-to-End Scene Graph Generation With Transformer.

Scene Graph Generation (SGG) remains a challenging visual understanding task due to its compositional property. Most previous works adopt a bottom-up, two-stage or point-based, one-stage approach, which often suffers from high time complexity or suboptimal designs. In this paper, we propose a novel SGG method to address the aforementioned issues, formulating the task as a bipartite graph construction problem. To address the issues above, we create a transformer-based end-to-end framework to generate the entity and entity-aware predicate proposal set, and infer directed edges to form relation triplets. Moreover, we design a graph assembling module to infer the connectivity of the bipartite scene graph based on our entity-aware structure, enabling us to generate the scene graph in an end-to-end manner. Based on bipartite graph assembling paradigm, we further propose a new technical design to address the efficacy of entity-aware modeling and optimization stability of graph assembling. Equipped with the enhanced entity-aware design, our method achieves optimal performance and time-complexity. Extensive experimental results show that our design is able to achieve the state-of-the-art or comparable performance on three challenging benchmarks, surpassing most of the existing approaches and enjoying higher efficiency in inference.

Open-Vocabulary Video Relation Extraction

A comprehensive understanding of videos is inseparable from describing the action with its contextual action-object interactions. However, many current video understanding tasks prioritize general action classification and overlook the actors and relationships that shape the nature of the action, resulting in a superficial understanding of the action. Motivated by this, we introduce Open-vocabulary Video Relation Extraction (OVRE), a novel task that views action understanding through the lens of action-centric relation triplets. OVRE focuses on pairwise relations that take part in the action and describes these relation triplets with natural languages. Moreover, we curate the Moments-OVRE dataset, which comprises 180K videos with action-centric relation triplets, sourced from a multi-label action classification dataset. With Moments-OVRE, we further propose a cross-modal mapping model to generate relation triplets as a sequence. Finally, we benchmark existing cross-modal generation models on the new task of OVRE. Our code and dataset are available at https://github.com/Iriya99/OVRE.

An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction

In this paper, we propose a novel method for joint entity and relation extraction from unstructured text by framing it as a conditional sequence generation problem. In contrast to conventional generative information extraction models that are left-to-right token-level generators, our approach is \textit{span-based}. It generates a linearized graph where nodes represent text spans and edges represent relation triplets. Our method employs a transformer encoder-decoder architecture with pointing mechanism on a dynamic vocabulary of spans and relation types. Our model can capture the structural characteristics and boundaries of entities and relations through span representations while simultaneously grounding the generated output in the original text thanks to the pointing mechanism. Evaluation on benchmark datasets validates the effectiveness of our approach, demonstrating competitive results. Code is available at https://github.com/urchade/ATG.

An inter-modal attention-based deep learning framework using unified modality for multimodal fake news, hate speech and offensive language detection

Fake news, hate speech and offensive language are related evil triplets currently affecting modern societies. Text modality for the computational detection of these phenomena has been widely used. In recent times, multimodal studies in this direction are attracting a lot of interests because of the potentials offered by other modalities in contributing to the detection of these menaces. However, a major problem in multimodal content understanding is how to effectively model the complementarity of the different modalities due to their diverse characteristics and features. From a multimodal point of view, the three tasks have been studied mainly using image and text modalities. Improving the effectiveness of the diverse multimodal approaches is still an open research topic. In addition to the traditional text and image modalities, we consider image–texts which are rarely used in previous studies but which contain useful information for enhancing the effectiveness of a prediction model. In order to ease multimodal content understanding and enhance prediction, we leverage recent advances in computer vision and deep learning for these tasks. First, we unify the modalities by creating a text representation of the images and image–texts, in addition to the main text. Secondly, we propose a multi-layer deep neural network with inter-modal attention mechanism to model the complementarity among these modalities. We conduct extensive experiments involving three standard datasets covering the three tasks. Experimental results show that detection of fake news, hate speech and offensive language can benefit from this approach. Furthermore, we conduct robust ablation experiments to show the effectiveness of our approach. Our model predominantly outperforms prior works across the datasets.

Renewable Electrochemiluminescence Biosensor Based on Eu-MOGs as a Highly Efficient Emitter and a DNAzyme-Mediated Dual-drive DNA Walker as a Signal Amplifier for Ultrasensitive Detection of miRNA-222.

Herein, novel europium metal-organic gels (Eu-MOGs) with excellent cathode electrochemiluminescence (ECL) emission are first used to construct biosensors for the ultrasensitive detection of miRNA-222. Impressively, N and O elements of organic ligand 2,2':6,2″-terpyridine 4,4',4″-tricarboxylic acid (H3-tctpy) can perfectly coordinate with Eu3+ to form Eu-MOGs, which not only reduce nonradiative transition caused by the intramolecular free rotation of phenyl rings in other MOGs to enhance the ECL signal with extraordinary ECL efficiency as high as 37.2% (vs the [Ru(bpy)3]2+/S2O82- ECL system) but also reinforce ligand-to-metal charge transfer (LMCT) by the strong affinity between Eu3+ and N and O elements to greatly improve the stability of ECL signals. Besides, an improved nucleic acid cascade amplification reaction is developed to greatly raise the conversion efficiency from target miRNA-222 to a DNAzyme-mediated dual-drive DNA walker as output DNA, which can simultaneously shear the specific recognition sites from two directions. In that way, the proposed biosensor can further enhance the detection sensitivity of miRNA-222 with a linear range of 10 aM-1 nM and a detection limit (LOD) of 8.5 aM, which can also achieve an accurate response in cancer cell lysates of MHCC-97L and HeLa. Additionally, the biosensor can be self-regenerated by the folding/unfolding of related triplets with pH changes to simplify experimental operations and reduce the cost. Hence, this work proposed novel MOGs with stable and intense ECL signals for the construction of a renewable ECL biosensor, supplying a reliable detection method in biomarker analysis and disease diagnosis.

The Mathematical Formulas of 2-Cyclic Refined Duplets and Triplets

This work is dedicated to studying the problem of computing 2-cyclic refined neutrosophic duplets and triplets in the 2-cyclic refined neutrosophic ring of real numbers, where we present four different formulas that describe all possible duplets in this extended ring. Also, we present four different formulas for the computation of related triplets in the same ring.

MatSciRE: Leveraging pointer networks to automate entity and relation extraction for material science knowledge-base construction

Material science literature is a rich source of factual information about various categories of entities (like materials and compositions) and various relations between these entities, such as conductivity, voltage, etc. Automatically extracting this information to generate a material science knowledge base is a challenging task. In this paper, we propose MatSciRE (Material Science Relation Extractor), a Pointer Network-based encoder–decoder framework, to jointly extract entities and relations from material science articles as a triplet (entity1,relation,entity2). Specifically, we target the battery materials and identify five relations to work on — conductivity, coulombic efficiency, capacity, voltage, and energy. Our proposed approach achieved a much better F1-score (0.771) than a previous attempt using ChemDataExtractor (0.716). The overall graphical framework of MatSciRE is shown in Fig. 1. The material information is extracted from material science literature in the form of entity–relation triplets using MatSciRE.

A Novel Method for Constructing Spatiotemporal Knowledge Graph for Maritime Ship Activities

This study focused on the construction of a spatiotemporal knowledge graph for ship activities. First, a ship activity ontology model was proposed to describe the entities and relations of ship activities. Then, maritime event text data were utilized as the ship activity dataset, where entities and relations were extracted to form triplets. Thus, the data layer was populated, completing the construction of the ship activity spatiotemporal knowledge graph. The process of extracting triplets involved initially inputting the text sentences into the Bidirectional Encoder Representations from Transformers (BERT) model for pretraining to obtain vector representations of characters. These representations were then fed into a lattice long short-term memory network (Lattice-LSTM) for further processing. The resulting hidden vectors h1,h2,⋯,hn were input into the conditional random field (CRF) to perform named entity recognition. The recognized entities were then labeled in the original sentences and input into another BERT-Lattice-LSTM network. The resulting hidden vectors h′1,h′2,⋯,h′n were fed into a relation classifier, which output the relation between the two labeled entities, completing the extraction of entity–relation triplets. In experiments, the proposed method achieved triplet extraction performance exceeding 90% for three different evaluation metrics: Precision, Recall, and F1-measure.

Synthesis of Heavy-Atom-Free Triplet Photosensitizers Based on Organoboron Complexes

AbstractA new class of organoboron complexes have been developed as heavy-atom-free triplet photosensitizers. A methodology was developed for the synthesis of an indolocarbazole–imine boron difluoride (IIBD) dye and its dimer from a 6-formylindolocarbazole. The IIBD dye was then coupled with a BODIPY dye through the C-2 or C-8 position of the latter to synthesize two dyads. Both dyads showed superior photophysical properties to those of the IIBD dyes. The relative triplet conversion efficiencies of these dyes were determined by measuring their singlet-oxygen (1O2) generation capacities. All the synthesized dyes showed high 1O2 generation compared with the BODIPY dye PM567. The dyad linked through the C-2 position of the BODIPY core showed the highest 1O2 generation efficiency, which could be useful for photodynamic therapy of cancers.