Entity Pairs Research Articles

Extraction of entity relations from Chinese medical literature based on multi-scale CRNN.

BackgroundEntity relation extraction technology can be used to extract entities and relations from medical literature, and automatically establish professional mapping knowledge domains. The classical text classification model, convolutional neural networks for sentence classification (TEXTCNN), has been shown to have good classification performance, but also has a long-distance dependency problem, which is a common problem of convolutional neural networks (CNNs). Recurrent neural networks (RNN) address the long-distance dependency problem but cannot capture text features at a specific scale in the text.MethodsTo solve these problems, this study sought to establish a model with a multi-scale convolutional recurrent neural network for Sentence Classification (TEXTCRNN) to address the deficiencies in the 2 neural network structures. In entity relation extraction, the entity pair is generally composed of a subject and an object, but as the subject in the entity pair of medical literature is always omitted, it is difficult to use this coding method to obtain general entity position information. Thus, we proposed a new coding method to obtain entity position information to re-establish the relationship between subject and object and complete the entity relation extraction.ResultsBy comparing the benchmark neural network model and 2 typical multi-scale TEXTCRNN models, the TEXTCRNN [bidirectional long- and short-term memory (BiLSTM)] and TEXTCRNN [double-layer stacking gated recurrent unit (GRU)], the results showed that the multi-scale CRNN model had the best F1 value performance, and the TEXTCRNN (double-layer stacking GRU) was more capable of entity relation classification when the same entity word did not belong to the same entity relation.ConclusionsThe experimental results of the entity relation extraction from Pharmacopoeia of the People’s Republic of China—Guidelines for Clinical Drug Use—Volume of Chemical Drugs and Biological Products showed that entity relation extraction could effectively proceed using the new labeling method. Additionally, compared to typical neural network models, including the TEXTCNN, GRU, and BiLSTM, the multi-scale convolutional recurrent neural network structure had advantages across several evaluation indicators.

Generalized supervised meta-blocking

Entity Resolution is a core data integration task that relies on Blocking to scale to large datasets. Schema-agnostic blocking achieves very high recall, requires no domain knowledge and applies to data of any structuredness and schema heterogeneity. This comes at the cost of many irrelevant candidate pairs (i.e., comparisons), which can be significantly reduced by Meta-blocking techniques that leverage the entity co-occurrence patterns inside blocks: first, pairs of candidate entities are weighted in proportion to their matching likelihood, and then, pruning discards the pairs with the lowest scores. Supervised Meta-blocking goes beyond this approach by combining multiple scores per comparison into a feature vector that is fed to a binary classifier. By using probabilistic classifiers, Generalized Supervised Meta-blocking associates every pair of candidates with a score that can be used by any pruning algorithm. For higher effectiveness, new weighting schemes are examined as features. Through extensive experiments, we identify the best pruning algorithms, their optimal sets of features, as well as the minimum possible size of the training set.

An Entity Relationship Extraction Model Based on BERT-BLSTM-CRF for Food Safety Domain.

Dealing with food safety issues in time through online public opinion incidents can reduce the impact of incidents and protect human health effectively. Therefore, by the smart technology of extracting the entity relationship of public opinion events in the food field, the knowledge graph of the food safety field is constructed to discover the relationship between food safety issues. To solve the problem of multi-entity relationships in food safety incident sentences for few-shot learning, this paper adopts the pipeline-type extraction method. Entity relationship is extracted from Bidirectional Encoder Representation from Transformers (BERTs) joined Bidirectional Long Short-Term Memory (BLSTM), namely, the BERT-BLSTM network model. Based on the entity relationship types extracted from the BERT-BLSTM model and the introduction of Chinese character features, an entity pair extraction model based on the BERT-BLSTM-conditional random field (CRF) is established. In this paper, several common deep neural network models are compared with the BERT-BLSTM-CRF model with a food public opinion events dataset. Experimental results show that the precision of the entity relationship extraction model based on BERT-BLSTM-CRF is 3.29%∼23.25% higher than that of other models in the food public opinion events dataset, which verifies the validity and rationality of the model proposed in this paper.

Learning to classify relations between entities from noisy data - A meta instance reweighting approach

Relation classification between entities is a fundamental problem in knowledge extraction. It aims at determining if a semantic relation holds between a pair of entities based on textual descriptions. In general, the training data for each relation is limited. Distant supervision has thus been widely used to generate abundant weakly labeled data for the task. These data are noisy, containing many errors. A crucial problem is to select reliable instances for training or weigh them adequately. However, most existing approaches rely solely on statistics among the noisy data to generate classification features. They could be easily trapped by frequent wrong instances. To deal with this problem, we use a small set of manually annotated samples as reference data to guide the selection/weighting process. In this paper, we propose a new meta instance reweighting framework, which automatically adjusts the instance weights under the guidance of the reference data. The principle is that the classifier trained with the weighted data should perform well on the reference data. To further improve the approach, we propose to augment the limited reference data with a set of highly reliable instances — elite instances, selected from noisy data. The selection of elite instances is based on both their labeling quality and diversity so as to maximize the coverage of diverse expressions by the reference set. Extensive experiments on two public datasets by applying our method to two base neural models demonstrate the effectiveness of our method: It outperforms previous state-of-the-art approaches. We also show that the use of both reference data and elite instances is beneficial. The proposed approach is model-agnostic: it can be used on other classification models. It can also be applied to other classification tasks under distant supervision.

Enhance prototypical networks with hybrid attention and confusing loss function for few-shot relation classification

Relation classification (RC) is a fundamental task to building knowledge graphs and describing semantic formalization. It aims to classify a relation between the head and the tail entities in a sentence. The existing RC method mainly adopts the distant supervision (DS) scheme. However, DS still has the problem of long-tail and suffers from data sparsity. Recently, few-shot learning (FSL) has attracted people’s attention. It solves the long-tail problem by learning from few-shot samples. The prototypical networks have a better effect on FSL, which classifies a relation by distance. However, the prototypical networks and their related variants did not consider the critical role of entity words. In addition, not all sentences in support set equally contributed to classifying relations. Furthermore, an entity pair in a sentence may have true and confusing relations, which is difficult for the RC model to distinguish them. A new context encoder BERT_FE is proposed to address those problems, which uses the BERT model as pre-training and fuses the information of head and tail entities by entity word-level attention (WLA). At the same time, the sentence-level attention (SLA) is proposed to give more weight to sentences of the support set similar to the query instance and improve the classification accuracy. A confusing loss function (CLF) is designed to enhance the model’s ability to distinguish between true and confusing relations. The experiment results demonstrate that our proposed model (HACLF) is better than several baseline models.

Unsupervised Time Series Classification for Climate Data


 
 
 Outstanding success of Convolutional Neural Net- work image classification in the last few years in- fluenced application of this technique to a vari- ety of embeddable entities. CNN image classifica- tion methods are getting high accuracies but they are based on supervised machine learning that re- quires labeling of input data and do not help to understand unknown data. In this study we in- troduce unsupervised machine learning model that categorizes entity pairs to classes of similar and non similar pairs by converting pairs of entities to mirror vectors, transforming mirror vectors to Gramian Angular Fields (GAF) images and clas- sifying images using CNN transfer learning classi- fication. Based on climate data we demonstrated several scenarios that show when this model is re- liable for time series pair classification.
 
 

SLAN: Similarity-aware aggregation network for embedding out-of-knowledge-graph entities

Knowledge graph embedding acts as a pivotal role in predicting the missing information in knowledge graphs (KGs). Due to the evolving nature of real-world KGs, one requires the ability to make predictions for newly emerging entities besides those already observed at training time. Current studies have made great efforts to develop a neighborhood aggregator and embed out-of-knowledge-graph (OOKG) entities inductively, with less focus on exploiting the similarity between the existing and newly emerging entities. Attaching importance to such similarity helps facilitate semantic transfer. In this work, we propose a similarity-aware aggregation network for embedding out-of-knowledge-graph entities. Motivated by the fact that similar entities are likely to occur in common graph context, we skillfully design a similarity-aware function, which measures the distance of each entity pair based on the contextual gap. Moreover, we aggregate the neighborhood surrounding the target entity and its similarity information by query-specific attention weights, which are optimized during the learning process. Extensive experiments on knowledge graph completion task show that our method achieves substantial improvements over baselines.

Joint Extraction of Long-Distance Entity Relation by Aggregating Local- and Semantic-Dependent Features

The relations among long-distance entities are important for text understanding. Long-distance entities mean a pair of entities separated by more words in the text. Most current work of the joint extraction of entity relation is between entities that are close together. These methods are not suitable for long-distance entity relation extraction for they usually ignore the interaction between long-distance entities. In this paper, we propose Long-Distance Feature Aggregation Model (LDFAM), an end-to-end long-distance entity relation joint extraction model, which uses a Bi-GRU and weighted graph neural network to model text and learn its local- and semantic-dependent features. The long-distance dependence between entities is obtained by aggregating these two types of features. For entity recognition, we use conditional random fields to further obtain the dependency information between tags. For relation extraction, we fuse entity tag information for biaffine pairwise scoring. Experimental results on NYT and SemEval2010-Task8 datasets show that our model has achieved significant improvements compared to the baseline model. Finally, we apply the model to the field of material science to extract entity relationship knowledge from material literature. Experimental results show that benefiting from the long-distance entity relation extraction, the proposed method can get more useful knowledge from material literature and improve the knowledge coverage of material domain knowledge acquisition.

Solving Sensor Ontology Metamatching Problem with Compact Flower Pollination Algorithm

To implement co-operation among applications on the Internet of Things (IoT), we need to describe the meaning of diverse sensor data with the sensor ontology. However, there exists a heterogeneity issue among different sensor ontologies, which hampers their communications. Sensor ontology matching is a feasible solution to this problem, which is able to map the identical ontology entity pairs. This work investigates the sensor ontology meta-matching problem, which indirectly optimizes the sensor ontology alignment’s quality by tuning the weights to aggregate different ontology matchers. Due to the largescale entity and their complex semantic relationships, swarm intelligence (SI) based techniques are emerging as a popular approach to optimize the sensor ontology alignment. Inspired by the success of the flower pollination algorithm (FPA) in the IoT domain, this work further proposes a compact FPA (CFPA), which introduces the compact encoding mechanism to improve the algorithm’s efficiency, and on this basis, the compact exploration and exploitation operators are proposed, and an adaptive switching probability is presented to trade-off these two searching strategies. The experiment uses the ontology alignment evaluation initiative (OAEI)’s benchmark and the real sensor ontologies to test CFPA’s performance. The statistical comparisons show that CFPA significantly outperforms other state-of-the-art sensor ontology matching techniques.

Novel target attention convolutional neural network for relation classification

Relation classification (RC) is an essential task in natural language processing (NLP), which extracts relationships of entity pairs in sentences of text. In the paper, a novel target attention convolutional neural network (TACNN) is proposed for the RC by fully utilizing word embedding information and position embedding information. Simultaneously, a target attention mechanism (TAM) is applied into a context layer of the convolutional neural network (CNN) model, which increases the effect of the relationship matrix weights of two entities in the sentence, while ignoring the calculation of irrelevant terms. And the TACNN is essentially to modify the weight of the relationship matrix of entities in the sentence at the context layer and connect the relationship feature composed of the lexical layer feature with the target attention layer feature. Therefore, the TACNN simplifies the structure of the CNN and improves the computational efficiency. On SemEval-2010 Task 8 dataset and Conll04 dataset, the TACNN obtains 85.3% and 71.4% of the F1-score, respectively. In contrast to previously available public models, the TACNN achieves a state-of-the-art level in the F1-score of the RC.

A Keyword Detection and Context Filtering Method for Document Level Relation Extraction

Relation extraction (RE) is the core link of downstream tasks, such as information retrieval, question answering systems, and knowledge graphs. Most of the current mainstream RE technologies focus on the sentence-level corpus, which has great limitations in practical applications. Moreover, the previously proposed models based on graph neural networks or transformers try to obtain context features from the global text, ignoring the importance of local features. In practice, the relation between entity pairs can usually be inferred just through a few keywords. This paper proposes a keyword detection and context filtering method based on the Self-Attention mechanism for document-level RE. In addition, a Self-Attention Memory (SAM) module in ConvLSTM is introduced to process the document context and capture keyword features. By searching for word embeddings with high cross-attention of entity pairs, we update and record critical local features to enhance the performance of the final classification model. The experimental results on three benchmark datasets (DocRED, CDR, and GBA) show that our model achieves advanced performance within open and specialized domain relationship extraction tasks, with up to 0.87% F1 value improvement compared to the state-of-the-art methods. We have also designed experiments to demonstrate that our model can achieve superior results by its stronger contextual filtering capability compared to other methods.

Toward Entity Alignment in the Open World: An Unsupervised Approach with Confidence Modeling

Entity alignment (EA) aims to discover the equivalent entities in different knowledge graphs (KGs). It is a pivotal step for integrating KGs to increase knowledge coverage and quality. Recent years have witnessed a rapid increase of EA frameworks. However, state-of-the-art solutions tend to rely on labeled data for model training. Additionally, they work under the closed-domain setting and cannot deal with entities that are unmatchable. To address these deficiencies, we offer an unsupervised framework UEA that performs entity alignment in the open world. Specifically, we first mine useful features from the side information of KGs. Then, we devise an unmatchable entity prediction module to filter out unmatchable entities and produce preliminary alignment results. These preliminary results are regarded as the pseudo-labeled data and forwarded to the progressive learning framework to generate structural representations, which are integrated with the side information to provide a more comprehensive view for alignment. Finally, the progressive learning framework gradually improves the quality of structural embeddings and enhances the alignment performance. Furthermore, noticing that the pseudo-labeled data are of various qualities, we introduce the concept of confidence to measure the probability of an entity pair of being true and develop a confidence-based unsupervised EA framework CUEA. Our solutions do not require labeled data and can effectively filter out unmatchable entities. Comprehensive experimental evaluations validate the superiority of our proposals .

A better index for analysis of co-occurrence and similarity.

Scientists often need to know whether pairs of entities tend to occur together or independently. Standard approaches to this issue use co-occurrence indices such as Jaccard, Sørensen-Dice, and Simpson. We show that these indices are sensitive to the prevalences of the entities they describe and that this invalidates their interpretability. We propose an index, α, that is insensitive to prevalences. Published datasets reanalyzed with both α and Jaccard's index (J) yield profoundly different biological inferences. For example, a published analysis using J contradicted predictions of the island biogeography theory finding that community stability increased with increasing physical isolation. Reanalysis of the same dataset with the estimator [Formula: see text] reversed that result and supported theoretical predictions. We found similarly marked effects in reanalyses of antibiotic cross-resistance and human disease biomarkers. Our index α is not merely an improvement; its use changes data interpretation in fundamental ways.

Coarse-to-Fine Entity Alignment for Chinese Heterogeneous Encyclopedia Knowledge Base

Entity alignment (EA) aims to automatically determine whether an entity pair in different knowledge bases or knowledge graphs refer to the same entity in reality. Inspired by human cognitive mechanisms, we propose a coarse-to-fine entity alignment model (called CFEA) consisting of three stages: coarse-grained, middle-grained, and fine-grained. In the coarse-grained stage, a pruning strategy based on the restriction of entity types is adopted to reduce the number of candidate matching entities. The goal of this stage is to filter out pairs of entities that are clearly not the same entity. In the middle-grained stage, we calculate the similarity of entity pairs through some key attribute values and matched attribute values, the goal of which is to identify the entity pairs that are obviously not the same entity or are obviously the same entity. After this step, the number of candidate entity pairs is further reduced. In the fine-grained stage, contextual information, such as abstract and description text, is considered, and topic modeling is carried out to achieve more accurate matching. The basic idea of this stage is to use more information to help judge entity pairs that are difficult to distinguish using basic information from the first two stages. The experimental results on real-world datasets verify the effectiveness of our model compared with baselines.

A mutually-exclusive binary cross tagging framework for joint extraction of entities and relations.

Joint extraction from unstructured text aims to extract relational triples composed of entity pairs and their relations. However, most existing works fail to process the overlapping issues that occur when the same entities are utilized to generate different relational triples in a sentence. In this work, we propose a mutually exclusive Binary Cross Tagging (BCT) scheme and develop the end-to-end BCT framework to jointly extract overlapping entities and triples. Each token of entities is assigned a mutually exclusive binary tag, and then these tags are cross-matched in all tag sequences to form triples. Our method is compared with other state-of-the-art models in two English public datasets and a large-scale Chinese dataset. Experiments show that our proposed framework achieves encouraging performance in F1 scores for the three datasets investigated. Further detailed analysis demonstrates that our method achieves strong performance overall with three overlapping patterns, especially when the overlapping problem becomes complex.

Spontaneous Resolution of Racemic Cage-Catenanes via Diastereomeric Enrichment at the Molecular Level and Subsequent Narcissistic Self-Sorting at the Supramolecular Level.

The spontaneous resolution of racemates, from natural compounds to artificial structures, has long been pursued to shed light on the origin of homochirality in life. Even though diverse synthetic systems have been elegantly devised to elaborate the underlying principles of spontaneous symmetry breaking, their complexity is still unparalleled to the natural masterpieces including DNA helix and proteins, which convey remarkable coalescence at both molecular and supramolecular levels. Here, we report on the spontaneous resolution of a pair of homochiral entities from a racemic mixture of a triply interlocked cage-catenane comprising 720 possible stereoisomers. This cage-catenane comprises six methyldithiane ring-containing linkers (denoted rac-2). As each methyldithiane ring has two chiral centers, it exhibits four possible diastereomers. These otherwise equimolar diastereomers are preferentially differentiated with the equatorial conformers over their axial analogues, leading to the dominant formation of (S, R)-2 and (R, S)-2, i.e., diastereomeric enrichment at the molecular level. This diastereomeric enrichment is unbiasedly transferred from precursor rac-2 to cage-catenane rac-4, from which a pair of homochirals (S, R)6-4 and (R, S)6-4 is narcissistically self-sorted upon crystallization. This powerful symmetry breaking is attributed to a supramolecular synergy of directional π-π stacking with the multivalency of erstwhile weak S···S contacts (with an unusual distance of 3.09 Å) that are cooperatively arranged in a helical fashion. This work highlights the attainability of complex homochiral entities by resorting to coalesced covalent and noncovalent contributions and therefore provides additional clues to the symmetry breaking of sophisticated yet well-defined architectures.

MiDTD: A Simple and Effective Distillation Framework for Distantly Supervised Relation Extraction

Relation extraction (RE), an important information extraction task, faced the great challenge brought by limited annotation data. To this end, distant supervision was proposed to automatically label RE data, and thus largely increased the number of annotated instances. Unfortunately, lots of noise relation annotations brought by automatic labeling become a new obstacle. Some recent studies have shown that the teacher-student framework of knowledge distillation can alleviate the interference of noise relation annotations via label softening. Nevertheless, we find that they still suffer from two problems: propagation of inaccurate dark knowledge and constraint of a unified distillation temperature . In this article, we propose a simple and effective Multi-instance Dynamic Temperature Distillation (MiDTD) framework, which is model-agnostic and mainly involves two modules: multi-instance target fusion (MiTF) and dynamic temperature regulation (DTR). MiTF combines the teacher’s predictions for multiple sentences with the same entity pair to amend the inaccurate dark knowledge in each student’s target. DTR allocates alterable distillation temperatures to different training instances to enable the softness of most student’s targets to be regulated to a moderate range. In experiments, we construct three concrete MiDTD instantiations with BERT, PCNN, and BiLSTM-based RE models, and the distilled students significantly outperform their teachers and the state-of-the-art (SOTA) methods.

Modelling Multi-relations for Convolutional-based Knowledge Graph Embedding

Representation learning of knowledge graphs aims to embed entities and relations into low-dimensional vectors. Most existing works only consider the direct relations or paths between an entity pair. It is considered that such approaches disconnect the semantic connection of multi-relations between an entity pair, and we propose a convolutional and multi-relational representation learning model, ConvMR. The proposed ConvMR model addresses the multi-relation issue in two aspects: (1) Encoding the multi-relations between an entity pair into a unified vector that maintains the semantic connection. (2) Since not all relations are necessary while joining multi-relations, we propose an attention-based relation encoder to automatically assign weights to different relations based on semantic hierarchy. Experimental results on two popular datasets, FB15k-237 and WN18RR, achieved consistent improvements on the mean rank. We also found that ConvMR is efficient to deal with less frequent entities.

Powerful multiple testing of paired null hypotheses using a latent graph model

In this paper, we explore the multiple testing problem of paired null hypotheses, for which the data are collected on pairs of entities and tests have to be performed for each pair. Typically, for each pair (i,j), we observe some interaction/association score between i and j and the aim is to detect the pairs with a significant score. In this setting, it is natural to assume that the true/false null constellation is structured according to an unobserved graph, where present edges correspond to a significant association score. The point of this work is to build an improved multiple testing decision by learning the graph structure. Our approach is in line with the seminal work of Sun and Cai [46], that uses the hidden Markov model to structure the dependencies between null hypotheses. Here, we adapt this strategy by considering the stochastic block model for the latent graph. Under appropriate assumptions, the new proposed procedure is shown to control the false discovery rate, up to remainder terms that vanish when the size of the number of hypotheses increases. The procedure is also shown to be nearly optimal in the sense that it is close to the procedure maximizing the true discovery rate. Numerical experiments reveal that our method outperforms state-of-the-art methods and is robust to model mis-specification. Finally, the applicability of the new method is demonstrated on data concerning the usage of self-service bicycles in London.

Application of cascade binary pointer tagging in joint entity and relation extraction of Chinese medical text.

Extracting relational triples from unstructured medical texts can provide a basis for the construction of large-scale medical knowledge graphs. The cascade binary pointer tagging network (CBPTN) shows excellent performance in the joint entity and relation extraction, so we try to explore its effectiveness in the joint entity and relation extraction of Chinese medical texts. In this paper, we propose two models based on the CBPTN: CBPTN with conditional layer normalization (Cas-CLN) and biaffine transformation-based CBPTN with multi-head selection (BTCAMS). Cas-CLN uses the CBPTN to decode the head entity and relation-tail entity successively and utilizes conditional layer normalization to enhance the connection between the two steps. BTCAMS detects all possible entities in a sentence by using the CBPTN and then determines the relation between each entity pair through biaffine transformation. We test the performance of the two models on two Chinese medical datasets: CMeIE and CEMRDS. The experimental results prove the effectiveness of the two models. Compared with the baseline CasREL, the F1 value of Cas-CLN and BTCAMS on the test data of CMeIE improved by 1.01 and 2.13%; on the test data of CEMRDS, the F1 value improved by 1.99 and 0.68%.