Entity Pairs Research Articles

Syntax-aware entity representations for neural relation extraction

Distantly supervised relation extraction has been widely used to find novel relational facts between entities from text, and can be easily scaled to very large corpora. Previous studies on neural relation extraction treat this task as a multi-instance learning problem, and encode the sentences in low-dimensional spaces via neural networks. Although great progress has been made, they seldom consider the information represented by entities, which are of great significance to relation extraction. In this article, we propose several methods based on different tree-based models to learn syntax-aware entity representations for neural relation extraction. First, we encode the context of entities on dependency trees as sentence-level entity embedding based on tree-structured neural network models. Then, we utilize inter-sentence attention mechanism to obtain sentence bag level entity embedding over all sentences containing the specified entity pair. Finally, we combine both sentence embedding and entity embedding for relation classification. Experimental results on a widely used real-world dataset indicate that our system performs better than the state-of-the-art systems of relation extraction.

SRX: efficient management of spatial RDF data

We present a general encoding scheme for the efficient management of spatial RDF data. The scheme approximates the geometries of the RDF entities inside their (integer) IDs and can be used, along with several operators and optimizations we introduce, to accelerate queries with spatial predicates and to re-encode entities dynamically in case of updates. We implement our ideas in SRX, a system built on top of the popular RDF-3X system. SRX extends RDF-3X with support for three types of spatial queries: range selections (e.g., find entities within a given polygon), spatial joins (e.g., find pairs of entities whose locations are close to each other), and spatial k-nearest neighbors (e.g., find the three closest entities from a given location). We evaluate SRX on spatial queries and updates with real RDF data, and we also compare its performance with the latest versions of three popular RDF stores. The results show SRX ’s superior performance over the competitors; compared to RDF-3X, SRX improves its performance for queries with spatial predicates while incurring little overhead during updates.

Neural Relation Extraction within and across Sentence Boundaries

Past work in relation extraction mostly focuses on binary relation between entity pairs within single sentence. Recently, the NLP community has gained interest in relation extraction in entity pairs spanning multiple sentences. In this paper, we propose a novel architecture for this task: inter-sentential dependency-based neural networks (iDepNN). iDepNN models the shortest and augmented dependency paths via recurrent and recursive neural networks to extract relationships within (intra-) and across (inter-) sentence boundaries. Compared to SVM and neural network baselines, iDepNN is more robust to false positives in relationships spanning sentences. We evaluate our models on four datasets from newswire (MUC6) and medical (BioNLP shared task) domains that achieve state-of-the-art performance and show a better balance in precision and recall for inter-sentential relationships. We perform better than 11 teams participating in the BioNLP shared task 2016 and achieve a gain of 5.2% (0.587 vs 0.558) in F1 over the winning team. We also release the crosssentence annotations for MUC6.

Learning to Transfer Relational Representations through Analogy

We propose a novel approach to learn representations of relations expressed by their textual mentions. In our assumption, if two pairs of entities belong to the same relation, then those two pairs are analogous. We collect a large set of analogous pairs by matching triples in knowledge bases with web-scale corpora through distant supervision. This dataset is adopted to train a hierarchical siamese network in order to learn entity-entity embeddings which encode relational information through the different linguistic paraphrasing expressing the same relation. The model can be used to generate pre-trained embeddings which provide a valuable signal when integrated into an existing neural-based model by outperforming the state-of-the-art methods on a relation extraction task.

Improving Distantly Supervised Relation Extraction with Neural Noise Converter and Conditional Optimal Selector

Distant supervised relation extraction has been successfully applied to large corpus with thousands of relations. However, the inevitable wrong labeling problem by distant supervision will hurt the performance of relation extraction. In this paper, we propose a method with neural noise converter to alleviate the impact of noisy data, and a conditional optimal selector to make proper prediction. Our noise converter learns the structured transition matrix on logit level and captures the property of distant supervised relation extraction dataset. The conditional optimal selector on the other hand helps to make proper prediction decision of an entity pair even if the group of sentences is overwhelmed by no-relation sentences. We conduct experiments on a widely used dataset and the results show significant improvement over competitive baseline methods.

Semantic Relation Classification via Bidirectional LSTM Networks with Entity-Aware Attention Using Latent Entity Typing

Classifying semantic relations between entity pairs in sentences is an important task in natural language processing (NLP). Most previous models applied to relation classification rely on high-level lexical and syntactic features obtained by NLP tools such as WordNet, the dependency parser, part-of-speech (POS) tagger, and named entity recognizers (NER). In addition, state-of-the-art neural models based on attention mechanisms do not fully utilize information related to the entity, which may be the most crucial feature for relation classification. To address these issues, we propose a novel end-to-end recurrent neural model that incorporates an entity-aware attention mechanism with a latent entity typing (LET) method. Our model not only effectively utilizes entities and their latent types as features, but also builds word representations by applying self-attention based on symmetrical similarity of a sentence itself. Moreover, the model is interpretable by visualizing applied attention mechanisms. Experimental results obtained with the SemEval-2010 Task 8 dataset, which is one of the most popular relation classification tasks, demonstrate that our model outperforms existing state-of-the-art models without any high-level features.

Development of a Consumer Health Vocabulary by Mining Health Forum Texts Based on Word Embedding: Semiautomatic Approach.

BackgroundThe vocabulary gap between consumers and professionals in the medical domain hinders information seeking and communication. Consumer health vocabularies have been developed to aid such informatics applications. This purpose is best served if the vocabulary evolves with consumers’ language.ObjectiveOur objective is to develop a method for identifying and adding new terms to consumer health vocabularies, so that it can keep up with the constantly evolving medical knowledge and language use.MethodsIn this paper, we propose a consumer health term–finding framework based on a distributed word vector space model. We first learned word vectors from a large-scale text corpus and then adopted a supervised method with existing consumer health vocabularies for learning vector representation of words, which can provide additional supervised fine tuning after unsupervised word embedding learning. With a fine-tuned word vector space, we identified pairs of professional terms and their consumer variants by their semantic distance in the vector space. A subsequent manual review of the extracted and labeled pairs of entities was conducted to validate the results generated by the proposed approach. The results were evaluated using mean reciprocal rank (MRR).ResultsManual evaluation showed that it is feasible to identify alternative medical concepts by using professional or consumer concepts as queries in the word vector space without fine tuning, but the results are more promising in the final fine-tuned word vector space. The MRR values indicated that on an average, a professional or consumer concept is about 14th closest to its counterpart in the word vector space without fine tuning, and the MRR in the final fine-tuned word vector space is 8. Furthermore, the results demonstrate that our method can collect abbreviations and common typos frequently used by consumers.ConclusionsBy integrating a large amount of text information and existing consumer health vocabularies, our method outperformed several baseline ranking methods and is effective for generating a list of candidate terms for human review during consumer health vocabulary development.

Graph-based clustering of extracted paraphrases for labelling crime reports

Paraphrases are well-known as synonyms that express the same context in different articulations. Extracting paraphrases from a large text corpus is a challenging task in Natural Language Processing applications. The present work proposes a graph based clustering technique for discovering labels of crime reports based on extracted paraphrases from large untagged crime corpora. Initially, the entity pairs are represented as shallow parse trees where the headword in each tree reflects the actual meaning of the phrase between the entities. Though the phrases having similar headwords have been collected together, there exist many phrases between the entities that express similar context without sharing the same headword. Therefore, clustering is done to create a group of similar meaning phrases termed as paraphrases. A complete weighted graph is constructed with the phrases as nodes and cosine similarity between pair of phrases as the weight of an edge with the phrases as terminal nodes. The graph is made sparse by removing edges with weights less than a threshold value and clustering coefficient has been calculated for each node. The subgraph(s) comprising node(s) with the highest clustering coefficient has been extracted with their adjacent edges. The remaining nodes with their adjacent edges in the graph are added one at a time to an extracted subgraph, if and only if the average clustering coefficient of the resultant subgraph increases and an agglomerative merging technique is applied to merge the extracted subgraphs until no merging takes place. Finally, each subgraph represents a cluster of phrases, yields one aspect of crime. Based on the extracted paraphrases, the reports can be easily labelled. The proposed work deals with crime reports for United States of America (USA), United Arab Emirates (UAE) and India and the evaluation is performed in terms of various supervised and unsupervised techniques.

A framework for crime data analysis using relationship among named entities

Many crime reports are available online in various blogs and Newswire. Though manual annotation of these massive reports is quite tedious for crime data analysis, it gives an overall crime scenario of all over the world. This motivates us to propose a framework for crime data analysis based on the online reports. Initially, the method extracts the crime reports and identifies named entities. The intermediate sequence of context words between every consecutive pair of named entities is termed as a crime vector that provides relationships between the entities. The feature vectors for each entity pair are generated from these crime vectors using the Word2Vec model. The paper considers three different types of named entity pairs to facilitate the major crime data analysis task, and for each type, similarity between every pair of entities is measured using respective feature vectors. For each type of named entity pair, a separate weighted graph is generated with entity pairs as vertices and similarity score between them as the weight of the corresponding edge. Then, Infomap, a graph-based clustering algorithm, is applied to obtain optimal set of clusters of entity pairs and a representative entity pair of each cluster. Each cluster is labelled by the relationship, represented by the crime vector, of its representative entity pair. In reality, all the entity pairs in a cluster may not reflect contextual similarity with their representative entity pair. So the clusters are further partitioned into subclusters based on WordNet-based path similarity measure which makes the entity pairs in each subcluster more contextually similar compared to their original cluster. These subclusters provide us various statistical crime information over the time period. The method is experimented only using the crime reports related to crime against women in India. The experimental results demonstrate the effectiveness and superiority of the method compared to others for analysing the crime data.

Information-Theoretic Remodularization of Object-Oriented Software Systems

Software remodularization consists in reorganizing software entities into modules such that pairs of entities belonging to the same modules are more similar than those belonging to different modules. In recent years, Search-Based Software Engineering (SBSE) approach has gained unprecedented growth for solving software remodularization problem. Most of the previous studies remodularize the software system by optimizing the structural coupling and cohesion metrics as objective functions. These metrics are defined in terms of the number of structural relationships counts, rather than taking patterns of relationships. It has been observed that the computation of coupling and cohesion based on patterns of relationships (i.e., information-theory based) are more accurate than the number of relationships. This paper proposes an information-theoretic software remodularization where an entropy-based similarity measure is introduced as an objective function along with other objective functions i.e., inter-module class change coupling, intra-module class change coupling, module size index (MSI), and module count index (MCI) and is further optimized using many-objective meta-heuristic algorithm. To evaluate the effectiveness of the proposed approach, seven object-oriented software systems have been remodularized using NSGA-III, MOEA/D, IBEA, and TAA algorithms. The results are compared with existing multi-objective formulation of remodularization problem in terms of authoritative software remodularization, non-extreme distribution, and stability. The experimentation results suggest that the proposed approach can be a good alternative to improve the quality of software systems. The findings suggest that the approach is more suitable for generating remodularization solution good from both quality metrics and developers perspective.

Design of a Secure Password-Based Authentication Scheme for M2M Networks in IoT Enabled Cyber-Physical Systems

The Internet of Things (IoT) forms a foundation for cyber-physical systems. We propose an efficient and secure authentication scheme for machine-to-machine (M2M) networks in IoT enabled cyber-physical systems. Smart objects and smart devices over CPS are capable of capturing a variety of multimedia contents; interact with each other and also with the physical world in a fully automatic manner without human interference. The proposed scheme allows any pair of entities in an M2M network to mutually authenticate each other and agree on a session key for communicating data in a secure and efficient way. The authentication process does not incorporate the M2M service provider, and hence eliminates the burden of managing the authentication of massive scale devices at the edge of the network. The burden of the authentication process is offloaded and distributed on the gateways under the authority of this M2M service provider. The proposed scheme requires the mobile user to hold only one secret key provided by the M2M service provider, by which, he can roam randomly in the M2M network and authenticate to any of the gateways in the domain. Then, this authenticated gateway allows the mobile user to authenticate with any sensor node in the domain. In the proposed scheme, the authentication process does not rely on any public key cryptographic operations. Authentication is achieved using very few hash invocations and symmetric key encryptions. Therefore, the scheme is suitable for environmental sensors which are limited in resources (computation, storage, and energy). We analyze the security of the proposed scheme using BAN logic, which is widely accepted as a framework for the assessment of authentication protocols and also using ProVerif. We assess the efficiency of the proposed scheme and compare with some recently proposed schemes.

Formal Methods Based Security for Cloud-based Manufacturing Cyber Physical System

With the advent of the Internet of Things (IoT), Cloud-based Cyber-Physical Systems (C2PS) are becoming more prevalent in manufacturing systems. Verifying the security of C2PS is a challenging task due to the inclusion of newer entities such as mobile phones and IoT devices. Therefore, scalable methods for verifying security properties of manufacturing systems are paramount. This paper presents an approach to guarantee security of C2PS using concepts from Communicating Sequential Processes (CSP) to describe the security properties of cloud constructs. We prove that composability is an essential property for enhancing the security of the C2PS. Furthermore, it also provides good scalability to the system. We propose formalisms required for modelling security properties in the Industrial IoT-based manufacturing system. Using this formalism and Hoares’ logic we provide conditions for composing systems. Finally, we provide strong authentication conditions among entity pairs in C2PS based manufacturing systems. We demonstrate the message exchanges, methodology, and different steps required for verifying C2PS.

A Graph Based Clustering Approach for Relation Extraction From Crime Data

Application of natural language processing techniques based on crime data can prove to be beneficial in several processes of the criminal justice industry. The availability of massive crime reports helps law enforcement agencies when a criminal investigation is launched. While investigating a crime, questions like what type of crime, who committed the crime, what happened at which place, on what time, and what actions are taken, keep arising. Now, it is not feasible for the law enforcement agencies to get into the detail of these available massive crime reports and get the answers. To tackle these problems associated with criminal justice industry, the proposed work considers a textual corpus containing information of crime against women in India and extracts substantial relations between the named entities present in the corpus by a hierarchical graph-based clustering technique. For extracting the relations, different types of entity pairs have been chosen and similarities among them have been measured based on the intermediate context words. Depending on the similarity score, a weighted graph has been formed and a similarity threshold is set to partition the graph based on the edge weights. With the iterative application of the clustering algorithm, all the named entity pairs are grouped into clusters, each of which signifies different crime aspects. Each cluster is characterized using the most frequent context word present in it. The proposed relation extraction scheme helps in crime pattern analysis that can aid in various criminal investigation requirements. The results with optimal cluster validation indices depict the effectiveness of this method.

Reinforcement Learning for Distantly Supervised Relation Extraction

Relation extraction is a task of identifying semantic relations between entity pairs from plain text, which can benefit a lot of AI applications such as knowledge base construction and answer questioning. The distant supervision strategy is introduced to automatically create large-scale training data, which inevitably suffers from noisy label problem. Recent works handle the sentence-level denoising by reinforcement learning, which regards the labels from distant supervision as the ground-truth. However, few works focus on the label-level denoising that corrects noisy labels directly. In this paper, we propose a reinforcement learning-based label denoising method for distantly supervised relation extraction. The model consists of two modules: extraction network (ENet) and policy network (PNet). The core of our label denoising is designing a policy in the PNet to obtain latent labels, where we can select the actions of using the distantly supervised labels or the predicted labels from the ENet. More concretely, the task can be modeled as an iterative process. First, the ENet predicts the relation probability, through which the model generates state representation. Second, the PNet learns the latent labels with taken actions and uses them to update the ENet. Then the optimized ENet gives the rewards to the PNet. The joint learning of two modules can obtain a reliable latent label and effectively improve the classification performance. The experimental results show that reinforcement learning is effective for noisy label correction and the proposed method can outperform the state-of-the-art relation extraction systems.

Knowledge Base Reasoning with Convolutional-based Recurrent Neural Networks

Recurrent neural network(RNN) has achieved remarkable performances in complex reasoning on knowledge bases, which usually takes as inputs vector embeddings of relations along a path between an entity pair. However, it is insufficient to extract local correlations of a path due to RNN is better at capturing global sequential information of a path. In this paper, we take full advantages of convolutional neural network that can effectively extract local features, and propose a convolutional-based RNN architecture denoted as C-RNN to perform reasoning. C-RNN first utilizes CNN to extract local high-level correlation features of a path, and then feeds the correlation features into recurrent neural network to model the path representation. Our C-RNN architecture is adaptable to obtain not only local features but also global sequential features of a path. Based on C-RNN architecture, we devise two models, the unidirectional C-RNN and bidirectional C-RNN. We empirically evaluate them on a large-scale FreeBase+ClueWeb prediction task. Experimental results show that C-RNN models achieve state-of-the-art predictive performance.

Distant Supervision for Relation Extraction via Piecewise Attention and Bag-Level Contextual Inference

Distant supervision (DS) has become an efficient approach for relation extraction (RE) to alleviate the lack of labeled examples in supervised learning. In this paper, we propose a novel neural RE model that combines a bidirectional gated recurrent unit model with a form of hierarchical attention that is better suited to RE. We demonstrate that an additional attention mechanism called piecewise attention, which builds itself upon segment level representations, significantly enhances the performance of the distantly supervised relation extraction task. Our piecewise attention mechanism not only captures crucial segments in each sentence but also reflects the direction of relations between two entities. Furthermore, we propose a contextual inference method that can infer the most likely positive examples of an entity pair in bags with very limited contextual information. In addition, we provide an annotated dataset without false positive examples based on the Riedel testing dataset, and report on the actual performance of several RE models. The experimental results show that our proposed methods outperform the previous state-of-the-art baselines on both original and annotated datasets for the distantly supervised RE task.

Learning representations of Web entities for entity resolution

PurposeMatching instances of the same entity, a task known as entity resolution, is a key step in the process of data integration. This paper aims to propose a deep learning network that learns different representations of Web entities for entity resolution.Design/methodology/approachTo match Web entities, the proposed network learns the following representations of entities: embeddings, which are vector representations of the words in the entities in a low-dimensional space; convolutional vectors from a convolutional layer, which capture short-distance patterns in word sequences in the entities; and bag-of-word vectors, created by a bow layer that learns weights for words in the vocabulary based on the task at hand. Given a pair of entities, the similarity between their learned representations is used as a feature to a binary classifier that identifies a possible match. In addition to those features, the classifier also uses a modification of inverse document frequency for pairs, which identifies discriminative words in pairs of entities.FindingsThe proposed approach was evaluated in two commercial and two academic entity resolution benchmarking data sets. The results have shown that the proposed strategy outperforms previous approaches in the commercial data sets, which are more challenging, and have similar results to its competitors in the academic data sets.Originality/valueNo previous work has used a single deep learning framework to learn different representations of Web entities for entity resolution.

Pitfalls of Differential Diagnosis in Dermatologic Surgery

Evaluation of skin diseases can be challenging for non-dermatologists. Even obvious well-characterized skin pathologies might be misleading and thus treatment can fail. Particularly the differentiation of surgical treated entities is important, for example the management of a wound healing disturbance profoundly differs from that of a pyoderma gangrenosum. This article outlines several easily mistaken pairs of dermatologic entities on one hand and surgical on the other. For example, a livedo vasculopathy can be confused with a leg ulcer, a nail melanoma with a simple hematoma and finally a hidradenitis suppurativa with an axillary abscess. Typical clinical signs and anamnestic data may often lead to the right diagnosis also assisted by the simple fact to "keep it in mind".

Combining entity co-occurrence information and sentence semantic features for relation extraction

Relation extraction is one of the most important tasks in information extraction and a key step in knowledge graph construction. The existing relation extraction approaches mostly try to capture semantic features for entity pairs at the sentence level, which might ignore the global context information of the entities in the entire corpus. In this paper, we propose a novel neural network model for relation extraction, named CNSSNN, which combines the information of entity co-occurrences with sentences semantic features. In this model, we first build an entity co-occurrence network from the corpus. Then, we introduce a network-level attention mechanism to capture network environmental information selectively and generate the corpus-level global context features for the entities. At the same time, we employ a bi-directional gated recurrent unit (bi-GRU) network to extract sentence-level semantic features for entity pairs. Finally, we combine the corpus-level features and the sentence-level features to classify relations. The experimental results, over a manually labeled dataset, show that our approach consistently outperforms other existing approaches in terms of both precision and recall.

Encoding implicit relation requirements for relation extraction: A joint inference approach

Relation extraction is the task of identifying predefined relationship between entities, and plays an essential role in information extraction, knowledge base construction, question answering and so on. Most existing relation extractors make predictions for each entity pair locally and individually, while ignoring implicit global clues available across different entity pairs and in the knowledge base, which often leads to conflicts among local predictions from different entity pairs. This paper proposes a joint inference framework that employs such global clues to resolve disagreements among local predictions. We exploit two kinds of clues to generate constraints which can capture the implicit type and cardinality requirements of a relation. Those constraints can be examined in either hard style or soft style, both of which can be effectively explored in an integer linear program formulation. Experimental results on both English and Chinese datasets show that our proposed framework can effectively utilize those two categories of global clues and resolve the disagreements among local predictions, thus improve various relation extractors when such clues are applicable to the datasets. Our experiments also indicate that the clues learnt automatically from existing knowledge bases perform comparably to or better than those refined by human.