Piecewise Convolutional Neural Network Research Articles

A Construction of Knowledge Graph for Semiconductor Industry Chain Based on Lattice-LSTM and PCNN Models

<p>This paper mainly focuses on building the knowledge graph of semiconductor industry chain. The main research contents include knowledge extraction, knowledge storage, and construction of knowledge graph in semiconductor field. The crawler technology and character recognition technology are used to obtain semiconductor industry chain information from the Internet, magazines, and institutions to establish the original data set. Then, Lattice Long Short-Term Memory (Lattice-LSTM) model is used to implement the entity extraction and recognition. The piecewise convolutional neural network (PCNN) model based on the sentence-level attention mechanism is used to extract relationships and obtain entity triples. The semiconductor dictionary library is constructed through the obtained structured data. The dictionary library and Chinese natural language toolkit HanLP are combined to annotate unstructured text data for knowledge extraction. Neo4j graph database is used to store the extracted data of semiconductor industry chain. Finally, Spring Boot and Vue technology are used to create a knowledge graph system.</p> <p> </p>

Improving Drug-Drug Interaction Extraction with Gaussian Noise.

Drug-Drug Interactions (DDIs) produce essential and valuable insights for healthcare professionals, since they provide data on the impact of concurrent administration of medications to patients during therapy. In that sense, some relevant works, related to the DDIExtraction2013 Challenge, are available in the current technical literature. This study aims to improve previous results, using two models, where a Gaussian noise layer is added to achieve better DDI relationship extraction. (1) A Piecewise Convolutional Neural Network (PW-CNN) model is used to capture relationships among pharmacological entities described in biomedical databases. Additionally, the model incorporates multichannel words to enrich a person's vocabulary and reduce unfamiliar words. (2) The model uses the pre-trained BERT language model to classify relationships, while also integrating data from the target entities. After identifying the target entities, the model transfers the relevant information through the pre-trained architecture and integrates the encoded data for both entities. The results of the experiment show an improved performance, with respect to previous models.

Semantic piecewise convolutional neural network with adaptive negative training for distantly supervised relation extraction

Distantly Supervised Relation Extraction (DSRE) aligns existing knowledge bases with unstructured text to extract relation facts, and its automatically generated training data is inevitably noisy. Most existing works identify and reduce the impact of noise by enhancing semantic features. However, they only consider the semantic information in a single instance and ignore the semantic information between different instances. In this work, we propose a Semantic Piecewise Convolutional Neural Network (SPCNN), which uses the similarity between different entity pairs as semantic information to improve relation extraction. Specifically, to learn better semantic vector representations, we combine position features with entity pair features and entity similarity features in a high-dimensional space respectively, and generating two different semantic-aware representations. Then we unify these two representations to form a high-quality bag representation for training. Moreover, we design an Adaptive Negative Training (ANT) strategy, which facilitates the network to further exploit the rich semantic features to reduce the interference of noisy labels. Extensive experimental results on a large-scale benchmark dataset show that our method significantly outperforms other baselines.

Reinforcement learning-based distant supervision relation extraction for fault diagnosis knowledge graph construction under industry 4.0

Fault diagnosis is the key concern in the operation and maintenance of industrial assets. A fault diagnosis knowledge graph (KG) can provide decision support to the engineers to efficiently conduct maintenance tasks. However, as a type of domain KG, it would be time-consuming to manually label the corpus collected from the multi-source including the maintenance log, handbook and article. Meanwhile, the existence of the noisy sentence in the multi-source corpus jeopardises the performance of relation extraction modelling. In order to address this issue, this paper proposes a distant supervision relation extraction (DSRE)-based approach to construct a fault diagnosis KG. In this approach, the ontology of the fault diagnosis KG is firstly designed. Subsequently, a DSRE algorithm named relation-aware-based sentence-level attention enhanced piecewise convolutional neural network with reinforcement learning strategy (PCNN-ATTRA-RL) is proposed. The algorithm can effectively lower the impact of noisy sentences and accurately label the relation of different entities when the labelled data is insufficient. In this algorithm, PCNN-ATTRA is designed as the DSRE classifier to effectively extract the relation between entity pairs. RL is conducted to remove the noisy sentence so as to further improve the performance. An experimental study based on the multi-source corpus collected from the real world reveals that the proposed approach shows merits in comparison with the state-of-the-art algorithms. Meanwhile, a fault diagnosis KG, which can greatly support the decision-making of the engineers in the fault diagnosis, is established via the proposed approach.

Spatially oriented convolutional neural network for spatial relation extraction from natural language texts

AbstractSpatial relation extraction (e.g., topological relations, directional relations, and distance relations) from natural language descriptions is a fundamental but challenging task in several practical applications. Current state‐of‐the‐art methods rely on rule‐based metrics, either those specifically developed for extracting spatial relations or those integrated in methods that combine multiple metrics. However, these methods all rely on developed rules and do not effectively capture the characteristics of natural language spatial relations because the descriptions may be heterogeneous and vague and may be context sparse. In this article, we present a spatially oriented piecewise convolutional neural network (SP‐CNN) that is specifically designed with these linguistic issues in mind. Our method extends a general piecewise convolutional neural network with a set of improvements designed to tackle the task of spatial relation extraction. We also propose an automated workflow for generating training datasets by integrating new sentences with those in a knowledge base, based on string similarity and semantic similarity, and then transforming the sentences into training data. We exploit a spatially oriented channel that uses prior human knowledge to automatically match words and understand the linguistic clues to spatial relations, finally leading to an extraction decision. We present both the qualitative and quantitative performance of the proposed methodology using a large dataset collected from Wikipedia. The experimental results demonstrate that the SP‐CNN, with its supervised machine learning, can significantly outperform current state‐of‐the‐art methods on constructed datasets.

Piecewise convolutional neural networks with position attention and similar bag attention for distant supervision relation extraction

Piecewise convolutional neural networks with position attention and similar bag attention for distant supervision relation extraction

A gated piecewise CNN with entity-aware enhancement for distantly supervised relation extraction

The piecewise convolutional neural network (PCNN) is an important method for distant supervision relation extraction. However, the existing methods based on the PCNN still have the following shortcomings: these methods lack the consideration of the impacts of entity pairs and the sentence context on word encoding and do not distinguish the different contributions of the three segments in PCNN to relation classification. To solve these problems, we propose a novel gated piecewise CNN with entity-aware enhancement for distantly supervised relation extraction. First, we use a multi-head self-attention mechanism to combine the word embedding with the head/tail entity embedding and relative position embedding to generate an entity-aware enhanced word representation, which is capable of capturing the semantic dependency between each word and entity pair. Then we introduce a global gate to combine each entity-aware enhanced word representation with their average in the input sentence to form the final word representation of the PCNN input. Moreover, to determine the key segments where the most important information for relation classification appears, we design another gate mechanism to assign a different weight to each sentence segment to highlight the effects of key segments on the PCNN. Experiments on New York Times dataset demonstrate that our model significantly outperforms most of the state-of-the-art models.

Combined Networks with Multi-level Attention for Distantly-Supervised Relation Extraction

Distantly-supervised relation extraction is expected to extract relational facts from very large corpora. However, it is inevitably accompanied by the problem of mislabeling, which affects the performance of relation extraction. To obtain richer information from sentences and make full use of the information of entity pairs, we propose the combined networks containing piecewise convolutional neural networks (PCNN) and bidirectional gated recurrent units (BiGRU) with multi-level attention. In this model, PCNN and BiGRU are utilized to obtain local features and sequence information. Then the multi-level attention is proposed to extract the most correlated information, which can affect the assignment of sentence attention weight. And it reduces the influence of wrong labeled instances indirectly. The experimental results show that our method can enhance the representation power of the network and prove the effectiveness of our model compared with several baseline methods.

Sentiment Analysis Method based on Piecewise Convolutional Neural Network and Generative Adversarial Network

Text sentiment analysis is one of the most important tasks in the field of public opinion monitoring, service evaluation and satisfaction analysis under network environments. Compared with the traditional Natural Language Processing analysis tools, convolution neural networks can automatically learn useful features from sentences and improve the performance of the affective analysis model. However, the original convolution neural network model ignores sentence structure information which is very important for text sentiment analysis. In this paper, we add piece-wise pooling to the convolution neural network, which allows the model to obtain the sentence structure. And the main features of different sentences are extracted to analyze the emotional tendencies of the text. At the same time, the user’s feedback involves many different fields, and there is less labeled data. In order to alleviate the sparsity of the data, this paper also uses the generative adversarial network to make common feature extractions, so that the model can obtain the common features associated with emotions in different fields, and improves the model’s Generalization ability with less training data. Experiments on different datasets demonstrate the effectiveness of this method.

Improved Distant Supervised Model in Tibetan Relation Extraction Using ELMo and Attention

The task of relation extraction is classifying the relations between two entities in a sentence. Distant supervision relation extraction can automatically align entities in texts based on Knowledge Base without labeled training data. For low-resource language relation extraction, such as Tibetan, the main problem is the lack of labeled training data. In this paper, we propose an improved distant supervised relation extraction model based on Piecewise Convolutional Neural Network (PCNN) to expand the Tibetan corpus. We add self-attention mechanism and soft-label method to decrease wrong labels, and use Embeddings from Language Models (ELMo) to solve the semantic ambiguity problem. Meanwhile, according to the Tibetan characteristics, we combine the word vector and part of speech vector to extract deeply feature of words. Finally, the experimental results show that P@avg value increases by 14.4% than baseline.

Shared Representation Generator for Relation Extraction With Piecewise-LSTM Convolutional Neural Networks

Traditional distant supervision for relation extraction is faced with the problem of introducing noises. In this paper, we present a shared representation generator to de-emphasize the noisy expressions by extracting common features in relation. Different from computing weighted sum in widespread attention mechanism, we directly generate bag representation in multi-instance learning by feature transformation, which only remains the semantics related to predict relation. We introduce the generator loss into objective function to improve the performance of shared representation. Also, the structure of our proposed generator is flexible and scalable. To capture more structural information, piecewise convolutional neural network (PCNN) is widely used to divide the output of convolutional layer into three segments, but this approach breaks the consistence and inner relationship of the sentence. We encode the sentence with piecewise-LSTM convolutional neural network (PLSTM-CNN) to alleviate this issue, which adopts BiLSTM after the pooling layer of PCNN. The experimental results show that we achieve significant improvement on relation extraction as compared with the baselines.

Utilizing soft constraints to enhance medical relation extraction from the history of present illness in electronic medical records

Relation extraction between medical concepts from electronic medical records has pervasive applications as well as significance. However, previous researches utilizing machine learning algorithms judge the semantic types of medical concept pair mentions independently. In fact, different concept pair mentions in the same context are of dependencies which can provide beneficial evidences for identifying their relation types. To the best of our knowledge, only one study has considered such dependencies in discharge summaries. However, its hard constraints are not applied effectively to the History of Present Illness (HPI) in electronic Medical Records. According to the writing characteristics of HPI records, we generalize two regularities of dependencies among concept pairs mentioned in an HPI record to enhance the performance of relation extraction. We incorporate the two soft constraints corresponding to the regularities and the posterior probabilities returned by a local classifier into a joint inference process which applies Integer Quadratic Programming method to carry out collective classification for all concept pair mentions in an HPI record. We implement four local classification models including support vector machine, logistics regression, random forest and piecewise convolutional neural networks to examine the performance of our approach. A series of experimental results demonstrate that our collective classification method has made a principal improvement and outperforms the other state-of-the-art methods.

Chemical-induced disease extraction via recurrent piecewise convolutional neural networks

BackgroundExtracting relationships between chemicals and diseases from unstructured literature have attracted plenty of attention since the relationships are very useful for a large number of biomedical applications such as drug repositioning and pharmacovigilance. A number of machine learning methods have been proposed for chemical-induced disease (CID) extraction due to some publicly available annotated corpora. Most of them suffer from time-consuming feature engineering except deep learning methods. In this paper, we propose a novel document-level deep learning method, called recurrent piecewise convolutional neural networks (RPCNN), for CID extraction.ResultsExperimental results on a benchmark dataset, the CDR (Chemical-induced Disease Relation) dataset of the BioCreative V challenge for CID extraction show that the highest precision, recall and F-score of our RPCNN-based CID extraction system are 65.24, 77.21 and 70.77%, which is competitive with other state-of-the-art systems.ConclusionsA novel deep learning method is proposed for document-level CID extraction, where domain knowledge, piecewise strategy, attention mechanism, and multi-instance learning are combined together. The effectiveness of the method is proved by experiments conducted on a benchmark dataset.

Adversarial learning for distant supervised relation extraction

Recently, many researchers have concentrated on using neural networks to learn features for Distant Supervised Relation Extraction (DSRE). These approaches generally use a softmax classifier with cross-entropy loss, which inevitably brings the noise of artificial class NA into classification process. To address the shortcoming, the classifier with ranking loss is employed to DSRE. Uniformly randomly selecting a relation or heuristically selecting the highest score among all incorrect relations are two common methods for generating a negative class in the ranking loss function. However, the majority of the generated negative class can be easily discriminated from positive class and will contribute little towards the training. Inspired by Generative Adversarial Networks (GANs), we use a neural network as the negative class generator to assist the training of our desired model, which acts as the discriminator in GANs. Through the alternating optimization of generator and discriminator, the generator is learning to produce more and more discriminable negative classes and the discriminator has to become better as well. This framework is independent of the concrete form of generator and discriminator. In this paper, we use a two layers fully-connected neural network as the generator and the Piecewise Convolutional Neural Networks (PCNNs) as the discriminator. Experiment results show that our proposed GAN-based method is effective and performs better than state-of-the-art methods.