Biomedical Event Trigger Research Articles

A context-enhanced neural network model for biomedical event trigger detection

As an important component of biomedical event extraction, biomedical event trigger detection has received extensive research attention in recent years. Most studies focus on designing various models or features according to the original text itself, but fail to leverage contextual information of the original text from external knowledge base such as Wikipedia, which is publicly available. To address the issue, we propose a context-enhanced neural network model that automatically integrates the related information from external knowledge base for biomedical event trigger detection. Specifically, the proposed model first extracts the related context of the original text from external knowledge base. Then the original text and its context are sequentially fed into the BERT embedding layer and Transformer convolution layer to learn high-level semantic representation. Finally, the probability of possible tags is calculated using the CRF layer. Experimental results on the MLEE dataset show our proposed model achieves 86.83% F1 score, outperforming the existing methods and context-enhanced baseline systems significantly. Experimental analysis also indicates the effectiveness of contextual information for trigger detection in biomedical domain.

Effective type label-based synergistic representation learning for biomedical event trigger detection

BackgroundDetecting event triggers in biomedical texts, which contain domain knowledge and context-dependent terms, is more challenging than in general-domain texts. Most state-of-the-art models rely mainly on external resources such as linguistic tools and knowledge bases to improve system performance. However, they lack effective mechanisms to obtain semantic clues from label specification and sentence context. Given its success in image classification, label representation learning is a promising approach to enhancing biomedical event trigger detection models by leveraging the rich semantics of pre-defined event type labels.ResultsIn this paper, we propose the Biomedical Label-based Synergistic representation Learning (BioLSL) model, which effectively utilizes event type labels by learning their correlation with trigger words and enriches the representation contextually. The BioLSL model consists of three modules. Firstly, the Domain-specific Joint Encoding module employs a transformer-based, domain-specific pre-trained architecture to jointly encode input sentences and pre-defined event type labels. Secondly, the Label-based Synergistic Representation Learning module learns the semantic relationships between input texts and event type labels, and generates a Label-Trigger Aware Representation (LTAR) and a Label-Context Aware Representation (LCAR) for enhanced semantic representations. Finally, the Trigger Classification module makes structured predictions, where each label is predicted with respect to its neighbours. We conduct experiments on three benchmark BioNLP datasets, namely MLEE, GE09, and GE11, to evaluate our proposed BioLSL model. Results show that BioLSL has achieved state-of-the-art performance, outperforming the baseline models.ConclusionsThe proposed BioLSL model demonstrates good performance for biomedical event trigger detection without using any external resources. This suggests that label representation learning and context-aware enhancement are promising directions for improving the task. The key enhancement is that BioLSL effectively learns to construct semantic linkages between the event mentions and type labels, which provide the latent information of label-trigger and label-context relationships in biomedical texts. Moreover, additional experiments on BioLSL show that it performs exceptionally well with limited training data under the data-scarce scenarios.

Child-Sum EATree-LSTMs: enhanced attentive Child-Sum Tree-LSTMs for biomedical event extraction

BackgroundTree-structured neural networks have shown promise in extracting lexical representations of sentence syntactic structures, particularly in the detection of event triggers using recursive neural networks.MethodsIn this study, we introduce an attention mechanism into Child-Sum Tree-LSTMs for the detection of biomedical event triggers. We incorporate previous researches on assigning attention weights to adjacent nodes and integrate this mechanism into Child-Sum Tree-LSTMs to improve the detection of event trigger words. We also address a limitation of shallow syntactic dependencies in Child-Sum Tree-LSTMs by integrating deep syntactic dependencies to enhance the effect of the attention mechanism.ResultsOur proposed model, which integrates an enhanced attention mechanism into Tree-LSTM, shows the best performance for the MLEE and BioNLP’09 datasets. Moreover, our model outperforms almost all complex event categories for the BioNLP’09/11/13 test set.ConclusionWe evaluate the performance of our proposed model with the MLEE and BioNLP datasets and demonstrate the advantage of an enhanced attention mechanism in detecting biomedical event trigger words.

Gated tree-structured RecurNN for Detecting Biomedical Event Trigger

It is critical to deeply mine semantic features for information extraction. Tree-structured model is a linguistically attractive option due to its linguistic representations of sentence syntactic structure. Tree-LSTM has been introduced to represent tree-structured network topologies for the syntactic properties. To alleviate the limitation of the Tree-LSTM, we work towards addressing the issue by developing gated mechanism variants for the tree-structured network. The gated mechanism is more complex and diverse for the tree-structured model. We apply Child-Sum Tree-LSTM and Child-Sum Tree-GRU for recognizing biomedical event triggers, and develop two new gated mechanism variants incorporating peephole connection and coupled mechanism into the tree-structured model. The experimental results showed the advantage of gated units. The Child-Sum Tree-LSTM achieved the best results among the gated tree-structured models, and the performance of variants is nearly the same as Child-Sum Tree-LSTM. However, Child-Sum Tree-GRU and Child-Sum Tree-coupled reach reduction in computation time.

Gated Tree-Structured Recurnn for Detecting Biomedical Event Trigger

Gated Tree-Structured Recurnn for Detecting Biomedical Event Trigger

SWACG: A Hybrid Neural Network Integrating Sliding Window for Biomedical Event Trigger Extraction

Abstract As an important part of biomedical text mining, biomedical events play a key role in improving biomedical research and disease prevention. Trigger identification, extracting the words describing the event types, is a critical and prerequisite step for biomedical event extraction. Traditional methods excessively rely on natural language processing tools in the feature extraction process, incurring a significant manual cost. In addition, because of the particularity of the biomedical literature, the problem of long-distance dependency is obvious. To solve these problems, we propose a hybrid structure SWACG, which consists of the ReCNN-BiGRU (Residual CNN and Bidirectional Gated Recurrent Unit) hybrid neural network and MH-attention (Multi-Head attention) mechanism. The proposed model uses ReCNN to extract vocabulary-level features and BiGRU to obtain contextual semantic information. Furthermore, sliding window divides long sentences into equal-length short sentences without destroying context information, which can avoid long-distance dependency. Experimental results show that our method advances the state-of-the-art performance on the commonly used Multi-Level Event Extraction (MLEE) corpus, achieving 82.20% F-score.

Biomedical event trigger extraction based on multi-layer residual BiLSTM and contextualized word representations

Biomedical event extraction is an important branch of biomedical information extraction. Trigger extraction is the most essential sub-task in event extraction, which has been widely concerned. Existing trigger extraction studies are mostly based on conventional machine learning or neural networks. But they neglect the ambiguity of word representations and the insufficient feature extraction by shallow hidden layers. In this paper, trigger extraction is treated as a sequence labeling problem. We introduce the language model to dynamically compute contextualized word representations and propose a multi-layer residual bidirectional long short-term memory (BiLSTM) architecture. First, we concatenate contextualized word embedding, pretrained word embedding and character-level embedding as the feature representations, which effectively solves the tokens’ ambiguity in biomedical corpora. Then, the designed BiLSTM block with residual connection and gated multi-layer perceptron is adopted to extract features iteratively. This architecture improves the ability of our model to capture information and avoids gradient exploding or vanishing. Finally, we combine the multi-layer residual BiLSTM with CRF layer to obtain more reasonable label sequences. Comparing with other state-of-the-art methods, the proposed model achieves the competitive performance (F1-score: 80.74%) on the biomedical multi-level event extraction (MLEE) corpus without any manual participation and feature engineering.

A transfer learning model with multi-source domains for biomedical event trigger extraction

BackgroundAutomatic extraction of biomedical events from literature, that allows for faster update of the latest discoveries automatically, is a heated research topic now. Trigger word recognition is a critical step in the process of event extraction. Its performance directly influences the results of the event extraction. In general, machine learning-based trigger recognition approaches such as neural networks must to be trained on a dataset with plentiful annotations to achieve high performances. However, the problem of the datasets in wide coverage event domains is that their annotations are insufficient and imbalance. One of the methods widely used to deal with this problem is transfer learning. In this work, we aim to extend the transfer learning to utilize multiple source domains. Multiple source domain datasets can be jointly trained to help achieve a higher recognition performance on a target domain with wide coverage events.ResultsBased on the study of previous work, we propose an improved multi-source domain neural network transfer learning architecture and a training approach for biomedical trigger detection task, which can share knowledge between the multi-source and target domains more comprehensively. We extend the ability of traditional adversarial networks to extract common features between source and target domains, when there is more than one dataset in the source domains. Multiple feature extraction channels to simultaneously capture global and local common features are designed. Moreover, under the constraint of an extra classifier, the multiple local common feature sub-channels can extract and transfer more diverse common features from the related multi-source domains effectively. In the experiments, MLEE corpus is used to train and test the proposed model to recognize the wide coverage triggers as a target dataset. Other four corpora with the varying degrees of relevance with MLEE from different domains are used as source datasets, respectively. Our proposed approach achieves recognition improvement compared with traditional adversarial networks. Moreover, its performance is competitive compared with the results of other leading systems on the same MLEE corpus.ConclusionsThe proposed Multi-Source Transfer Learning-based Trigger Recognizer (MSTLTR) can further improve the performance compared with the traditional method, when the source domains are more than one. The most essential improvement is that our approach represents common features in two aspects: the global common features and the local common features. Hence, these more sharable features improve the performance and generalization of the model on the target domain effectively.

Design and analysis on molecular level biomedical event trigger extraction using recurrent neural network-based particle swarm optimisation for COVID-19 research

Design and analysis on molecular level biomedical event trigger extraction using recurrent neural network-based particle swarm optimisation for COVID-19 research

Design and analysis on molecular level biomedical event trigger extraction using recurrent neural network-based particle swarm optimisation for COVID-19 research

Design and analysis on molecular level biomedical event trigger extraction using recurrent neural network-based particle swarm optimisation for COVID-19 research

A Two-Stage Biomedical Event Trigger Detection Method Based on Hybrid Neural Network and Sentence Embeddings

Biomedical event extraction is a challenging task in biomedical text mining, which plays an important role in improving biomedical research and disease prevention. As the crucial and prerequisite step in event extraction, biomedical trigger detection has attracted much attention. Previous approaches usually depended on feature engineering with unbalanced data. In this paper, we propose a two-stage method based on hybrid neural network for trigger detection, which divides trigger detection into recognition stage and classification stage. In the first stage, we build a BiLSTM based recognition model integrating attention mechanism (Att-BiLSTM). In the second stage, the classification model based on Passive-Aggressive online algorithm is constructed. Furthermore, to enrich sentence-level features, we establish sentence embeddings and add reading gate. On the multi-level event extraction (MLEE) corpus test dataset, our method achieves an F-score of 80.26%, which achieves the state-of-the-art systems.

Extraction of the molecular level biomedical event trigger based on gene ontology using radial belief neural network techniques

Detection of molecular level biomedical event extraction plays a vital role in creating and visualizing the applications related to natural language processing. Cystic Fibrosis is an inherited genetic and debilitating pathology involving the respiratory and digestive systems. The excessive production of thick sticky mucus on the outside of the cells is the main consequence of such disease. This includes disease prevention and medical search to signify the occurrence and detection of event triggers, which is regarded as a proper step in an event extraction of molecular level in biomedical applications. In this model, use a rich set of extracted features to feed the machine learning classifier that helps in better extraction of events. The study uses an automatic feature selection and a classification model using Radial Belief Neural Network (RBNN) for the optimal detection of molecular biomedical event detection. The Radial Belief Neural Network (RBNN) is the proposed system is implemented and it is the classifier to give accurate result of the disease detection. These three algorithms are used to enhance the generalization performance and scalability of detecting the molecular event triggers. The validation is conducted on the cystic fibrosis event trigger based on the gene ontology bio system using the RBNN model with a lung molecular event-level extraction dataset. The extensive computation shows that the Radial Belief Neural Network (RBNN) is proposed to given the better performance results like Accuracy, Sensitivity, Specificity, F-measure and Execution time.

FBSN: A hybrid fine-grained neural network for biomedical event trigger identification

Biomedical event extraction is one of the fundamental tasks in medical research and disease prevention. Event trigger usually signifies the occurrence of a biomedical event by adopting a word or a phrase. Meanwhile, the task of biomedical event trigger identification is a critical and prerequisite step for biomedical event extraction. The existing methods generally rely on the complex and unobtainable features engineering. To alleviate this problem, we propose a hybrid structure FBSN which consists of Fine-grained Bidirectional Long Short Term Memory (FBi-LSTM) and Support Vector Machine (SVM) to deal with the event trigger identification. The hybrid architecture makes the most of their advantages: FBi-LSTM is to mainly extract the higher level features by the fine-grained representations, and SVM is largely appropriate for small dataset for classifying the results of biomedical event trigger. After that, the popular dataset Multi Level Event Extraction (MLEE) is employed to verify our hybrid structure. Experimental results show that our method is able to achieve the state-of-the-art baseline approaches. Meanwhile, we also discuss the detailed experiments in trigger identification task.

A tree-based neural network model for biomedical event trigger detection

Biomedical event trigger detection is a heated research topic since its important role in biomedical event extraction. Previous studies show that syntactic features are very crucial for the task. However, existing methods largely focus on traditional statistical models, and usually capture syntactic features by extracting a set of manually-crafted features based on dependency tree. This limits the performance of the task. In this paper, we propose a tree-based neural network model, which can automatically learn syntactic features from dependency tree for trigger detection. Specifically, we use a recursive neural network to represent whole dependency tree globally, to better incorporate dependency-based syntax information. Results on MLEE and BioNLP-09 datasets show that the proposed model achieves 80.28% and 73.24% F1 score, respectively, outperforming traditional statistical models and neural baseline systems.

Multiple-level biomedical event trigger recognition with transfer learning

BackgroundAutomatic extraction of biomedical events from literature is an important task in the understanding biological systems, allowing for faster update of the latest discoveries automatically. Detecting trigger words which indicate events is a critical step in the process of event extraction, because following steps depend on the recognized triggers. The task in this study is to identify event triggers from the literature across multiple levels of biological organization. In order to achieve high performances, the machine learning based approaches, such as neural networks, must be trained on a dataset with plentiful annotations. However, annotations might be difficult to obtain on the multiple levels, and annotated resources have so far mainly focused on the relations and processes at the molecular level. In this work, we aim to apply transfer learning for multiple-level trigger recognition, in which a source dataset with sufficient annotations on the molecular level is utilized to improve performance on a target domain with insufficient annotations and more trigger types.ResultsWe propose a generalized cross-domain neural network transfer learning architecture and approach, which can share as much knowledge as possible between the source and target domains, especially when their label sets overlap. In the experiments, MLEE corpus is used to train and test the proposed model to recognize the multiple-level triggers as a target dataset. Two different corpora having the varying degrees of overlapping labels with MLEE from the BioNLP’09 and BioNLP’11 Shared Tasks are used as source datasets, respectively. Regardless of the degree of overlap, our proposed approach achieves recognition improvement. Moreover, its performance exceeds previously reported results of other leading systems on the same MLEE corpus.ConclusionsThe proposed transfer learning method can further improve the performance compared with the traditional method, when the labels of the source and target datasets overlap. The most essential reason is that our approach has changed the way parameters are shared. The vertical sharing replaces the horizontal sharing, which brings more sharable parameters. Hence, these more shared parameters between networks improve the performance and generalization of the model on the target domain effectively.

Biomedical event trigger detection with convolutional highway neural network and extreme learning machine

Detecting biomedical events in text plays a critical role in building natural language processing applications, such as in medical search, disease prevention, and pharmacovigilance. Since an event trigger can signify the occurrence of the event, the detection of biomedical event triggers is a critical step in biomedical event extraction. Current methods usually extract rich features and then feed these features to a classifier. To enhance both automatic feature selection and classification, this paper presented an end-to-end convolutional highway neural network and extreme learning machine (CHNN–ELM) framework to detect biomedical event triggers. This structure has two stages. In the first stage, CHNN is used to efficiently select higher level semantic features based on four different dimensions: embedding, convolutional layer, pooling layer, and highway layer. In the second stage, the proposed model leverages ELM, which has great scalability and generalization performance, to identify various types of biomedical event triggers. Extensive experiments are conducted on the Multi-Level Event Extraction (MLEE) dataset. To the best of our knowledge, this paper is the first to introduce ELM into this task. The results demonstrated that with better feature selection and classification, our approach outperforms several current state-of-the-art methods.

Contextual label sensitive gated network for biomedical event trigger extraction.

Biomedical events play a key role in improving biomedical research. Event trigger identification, extracting the words describing the event types, is a crucial and prerequisite step in the pipeline process of biomedical event extraction. There exist two main problems in previous methods: (1) The association among contextual trigger labels which can provide significant clues is ignored. (2)The weight between word embeddings and contextual features needs to be adjusted dynamically according to the trigger candidate. In this paper, we propose a novel contextual label sensitive gated network for biomedical event trigger extraction to solve the above two problems, which can mix the two parts dynamically and capture the contextual label clues automatically. Furthermore, we also introduce the dependency-based word embeddings to represent dependency-based semantic information as well as attention mechanism to get more focused representations. Experimental results show that our approach advances state-of-the-arts and achieves the best F1-score on the commonly used Multi-Level Event Extraction (MLEE) corpus.

An Integrated Biomedical Event Trigger Identification Approach With a Neural Network and Weighted Extreme Learning Machine

Biomedical event trigger identification is a sub-task in biomedical event extraction that aims to recognize the trigger label of biomedical events in context. It is a fundamental task in natural language processing. Previous approaches usually depended on feature engineering with unbalanced data. In this paper, we present a bidirectional long short-term memory convolution neural network weighted extreme learning machine (BC-WELM) to identify the biomedical event trigger. Using the different dimensions of embeddings as input, this model considers the contextual modeling by the Bi-LSTM and the local modeling by CNN and, then, classifies the trigger label to settle the unbalanced problem by the WELM. With this design, the BC-WELM model is helpful for biomedical event trigger identification. The experimental results on the MLEE dataset demonstrate that our approach is capable of outperforming the state-of-the-art baselines on an F1 score.

Bidirectional long short-term memory with CRF for detecting biomedical event trigger in FastText semantic space

BackgroundIn biomedical information extraction, event extraction plays a crucial role. Biological events are used to describe the dynamic effects or relationships between biological entities such as proteins and genes. Event extraction is generally divided into trigger detection and argument recognition. The performance of trigger detection directly affects the results of the event extraction. In general, the traditional method is used to address the trigger detection as a classification task, as well as the use of machine learning or rules method, which construct many features to improve the classification results. Moreover, the classification model only recognizes triggers composed of single words, whereas for multiple words, the result is unsatisfactory.ResultsThe corpus of our model is MLEE. If we were to only use the biomedical LSTM and CRF model without other features, the F-score would reach about 78.08%. Comparing entity to part of speech (POS), we find the entity features more conducive to the improvement of performance of detection, with the F-score potentially reaching about 80%. Furthermore, we also experiment on the other three corpora (BioNLP 2009, BioNLP 2011, and BioNLP 2013) to verify the generalization of our model. Hence, F-scores can reach more than 60%, which are better than the comparative experiments.ConclusionsThe trigger recognition method based on the sequence annotation model does not require initial complex feature engineering, and only requires a simple labeling mechanism to complete the training. Therefore, generalization of our model is better compared to other traditional models. Secondly, this method can identify multi-word triggers, thereby improving the F-scores of trigger recognition. Thirdly, details on the entity have a crucial impact on trigger detection. Finally, the combination of character-level word embedding and word-level word embedding provides increasingly effective information for the model; therefore, it is a key to the success of the experiment.

A Two-Stage Biomedical Event Trigger Detection Method Integrating Feature Selection and Word Embeddings.

Extracting biomedical events from biomedical literature plays an important role in the field of biomedical text mining, and the trigger detection is a key step in biomedical event extraction. We propose a two-stage method for trigger detection, which divides trigger detection into recognition stage and classification stage, and different features are selected in each stage. In the first stage, we select the features which are more suitable for recognition, and in the second stage, the features that are more helpful to classification are adopted. Furthermore, we integrate word embeddings to represent words semantically and syntactically. On the multi-level event extraction (MLEE) corpus test dataset, our method achieves an F-score of 79.75 percent, which outperforms the state-of-the-art systems.