Drug-drug Interaction Extraction Research Articles

TP-DDI: Transformer-based pipeline for the extraction of Drug-Drug Interactions

Drug-Drug Interaction (DDI) extraction is the task of identifying drug entities and the potential interactions between drug pairs from biomedical literature. Computer-aided extraction of DDIs is vital for drug discovery, as this process remains extremely expensive and time consuming. Therefore, Machine Learning-based approaches can reduce the laborious task during the drug development cycle. Numerous traditional and Neural Network-based approaches for Drug Named Entity Recognition (DNER) and the classification of DDIs have been proposed over the years. However, despite the development of many effective methods, achieving good prediction accuracy is an area where significant improvement can be made. In this article, we present a novel end-to-end approach that tackles the overall DDI extraction task as a pipelined method via the Transformer model architecture and biomedical domain pre-trained weights. In our approach, the tasks of DNER and DDI classification are executed successively to extract the drug entities and to classify their relationship respectively. The proposed approach, TP-DDI, integrates prior knowledge by using pre-trained weights from BioBERT and improves in both the Drug Named Entity Recognition and the overall DDI extraction task over the current state-of-the-art approaches on the DDI Extraction 2013 corpus.

Exploring the Adaptation of Recurrent Neural Network Approaches for Extracting Drug–Drug Interactions from Biomedical Text

Information extraction (IE) is the process of automatically identifying structured information from unstructured or partially structured text. IE processes can involve several activities, such as named entity recognition, event extraction, relationship discovery, and document classification, with the overall goal of translating text into a more structured form. Information on the changes in the effect of a drug, when taken in combination with a second drug, is known as drug–drug interaction (DDI). DDIs can delay, decrease, or enhance absorption of drugs and thus decrease or increase their efficacy or cause adverse effects. Recent research trends have shown several adaptation of recurrent neural networks (RNNs) from text. In this study, we highlight significant challenges of using RNNs in biomedical text processing and propose automatic extraction of DDIs aiming at overcoming some challenges. Our results show that the system is competitive against other systems for the task of extracting DDIs.

Drug-Drug interaction extraction using a position and similarity fusion-based attention mechanism

Taking multiple drugs at the same time can increase or decrease each drug’s effectiveness or cause side effects. These drug-drug interactions (DDIs) may lead to an increase in the cost of medical care or even threaten patients’ health and life. Thus, automatic extraction of DDIs is an important research field to improve patient safety. In this work, a deep neural network model is presented for extracting DDIs from medical texts. This model utilizes a novel attention mechanism for improving the discrimination of important words from others, based on the word similarities and their relative position with respect to candidate drugs. This approach is applied for calculating the attention weights for the outputs of a bi-directional long short-term memory (Bi-LSTM) model in the deep network structure before detecting the type of DDIs. The proposed method was tested on the standard DDI Extraction 2013 dataset and according to experimental results was able to achieve an F1-Score of 78.30 which is comparable to the best results reported for the state-of-the-art methods. A detailed study of the proposed method and its components is also provided.

Drug-Drug Interaction Extraction Based on Bidirectional Gated Recurrent Unit networks and Capsule Networks

Drug-Drug Interaction Extraction Based on Bidirectional Gated Recurrent Unit networks and Capsule Networks

Attention-Gated Graph Convolutions for Extracting Drug Interaction Information from Drug Labels

Preventable adverse events as a result of medical errors present a growing concern in the healthcare system. As drug-drug interactions (DDIs) may lead to preventable adverse events, being able to extract DDIs from drug labels into a machine-processable form is an important step toward effective dissemination of drug safety information. Herein, we tackle the problem of jointly extracting mentions of drugs and their interactions, including interaction outcome, from drug labels. Our deep learning approach entails composing various intermediate representations, including graph-based context derived using graph convolutions (GCs) with a novel attention-based gating mechanism (holistically called GCA), which are combined in meaningful ways to predict on all subtasks jointly. Our model is trained and evaluated on the 2018 TAC DDI corpus. Our GCA model in conjunction with transfer learning performs at 39.20% F1 and 26.09% F1 on entity recognition (ER) and relation extraction (RE), respectively, on the first official test set and at 45.30% F1 and 27.87% F1 on ER and RE, respectively, on the second official test set. These updated results lead to improvements over our prior best by up to 6 absolute F1 points. After controlling for available training data, the proposed model exhibits state-of-the-art performance for this task.

Using drug descriptions and molecular structures for drug-drug interaction extraction from literature.

MotivationNeural methods to extract drug–drug interactions (DDIs) from literature require a large number of annotations. In this study, we propose a novel method to effectively utilize external drug database information as well as information from large-scale plain text for DDI extraction. Specifically, we focus on drug description and molecular structure information as the drug database information.ResultsWe evaluated our approach on the DDIExtraction 2013 shared task dataset. We obtained the following results. First, large-scale raw text information can greatly improve the performance of extracting DDIs when combined with the existing model and it shows the state-of-the-art performance. Second, each of drug description and molecular structure information is helpful to further improve the DDI performance for some specific DDI types. Finally, the simultaneous use of the drug description and molecular structure information can significantly improve the performance on all the DDI types. We showed that the plain text, the drug description information and molecular structure information are complementary and their effective combination is essential for the improvement.Availability and implementation Our code is available at https://github.com/tticoin/DESC_MOL-DDIE.

The impact of learning Unified Medical Language System knowledge embeddings in relation extraction from biomedical texts

We explored how knowledge embeddings (KEs) learned from the Unified Medical Language System (UMLS) Metathesaurus impact the quality of relation extraction on 2 diverse sets of biomedical texts. Two forms of KEs were learned for concepts and relation types from the UMLS Metathesaurus, namely lexicalized knowledge embeddings (LKEs) and unlexicalized KEs. A knowledge embedding encoder (KEE) enabled learning either LKEs or unlexicalized KEs as well as neural models capable of producing LKEs for mentions of biomedical concepts in texts and relation types that are not encoded in the UMLS Metathesaurus. This allowed us to design the relation extraction with knowledge embeddings (REKE) system, which incorporates either LKEs or unlexicalized KEs produced for relation types of interest and their arguments. The incorporation of either LKEs or unlexicalized KE in REKE advances the state of the art in relation extraction on 2 relation extraction datasets: the 2010 i2b2/VA dataset and the 2013 Drug-Drug Interaction Extraction Challenge corpus. Moreover, the impact of LKEs is superior, achieving F1 scores of 78.2 and 82.0, respectively. REKE not only highlights the importance of incorporating knowledge encoded in the UMLS Metathesaurus in a novel way, through 2 possible forms of KEs, but it also showcases the subtleties of incorporating KEs in relation extraction systems. Incorporating LKEs informed by the UMLS Metathesaurus in a relation extraction system operating on biomedical texts shows significant promise. We present the REKE system, which establishes new state-of-the-art results for relation extraction on 2 datasets when using LKEs.

Relation Extraction From Biomedical and Clinical Text: Unified Multitask Learning Framework.

To minimize the accelerating amount of time invested on the biomedical literature search, numerous approaches for automated knowledge extraction have been proposed. Relation extraction is one such task where semantic relations between the entities are identified from the free text. In the biomedical domain, extraction of regulatory pathways, metabolic processes, adverse drug reaction or disease models necessitates knowledge from the individual relations, for example, physical or regulatory interactions between genes, proteins, drugs, chemical, disease or phenotype. In this paper, we study the relation extraction task from three major biomedical and clinical tasks, namely drug-drug interaction, protein-protein interaction, and medical concept relation extraction. Towards this, we model the relation extraction problem in a multi-task learning (MTL)framework, and introduce for the first time the concept of structured self-attentive network complemented with the adversarial learning approach for the prediction of relationships from the biomedical and clinical text. The fundamental notion of MTL is to simultaneously learn multiple problems together by utilizing the concepts of the shared representation. Additionally, we also generate the highly efficient single task model which exploits the shortest dependency path embedding learned over the attentive gated recurrent unit to compare our proposed MTL models. The framework we propose significantly improves over all the baselines (deep learning techniques)and single-task models for predicting the relationships, without compromising on the performance of all the tasks.

Single Concatenated Input is Better than Indenpendent Multiple-input for CNNs to Predict Chemical-induced Disease Relation from Literature

Single Concatenated Input is Better than Indenpendent Multiple-input for CNNs to Predict Chemical-induced Disease Relation from Literature

Extracting drug-drug interactions from texts with BioBERT and multiple entity-aware attentions.

Drug-drug interactions (DDIs) extraction is one of the important tasks in the field of biomedical relation extraction, which plays an important role in the field of pharmacovigilance. Previous neural network based models have achieved good performance in DDIs extraction. However, most of the previous models did not make good use of the information of drug entity names, which can help to judge the relation between drugs. This is mainly because drug names are often very complex, leading to the fact that neural network models cannot understand their semantics directly. To address this issue, we propose a DDIs extraction model using multiple entity-aware attentions with various entity information. We use an output-modified bidirectional transformer (BioBERT) and a bidirectional gated recurrent unit layer (BiGRU) to obtain the vector representation of sentences. The vectors of drug description documents encoded by Doc2Vec are used as drug description information, which is an external knowledge to our model. Then we construct three different kinds of entity-aware attentions to get the sentence representations with entity information weighted, including attentions using the drug description information. The outputs of attention layers are concatenated and fed into a multi-layer perception layer. Finally, we get the result by a softmax classifier. The F-score is used to evaluate our model, which is also adopted by most previous DDIs extraction models. We evaluate our proposed model on the DDIExtraction 2013 corpus, which is the benchmark corpus of this domain, and achieves the state-of-the-art result (80.9% in F-score).

AGCN: Attention-based graph convolutional networks for drug-drug interaction extraction

Extracting drug-drug interaction (DDI) relations is one of the most typical tasks in the field of biomedical relation extraction. Automatic DDI extraction from the biomedical corpus is central to the mining of knowledge hidden in the biomedical literature. Existing approaches for DDI extraction primarily focus on either the contextual or the structural information of the sentence, despite their complementary role. Also, previous studies do not even exploit the entire knowledge of the input sentence, which could lead to a loss of crucial clues. In this paper, we propose an Attention-based Graph Convolutional Networks (AGCN) to address these issues. In contrast to the existing DDI extraction methods, the AGCN is designed to leverage contextual and structural knowledge together, where GCN is employed in combination with encoders based on recurrent networks. Additionally, we apply a novel attention-based pruning strategy to optimallyuse syntactic information while ignoring irrelevant information, in contrast to previous rule-based pruning methods. Therefore, AGCN can take advantage of the context and structure of the input sentence as efficiently as possible. We evaluate our model using a dominant DDI extraction corpus. The experimental results demonstrate the effectiveness of our model, which outperforms existing approaches.

Drug-drug interaction extraction via hybrid neural networks on biomedical literature.

Adverse events caused by drug-drug interaction (DDI) not only pose a serious threat to health, but also increase additional medical care expenditure. However, despite the emergence of many excellent text mining-based DDI classification methods, achieving a balance between using simpler method and better model performance is still unsatisfactory. In this article, we present a deep learning method of stacked bidirectional Gated Recurrent Unit (GRU)- convolutional neural network (SGRU-CNN) model which apply stacked bidirectional GRU (BiGRU) network and convolutional neural network (CNN) on lexical information and entity position information respectively to conduct DDIs extraction task. Furthermore, SGRU-CNN model assigns the weights of each word feature to improve performance with one attentive pooling layer. On the condition that other values are not inferior to other algorithms, experimental results on the DDI Extraction 2013 corpus show that our model achieves a 1.54% improvement in recall value. And the proposed SGRU-CNN model reaches great performance (F1-score: 0.75) with the fewest features, indicating an excellent balance between avoiding redundant preprocessing task and higher accuracy in relation extraction on biomedical literature using our method.

BioConceptVec: Creating and evaluating literature-based biomedical concept embeddings on a large scale.

A massive number of biological entities, such as genes and mutations, are mentioned in the biomedical literature. The capturing of the semantic relatedness of biological entities is vital to many biological applications, such as protein-protein interaction prediction and literature-based discovery. Concept embeddings—which involve the learning of vector representations of concepts using machine learning models—have been employed to capture the semantics of concepts. To develop concept embeddings, named-entity recognition (NER) tools are first used to identify and normalize concepts from the literature, and then different machine learning models are used to train the embeddings. Despite multiple attempts, existing biomedical concept embeddings generally suffer from suboptimal NER tools, small-scale evaluation, and limited availability. In response, we employed high-performance machine learning-based NER tools for concept recognition and trained our concept embeddings, BioConceptVec, via four different machine learning models on ~30 million PubMed abstracts. BioConceptVec covers over 400,000 biomedical concepts mentioned in the literature and is of the largest among the publicly available biomedical concept embeddings to date. To evaluate the validity and utility of BioConceptVec, we respectively performed two intrinsic evaluations (identifying related concepts based on drug-gene and gene-gene interactions) and two extrinsic evaluations (protein-protein interaction prediction and drug-drug interaction extraction), collectively using over 25 million instances from nine independent datasets (17 million instances from six intrinsic evaluation tasks and 8 million instances from three extrinsic evaluation tasks), which is, by far, the most comprehensive to our best knowledge. The intrinsic evaluation results demonstrate that BioConceptVec consistently has, by a large margin, better performance than existing concept embeddings in identifying similar and related concepts. More importantly, the extrinsic evaluation results demonstrate that using BioConceptVec with advanced deep learning models can significantly improve performance in downstream bioinformatics studies and biomedical text-mining applications. Our BioConceptVec embeddings and benchmarking datasets are publicly available at https://github.com/ncbi-nlp/BioConceptVec.

A Modified Skip-Gram Algorithm for Extracting Drug-Drug Interactions from AERS Reports.

Drug-drug interactions (DDIs) are one of the indispensable factors leading to adverse event reactions. Considering the unique structure of AERS (Food and Drug Administration Adverse Event Reporting System (FDA AERS)) reports, we changed the scope of the window value in the original skip-gram algorithm, then propose a language concept representation model and extract features of drug name and reaction information from large-scale AERS reports. The validation of our scheme was tested and verified by comparing with vectors originated from the cooccurrence matrix in tenfold cross-validation. In the verification of description enrichment of the DrugBank DDI database, accuracy was calculated for measurement. The average area under the receiver operating characteristic curve of logistic regression classifiers based on the proposed language model is 6% higher than that of the cooccurrence matrix. At the same time, the average accuracy in five severe adverse event classes is 88%. These results indicate that our language model can be useful for extracting drug and reaction features from large-scale AERS reports.

BPR Based Drug Groups and Drug Interaction Extraction

The key step of drug discovery is the identification of interaction between drug and target proteins. This isn't just valuable to understand the disease, but also assist to distinguishing antagonistic symptoms of drugs. So, in drug repurposing [3] field the drug-target interaction (DTI) prediction is an essential tool. There are various methods to decipher unknown drug-target interaction [2], this is helped in the area of identifying the lead compound in the drug for a specific disease. In this paper proposes drug-target interaction extraction using Bayesian Personalized Ranking (BPR) method [5]. Here it is also solving the ranking problem by the implementation of matrix factorization method [5]. The proposed procedure can manage the occasion of new drugs and takes compound and hereditary resemblances of meds and targets and target tendency into account. [4].

One stage versus two stages deep learning approaches for the extraction of drug-drug interactions from texts

Drug-drug interactions (DDI) are a cause of adverse drug reactions. They occur when a drug has an impact on the effect of another drug. There is not a complete, up to date database where health care professionals can consult the interactions of any drug because most of the knowledge on DDI is hidden in unstructured text. In last years, deep learning has been succesfully applied to the extraction of DDI from texts, which requires the detection and later classification of DDI. Most of the deep learning systems for DDI extraction developed so far have addressed the detection and classification in one single step. In this study, we compare the performance of one-stage and two-stage architectures for DDI extraction. Our architectures are based on a bidirectional recurrent neural network layer composed of Gated Recurrent Units. The two-stage system obtained a 67.45 % micro-average F1 score on the test set.

Deep learning for drug-drug interaction extraction from the literature: a review.

Drug-drug interactions (DDIs) are crucial for drug research and pharmacovigilance. These interactions may cause adverse drug effects that threaten public health and patient safety. Therefore, the DDIs extraction from biomedical literature has been widely studied and emphasized in modern biomedical research. The previous rules-based and machine learning approaches rely on tedious feature engineering, which is labourious, time-consuming and unsatisfactory. With the development of deep learning technologies, this problem is alleviated by learning feature representations automatically. Here, we review the recent deep learning methods that have been applied to the extraction of DDIs from biomedical literature. We describe each method briefly and compare its performance in the DDI corpus systematically. Next, we summarize the advantages and disadvantages of these deep learning models for this task. Furthermore, we discuss some challenges and future perspectives of DDI extraction via deep learning methods. This review aims to serve as a useful guide for interested researchers to further advance bioinformatics algorithms for DDIs extraction from the literature.

Incorporating User Generated Content for Drug Drug Interaction Extraction Based on Full Attention Mechanism.

It is crucial for doctors to fully understand the interaction between drugs in prescriptions, especially when a patient takes multiple medications at the same time during treatment. The purpose of drug drug interaction (DDI) extraction is to automatically obtain the interaction between drugs from biomedical literature. Current state-of-the-art approaches for DDI extraction task are based on artificial intelligence and natural language processing. While such existing DDI extraction methods can provide more knowledge and enhance the performance through external resources such as biomedical databases or ontologies, due to the difficulty of updating, these external resources are delayed. In fact, user generated content (UGC) is another kind of external medical resources that can be quickly updated. We are trying to use UGC resources to provide more available information for our deep learning DDI extraction method. In this paper, we present a DDI extraction approach through a new attention mechanism called full-attention which can combine the UGC information with contextual information. We conducted a series of experiments on the DDI 2013 Evaluation dataset to evaluate our method. Experiments show improved performance compared with the state of the art and UGC-DDI model achieves a competitive F-score of 0.712.

Drug-Drug Interaction Extraction via Recurrent Hybrid Convolutional Neural Networks with an Improved Focal Loss.

Drug-drug interactions (DDIs) may bring huge health risks and dangerous effects to a patient’s body when taking two or more drugs at the same time or within a certain period of time. Therefore, the automatic extraction of unknown DDIs has great potential for the development of pharmaceutical agents and the safety of drug use. In this article, we propose a novel recurrent hybrid convolutional neural network (RHCNN) for DDI extraction from biomedical literature. In the embedding layer, the texts mentioning two entities are represented as a sequence of semantic embeddings and position embeddings. In particular, the complete semantic embedding is obtained by the information fusion between a word embedding and its contextual information which is learnt by recurrent structure. After that, the hybrid convolutional neural network is employed to learn the sentence-level features which consist of the local context features from consecutive words and the dependency features between separated words for DDI extraction. Lastly but most significantly, in order to make up for the defects of the traditional cross-entropy loss function when dealing with class imbalanced data, we apply an improved focal loss function to mitigate against this problem when using the DDIExtraction 2013 dataset. In our experiments, we achieve DDI automatic extraction with a micro F-score of 75.48% on the DDIExtraction 2013 dataset, outperforming the state-of-the-art approach by 2.49%.

Drug-Drug Interaction Extraction Based on Transfer Weight Matrix and Memory Network

Extracting drug-drug interaction (DDI) in the text is the process of identifying how two target drugs in a given sentence interact. Previous methods, which were limited to conventional machine learning techniques, we are susceptible to issues such as “vocabulary gap” and unattainable automation processes in feature extraction. Inspired by deep learning in natural language preprocessing, we addressed the aforementioned problems based on dynamic transfer matrix and memory networks. A TM-RNN method is proposed by adding the transfer weight matrix in multilayer bidirectional LSTM to improve robustness and introduce a memory network for feature fusion. We evaluated the TM-RNN model on the DDIExtraction 2013 Task. The proposed model achieved an overall F-score of 72.43, which outperforms the latest methods based on support vector machine and other neural networks. Meanwhile, the experimental results also indicated that the proposed model is more stable and less affected by negative samples.