Medical NLP Research Articles

Utilizing BERT for entity relationship extraction in Chinese medical texts

The Chinese medical sector has been somewhat lacking in knowledge graphs, a deficiency this study aims to address. By leveraging the prowess of the BERT pre-training model, a two-tier approach has been innovated that utilizes separate pre-trained encoders for both entity and relational models. These models are intricately linked: the output from the entity model seamlessly flows into the relational one, making it possible to adeptly extract entity relationships from Chinese medical texts. This research is anchored in the CMeIE dataset, sourced from the esteemed CHIP (China Health Information Processing) conference. This dataset stands as a recognized benchmark in evaluating Chinese medical texts. By harnessing this data, the methods have been rigorously tested and validated. The promising experimental results underscore the effectiveness of the approach in distilling relationships from Chinese medical literature. The implications of this research are profound. Beyond just enriching the Chinese medical domain, the boundaries of NLP technology are also being pushed. Potential applications are manifold: from constructing comprehensive Chinese medical knowledge graphs to assisting in early-stage medical diagnoses. This innovative approach not only addresses an existing gap but also sets the stage for future advancements in medical NLP.

GERNERMED++: Semantic annotation in German medical NLP through transfer-learning, translation and word alignment

We present a statistical model, GERNERMED++, for German medical natural language processing trained for named entity recognition (NER) as an open, publicly available model. We demonstrate the effectiveness of combining multiple techniques in order to achieve strong results in entity recognition performance by the means of transfer-learning on pre-trained deep language models (LM), word-alignment and neural machine translation, outperforming a pre-existing baseline model on several datasets. Due to the sparse situation of open, public medical entity recognition models for German texts, this work offers benefits to the German research community on medical NLP as a baseline model. The work serves as a refined successor to our first GERNERMED model. Similar to our previous work, our trained model is publicly available to other researchers. The sample code and the statistical model is available at: https://github.com/frankkramer-lab/GERNERMED-pp.

Advances in monolingual and crosslingual automatic disability annotation in Spanish

BackgroundUnlike diseases, automatic recognition of disabilities has not received the same attention in the area of medical NLP. Progress in this direction is hampered by obstacles like the lack of annotated corpus. Neural architectures learn to translate sequences from spontaneous representations into their corresponding standard representations given a set of samples. The aim of this paper is to present the last advances in monolingual (Spanish) and crosslingual (from English to Spanish and vice versa) automatic disability annotation. The task consists of identifying disability mentions in medical texts written in Spanish within a collection of abstracts from journal papers related to the biomedical domain.ResultsIn order to carry out the task, we have combined deep learning models that use different embedding granularities for sequence to sequence tagging with a simple acronym and abbreviation detection module to boost the coverage.ConclusionsOur monolingual experiments demonstrate that a good combination of different word embedding representations provide better results than single representations, significantly outperforming the state of the art in disability annotation in Spanish. Additionally, we have experimented crosslingual transfer (zero-shot) for disability annotation between English and Spanish with interesting results that might help overcoming the data scarcity bottleneck, specially significant for the disabilities.

Bridging the Gap between Medical Tabular Data and NLP Predictive Models: A Fuzzy-Logic-Based Textualization Approach

The increasing use of electronic health records (EHRs) generates a vast amount of data, which can be leveraged for predictive modeling and improving patient outcomes. However, EHR data are typically mixtures of structured and unstructured data, which presents two major challenges. While several studies have focused on using machine learning models to predict patient outcomes, these models often require data to be in a structured format, which may lead to the loss of important information. On the other hand, unstructured data, such as narrative reports, can be noisy and challenging for natural language processing applications and interoperability. Therefore, there is a need to bridge the gap between structured EHR data and NLP-based predictive models. In this paper, we propose a fuzzy-logic-based pipeline that generates medical narratives from structured EHR data and evaluates its performance in predicting patient outcomes. The pipeline includes a feature selection operation and a reasoning and inference function that generates medical narratives. We then extensively evaluate the generated narratives using transformer-based NLP models for a patient-outcome-prediction task. We furthermore assess the interpretability of the generated text using Shapley values. Our approach has demonstrated comparable performance to the benchmark baseline models with an F1-score of 93.7%, while exhibiting slightly improved results in terms of recall. The model demonstrated proficiency in the preservation of information and interpretability inherited from nuanced and structured narratives. To the best of our knowledge, this is the first study to demonstrate the ability to transform tabular data into text to apply NLP for a prediction task.

Deep Active Learning via Open-Set Recognition.

In many applications, data is easy to acquire but expensive and time-consuming to label, prominent examples include medical imaging and NLP. This disparity has only grown in recent years as our ability to collect data improves. Under these constraints, it makes sense to select only the most informative instances from the unlabeled pool and request an oracle (e.g., a human expert) to provide labels for those samples. The goal of active learning is to infer the informativeness of unlabeled samples so as to minimize the number of requests to the oracle. Here, we formulate active learning as an open-set recognition problem. In this paradigm, only some of the inputs belong to known classes; the classifier must identify the rest as unknown. More specifically, we leverage variational neural networks (VNNs), which produce high-confidence (i.e., low-entropy) predictions only for inputs that closely resemble the training data. We use the inverse of this confidence measure to select the samples that the oracle should label. Intuitively, unlabeled samples that the VNN is uncertain about contain features that the network has not been exposed to; thus they are more informative for future training. We carried out an extensive evaluation of our novel, probabilistic formulation of active learning, achieving state-of-the-art results on MNIST, CIFAR-10, CIFAR-100, and FashionMNIST. Additionally, unlike current active learning methods, our algorithm can learn even in the presence of out-of-distribution outliers. As our experiments show, when the unlabeled pool consists of a mixture of samples from multiple datasets, our approach can automatically distinguish between samples from seen vs. unseen datasets. Overall, our results show that high-quality uncertainty measures are key for pool-based active learning.

A Comparison of Word Embeddings to Study Complications in Neurosurgery.

Our study aimed to compare the capability of different word embeddings to capture the semantic similarity of clinical concepts related to complications in neurosurgery at the level of medical experts. Eighty-four sets of word embeddings (based on Word2vec, GloVe, FastText, PMI, and BERT algorithms) were benchmarked in a clustering task. FastText model showed the best close to the medical expertise capability to group medical terms by their meaning (adjusted Rand index = 0.682). Word embedding models can accurately reflect clinical concepts' semantic and linguistic similarities, promising their robust usage in medical domain-specific NLP tasks.

Year 2020 (with COVID): Observation of Scientific Literature on Clinical Natural Language Processing.

SummaryObjectives: To analyze the content of publications within the medical NLP domain in 2020. Methods: Automatic and manual preselection of publications to be reviewed, and selection of the best NLP papers of the year. Analysis of the important issues. Results: Three best papers have been selected in 2020. We also propose an analysis of the content of the NLP publications in 2020, all topics included. Conclusion: The two main issues addressed in 2020 are related to the investigation of COVID-related questions and to the further adaptation and use of transformer models. Besides, the trends from the past years continue, such as diversification of languages processed and use of information from social networks

DiLBERT: Cheap Embeddings for Disease Related Medical NLP

DiLBERT: Cheap Embeddings for Disease Related Medical NLP

Learning Conceptual-Contextual Embeddings for Medical Text

External knowledge is often useful for natural language understanding tasks. We introduce a contextual text representation model called Conceptual-Contextual (CC) embeddings, which incorporates structured knowledge into text representations. Unlike entity embedding methods, our approach encodes a knowledge graph into a context model. CC embeddings can be easily reused for a wide range of tasks in a similar fashion to pre-trained language models. Our model effectively encodes the huge UMLS database by leveraging semantic generalizability. Experiments on electronic health records (EHRs) and medical text processing benchmarks showed our model gives a major boost to the performance of supervised medical NLP tasks.

Fine-Grained Emotion Detection in Suicide Notes: A Thresholding Approach to Multi-Label Classification

We present a system to automatically identify emotion-carrying sentences in suicide notes and to detect the specific fine-grained emotion conveyed. With this system, we competed in Track 2 of the 2011 Medical NLP Challenge,14 where the task was to distinguish between fifteen emotion labels, from guilt, sorrow, and hopelessness to hopefulness and happiness.Since a sentence can be annotated with multiple emotions, we designed a thresholding approach that enables assigning multiple labels to a single instance. We rely on the probability estimates returned by an SVM classifier and experimentally set thresholds on these probabilities. Emotion labels are assigned only if their probability exceeds a certain threshold and if the probability of the sentence being emotion-free is low enough. We show the advantages of this thresholding approach by comparing it to a naïve system that assigns only the most probable label to each test sentence, and to a system trained on emotion-carrying sentences only.

Labeling Emotions in Suicide Notes: Cost-Sensitive Learning with Heterogeneous Features

This paper describes a system developed for Track 2 of the 2011 Medical NLP Challenge on identifying emotions in suicide notes. Our approach involves learning a collection of one-versus-all classifiers, each deciding whether or not a particular label should be assigned to a given sentence. We explore a variety of features types—syntactic, semantic and surface-oriented. Cost-sensitive learning is used for dealing with the issue of class imbalance in the data.

Clinical text classification under the Open and Closed Topic Assumptions

This paper investigates multi-topic aspects in automatic classification of clinical free text in comparison with general text. In this paper, we facilitate two different views on multi-topics: the Closed Topic Assumption (CTA) and the Open Topic Assumption (OTA). Experimental results show that the characteristics of multi-topic assignments in the Computational Medicine Centre (CMC) Medical NLP Challenge Data is strongly OTA-oriented but general text Reuters-21578 is characterised in the middle of the OTA and CTA spectrum.

Magic in practice. introducing medical nlp: the art and science of language in healing and health Garner Thomson Magic in practice. introducing medical nlp: the art and science of language in healing and health Khalid Khan Hammersmith Press London 208 £24.99 9-781905140190 1905140193

The curiously entitled Magic in Practice is a useful text that aims to unravel the elusive qualities that comprise good relationships between doctors and patients.

Semi-structured document categorization with a semantic kernel

Since a decade, text categorization has become an active field of research in the machine learning community. Most of the approaches are based on the term occurrence frequency. The performance of such surface-based methods can decrease when the texts are too complex, i.e., ambiguous. One alternative is to use the semantic-based approaches to process textual documents according to their meaning. Furthermore, research in text categorization has mainly focused on “flat texts” whereas many documents are now semi-structured and especially under the XML format. In this paper, we propose a semantic kernel for semi-structured biomedical documents. The semantic meanings of words are extracted using the unified medical language system (UMLS) framework. The kernel, with a SVM classifier, has been applied to a text categorization task on a medical corpus of free text documents. The results have shown that the semantic kernel outperforms the linear kernel and the naive Bayes classifier. Moreover, this kernel was ranked in the top 10 of the best algorithms among 44 classification methods at the 2007 Computational Medicine Center (CMC) Medical NLP International Challenge.

Medical i2b2 NLP Smoking Challenge: The A-Life System Architecture and Methodology

We describe the architecture of LifeCode® (A-Life Medical, Inc.), a natural language processing system for free-text clinical information extraction, our methodology in applying LifeCode® to the i2b2 smoking challenge, and statistical measures for performance evaluation. Due to the limited test size and the coefficient of variation in the test standard, it is difficult to draw conclusions regarding the relative efficacy of approaches that were applied to this challenge.