Natural Language Processing Pipeline Research Articles

Improving surveillance of emergency department activity through natural language processing

Objective and ApproachManagers, clinicians and policymakers have limited information on near-real time patterns of attendance at emergency departments (ED). Our aim was to develop a surveillance system through extracting information from the presenting complaint to develop and feed activity dashboards, improve injury surveillance and support research. We developed a Natural Language Processing (NLP) pipeline using the General Architecture for Text Engineering (GATE) system. The pipeline produced Unified Medical Language System (UMLS) codes from ED presenting complaint records and also implemented the Joint Action on Monitoring Injuries in Europe (JAMIE) Minimum Dataset. We classified complaints into 31 common problem presentations (Coughs, Bleeds, Mental Health, Dizziness...), along with categories from JAMIE (Mechanism of Injury, Location, Activity, Intent…). ResultsPreliminary results are from a pilot study of 86,416 presenting complaints from Morriston Hospital in 2014. The most common terms used for grouped symptoms/categories were “Injury” (38.6%, n=28691), “Pain” (22.8%, n=19420), “Unwell” (7.2%, n=6117), “Bleeding” (6.7%, n=5760), and “Shortness of Breath” (3.8%, n=3255). 32.3% of records contained Mechanism of Injury codes, including, falls (15.2%, n=13174), struck by/against (6.9%, n=5960), cuts (5.6%, n=4833), road traffic collisions (2.5%, n=2124) and poisoning (1%, n=900). We are continuing to refine and validate the categories using precision and recall and with comparison with coded diagnoses. Conclusions and Implications NLP can be used to extract valuable additional detailed information from ED data. We will further develop the algorithms for all hospitals in Wales for data from 2023 and design dashboards to produce a near-real time surveillance system.

An innovative linked Electronic Health Record derived data ecosystem for understanding the complexities of health and ageing.

ObjectiveWe describe the creation of an Electronic Health Record (EHR) derived linked data platform within the Australian National Centre for Healthy Ageing (NCHA). ApproachOur approach involved establishing a core set of EHR data suitable for research from the sole public health provider within a defined geographic region. Relevant items were selected based on published literature and consensus processes and curated within a specialised research data warehouse. Curation involved: data validation, quality assessment, internal linkage between episodes and data harmonisation/merging. An AI supported natural language processing pipeline was implemented to extract text-based data. Approvals were obtained from government agencies to link commonwealth claims data (primary care, medication, aged care, death) and state administrative hospital data, for all residents aged ≥60 years. Publicly available datasets were scoped for inclusion and collaborations established to incorporate local council data. ResultsA core of 113 curated items for >900,000 people over 10 years have been established across 4 hospitals and 10 community/outpatient services. Linked state and commonwealth data were obtained for 179,089 residents aged >60 years (Jan 2010-May 2021, linkage accuracy 98.4%). Environmental (greenery, air pollution, walkability) and Census data have been incorporated for each neighbourhood (Statistical Area 1). Data access, extraction and release has been tested through over 30 use-case projects in: dementia, residential aged care, medication use, homelessness, environmental impacts on health and ageing and health service redesign. ImplicationsThe NCHA Data Platform provides an international exemplar for the use of linked EHR data in advancing population health research.

Development of a natural language processing pipeline for assessment of cardiovascular risk in myeloproliferative neoplasms.

Development of a natural language processing pipeline for assessment of cardiovascular risk in myeloproliferative neoplasms.

Use of Natural Language Processing to Extract and Classify Papillary Thyroid Cancer Features From Surgical Pathology Reports

BackgroundWe aim to use Natural Language Processing to automate the extraction and classification of thyroid cancer risk factors from pathology reports. MethodsWe analyzed 1410 surgical pathology reports from adult papillary thyroid cancer patients from 2010 to 2019. Structured and nonstructured reports were used to create a consensus-based ground truth dictionary and categorized them into modified recurrence risk levels. Nonstructured reports were narrative, while structured reports followed standardized formats. We developed ThyroPath, a rule-based Natural Language Processing pipeline, to extract and classify thyroid cancer features into risk categories. Training involved 225 reports (150 structured, 75 unstructured), with testing on 170 reports (120 structured, 50 unstructured) for evaluation. The pipeline's performance was assessed using both strict and lenient criteria for accuracy, precision, recall, and F1-score; a metric that combines precision and recall evaluation. ResultsIn extraction tasks, ThyroPath achieved overall strict F-1 scores of 93% for structured reports and 90% for unstructured reports, covering 18 thyroid cancer pathology features. In classification tasks, ThyroPath-extracted information demonstrated an overall accuracy of 93% in categorizing reports based on their corresponding guideline-based risk of recurrence: 76.9% for high-risk, 86.8% for intermediate risk, and 100% for both low and very low-risk cases. However, ThyroPath achieved 100% accuracy across all risk categories with human extracted pathology information. ConclusionsThyroPath shows promise in automating the extraction and risk recurrence classification of thyroid pathology reports at large scale. It offers a solution to laborious manual reviews and advancing virtual registries. However, it requires further validation before implementation.

Improving biomedical entity linking for complex entity mentions with LLM-based text simplification.

Large amounts of important medical information are captured in free-text documents in biomedical research and within healthcare systems, which can be made accessible through natural language processing (NLP). A key component in most biomedical NLP pipelines is entity linking, i.e. grounding textual mentions of named entities to a reference of medical concepts, usually derived from a terminology system, such as the Systematized Nomenclature of Medicine Clinical Terms. However, complex entity mentions, spanning multiple tokens, are notoriously hard to normalize due to the difficulty of finding appropriate candidate concepts. In this work, we propose an approach to preprocess such mentions for candidate generation, building upon recent advances in text simplification with generative large language models. We evaluate the feasibility of our method in the context of the entity linking track of the BioCreative VIII SympTEMIST shared task. We find that instructing the latest Generative Pre-trained Transformer model with a few-shot prompt for text simplification results in mention spans that are easier to normalize. Thus, we can improve recall during candidate generation by 2.9percentage points compared to our baseline system, which achieved the best score in the original shared task evaluation. Furthermore, we show that this improvement in recall can be fully translated into top-1 accuracy through careful initialization of a subsequent reranking model. Our best system achieves an accuracy of 63.6% on the SympTEMIST test set. The proposed approach has been integrated into the open-source xMEN toolkit, which is available online via https://github.com/hpi-dhc/xmen.

Demystifying large language models in second language development research

Evaluating students' textual response is a common and critical task in language research and education practice. However, manual assessment can be tedious and may lack consistency, posing challenges for both scientific discovery and frontline teaching. Leveraging state-of-the-art large language models (LLMs), we aim to define and operationalize LLM-Surprisal, a numeric representation of the interplay between lexical diversity and syntactic complexity, and to empirically and theoretically demonstrate its relevance for automatic writing assessment and Chinese L2 (second language) learners’ English writing development. We developed an LLM-based natural language processing pipeline that can automatically compute text Surprisal scores. By comparing Surprisal metrics with the widely used classic indices in L2 studies, we extended the usage of computational metrics in Chinese learners’ L2 English writing. Our analyses suggested that LLM-Surprisals can distinguish L2 from L1 (first language) writing, index L2 development stages, and predict scores provided by human professionals. This indicated that the Surprisal dimension may manifest itself as critical aspects in L2 development. The relative advantages and disadvantages of these approaches were discussed in depth. We concluded that LLMs are promising tools that can enhance L2 research. Our showcase paves the way for more nuanced approaches to computationally assessing and understanding L2 development. Our pipelines and findings will inspire language teachers, learners, and researchers to operationalize LLMs in an innovative and accessible manner.

Representation of Social Determinants of Health terminology in medical subject headings: impact of added terms.

To enhance and evaluate the quality of PubMed search results for Social Determinants of Health (SDoH) through the addition of new SDoH terms to Medical Subject Headings (MeSH). High priority SDoH terms and definitions were collated from authoritative sources, curated based on publication frequencies, and refined by subject matter experts. Descriptive analyses were used to investigate how PubMed search details and best match results were affected by the addition of SDoH concepts to MeSH. Three information retrieval metrics (Precision, Recall, and F measure) were used to quantitatively assess the accuracy of PubMed search results. Pre- and post-update documents were clustered into topic areas using a Natural Language Processing pipeline, and SDoH relevancy assessed. Addition of 35 SDoH terms to MeSH resulted in more accurate algorithmic translations of search terms and more reliable best match results. The Precision, Recall, and F measures of post-update results were significantly higher than those of pre-update results. The percentage of retrieved publications belonging to SDoH clusters was significantly greater in the post- than pre-update searches. This evaluation confirms that inclusion of new SDoH terms in MeSH can lead to qualitative and quantitative enhancements in PubMed search retrievals. It demonstrates the methodology for and impact of suggesting new terms for MeSH indexing. It provides a foundation for future efforts across behavioral and social science research (BSSR) domains. Improving the representation of BSSR terminology in MeSH can improve PubMed search results, thereby enhancing the ability of investigators and clinicians to build and utilize a cumulative BSSR knowledge base.

Combining Federated Machine Learning and Qualitative Methods to Investigate Novel Pediatric Asthma Subtypes: Protocol for a Mixed Methods Study.

Pediatric asthma is a heterogeneous disease; however, current characterizations of its subtypes are limited. Machine learning (ML) methods are well-suited for identifying subtypes. In particular, deep neural networks can learn patient representations by leveraging longitudinal information captured in electronic health records (EHRs) while considering future outcomes. However, the traditional approach for subtype analysis requires large amounts of EHR data, which may contain protected health information causing potential concerns regarding patient privacy. Federated learning is the key technology to address privacy concerns while preserving the accuracy and performance of ML algorithms. Federated learning could enable multisite development and implementation of ML algorithms to facilitate the translation of artificial intelligence into clinical practice. The aim of this study is to develop a research protocol for implementation of federated ML across a large clinical research network to identify and discover pediatric asthma subtypes and their progression over time. This mixed methods study uses data and clinicians from the OneFlorida+ clinical research network, which is a large regional network covering linked and longitudinal patient-level real-world data (RWD) of over 20 million patients from Florida, Georgia, and Alabama in the United States. To characterize the subtypes, we will use OneFlorida+ data from 2011 to 2023 and develop a research-grade pediatric asthma computable phenotype and clinical natural language processing pipeline to identify pediatric patients with asthma aged 2-18 years. We will then apply federated learning to characterize pediatric asthma subtypes and their temporal progression. Using the Promoting Action on Research Implementation in Health Services framework, we will conduct focus groups with practicing pediatric asthma clinicians within the OneFlorida+ network to investigate the clinical utility of the subtypes. With a user-centered design, we will create prototypes to visualize the subtypes in the EHR to best assist with the clinical management of children with asthma. OneFlorida+ data from 2011 to 2023 have been collected for 411,628 patients aged 2-18 years along with 11,156,148 clinical notes. We expect to complete the computable phenotyping within the first year of the project, followed by subtyping during the second and third years, and then will perform the focus groups and establish the user-centered design in the fourth and fifth years of the project. Pediatric asthma subtypes incorporating RWD from diverse populations could improve patient outcomes by moving the field closer to precision pediatric asthma care. Our privacy-preserving federated learning methodology and qualitative implementation work will address several challenges of applying ML to large, multicenter RWD data. DERR1-10.2196/57981.

Clinical efficacy of pre-trained large language models through the lens of aphasia

The rapid development of large language models (LLMs) motivates us to explore how such state-of-the-art natural language processing systems can inform aphasia research. What kind of language indices can we derive from a pre-trained LLM? How do they differ from or relate to the existing language features in aphasia? To what extent can LLMs serve as an interpretable and effective diagnostic and measurement tool in a clinical context? To investigate these questions, we constructed predictive and correlational models, which utilize mean surprisals from LLMs as predictor variables. Using AphasiaBank archived data, we validated our models’ efficacy in aphasia diagnosis, measurement, and prediction. Our finding is that LLMs-surprisals can effectively detect the presence of aphasia and different natures of the disorder, LLMs in conjunction with the existing language indices improve models’ efficacy in subtyping aphasia, and LLMs-surprisals can capture common agrammatic deficits at both word and sentence level. Overall, LLMs have potential to advance automatic and precise aphasia prediction. A natural language processing pipeline can be greatly benefitted from integrating LLMs, enabling us to refine models of existing language disorders, such as aphasia.

Early evaluation of a natural language processing tool to improve access to educational resources for surgical patients.

Accessible patient information sources are vital in educating patients about the benefits and risks of spinal surgery, which is crucial for obtaining informed consent. We aim to assess the effectiveness of a natural language processing (NLP) pipeline in recognizing surgical procedures from clinic letters and linking this with educational resources. Retrospective examination of letters from patients seeking surgery for degenerative spinal disease at a single neurosurgical center. We utilized MedCAT, a named entity recognition and linking NLP, integrated into the electronic health record (EHR), which extracts concepts and links them to systematized nomenclature of medicine-clinical terms (SNOMED-CT). Investigators reviewed clinic letters, identifying words or phrases that described or identified operations and recording the SNOMED-CT terms as ground truth. This was compared to SNOMED-CT terms identified by the model, untrained on our dataset. A pipeline linking clinic letters to patient-specific educational resources was established, and precision, recall, and F1 scores were calculated. Across 199 letters the model identified 582 surgical procedures, and the overall pipeline after adding rules a total of 784 procedures (precision = 0.94, recall = 0.86, F1 = 0.91). Across 187 letters with identified SNOMED-CT terms the integrated pipeline linking education resources directly to the EHR was successful in 157 (78%) patients (precision = 0.99, recall = 0.87, F1 = 0.92). NLP accurately identifies surgical procedures in pre-operative clinic letters within an untrained subspecialty. Performance varies among letter authors and depends on the language used by clinicians. The identified procedures can be linked to patient education resources, potentially improving patients' understanding of surgical procedures.

Advancing equity in breast cancer care: natural language processing for analysing treatment outcomes in under-represented populations

ObjectiveThe study aimed to develop natural language processing (NLP) algorithms to automate extracting patient-centred breast cancer treatment outcomes from clinical notes in electronic health records (EHRs), particularly for women from...

Age modulates the predictive value of self-reported sleepiness for all-cause mortality risk: insights from a comprehensive national database of veterans.

Excessive daytime sleepiness (EDS) is prevalent and overwhelmingly stems from disturbed sleep. We hypothesized that age modulates the association between EDS and increased all-cause mortality. We utilized the Veterans' Health Administration data from 1999-2022. We enrolled participants with sleep related ICD9/10 codes or sleep services. A natural language processing (NLP) pipeline was developed and validated to extract the Epworth Sleepiness Scale (ESS) as a self-reported tool to measure EDS from physician progress notes. The NLP's accuracy was assessed through manual annotation of 470 notes. Participants were categorized into Normal-ESS, n-ESS, (ESS 0-10) and high-ESS, h-ESS, (ESS 11-24). We created three age groups: < 50 years; 50 to < 65 years; and ≥ 65 years. The adjusted odds ratio (aOR) of mortality was calculated for age, BMI, sex, race, ethnicity, and the Charlson Comorbidity Index (CCI), using n-ESS as the reference. Subsequently, we conducted age stratified analysis. The first ESS records were extracted from 423,087 veterans with a mean age of 54.8 (±14.6), mean BMI of 32.6 (±6.2), and 90.5% male. The aOR across all ages was 17% higher (1.15,1.19) in the h-ESS category. The aORs only became statistically significant for individuals aged ≥ 50 years in the h-ESS compared to the n-ESS category (< 50 years: 1.02 [0.96,1.08], 50 to < 65 years 1.13[1.10,1.16]; ≥ 65 years: 1.25 [1.21-1.28]). High ESS, predicted increased mortality only in participants aged 50 and older. Further research is required to identify this differential behavior in relation to age.

Age modifies the association between severe sleep apnea and all-cause mortality

PurposeWhile sleep apnea (SA) gets more prevalent with advancing age, the impact of age on the association between SA and health outcomes is not well known. We assessed the association between the severity of SA and all-cause mortality in different age groups using large longitudinal data. MethodWe applied a Natural Language Processing pipeline to extract the apnea-hypopnea index (AHI) from the physicians’ interpretation of sleep studies performed at the Veteran Health Administration (FY 1999–2022). We categorized the participants as no SA (n-SA, AHI< 5) and severe SA (s-SA, AHI≥30). We grouped the cohort based on age: Young≤40; Middle-aged:40–65; and Older adults≥65; and calculated the odds ratio (aOR) of mortality adjusted for age, sex, race, ethnicity, BMI, and Charlson-Comorbidity Index (CCI) using n-SA as the reference. ResultsWe identified 146,148 participants (age 52.23 ± 15.02; BMI 32.11 ± 6.05; male 86.7 %; White 66 %). Prevalence of s-SA increased with age. All-cause mortality was lower in s-SA compared to n-SA in the entire cohort (aOR,0.56; 95%CI: 0.54,0.58). Comparing s-SA to n-SA, the all-cause mortality rates (Young 1.86 % vs 1.49 %; Middle-aged 12.07 % vs 13.34 %; and Older adults 26.35 % vs 40.18 %) and the aOR diminished as the age increased (Young: 1.11, 95%CI: 0.93–1.32; Middle-aged: 0.64, 95%CI: 0.61–0.67; and Older adults: 0.44, 95%CI: 0.41–0.46). ConclusionThe prevalence of severe SA increased while the odds of all-cause mortality compared to n-SA diminished with age. SA may exert less harmful effects on the aged population. A causality analysis is warranted to assess the relationship between SA, aging, and all-cause mortality.

Natural language processing pipeline to extract prostate cancer-related information from clinical notes.

To develop an automated pipeline for extracting prostate cancer-related information from clinical notes. This retrospective study included 23,225 patients who underwent prostate MRI between 2017 and 2022. Cancer risk factors (family history of cancer and digital rectal exam findings), pre-MRI prostate pathology, and treatment history of prostate cancer were extracted from free-text clinical notes in English as binary or multi-class classification tasks. Any sentence containing pre-defined keywords was extracted from clinical notes within one year before the MRI. After manually creating sentence-level datasets with ground truth, Bidirectional Encoder Representations from Transformers (BERT)-based sentence-level models were fine-tuned using the extracted sentence as input and the category as output. The patient-level output was determined by compilation of multiple sentence-level outputs using tree-based models. Sentence-level classification performance was evaluated using the area under the receiver operating characteristic curve (AUC) on 15% of the sentence-level dataset (sentence-level test set). The patient-level classification performance was evaluated on the patient-level test set created by radiologists by reviewing the clinical notes of 603 patients. Accuracy and sensitivity were compared between the pipeline and radiologists. Sentence-level AUCs were ≥ 0.94. The pipeline showed higher patient-level sensitivity for extracting cancer risk factors (e.g., family history of prostate cancer, 96.5% vs. 77.9%, p < 0.001), but lower accuracy in classifying pre-MRI prostate pathology (92.5% vs. 95.9%, p = 0.002) and treatment history of prostate cancer (95.5% vs. 97.7%, p = 0.03) than radiologists, respectively. The proposed pipeline showed promising performance, especially for extracting cancer risk factors from patient's clinical notes. The natural language processing pipeline showed a higher sensitivity for extracting prostate cancer risk factors than radiologists and may help efficiently gather relevant text information when interpreting prostate MRI. When interpreting prostate MRI, it is necessary to extract prostate cancer-related information from clinical notes. This pipeline extracted the presence of prostate cancer risk factors with higher sensitivity than radiologists. Natural language processing may help radiologists efficiently gather relevant prostate cancer-related text information.

Novel approach to implementing natural language processing for clinical staging of non-small-cell lung cancer.

e13624 Background: Cancer staging is instrumental in driving clinical management and trial enrollment, but staging data are generally unreliable and unstructured in the electronic health record (EHR). Advances in natural language processing (NLP) may facilitate clinical staging and documentation [1], but challenges to real-world implementation include (1) automatically identifying appropriate patients and reports from the EHR and (2) developing an unbiased dataset for training and validation [2]. We describe our institution’s novel approach to overcome these barriers while building an in-house NLP pipeline for clinical tumor staging of non-small-cell lung cancer (NSCLC). Methods: We identified patients by searching our EHR (Epic) for a molecular analysis test ordered specifically for pathological diagnoses of NSCLC at our institution. We used the test order date as the diagnosis proxy date (DPD). For each patient, we extracted imaging reports up to 16 weeks before and 6 weeks after the DPD. To derive primary tumor size, we analyzed the CT Chest or PET/CT report closest to the DPD using an oncology-trained NLP text extraction and labeling tool (John Snow Labs). We cleaned all extracted tumor size entities and identified the largest measurement linked to the lungs. We compared primary tumor measurements from the NLP pipeline to those in a preexisting, manually compiled cancer registry (CNEXT). We manually analyzed discrepancies through chart review. Results: 542 patients with a DPD between 11/2016 - 9/2023 were processed through the NLP pipeline. Of 443 patients with valid values in both the pipeline and CNEXT, 53% (234) were exact matches, and 20% (90) had a close match (within 0-5mm), yielding a 73% accuracy rate for values within 5mm. When mismatched values were manually reviewed, several cases in CNEXT were found to have a DPD differing by more than 3 months and tumor sizes derived from external reports. When these cases were excluded, 320 of the remaining 349 patients had valid values in both the pipeline and the updated manual review. In this refined population, 66% (213) were exact matches, and 15% (48) had a close match, yielding an 82% accuracy rate for values within 5mm. Conclusions: To our knowledge, this is the first report of a pathology-based method to automatically and reliably identify patients with NSCLC and their relevant imaging reports directly from the EHR. We used a prebuilt NLP tool to derive primary tumor sizes with relatively high accuracy and found that adding flags for timeline discrepancies and external reports can further improve validity. As we near completion of analogous pipelines for node and metastasis staging, we will develop methodology to identify subgroups of patients that can be clinically staged with near-perfect accuracy, ultimately aiming to substantially limit manual staging of uncomplicated cases. 1. Puts 2023. 2. Wang 2022.

Implementing a natural language processing pipeline to expedite clinical chart review to assess germline genetic testing in veterans with breast cancer.

e13640 Background: The Making Advances in Mammography and Medical Options for Veterans Act of 2022 mandated that the Department of Veterans Affairs (VA) perform a study of germline genetic testing in Veterans with breast cancer (BC). Assessing whether a Veteran with BC was offered germline genetic testing based on electronic health records requires time-consuming review of unstructured clinical notes by clinical experts who are also extremely busy providing care. To expedite this review process, we implemented a productionized natural language processing (NLP) pipeline to identify relevant concepts and highlight them within a custom-made chart review interface. Methods: Clinical subject-matter experts (SMEs) were convened to identify keywords suggestive of genetic testing. These were categorized into 3 sets: gene-related concepts, VA-specific services, and exams and tests. We then used entity recognition and entity linking to link these concepts to a knowledge base. These were enriched with entities similarly extracted from an open-source reference clinical text on germline genetic testing in BC. The resulting concepts were visualized as a word cloud and reviewed by the SMEs. We implemented a Power BI and Power Apps interface to display clinical notes from 200 randomly selected Veterans, with relevant concepts highlighted. Each Veteran’s notes were reviewed independently by 2 of the 5 SMEs, who assessed whether the Veteran was offered testing, referred to a genetic counseling service, or otherwise had evidence of testing. Disagreements were adjudicated by the whole team. Results: 11622 notes (mean 58/Veteran, 3028 characters/note) were reviewed. Table shows the most prevalent entities among genes, mutations, and exam names. Of the 5 reviewers, 3 annotated 80 patients each in a single session, lasting less than 50 minutes on average. Reviewers had an inter-annotator agreement of 85.5%. Mention of target concepts was associated with germline genetic testing with an accuracy of 80.4%, precision of 76.4%, recall of 99.1%, and F1 of 86.3%. Germline genetic testing rates are detailed in a separate submission. Conclusions: Modern informatics tools such as productionized NLP, the Microsoft Power Platform, and high-performance computing environments allow for rapid implementation of health information technology tools that can enable clinicians to gain critical insights quickly and efficiently from electronic health record data. [Table: see text]

An NLP-based approach to assessing a company’s maturity level in the digital era

Conducting a maturity assessment allows companies to measure their readiness in implementing novel technologies. However, this task is challenging due to the multidimensional, complex, unpredictable, and non-linear nature of innovation. In this paper, we introduce an innovative approach to maturity assessment that enables both intra- and inter-company analysis. Our approach evaluates a company’s absolute maturity score concerning a specific technology or area. By leveraging a Natural Language Processing pipeline applied to a semi-structured questionnaire we extract popular concepts from the answers and present them to a human expert for analysis. The expert can refine the analysis by adding or removing concepts as needed. Subsequently, we compute a similarity metric for each answer to determine a company’s maturity in specific concepts. The output of our analysis is presented through human-readable plots, offering clear insights into the internal maturity level of the company and allowing for a comparison with competitors across the chosen concepts. To demonstrate the capabilities of our method, we provide a running example showcasing both quantitative and qualitative results of the analysis. Our approach demonstrates efficiency, with preprocessing completed in 1.967±0.758 s, and information extraction in 0.074±0.017 s on average, excluding human intervention time, and requiring low hardware resources.

Identification of patients' smoking status using an explainable AI approach: a Danish electronic health records case study.

Smoking is a critical risk factor responsible for over eight million annual deaths worldwide. It is essential to obtain information on smoking habits to advance research and implement preventive measures such as screening of high-risk individuals. In most countries, including Denmark, smoking habits are not systematically recorded and at best documented within unstructured free-text segments of electronic health records (EHRs). This would require researchers and clinicians to manually navigate through extensive amounts of unstructured data, which is one of the main reasons that smoking habits are rarely integrated into larger studies. Our aim is to develop machine learning models to classify patients' smoking status from their EHRs. This study proposes an efficient natural language processing (NLP) pipeline capable of classifying patients' smoking status and providing explanations for the decisions. The proposed NLP pipeline comprises four distinct components, which are; (1) considering preprocessing techniques to address abbreviations, punctuation, and other textual irregularities, (2) four cutting-edge feature extraction techniques, i.e. Embedding, BERT, Word2Vec, and Count Vectorizer, employed to extract the optimal features, (3) utilization of a Stacking-based Ensemble (SE) model and a Convolutional Long Short-Term Memory Neural Network (CNN-LSTM) for the identification of smoking status, and (4) application of a local interpretable model-agnostic explanation to explain the decisions rendered by the detection models. The EHRs of 23,132 patients with suspected lung cancer were collected from the Region of Southern Denmark during the period 1/1/2009-31/12/2018. A medical professional annotated the data into 'Smoker' and 'Non-Smoker' with further classifications as 'Active-Smoker', 'Former-Smoker', and 'Never-Smoker'. Subsequently, the annotated dataset was used for the development of binary and multiclass classification models. An extensive comparison was conducted of the detection performance across various model architectures. The results of experimental validation confirm the consistency among the models. However, for binary classification, BERT method with CNN-LSTM architecture outperformed other models by achieving precision, recall, and F1-scores between 97% and 99% for both Never-Smokers and Active-Smokers. In multiclass classification, the Embedding technique with CNN-LSTM architecture yielded the most favorable results in class-specific evaluations, with equal performance measures of 97% for Never-Smoker and measures in the range of 86 to 89% for Active-Smoker and 91-92% for Never-Smoker. Our proposed NLP pipeline achieved a high level of classification performance. In addition, we presented the explanation of the decision made by the best performing detection model. Future work will expand the model's capabilities to analyze longer notes and a broader range of categories to maximize its utility in further research and screening applications.

The application of natural language processing for the extraction of mechanistic information in toxicology.

To study the ways in which compounds can induce adverse effects, toxicologists have been constructing Adverse Outcome Pathways (AOPs). An AOP can be considered as a pragmatic tool to capture and visualize mechanisms underlying different types of toxicity inflicted by any kind of stressor, and describes the interactions between key entities that lead to the adverse outcome on multiple biological levels of organization. The construction or optimization of an AOP is a labor intensive process, which currently depends on the manual search, collection, reviewing and synthesis of available scientific literature. This process could however be largely facilitated using Natural Language Processing (NLP) to extract information contained in scientific literature in a systematic, objective, and rapid manner that would lead to greater accuracy and reproducibility. This would support researchers to invest their expertise in the substantive assessment of the AOPs by replacing the time spent on evidence gathering by a critical review of the data extracted by NLP. As case examples, we selected two frequent adversities observed in the liver: namely, cholestasis and steatosis denoting accumulation of bile and lipid, respectively. We used deep learning language models to recognize entities of interest in text and establish causal relationships between them. We demonstrate how an NLP pipeline combining Named Entity Recognition and a simple rules-based relationship extraction model helps screen compounds related to liver adversities in the literature, but also extract mechanistic information for how such adversities develop, from the molecular to the organismal level. Finally, we provide some perspectives opened by the recent progress in Large Language Models and how these could be used in the future. We propose this work brings two main contributions: 1) a proof-of-concept that NLP can support the extraction of information from text for modern toxicology and 2) a template open-source model for recognition of toxicological entities and extraction of their relationships. All resources are openly accessible via GitHub (https://github.com/ontox-project/en-tox).

Text mining and portal development for gene-specific publications on Alzheimer’s disease and other neurodegenerative diseases

BackgroundTremendous research efforts have been made in the Alzheimer’s disease (AD) field to understand the disease etiology, progression and discover treatments for AD. Many mechanistic hypotheses, therapeutic targets and treatment strategies have been proposed in the last few decades. Reviewing previous work and staying current on this ever-growing body of AD publications is an essential yet difficult task for AD researchers.MethodsIn this study, we designed and implemented a natural language processing (NLP) pipeline to extract gene-specific neurodegenerative disease (ND) -focused information from the PubMed database. The collected publication information was filtered and cleaned to construct AD-related gene-specific publication profiles. Six categories of AD-related information are extracted from the processed publication data: publication trend by year, dementia type occurrence, brain region occurrence, mouse model information, keywords occurrence, and co-occurring genes. A user-friendly web portal is then developed using Django framework to provide gene query functions and data visualizations for the generalized and summarized publication information.ResultsBy implementing the NLP pipeline, we extracted gene-specific ND-related publication information from the abstracts of the publications in the PubMed database. The results are summarized and visualized through an interactive web query portal. Multiple visualization windows display the ND publication trends, mouse models used, dementia types, involved brain regions, keywords to major AD-related biological processes, and co-occurring genes. Direct links to PubMed sites are provided for all recorded publications on the query result page of the web portal.ConclusionThe resulting portal is a valuable tool and data source for quick querying and displaying AD publications tailored to users’ interested research areas and gene targets, which is especially convenient for users without informatic mining skills. Our study will not only keep AD field researchers updated with the progress of AD research, assist them in conducting preliminary examinations efficiently, but also offers additional support for hypothesis generation and validation which will contribute significantly to the communication, dissemination, and progress of AD research.