Deep Learning In Natural Language Processing Research Articles

A deep learning classification framework for research methods of marine protected area management

The latest emerging transdisciplinary marine protected area (MPA) research scheme requires efficient approaches for theoretically based and data-driven method integration. However, due to the rapid development and diversification of research methods, it is growingly difficult to locate new methods in methodological dimensions and integrate them to the utmost utility. This study proposes a deep learning-based classification framework for MPA management methods focused particularly on data and theory capabilities using natural language processing (NLP). It extracted keywords from academic sources and performed clustering based on semantic similarity, generating benchmark texts for abstract labeling. By training the deep learning NLP model and analyzing the abstracts of 9049 MPA management empirical research articles from 1986 to 2024, the data and theory scores were attributed to each article, and a total of 19 major method categories and 110 segment branches were identified in qualitative, quantitative, and mixed genres. Combination types of research methods were summarized, yielding the data-theory neutralization principle where the average data and theory scores tend to approximate 0.50. Applying the principle broadens traditional boundaries for method integration and extends method synthesis to higher numbers, generating a practical research 2paradigm for future MPA research. Implications include bridging social and ecological data, theorizing emergent challenges in complex systems and integrating theory construction and data science. The framework is applicable to quantification of other environmental management disciplines and can serve as guidance for multidisciplinary method integration.© 2017 Elsevier Inc. All rights reserved.

Evolution and advancements in deep learning models for Natural Language Processing

This paper provides a comprehensive review of the evolution and advancements in deep learning models for Natural Language Processing (NLP). It explores the transition from statistical models to neural networks, highlighting the paradigm shift towards data-driven methodologies and the implications for NLP tasks. The emergence of neural network architectures, such as Recurrent Neural Networks (RNNs) and transformer-based models like BERT and GPT, has revolutionized language understanding and generation. Furthermore, the integration of deep learning in traditional NLP tasks, such as part-of-speech tagging and named entity recognition, has led to significant improvements in accuracy and efficiency. The paper also discusses the quantitative analysis of deep learning models, including performance metrics, computational efficiency, and mathematical modeling of language tasks. Case studies and applications, including sentiment analysis, machine translation, and automated content generation, exemplify the transformative impact of deep learning in NLP.

A deep learning–enabled workflow to estimate real world progression-free survival in patients with metastatic breast cancer.

11176 Background: Progression-free survival (PFS) is frequently measured in oncology clinical trials. In analyses outside of the trial setting, various strategies have been utilized to assess real-world PFS (rwPFS), including manual abstraction of clinical records, natural language processing (NLP) of oncology notes and radiology reports, and longitudinal analyses of radiologic images. Here we develop and validate a new semi-automated workflow that combines NLP of clinical notes with structured electronic health record data to facilitate the evaluation of rwPFS in patients with metastatic breast cancer (mBC). Methods: This study analyzes de-identified EHR data using nference nSights. The data is extracted following privacy-preserving protocols and is HIPAA compliant. The overall cohort included 316 patients with HR-positive, HER2-negative mBC who initiated Palbociclib and Letrozole combination therapy between January 1, 2015 and December 31, 2021. We developed and implemented an ensemble of deep-learning NLP frameworks to capture progression events from unstructured clinical notes and radiology reports. A change in the line of therapy, as determined by structured drug order/administration records, was also considered a progression event. We used manually curated “ground-truth” datasets to evaluate the performance of the progression-event capture workflow at the levels of both sentences (N = 1000) and patients (N = 100) by calculating sensitivity, specificity, precision, accuracy, and F1 scores. Progression events and censoring events (death, loss to follow-up, end of study period) were considered to compute rwPFS. Results: At the sentence level, progression events were captured from clinical notes and radiology reports with a sensitivity of 99.8%, specificity of 96.7%, and accuracy of 98.2% (Table). At the patient level, initial progression was correctly captured within a window of +/-30 days with a sensitivity of 92.5%, specificity of 83.0%, and accuracy of 88.0% (Table). In a sample of 100 patients, the median rwPFS was determined to be 25 months (95% CI; 15-35 months) by manual curation and 22 months (95% CI; 15-35 months) by the semi-automated workflow. In the overall cohort, median rwPFS was 20 months (95% CI; 18-25 months). Conclusions: An ensemble of NLP algorithms extracted progression events from clinical notes and radiology reports with high accuracy. A semi-automated workflow enabled rapid and accurate determination of rwPFS in mBC patients receiving a combination chemotherapy regimen. Further evaluation of this workflow to estimate rwPFS in other cancers and therapeutic settings is warranted. [Table: see text]

Assessment of Gender Differences in Letters of Recommendation for Physical Therapy Residency Applications.

Letters of recommendation (LOR) are an integral component of physical therapy residency applications. Identifying the influence of applicant and writer gender in LOR will help identify whether potential implicit gender bias exists in physical therapy residency application processes. Several medical and surgical residency education programs have reported positive, neutral, or negative LOR female gender bias among applicants and writers. Little research exists on gender differences in LOR to physical therapy education programs or physical therapy residency programs. Seven hundred sixty-eight LOR were analyzed from 256 applications to 3 physical therapy residency programs (neurologic, orthopaedic, sports) at one institution from 2014 to 2020. Thematic categories were developed to identify themes in a sample of LOR. Associations between writer and applicant gender were analyzed using summary statistics, word counts, thematic and psycholinguistic extraction, and rule-based and deep learning Natural Language Processing . No significant difference in LOR word counts were found based on writer or applicant gender. Increased word counts were seen in sports residency LOR compared with the orthopaedic residency. Thematic analysis showed LOR gender differences with male applicants receiving more positive generalized recommendations and female applicants receiving more comments regarding interpersonal relationship skills. No thematic or psycholinguistic gender differences were seen by LOR writer. Male applicants were 1.9 times more likely to select all male LOR writers, whereas female applicants were 2.1 times more likely to choose all female LOR writers. Gender differences in LORs for physical therapy residencies were found using a comprehensive Natural Language Processing approach that identified both a positive recommendation male applicant gender bias and a positive interpersonal relationship skill female applicant gender bias. Applicants were not harmed nor helped by selecting LOR writers of the opposite gender. Admissions committees and LOR writers should be mindful of potential implicit gender biases in LOR submitted to physical therapy residency programs.

A Combined Manual Annotation and Deep-Learning Natural Language Processing Study on Accurate Entity Extraction in Hereditary Disease Related Biomedical Literature

We report a combined manual annotation and deep-learning natural language processing study to make accurate entity extraction in hereditary disease related biomedical literature. A total of 400 full articles were manually annotated based on published guidelines by experienced genetic interpreters at Beijing Genomics Institute (BGI). The performance of our manual annotations was assessed by comparing our re-annotated results with those publicly available. The overall Jaccard index was calculated to be 0.866 for the four entity types—gene, variant, disease and species. Both a BERT-based large name entity recognition (NER) model and a DistilBERT-based simplified NER model were trained, validated and tested, respectively. Due to the limited manually annotated corpus, Such NER models were fine-tuned with two phases. The F1-scores of BERT-based NER for gene, variant, disease and species are 97.28%, 93.52%, 92.54% and 95.76%, respectively, while those of DistilBERT-based NER are 95.14%, 86.26%, 91.37% and 89.92%, respectively. Most importantly, the entity type of variant has been extracted by a large language model for the first time and a comparable F1-score with the state-of-the-art variant extraction model tmVar has been achieved.Graphical

Fine-Tuning IndoBERT for Indonesian Exam Question Classification Based on Bloom's Taxonomy

Background: The learning assessment of elementary schools has recently incorporated Bloom's Taxonomy, a structure in education that categorizes different levels of cognitive learning and thinking skills, as a fundamental framework. This assessment now includes High Order Thinking Skill (HOTS) questions, with a specific focus on Indonesian topics. The implementation of this system has been observed to require teachers to manually categorize or classify questions, and this process typically requires more time and resources. To address the associated difficulty, automated categorization and classification are required to streamline the process. However, despite various research efforts in questions classification, there is still room for improvement in terms of performance, particularly in precision and accuracy. Numerous investigations have explored the use of Deep Learning Natural Language Processing models such as BERT for classification, and IndoBERT is one such pre-trained model for text analysis. Objective: This research aims to build classification system that is capable of classifying Indonesian exam questions in multiple-choice form based on Bloom's Taxonomy using IndoBERT pre-trained model. Methods: The methodology used includes hyperparameter fine-tuning, which was carried out to identify the optimal model performance. This performance was subsequently evaluated based on accuracy, F1 Score, Precision, Recall, and the time required for the training and validation of the model. Results: The proposed Fine Tuned IndoBERT Model showed that the accuracy rate was 97%, 97% F1 Score, 97% Recall, and 98% Precision with an average training time per epoch of 1.55 seconds and an average validation time per epoch of 0.38 seconds. Conclusion: Fine Tuned IndoBERT model was observed to have a relatively high classification performance, and based on this observation, the system was considered capable of classifying Indonesian exam questions at the elementary school level. Keywords: IndoBERT, Fine Tuning, Indonesian Exam Question, Model Classifier, Natural Language Processing, Bloom’s Taxonomy

Poverty and Suicidal Ideation Among Hispanic Mental Health Care Patients Leading up to the COVID-19 Pandemic.

Introduction: Suicide rates have risen in Hispanic communities since 2015, and poverty rates among Hispanics often exceed the national average. Suicidality is a complex phenomenon. Mental illness may not alone explain whether suicidal thoughts or behaviors will occur; it remains uncertain how poverty affects suicidality among Hispanic persons with known mental health conditions. Our objective was to examine whether poverty was associated with suicidal ideation among Hispanic mental healthcare patients from 2016 to 2019. Methods: We used de-identified electronic health record (EHR) data from Holmusk, captured using the MindLinc EHR system. Our analytic sample included 4,718 Hispanic patient-year observations from 13 states. Holmusk uses deep-learning natural language processing (NLP) algorithms to quantify free-text patient assessment data and poverty for mental health patients. We conducted a pooled cross-sectional analysis and estimated logistic regression models. Results: Hispanic mental health patients who experienced poverty had 1.55 greater odds of having suicidal thoughts in a given year than patients who did not experience poverty. Conclusion: Poverty may put Hispanic patients at greater risk for suicidal thoughts even when they are already receiving treatment for psychiatric conditions. NLP appears to be a promising approach for categorizing free-text information on social circumstances affecting suicidality in clinical settings.

Detecting Ground Glass Opacity Features in Patients With Lung Cancer: Automated Extraction and Longitudinal Analysis via Deep Learning-Based Natural Language Processing.

Ground-glass opacities (GGOs) appearing in computed tomography (CT) scans may indicate potential lung malignancy. Proper management of GGOs based on their features can prevent the development of lung cancer. Electronic health records are rich sources of information on GGO nodules and their granular features, but most of the valuable information is embedded in unstructured clinical notes. We aimed to develop, test, and validate a deep learning-based natural language processing (NLP) tool that automatically extracts GGO features to inform the longitudinal trajectory of GGO status from large-scale radiology notes. We developed a bidirectional long short-term memory with a conditional random field-based deep-learning NLP pipeline to extract GGO and granular features of GGO retrospectively from radiology notes of 13,216 lung cancer patients. We evaluated the pipeline with quality assessments and analyzed cohort characterization of the distribution of nodule features longitudinally to assess changes in size and solidity over time. Our NLP pipeline built on the GGO ontology we developed achieved between 95% and 100% precision, 89% and 100% recall, and 92% and 100% F1-scores on different GGO features. We deployed this GGO NLP model to extract and structure comprehensive characteristics of GGOs from 29,496 radiology notes of 4521 lung cancer patients. Longitudinal analysis revealed that size increased in 16.8% (240/1424) of patients, decreased in 14.6% (208/1424), and remained unchanged in 68.5% (976/1424) in their last note compared to the first note. Among 1127 patients who had longitudinal radiology notes of GGO status, 815 (72.3%) were reported to have stable status, and 259 (23%) had increased/progressed status in the subsequent notes. Our deep learning-based NLP pipeline can automatically extract granular GGO features at scale from electronic health records when this information is documented in radiology notes and help inform the natural history of GGO. This will open the way for a new paradigm in lung cancer prevention and early detection.

Movie information AI question answering system based on knowledge graph

Knowledge is essential for understanding. The ongoing explosion of information highlights the need for electronic texts that allow machines to better understand human language. The explosion of information in various fields has greatly changed our lives. Among them, the information explosion of the film is particularly prominent. Nowadays, people's requirements for high quality of spiritual life are increasing, and people's demand for movies is also increasing. With the extensive coverage of the Internet, the amount of movie-related information is also increasing, so how to design and build a question-and-answer system about movies is more and more urgent and important. At the same time, the original intention of the knowledge graph is to express various entities, concepts that exist objectively in the real world, and various relationships between them. It can better understand the abstract components in the language, and better to confirm the information through the interaction of the ambiguous parts of the language. In addition, the upgrade of deep learning natural language processing technology allows us to better understand and analyse language. In view of the above advantages, the movie question answering system in this paper is constructed based on the knowledge graph and uses natural language processing technology.

Natural language processing to identify reasons for sex disparity in statin prescriptions

BackgroundStatins are the cornerstone of treatment of patients with atherosclerotic cardiovascular disease (ASCVD). Despite this, multiple studies have shown that women with ASCVD are less likely to be prescribed statins than men. The objective of this study was to use Natural Language Processing (NLP) to elucidate factors contributing to this disparity. MethodsOur cohort included adult patients with two or more encounters between 2014 and 2021 with an ASCVD diagnosis within a multisite electronic health record (EHR) in Northern California. After reviewing structured EHR prescription data, we used a benchmark deep learning NLP approach, Clinical Bidirectional Encoder Representations from Transformers (BERT), to identify and interpret discussions of statin prescriptions documented in clinical notes. Clinical BERT was evaluated against expert clinician review in 20% test sets. ResultsThere were 88,913 patients with ASCVD (mean age 67.8±13.1 years) and 35,901 (40.4%) were women. Women with ASCVD were less likely to be prescribed statins compared with men (56.6% vs 67.6%, p <0.001), and, when prescribed, less likely to be prescribed guideline-directed high-intensity dosing (41.4% vs 49.8%, p <0.001). These disparities were more pronounced among younger patients, patients with private insurance, and those for whom English is their preferred language. Among those not prescribed statins, women were less likely than men to have statins mentioned in their clinical notes (16.9% vs 19.1%, p <0.001). Women were less likely than men to have statin use reported in clinical notes despite absence of recorded prescription (32.8% vs 42.6%, p <0.001). Women were slightly more likely than men to have statin intolerance documented in structured data or clinical notes (6.0% vs 5.3%, p=0.003). ConclusionsWomen with ASCVD were less likely to be prescribed guideline-directed statins compared with men. NLP identified additional sex-based statin disparities and reasons for statin non-prescription in clinical notes of patients with ASCVD.

Identifying Reasons for Statin Nonuse in Patients With Diabetes Using Deep Learning of Electronic Health Records.

Background Statins are guideline-recommended medications that reduce cardiovascular events in patients with diabetes. Yet, statin use is concerningly low in this high-risk population. Identifying reasons for statin nonuse, which are typically described in unstructured electronic health record data, can inform targeted system interventions to improve statin use. We aimed to leverage a deep learning approach to identify reasons for statin nonuse in patients with diabetes. Methods and Results Adults with diabetes and no statin prescriptions were identified from a multiethnic, multisite Northern California electronic health record cohort from 2014 to 2020. We used a benchmark deep learning natural language processing approach (Clinical Bidirectional Encoder Representations from Transformers) to identify statin nonuse and reasons for statin nonuse from unstructured electronic health record data. Performance was evaluated against expert clinician review from manual annotation of clinical notes and compared with other natural language processing approaches. Of 33 461 patients with diabetes (mean age 59±15 years, 49% women, 36% White patients, 24% Asian patients, and 15% Hispanic patients), 47% (15 580) had no statin prescriptions. From unstructured data, Clinical Bidirectional Encoder Representations from Transformers accurately identified statin nonuse (area under receiver operating characteristic curve [AUC] 0.99 [0.98-1.0]) and key patient (eg, side effects/contraindications), clinician (eg, guideline-discordant practice), and system reasons (eg, clinical inertia) for statin nonuse (AUC 0.90 [0.86-0.93]) and outperformed other natural language processing approaches. Reasons for nonuse varied by clinical and demographic characteristics, including race and ethnicity. Conclusions A deep learning algorithm identified statin nonuse and actionable reasons for statin nonuse in patients with diabetes. Findings may enable targeted interventions to improve guideline-directed statin use and be scaled to other evidence-based therapies.

Assessment of Natural Language Processing of Electronic Health Records to Measure Goals-of-Care Discussions as a Clinical Trial Outcome

Many clinical trial outcomes are documented in free-text electronic health records (EHRs), making manual data collection costly and infeasible at scale. Natural language processing (NLP) is a promising approach for measuring such outcomes efficiently, but ignoring NLP-related misclassification may lead to underpowered studies. To evaluate the performance, feasibility, and power implications of using NLP to measure the primary outcome of EHR-documented goals-of-care discussions in a pragmatic randomized clinical trial of a communication intervention. This diagnostic study compared the performance, feasibility, and power implications of measuring EHR-documented goals-of-care discussions using 3 approaches: (1) deep-learning NLP, (2) NLP-screened human abstraction (manual verification of NLP-positive records), and (3) conventional manual abstraction. The study included hospitalized patients aged 55 years or older with serious illness enrolled between April 23, 2020, and March 26, 2021, in a pragmatic randomized clinical trial of a communication intervention in a multihospital US academic health system. Main outcomes were natural language processing performance characteristics, human abstractor-hours, and misclassification-adjusted statistical power of methods of measuring clinician-documented goals-of-care discussions. Performance of NLP was evaluated with receiver operating characteristic (ROC) curves and precision-recall (PR) analyses and examined the effects of misclassification on power using mathematical substitution and Monte Carlo simulation. A total of 2512 trial participants (mean [SD] age, 71.7 [10.8] years; 1456 [58%] female) amassed 44 324 clinical notes during 30-day follow-up. In a validation sample of 159 participants, deep-learning NLP trained on a separate training data set from identified patients with documented goals-of-care discussions with moderate accuracy (maximal F1 score, 0.82; area under the ROC curve, 0.924; area under the PR curve, 0.879). Manual abstraction of the outcome from the trial data set would require an estimated 2000 abstractor-hours and would power the trial to detect a risk difference of 5.4% (assuming 33.5% control-arm prevalence, 80% power, and 2-sided α = .05). Measuring the outcome by NLP alone would power the trial to detect a risk difference of 7.6%. Measuring the outcome by NLP-screened human abstraction would require 34.3 abstractor-hours to achieve estimated sensitivity of 92.6% and would power the trial to detect a risk difference of 5.7%. Monte Carlo simulations corroborated misclassification-adjusted power calculations. In this diagnostic study, deep-learning NLP and NLP-screened human abstraction had favorable characteristics for measuring an EHR outcome at scale. Adjusted power calculations accurately quantified power loss from NLP-related misclassification, suggesting that incorporation of this approach into the design of studies using NLP would be beneficial.

Urban landscape and climate affect residents’ sentiments based on big data

Residents’ sentiments are a quantitative indicator of human feelings, which is useful for optimizing urban residential environments. Little is known about the spatiotemporal variations and potential drivers of sentiment based on big data. A total of 221,104 Weibo social media data were used to quantify daily sentiment in the Beijing metropolitan area during the COVID-19 pandemic from January 1, 2021 to March 8, 2022. Deep-learning natural language processing was used to extract this dataset to investigate the spatiotemporal sentiment patterns. The density of roads and buildings, normalized difference vegetation index (NDVI), population, sky visibility factor, daily land surface temperature (LST), daily precipitation, and daily air pollution concentrations (CO, NO2, PM2.5, SO2, and O3) were explored as potential drivers of sentiment. Results show that (1) the holiday sentiment was 1.31% higher than on weekends and 4.61% higher than on weekdays. Extreme precipitation, air pollution, and COVID-19 lockdown measures have reduced sentiment. (2) The sentiment in spring was found to be the highest. The numbers of functional zones with high sentiment values (>0.8) in spring were 13.59%, in summer 34.48%, and in autumn 14.71%. (3) Sentiment was highest under conditions of moderate greenness (0.4<NDVI<0.6) and comfortable daily temperature (25 °C < LST<30 °C). (4) Sentiment was negatively associated with daily air pollutants, such as PM2.5, NO2, and CO. This paper presents the effectiveness of sentiment quantification based on social media data and deep-learning techniques. The results provide practical implications and support decisions for sustainable urban health development.

Hindsight2020: Characterizing Uncertainty in the COVID-19 Scientific Literature.

Following emerging, re-emerging, and endemic pathogen outbreaks, the rush to publish and the risk of data misrepresentation, misinterpretation, and even misinformation puts an even greater onus on methodological rigor, which includes revisiting initial assumptions as new evidence becomes available. This study sought to understand how and when early evidence emerges and evolves when addressing different types of recurring pathogen-related questions. By applying claim-matching by means of deep learning Natural Language Processing (NLP) of coronavirus disease 2019 (COVID-19) scientific literature against a set of expert-curated evidence, patterns in timing across different COVID-19 questions-and-answers were identified, to build a framework for characterizing uncertainty in emerging infectious disease (EID) research over time. COVID-19 was chosen as a use case for this framework given the large and accessible datasets curated for scientists during the beginning of the pandemic. Timing patterns in reliably answering broad COVID-19 questions often do not align with general publication patterns, but early expert-curated evidence was generally stable. Because instability in answers often occurred within the first 2 to 6 mo for specific COVID-19 topics, public health officials could apply more conservative policies at the start of future pandemics, to be revised as evidence stabilizes.

Towards Adversarial Attacks for Clinical Document Classification

Regardless of revolutionizing improvements in various domains thanks to recent advancements in the field of Deep Learning (DL), recent studies have demonstrated that DL networks are susceptible to adversarial attacks. Such attacks are crucial in sensitive environments to make critical and life-changing decisions, such as health decision-making. Research efforts on using textual adversaries to attack DL for natural language processing (NLP) have received increasing attention in recent years. Among the available textual adversarial studies, Electronic Health Records (EHR) have gained the least attention. This paper investigates the effectiveness of adversarial attacks on clinical document classification and proposes a defense mechanism to develop a robust convolutional neural network (CNN) model and counteract these attacks. Specifically, we apply various black-box attacks based on concatenation and editing adversaries on unstructured clinical text. Then, we propose a defense technique based on feature selection and filtering to improve the robustness of the models. Experimental results show that a small perturbation to the unstructured text in clinical documents causes a significant drop in performance. Performing the proposed defense mechanism under the same adversarial attacks, on the other hand, avoids such a drop in performance. Therefore, it enhances the robustness of the CNN model for clinical document classification.

Information Analysis on Foreign Institution for International R&amp;D Collaboration Using Natural Language Processing

The number of international collaborations in research and development (R&amp;D) has been increasing in the energy sector to solve global environmental problems—such as climate change and the energy crisis—and to reduce the time, cost, and risk of failure. Successful international project planning requires the analysis of research fields and the technology expertise of cooperative partner institutions or countries, but this takes time and resources. In this study, we developed a method to analyze the information on research organizations and topics, taking advantage of data analysis as well as deep learning natural language processing (NLP) models. A method to evaluate the relative superiority of efficient international collaboration was suggested, assuming international collaboration of the National Renewable Energy Laboratory (NREL) and the Korea Institute of Energy Research (KIER). Additionally, a workflow of an automated executive summary and a translation of tens of web-posted articles is also suggested for a quick glance. The valuation of the suggested methodology is estimated as much as the annual salary of an experienced employee.

Natural Language Processing and Graph Theory: Making Sense of Imaging Records in a Novel Representation Frame.

A concise visualization framework of related reports would increase readability and improve patient management. To this end, temporal referrals to prior comparative exams are an essential connection to previous exams in written reports. Due to unstructured narrative texts' variable structure and content, their extraction is hampered by poor computer readability. Natural language processing (NLP) permits the extraction of structured information from unstructured texts automatically and can serve as an essential input for such a novel visualization framework. This study proposes and evaluates an NLP-based algorithm capable of extracting the temporal referrals in written radiology reports, applies it to all the radiology reports generated for 10 years, introduces a graphical representation of imaging reports, and investigates its benefits for clinical and research purposes. In this single-center, university hospital, retrospective study, we developed a convolutional neural network capable of extracting the date of referrals from imaging reports. The model's performance was assessed by calculating precision, recall, and F1-score using an independent test set of 149 reports. Next, the algorithm was applied to our department's radiology reports generated from 2011 to 2021. Finally, the reports and their metadata were represented in a modulable graph. For extracting the date of referrals, the named-entity recognition (NER) model had a high precision of 0.93, a recall of 0.95, and an F1-score of 0.94. A total of 1,684,635 reports were included in the analysis. Temporal reference was mentioned in 53.3% (656,852/1,684,635), explicitly stated as not available in 21.0% (258,386/1,684,635), and omitted in 25.7% (317,059/1,684,635) of the reports. Imaging records can be visualized in a directed and modulable graph, in which the referring links represent the connecting arrows. Automatically extracting the date of referrals from unstructured radiology reports using deep learning NLP algorithms is feasible. Graphs refined the selection of distinct pathology pathways, facilitated the revelation of missing comparisons, and enabled the query of specific referring exam sequences. Further work is needed to evaluate its benefits in clinics, research, and resource planning.

Predicting medical specialty from text based on a domain-specific pre-trained BERT

BackgroundOwing to the prevalence of the coronavirus disease (COVID-19), coping with clinical issues at the individual level has become important to the healthcare system. Accordingly, precise initiation of treatment after a hospital visit is required for expedited processes and effective diagnoses of outpatients. To achieve this, artificial intelligence in medical natural language processing (NLP), such as a healthcare chatbot or a clinical decision support system, can be suitable tools for an advanced clinical system. Furthermore, support for decisions on the medical specialty from the initial visit can be helpful. Materials and methodsIn this study, we propose a medical specialty prediction model from patient-side medical question text based on pre-trained bidirectional encoder representations from transformers (BERT). The dataset comprised pairs of medical question texts and labeled specialties scraped from a website for the medical question-and-answer service. The model was fine-tuned for predicting the required medical specialty labels among 27 labels from medical question texts. To demonstrate the feasibility, we conducted experiments on a real-world dataset and elaborately evaluated the predictive performance compared with four deep learning NLP models through cross-validation and test set evaluation. ResultsThe proposed model showed improved performance compared with competitive models in terms of overall specialties. In addition, we demonstrate the usefulness of the proposed model by performing case studies for visualization applications. ConclusionThe proposed model can benefit hospital patient management and reasonable recommendations for specialties for patients.

Developing a deep learning natural language processing algorithm for automated reporting of adverse drug reactions

The detection of adverse drug reactions (ADRs) is critical to our understanding of the safety and risk-benefit profile of medications. With an incidence that has not changed over the last 30 years, ADRs are a significant source of patient morbidity, responsible for 5%–10% of acute care hospital admissions worldwide. Spontaneous reporting of ADRs has long been the standard method of reporting, however this approach is known to have high rates of under-reporting, a problem that limits pharmacovigilance efforts. Automated ADR reporting presents an alternative pathway to increase reporting rates, although this may be limited by over-reporting of other drug-related adverse events.We developed a deep learning natural language processing algorithm to identify ADRs in discharge summaries at a single academic hospital centre. Our model was developed in two stages: first, a pre-trained model (DeBERTa) was further pre-trained on 1.1 million unlabelled clinical documents; secondly, this model was fine-tuned to detect ADR mentions in a corpus of 861 annotated discharge summaries. This model was compared to a version without the pre-training step, and a previously published RoBERTa model pretrained on MIMIC III, which has demonstrated strong performance on other pharmacovigilance tasks. To ensure that our algorithm could differentiate ADRs from other drug-related adverse events, the annotated corpus was enriched for both validated ADR reports and confounding drug-related adverse events using. The final model demonstrated good performance with a ROC–AUC of 0.955 (95% CI 0.933 - 0.978) for the task of identifying discharge summaries containing ADR mentions, significantly outperforming the two comparator models.

Development and Evaluation of a Natural Language Processing Annotation Tool (NAT) to Facilitate Phenotyping of Cognitive Status in Electronic Health Records

AbstractBackgroundElectronic Health Records (EHR) with large sample sizes and rich information offer great potential for dementia research, but current methods of phenotyping cognitive status are either not scalable or suffer from inaccuracies. Our objective is to evaluate whether deep learning Natural Language Processing (NLP)‐powered semi‐automated annotation can improve the speed and reliability of chart reviews for phenotyping cognitive status.MethodIn this study we developed and evaluated a semi‐automated NLP‐powered annotation tool (NAT) to facilitate phenotyping of cognitive status. Clinical experts adjudicated the cognitive status of 627 patients at Mass General Brigham (MGB) Healthcare using NAT or traditional chart reviews. Patient charts contained EHR data from two datasets: (1) Records from January 1, 2017 to December 31, 2018 for 100 Medicare beneficiaries from the MGB Accountable Care Organization (ACO), and (2) Records from 2‐years pre‐COVID diagnosis to the date of COVID diagnosis for 527 MGB patients. All EHR data from the relevant period were extracted; diagnosis codes, medications, and laboratory test values were processed and summarized. Clinical notes were processed through a deep learning NLP algorithm and a web tool was developed to present an integrated view of all data. Cognitive status was rated as cognitively normal, cognitively impaired, or undetermined. Assessment time and interrater agreement of NAT compared to manual chart reviews for cognitive status phenotyping was evaluated.ResultNAT adjudication provided higher interrater agreement (Cohen k = 0.89 vs. k = 0.80) and significant speed up (time difference mean [SD]: 1.4 [1.3] minutes, P &lt; 0.001; ratio median [min, max]: 2.2 [0.4, 20]) over manual chart reviews. There was moderate agreement with manual chart reviews (Cohen k = 0.67). In the cases that exhibited disagreement with manual chart review, NAT adjudication was able to produce assessments that had broader clinical consensus due to its integrated view of highlighted relevant information and semi‐automated NLP features.ConclusionNAT adjudication improves the speed and reliability for phenotyping cognitive status compared to manual chart reviews. This study underscores the potential of an NLP‐based clinically adjudicated method to build large‐scale dementia research cohorts from EHR.