Natural Language Processing Learning Research Articles

Exploring the Potential of Meta-Learning in Natural Language Processing

-Meta-learning has emerged as a powerful technique for enabling machines to learn to learn. It enables algorithms to leverage past experience and adapt to new tasks with limited labeled data. In natural language processing (NLP), meta-learning has been shown to improve the performance of models on a range of tasks. However, the full potential of meta-learning in NLP has not been explored. In this paper, we review recent work on meta-learning in NLP and propose new research directions to further explore the potential of this technique. Specifically, we investigate the use of meta-learning for few-shot learning, domain adaptation, and cross-lingual learning. We also explore the challenges associated with meta- learning in NLP, such as the choice of meta-features and the need for large-scale meta-learning datasets. Our experiments demonstrate that meta-learning has the potential to significantly improve the performance of NLP models, particularly on low-resource and cross-lingual tasks. Keywords: NLP, meta-learning, few-shot learning, language modeling, sentiment analysis, machine translation, BERT, transformer models, deep neural networks.

Can Natural Language Processing Improve Adnexal Torsion Predictions?

To create a decision support tool based on machine learning algorithms and natural language processing (NLP) technology, to augment clinicians' ability to predict cases of suspected adnexal torsion. Retrospective cohort study SETTING: Gynecology department, university-affiliated teaching medical center, 2014-2022. This study assessed risk-factors for adnexal torsion among women managed surgically for suspected adnexal torsion based on clinical and sonographic data. None. The dataset included demographic, clinical, sonographic, and surgical information obtained from electronic medical records. NLP was used to extract insights from unstructured free text and unlock them for automated reasoning. The machine learning model was a CatBoost classifier that utilizes gradient boosting on decision trees. The study cohort included 433 women who met inclusion criteria and underwent laparoscopy. Among them, 320 (74%) had adnexal torsion diagnosed during laparoscopy, and 113 (26%) did not. The model developed improved prediction of adnexal torsion to 84%, with a recall of 95%. The model ranked several parameters as important for prediction. Age, difference in size between ovaries, and the size of each ovary were the most significant. The precision for the "no torsion" class was 77%, with a recall of 45%. Using machine learning algorithms and NLP technology as a decision-support tool for the diagnosis of adnexal torsion is feasible. It improved true prediction of adnexal torsion to 84% and decreased cases of unnecessary laparoscopy.

Natural language processing to identify reasons for sex disparity in statin prescriptions

BackgroundStatins are the cornerstone of treatment of patients with atherosclerotic cardiovascular disease (ASCVD). Despite this, multiple studies have shown that women with ASCVD are less likely to be prescribed statins than men. The objective of this study was to use Natural Language Processing (NLP) to elucidate factors contributing to this disparity. MethodsOur cohort included adult patients with two or more encounters between 2014 and 2021 with an ASCVD diagnosis within a multisite electronic health record (EHR) in Northern California. After reviewing structured EHR prescription data, we used a benchmark deep learning NLP approach, Clinical Bidirectional Encoder Representations from Transformers (BERT), to identify and interpret discussions of statin prescriptions documented in clinical notes. Clinical BERT was evaluated against expert clinician review in 20% test sets. ResultsThere were 88,913 patients with ASCVD (mean age 67.8±13.1 years) and 35,901 (40.4%) were women. Women with ASCVD were less likely to be prescribed statins compared with men (56.6% vs 67.6%, p <0.001), and, when prescribed, less likely to be prescribed guideline-directed high-intensity dosing (41.4% vs 49.8%, p <0.001). These disparities were more pronounced among younger patients, patients with private insurance, and those for whom English is their preferred language. Among those not prescribed statins, women were less likely than men to have statins mentioned in their clinical notes (16.9% vs 19.1%, p <0.001). Women were less likely than men to have statin use reported in clinical notes despite absence of recorded prescription (32.8% vs 42.6%, p <0.001). Women were slightly more likely than men to have statin intolerance documented in structured data or clinical notes (6.0% vs 5.3%, p=0.003). ConclusionsWomen with ASCVD were less likely to be prescribed guideline-directed statins compared with men. NLP identified additional sex-based statin disparities and reasons for statin non-prescription in clinical notes of patients with ASCVD.

Machine Learning for Natural Language Processing: Techniques and Evaluation

The purpose of this research study is to analyse the approaches that are used in machine learning for the solution of natural language processing (NLP) issues. Also, the assessment criteria that are used in the performance analysis of these models are looked at as part of this investigation. We investigate the challenges that come with using machine learning for natural language processing (NLP), including the issue of data bias and the need that these models have the capacity to be explained. Research is conducted into a variety of supervised and unsupervised learning approaches, including as neural networks, topic modelling, and clustering. Also, a number of evaluation metrics, such as accuracy, precision, recall, and F1 score, are discussed. As the research comes to a conclusion, it is highlighted how important it is to find solutions to problems like data bias and the inability to explain results. This is done in order to ensure that the results generated by machine learning models for natural language processing are accurate and unbiased.

Identifying Reasons for Statin Nonuse in Patients With Diabetes Using Deep Learning of Electronic Health Records.

Background Statins are guideline-recommended medications that reduce cardiovascular events in patients with diabetes. Yet, statin use is concerningly low in this high-risk population. Identifying reasons for statin nonuse, which are typically described in unstructured electronic health record data, can inform targeted system interventions to improve statin use. We aimed to leverage a deep learning approach to identify reasons for statin nonuse in patients with diabetes. Methods and Results Adults with diabetes and no statin prescriptions were identified from a multiethnic, multisite Northern California electronic health record cohort from 2014 to 2020. We used a benchmark deep learning natural language processing approach (Clinical Bidirectional Encoder Representations from Transformers) to identify statin nonuse and reasons for statin nonuse from unstructured electronic health record data. Performance was evaluated against expert clinician review from manual annotation of clinical notes and compared with other natural language processing approaches. Of 33 461 patients with diabetes (mean age 59±15 years, 49% women, 36% White patients, 24% Asian patients, and 15% Hispanic patients), 47% (15 580) had no statin prescriptions. From unstructured data, Clinical Bidirectional Encoder Representations from Transformers accurately identified statin nonuse (area under receiver operating characteristic curve [AUC] 0.99 [0.98-1.0]) and key patient (eg, side effects/contraindications), clinician (eg, guideline-discordant practice), and system reasons (eg, clinical inertia) for statin nonuse (AUC 0.90 [0.86-0.93]) and outperformed other natural language processing approaches. Reasons for nonuse varied by clinical and demographic characteristics, including race and ethnicity. Conclusions A deep learning algorithm identified statin nonuse and actionable reasons for statin nonuse in patients with diabetes. Findings may enable targeted interventions to improve guideline-directed statin use and be scaled to other evidence-based therapies.

A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned, and Perspectives

Modern natural language processing (NLP) methods employ self-supervised pretraining objectives such as masked language modeling to boost the performance of various downstream tasks. These pretraining methods are frequently extended with recurrence, adversarial, or linguistic property masking. Recently, contrastive self-supervised training objectives have enabled successes in image representation pretraining by learning to contrast input-input pairs of augmented images as either similar or dissimilar. In NLP however, a single token augmentation can invert the meaning of a sentence during input-input contrastive learning, which led to input-output contrastive approaches that avoid the issue by instead contrasting over input-label pairs. In this primer, we summarize recent self-supervised and supervised contrastive NLP pretraining methods and describe where they are used to improve language modeling, zero to few-shot learning, pretraining data-efficiency, and specific NLP tasks. We overview key contrastive learning concepts with lessons learned from prior research and structure works by applications. Finally, we point to open challenges and future directions for contrastive NLP to encourage bringing contrastive NLP pretraining closer to recent successes in image representation pretraining.

Context parameters retrieval framework from electronic healthcare record through biomedical NLP for clinical support

This paper presents architecture for extracting context features from electronic health records (EHR) to design a clinical support system using transfer learning for natural language processing (NLP). The system is trained to provide a supporting summary to the medical practioners on the basis of the ICD 9 codes and respective symptoms of the patient. The BioALBERT model has been trained over biomedical corpora and the proposed model uses improvised parameter sharing techniques and requires less physical memory. The theoretical analysis of the proposed system is supported by the experimental analysis. MIMIC-III database has been used to fine train the proposed models and to assess the efficiency and efficacy of the proposed work. This study introduced a context-aware approach for extracting useful context from EHR, which can be used to acquire a basic understanding of the treatment path.

Algorithmic and meta-algorithmic machine learning natural language processing approaches for stakeholder requirements classification

Algorithmic and meta-algorithmic machine learning natural language processing approaches for stakeholder requirements classification

Hindsight2020: Characterizing Uncertainty in the COVID-19 Scientific Literature.

Following emerging, re-emerging, and endemic pathogen outbreaks, the rush to publish and the risk of data misrepresentation, misinterpretation, and even misinformation puts an even greater onus on methodological rigor, which includes revisiting initial assumptions as new evidence becomes available. This study sought to understand how and when early evidence emerges and evolves when addressing different types of recurring pathogen-related questions. By applying claim-matching by means of deep learning Natural Language Processing (NLP) of coronavirus disease 2019 (COVID-19) scientific literature against a set of expert-curated evidence, patterns in timing across different COVID-19 questions-and-answers were identified, to build a framework for characterizing uncertainty in emerging infectious disease (EID) research over time. COVID-19 was chosen as a use case for this framework given the large and accessible datasets curated for scientists during the beginning of the pandemic. Timing patterns in reliably answering broad COVID-19 questions often do not align with general publication patterns, but early expert-curated evidence was generally stable. Because instability in answers often occurred within the first 2 to 6 mo for specific COVID-19 topics, public health officials could apply more conservative policies at the start of future pandemics, to be revised as evidence stabilizes.

Natural Language Processing and Graph Theory: Making Sense of Imaging Records in a Novel Representation Frame.

A concise visualization framework of related reports would increase readability and improve patient management. To this end, temporal referrals to prior comparative exams are an essential connection to previous exams in written reports. Due to unstructured narrative texts' variable structure and content, their extraction is hampered by poor computer readability. Natural language processing (NLP) permits the extraction of structured information from unstructured texts automatically and can serve as an essential input for such a novel visualization framework. This study proposes and evaluates an NLP-based algorithm capable of extracting the temporal referrals in written radiology reports, applies it to all the radiology reports generated for 10 years, introduces a graphical representation of imaging reports, and investigates its benefits for clinical and research purposes. In this single-center, university hospital, retrospective study, we developed a convolutional neural network capable of extracting the date of referrals from imaging reports. The model's performance was assessed by calculating precision, recall, and F1-score using an independent test set of 149 reports. Next, the algorithm was applied to our department's radiology reports generated from 2011 to 2021. Finally, the reports and their metadata were represented in a modulable graph. For extracting the date of referrals, the named-entity recognition (NER) model had a high precision of 0.93, a recall of 0.95, and an F1-score of 0.94. A total of 1,684,635 reports were included in the analysis. Temporal reference was mentioned in 53.3% (656,852/1,684,635), explicitly stated as not available in 21.0% (258,386/1,684,635), and omitted in 25.7% (317,059/1,684,635) of the reports. Imaging records can be visualized in a directed and modulable graph, in which the referring links represent the connecting arrows. Automatically extracting the date of referrals from unstructured radiology reports using deep learning NLP algorithms is feasible. Graphs refined the selection of distinct pathology pathways, facilitated the revelation of missing comparisons, and enabled the query of specific referring exam sequences. Further work is needed to evaluate its benefits in clinics, research, and resource planning.

Information Analysis on Foreign Institution for International R&amp;D Collaboration Using Natural Language Processing

The number of international collaborations in research and development (R&amp;D) has been increasing in the energy sector to solve global environmental problems—such as climate change and the energy crisis—and to reduce the time, cost, and risk of failure. Successful international project planning requires the analysis of research fields and the technology expertise of cooperative partner institutions or countries, but this takes time and resources. In this study, we developed a method to analyze the information on research organizations and topics, taking advantage of data analysis as well as deep learning natural language processing (NLP) models. A method to evaluate the relative superiority of efficient international collaboration was suggested, assuming international collaboration of the National Renewable Energy Laboratory (NREL) and the Korea Institute of Energy Research (KIER). Additionally, a workflow of an automated executive summary and a translation of tens of web-posted articles is also suggested for a quick glance. The valuation of the suggested methodology is estimated as much as the annual salary of an experienced employee.

Predicting medical specialty from text based on a domain-specific pre-trained BERT

BackgroundOwing to the prevalence of the coronavirus disease (COVID-19), coping with clinical issues at the individual level has become important to the healthcare system. Accordingly, precise initiation of treatment after a hospital visit is required for expedited processes and effective diagnoses of outpatients. To achieve this, artificial intelligence in medical natural language processing (NLP), such as a healthcare chatbot or a clinical decision support system, can be suitable tools for an advanced clinical system. Furthermore, support for decisions on the medical specialty from the initial visit can be helpful. Materials and methodsIn this study, we propose a medical specialty prediction model from patient-side medical question text based on pre-trained bidirectional encoder representations from transformers (BERT). The dataset comprised pairs of medical question texts and labeled specialties scraped from a website for the medical question-and-answer service. The model was fine-tuned for predicting the required medical specialty labels among 27 labels from medical question texts. To demonstrate the feasibility, we conducted experiments on a real-world dataset and elaborately evaluated the predictive performance compared with four deep learning NLP models through cross-validation and test set evaluation. ResultsThe proposed model showed improved performance compared with competitive models in terms of overall specialties. In addition, we demonstrate the usefulness of the proposed model by performing case studies for visualization applications. ConclusionThe proposed model can benefit hospital patient management and reasonable recommendations for specialties for patients.

Development and Evaluation of a Natural Language Processing Annotation Tool (NAT) to Facilitate Phenotyping of Cognitive Status in Electronic Health Records

AbstractBackgroundElectronic Health Records (EHR) with large sample sizes and rich information offer great potential for dementia research, but current methods of phenotyping cognitive status are either not scalable or suffer from inaccuracies. Our objective is to evaluate whether deep learning Natural Language Processing (NLP)‐powered semi‐automated annotation can improve the speed and reliability of chart reviews for phenotyping cognitive status.MethodIn this study we developed and evaluated a semi‐automated NLP‐powered annotation tool (NAT) to facilitate phenotyping of cognitive status. Clinical experts adjudicated the cognitive status of 627 patients at Mass General Brigham (MGB) Healthcare using NAT or traditional chart reviews. Patient charts contained EHR data from two datasets: (1) Records from January 1, 2017 to December 31, 2018 for 100 Medicare beneficiaries from the MGB Accountable Care Organization (ACO), and (2) Records from 2‐years pre‐COVID diagnosis to the date of COVID diagnosis for 527 MGB patients. All EHR data from the relevant period were extracted; diagnosis codes, medications, and laboratory test values were processed and summarized. Clinical notes were processed through a deep learning NLP algorithm and a web tool was developed to present an integrated view of all data. Cognitive status was rated as cognitively normal, cognitively impaired, or undetermined. Assessment time and interrater agreement of NAT compared to manual chart reviews for cognitive status phenotyping was evaluated.ResultNAT adjudication provided higher interrater agreement (Cohen k = 0.89 vs. k = 0.80) and significant speed up (time difference mean [SD]: 1.4 [1.3] minutes, P &lt; 0.001; ratio median [min, max]: 2.2 [0.4, 20]) over manual chart reviews. There was moderate agreement with manual chart reviews (Cohen k = 0.67). In the cases that exhibited disagreement with manual chart review, NAT adjudication was able to produce assessments that had broader clinical consensus due to its integrated view of highlighted relevant information and semi‐automated NLP features.ConclusionNAT adjudication improves the speed and reliability for phenotyping cognitive status compared to manual chart reviews. This study underscores the potential of an NLP‐based clinically adjudicated method to build large‐scale dementia research cohorts from EHR.

Developing a deep learning natural language processing algorithm for automated reporting of adverse drug reactions

The detection of adverse drug reactions (ADRs) is critical to our understanding of the safety and risk-benefit profile of medications. With an incidence that has not changed over the last 30 years, ADRs are a significant source of patient morbidity, responsible for 5%–10% of acute care hospital admissions worldwide. Spontaneous reporting of ADRs has long been the standard method of reporting, however this approach is known to have high rates of under-reporting, a problem that limits pharmacovigilance efforts. Automated ADR reporting presents an alternative pathway to increase reporting rates, although this may be limited by over-reporting of other drug-related adverse events.We developed a deep learning natural language processing algorithm to identify ADRs in discharge summaries at a single academic hospital centre. Our model was developed in two stages: first, a pre-trained model (DeBERTa) was further pre-trained on 1.1 million unlabelled clinical documents; secondly, this model was fine-tuned to detect ADR mentions in a corpus of 861 annotated discharge summaries. This model was compared to a version without the pre-training step, and a previously published RoBERTa model pretrained on MIMIC III, which has demonstrated strong performance on other pharmacovigilance tasks. To ensure that our algorithm could differentiate ADRs from other drug-related adverse events, the annotated corpus was enriched for both validated ADR reports and confounding drug-related adverse events using. The final model demonstrated good performance with a ROC–AUC of 0.955 (95% CI 0.933 - 0.978) for the task of identifying discharge summaries containing ADR mentions, significantly outperforming the two comparator models.

Abstract 12092: Natural Language Processing to Identify Reasons for Gender Disparities in Statin Use

Introduction: Statins are life-saving medications for patients with atherosclerotic cardiovascular disease (ASCVD), but women with ASCVD are persistently less likely to be prescribed statins than men. This study aims to use Natural Language Processing (NLP) to further elucidate patient and provider factors contributing to this disparity. Methods: The study cohort included patients with &gt;2 ASCVD encounters between 2014 and 2021 within a multisite electronic health record (EHR) in Northern California. Data from a random sample of our cohort (N = 942) was manually annotated to develop a benchmark deep learning natural language processing (NLP) approach, Clinical Bidirectional Encoder Representations from Transformers (BERT); 80% of these were used for model training and 20% for testing. After reviewing structured EHR data (e.g. prescriptions, allergies), BERT was used to identify and interpret discussions of statins in clinical notes. Results: Of 88,913 patients with ASCVD (mean age 67.8 ± 13.1 years), 35,901 (40.4%) were women. Women were less likely to be prescribed statins (56.6% vs. 67.6%, p &lt; 0.001). Only 18.6% of nonallergic patients without a statin prescription had a mention of statins in unstructured EHR text. Statin use through unstructured text was less likely to be identified among women than men (32.8% vs. 42.6%, p &lt; 0.001). Reasons for statin nonuse did not significantly differ by gender (Figure). Conclusions: Women with ASCVD were less likely to be use statins, and this disparity became more pronounced with the inclusion of unstructured data. An NLP approach revealed actionable reasons for statin nonuse. Future studies should leverage these approaches to monitor and track statin adherence by combining structured and unstructured data in real-world populations.

Abstract 12957: Identifying Reasons for Statin Nonuse in Individuals With Diabetes Using Deep Learning of Unstructured Electronic Health Records

Introduction: Statins are guideline-recommended and potentially life-saving for individuals with diabetes. Yet, statin use is concerningly low in this group. Identifying reasons for statin nonuse can inform targeted interventions. We hypothesize that clinical reasoning around reasons for statin nonuse may be buried in unstructured, narrative portions of the electronic health record (EHR). We aimed to identify reasons for statin nonuse in patients with diabetes from unstructured EHRs using deep learning. Methods: Adults diagnosed with diabetes from 2014 to 2020 with no statin prescriptions were identified from a multisite EHR health system in Northern California. We used a benchmark deep learning natural language processing (NLP) approach, Clinical Bidirectional Encoder Representations from Transformers (BERT), to identify statin nonuse and reasons for statin nonuse from unstructured EHRs. Clinical BERT was evaluated against expert clinician review in 20% test sets. Results: Of 33,461 patients with diabetes (mean age 59+15y, 49% women, 36% White, 24% Asian, 15% Hispanic), nearly half (47%) lacked statin prescriptions. By leveraging unstructured data containing any mention of statin-related terms, Clinical BERT accurately identified statin nonuse (area under the receiver operating characteristic curve (AUC 0.99 [0.98-1.0]) and identified key patient, clinician, and system reasons for statin nonuse, including statin-associated side effects, guideline-discordant clinician practice, and patient hesitancy (AUC 0.90 [0.86-0.93]; Figure). Conclusions: In a multiethnic diabetes cohort, we identified actionable reasons for statin nonuse using a deep learning approach to mine unstructured EHRs. Findings may enable targeted interventions to improve guideline-directed statin use and be readily scaled to other evidence-based therapies.

The boundaries of meaning: a case study in neural machine translation

ABSTRACT The success of deep learning in natural language processing raises intriguing questions about the nature of linguistic meaning and ways in which it can be processed by natural and artificial systems. One such question has to do with subword segmentation algorithms widely employed in language modeling, machine translation, and other tasks since 2016. These algorithms often cut words into semantically opaque pieces, such as ‘period’, ‘on’, ‘t’, and ‘ist’ in ‘period|on|t|ist’. The system then represents the resulting segments in a dense vector space, which is expected to model grammatical relations among them. This representation may in turn be used to map ‘period|on|t|ist’ (English) to ‘par|od|ont|iste’ (French). Thus, instead of being modeled at the lexical level, translation is reformulated more generally as the task of learning the best bilingual mapping between the sequences of subword segments of two languages; and sometimes even between pure character sequences: ‘p|e|r|i|o|d|o|n|t|i|s|t’ → ‘p|a|r|o|d|o|n|t|i|s|t|e’. Such segmentations and alignments are at work in highly efficient end-to-end machine translation systems, despite their allegedly opaque nature. But do they have linguistic or philosophical plausibility? I attempt to cast light on this question, in the spirit of making artificial intelligence more transparent and explainable.

Psychological Analysis of Athletes during Basketball Games from the Perspective of Deep Learning

Due to the influence of psychological factors during sports, basketball players often change the shooting rhythm in sports, which will reduce the shooting rate and directly affect the training effect. Therefore, we need to analyze the basketball shooting process of some athletes, which is not a complete negation of traditional training methods, but a psychological analysis of some athletes’ shooting in the course of competition, so as to complement the feedback and training information after the game. With the great improvement of computer computing power, deep learning natural language processing can help people analyze and solve previously unsolvable problems in production and life. The psychological emotion of adolescence has a great influence on the study and life of middle school students. At present, the unified monitoring and analysis of the daily life of middle school students need not only a lot of manpower, but also slow speed. If the psychological problems are not found in time and feedback is given, it will cause a series of adverse effects on individual athletes. The neural network model based on deep learning can process students' daily mass text information quickly and accurately, and then give comprehensive judgment, which is a good solution. This paper applies the neural network algorithm of the Bi-LSTM model and CNN model to study the text data, and finally has 95.55% and 90.03% accuracy in the psychological analysis experiment, which provides a feasible solution to solve the batch rapid analysis of psychological changes reflected in the daily text of athletes during basketball. Some suggestions are put forward on how to strengthen and improve the psychological quality of college basketball players and their ability to bear pressure and difficulties.

Enhancing Botanical Knowledge Graphs with Machine Learning

Integrating sparse and incomplete biodiversity data into a global, coherent data space and generating machine-readable data infrastructures is a challenge in biodiversity informatics. In recent years, biodiversity data researchers have started proposing Knowledge Graphs (KGs) as one approach to connecting biodiversity data worldwide (Page 2019), representing the connections between the what, when, and where of objects in natural history collections. At the Natural History Museum (NHM) we have constructed a KG of botanical specimens and collectors, encoded into numerical representations, and using a Relational Graph Convolutional Network (RGCN) (Schlichtkrull et al. 2018) to infer gaps in the KG, forging new connections between nodes. The datasets involved in our botanical KG project are NHM Botany Collector database (105,780 entities) and NHM Indian Region Botanical Specimen Dataset (110,043 entities with geographical information). Our KG with RGCN enables the structured and contextual data to be reasoned across the knowledge content, allowing us to dynamically update its representation according to its closely related neighbours. Our work will explain why and how the KG with RGCN can offer a better way to link digitised botanical data. We use the prototype KG to demonstrate its potential for modelling botanical data and provide a graphical representation for other machine learning applications. For example, the combination of KG with RGCN and Metric Learning (Xing et al. 2002, a form of Machine Learning generally used to automatically construct task-specific distance metrics) supports data completion via entity classification and link prediction for a subset of botanical specimens within a geographic region. These data augmentation models with KGs allow us to identify gaps in specimen provenance, and fill in missing data. After phase one training, our model can achieve 88% accuracy in entity classification and report a reasonable Mean Reciprocal Rank (MRR) in raw ranking link prediction for the Indian Region Botanical Specimen Dataset. Our research also evaluates the use of the KG and RGCN to improve post-OCR (Optical Character Recognition) correction algorithms as part of automatic specimen digitisation pipelines. This improves the accuracy of entity recognition on specimen label text identification and transcription, as part of machine learning natural language processing and human-in-the-loop transcription. Human-based transcription can be aided and improved by an interpretation recommendation system predicated on the specimen unit’s RGCN-inferred location in the KG. This methodology can also be used to explore the alignment of KGs from different institutions within the global biodiversity network, to identify the relative importance of collectors or determine strengths or gaps in different geographic regions or ecosystems, duplicate items in collections, or objects in collections that have potentially been misidentified.

A pre-trained BERT for Korean medical natural language processing

With advances in deep learning and natural language processing (NLP), the analysis of medical texts is becoming increasingly important. Nonetheless, despite the importance of processing medical texts, no research on Korean medical-specific language models has been conducted. The Korean medical text is highly difficult to analyze because of the agglutinative characteristics of the language, as well as the complex terminologies in the medical domain. To solve this problem, we collected a Korean medical corpus and used it to train the language models. In this paper, we present a Korean medical language model based on deep learning NLP. The model was trained using the pre-training framework of BERT for the medical context based on a state-of-the-art Korean language model. The pre-trained model showed increased accuracies of 0.147 and 0.148 for the masked language model with next sentence prediction. In the intrinsic evaluation, the next sentence prediction accuracy improved by 0.258, which is a remarkable enhancement. In addition, the extrinsic evaluation of Korean medical semantic textual similarity data showed a 0.046 increase in the Pearson correlation, and the evaluation for the Korean medical named entity recognition showed a 0.053 increase in the F1-score.