Novel Natural Language Processing Research Articles

CGP-based IO biomarker assessment and treatment utilization across a major US health system.

e23109 Background: Immune checkpoint inhibitors (IO) have become a powerful precision therapy option to treat advanced stage cancer with biomarkers such as PD-L1, tumor mutational burden (TMB), and microsatellite instability (MSI) associated with improved patient response rates. Despite this, many patients do not receive genomic testing for all IO biomarkers. Providence, a large US community health system, developed a pathologist-directed testing protocol where comprehensive genomic profiling (CGP) was routinely used at time of diagnosis for advanced cancer patients. Methods: Advanced cancer patients who received CGP (ProvSeq 523) and IHC testing for PD-L1 between 2019-2023 were included in the study. Patients were required to be treated at Providence and were assessed for presence of IO biomarkers and subsequent therapy selection. Real world data were curated from patient charts and genomic laboratory data by employing a novel natural-language processing (NLP) approach to accelerate abstraction. Results: The study included 2,502 patients (53% female, median age 68y, 83% white). Top 3 tumor types tested were lung (40%), colorectal (9%), and breast (8%). Overall, 58% (N = 1,455) of patients had presence of ≥1 IO biomarker, with 46% (N = 1,155) being PD-L1 positive, and 27% (N = 682) being TMB-H. In PD-L1 negative patients (N = 1,347; 54%), 300 (22%) were TMB-H and 18 (1%) were MSI-H. 53% (N = 767) of patients who harbored an actionable IO biomarker received IO-based therapy. Fewer patients with an IO biomarker received chemotherapy compared to patients without an IO biomarker (23% vs 35%, p < 0.001). Patients who possessed ≥2 IO biomarkers received an IO precision therapy 68% of the time vs 47% in patients with 1 IO biomarker (p < 0.001). In PD-L1 negative patients, 26% (N = 78) of TMB-H patients received IO monotherapy compared to 5% (N = 49) of TMB-L patients. Conclusions: IO biomarker presence is associated with increased precision therapy use. CGP identified many TMB-H patients for IO monotherapy that would have been missed with PD-L1 testing alone. More than half of patients eligible for IO therapy don’t receive it; ongoing analyses will evaluate the impact of pathology-directed reflex testing on IO as well as targeted therapy approaches. [Table: see text]

Hybrid Natural Language Processing Model for Sentiment Analysis during Natural Crisis

This paper introduces a novel natural language processing (NLP) model as an original approach to sentiment analysis, with a focus on understanding emotional responses during major disasters or conflicts. The model was created specifically for Croatian and is based on unigrams, but it can be used with any language that supports the n-gram model and expanded to multiple word sequences. The presented model generates a sentiment score aligned with discrete and dimensional emotion models, reliability metrics, and individual word scores using affective datasets Extended ANEW and NRC WordEmotion Association Lexicon. The sentiment analysis model incorporates different methodologies, including lexicon-based, machine learning, and hybrid approaches. The process of preprocessing includes translation, lemmatization, and data refinement, utilized automated translation services as well as the CLARIN Knowledge Centre for South Slavic languages (CLASSLA) library, with a particular emphasis on diacritical mark correction and tokenization. The presented model was experimentally evaluated on three simultaneous major natural crises that recently affected Croatia. The study’s findings reveal a significant shift in emotional dimensions during the COVID-19 pandemic, particularly a decrease in valence, arousal, and dominance, which corresponded with the two-month recovery period. Furthermore, the 2020 Croatian earthquakes elicited a wide range of negative discrete emotions, including anger, fear, and sadness, with the recuperation period much longer than in the case of COVID-19. This study represents an advancement in sentiment analysis, particularly in linguistically specific contexts, and provides insights into the emotional landscape shaped by major societal events.

Highway Construction Safety Analysis Using Large Language Models

The highway construction industry carries substantial safety risks for workers, necessitating thorough accident analyses to implement effective preventive measures. Current research lacks comprehensive investigations into safety incidents, relying heavily on conventional statistical methods and overlooking valuable textual information in publicly available databases. This study leverages a state-of-the-art large language model (LLM), specifically OpenAI’s GPT-3.5 model. The primary focus is to enhance text-based incident analysis that is sourced from OSHA’s Severe Injury Reports (SIR) database. By incorporating novel natural language processing (NLP) techniques, dimensionality reduction, clustering algorithms, and LLM prompting of incident narratives, the study aims to develop an approach to the analysis of major accident causes in highway construction. The resulting cluster analysis, coupled with LLM summarization and cause identification, reveals the major accident types, such as heat-related and struck-by injuries, as well as commonalities between incidents. This research showcases the potential of artificial intelligence (AI) and LLM technology in data-driven analysis. By efficiently processing textual data and providing insightful analysis, the study fosters practical implications for safety professionals and the development of more effective accident prevention and intervention strategies within the industry.

Bridging the gap in ESG measurement: Using NLP to quantify environmental, social, and governance communication

Environmental, social, and governance (ESG) criteria take a central role in fostering sustainable development in economies. This paper introduces a class of novel Natural Language Processing (NLP) models to assess corporate disclosures in the ESG subdomains. Using over 13.8 million texts from reports and news, specific E, S, and G models were pretrained. Additionally, three 2k datasets were developed to classify ESG-related texts. The models effectively explain variations in ESG ratings, showcasing a robust method for enhancing transparency and accuracy in evaluating corporate sustainability. This approach addresses the gap in precise, transparent ESG measurement, advancing sustainable development in economies.

A Computational Framework for Understanding Firm Communication During Disasters

Firms’ public communication on social media during disasters can benefit both disaster response efficiency and the perception of the corporate image. Despite its importance, limited guidelines are available to inform firms’ disaster communication strategies. The current study examines firms’ communication on social media in various disasters and how it impacts public engagement. We employ a novel natural language processing (NLP) approach, Semantic Projection with Active Retrieval (SPAR), to analyze Facebook posts made by Russell 3000 firms between 2009 and 2022 concerning various disasters. We show that firm communication can be measured based on two dimensions derived from the Competing Values Framework (CVF): internal versus external and stable versus flexible. We find that social media messages that emphasize operational continuity (internal/stable-oriented) are more popular during biological disasters. By contrast, messages that stress innovations and adaptations to disasters (external/flexible-oriented) elicit more engagement in weather-related disasters. The study offers a framework to characterize and guide firms’ design of disaster communication on social media in different disaster contexts. Our SPAR method is also available to firms to analyze their social media data and uncover the underlying patterns in communication across different contexts.

MmPose-NLP: A Natural Language Processing Approach to Precise Skeletal Pose Estimation Using mmWave Radars.

In this article, we presented mmPose-NLP, a novel natural language processing (NLP) inspired sequence-to-sequence (Seq2Seq) skeletal key-point estimator using millimeter-wave (mmWave) radar data. To the best of our knowledge, this is the first method to precisely estimate up to 25 skeletal key points using mmWave radar data alone. Skeletal pose estimation is critical in several applications ranging from autonomous vehicles, traffic monitoring, patient monitoring, and gait analysis, to defense security forensics, and aid both preventative and actionable decision making. The use of mmWave radars for this task, over traditionally employed optical sensors, provides several advantages, primarily its operational robustness to scene lighting and adverse weather conditions, where optical sensor performance degrade significantly. The mmWave radar point-cloud (PCL) data are first voxelized (analogous to tokenization in NLP) and N frames of the voxelized radar data (analogous to a text paragraph in NLP) is subjected to the proposed mmPose-NLP architecture, where the voxel indices of the 25 skeletal key points (analogous to keyword extraction in NLP) are predicted. The voxel indices are converted back to real-world 3-D coordinates using the voxel dictionary used during the tokenization process. Mean absolute error (MAE) metrics were used to measure the accuracy of the proposed system against the ground truth, with the proposed mmPose-NLP offering < 3 cm localization errors in the depth, horizontal, and vertical axes. The effect of the number of input frames versus performance/accuracy was also studied for N = {1,2, ... ,10} . A comprehensive methodology, results, discussions, and limitations are presented in this article. All the source codes and results are made available on GitHub for further research and development in this critical yet emerging domain of skeletal key-point estimation using mmWave radars.

Machine learning-based guilt detection in text

We introduce a novel Natural Language Processing (NLP) task called guilt detection, which focuses on detecting guilt in text. We identify guilt as a complex and vital emotion that has not been previously studied in NLP, and we aim to provide a more fine-grained analysis of it. To address the lack of publicly available corpora for guilt detection, we created VIC, a dataset containing 4622 texts from three existing emotion detection datasets that we binarized into guilt and no-guilt classes. We experimented with traditional machine learning methods using bag-of-words and term frequency-inverse document frequency features, achieving a 72% f1 score with the highest-performing model. Our study provides a first step towards understanding guilt in text and opens the door for future research in this area.

Improved outcomes from reflex comprehensive genomic profiling-guided precision therapeutic selection across a major US healthcare system.

6622 Background: An ever-increasing number of biomarker-guided therapies, some with pan-cancer indications have expanded the oncologist’s toolbox for fighting advanced cancer. Despite this, not all advanced cancer patients receive tumor genomic testing, or are tested for a limited number of targets. Still others are tested too late in their care journey to benefit from precision therapy. To assess the impact of removing testing barriers, we developed a reflex testing protocol where comprehensive genomic profiling (CGP) was routinely ordered by pathologists at time of diagnosis for advanced cancer patients. Methods: Reflex CGP testing was primarily initiated by the pathologist at the time of advanced cancer diagnosis. Testing was performed between 2019 and 2021 via CGP using the ProvSeq 523 lab-developed test, and testing was performed at no cost to the patient. Post-CGP, stage 4 patients were followed for ≥ 12 months. We assessed time to therapy, therapy selection, and overall survival (OS). Therapies were stratified by presence of biomarkers associated with approved targeted therapies (TT), presence of biomarkers for immunotherapies (IO), and/or therapies that were guideline based (GB) and not associated with a specific biomarker. As much key patient information is only consistently available in free-text medical charts, we implemented a novel natural-language processing (NLP) approach based on deep learning and large language models to accelerate abstraction. Results: A cohort of 1,423 advanced cancer patients met the study criteria. The median age was 66 years, 53% were female, and 82% white. The 3 most tested tumor types were 22% non-small cell lung cancer, 16% colorectal cancer, and 12% breast. Overall, 49% (N=704) of patients had a biomarker result considered actionable for an approved TT or IO. Median (IQR) time-to-treatment initiation post-CGP was 19 (2-70) days. In patients with no actionable TT or IO biomarkers (N=719), 63% were treated with chemotherapy-based regimens, 11% with GB, 17% with unmatched TT, and 8% with unmatched IO. Of patients who had only an actionable TT biomarker (N=287), 18% received matched TT and 13% received GB. 36% of patients with only an actionable IO biomarker (N=317) received matched IO monotherapy. 48% of patients with both actionable TT and IO biomarkers (N=100) received matched TT or IO monotherapy, and 5% received GB. Across all tumor types, patients receiving a TT had better OS compared to patients receiving chemotherapy with 12 months OS (%) of 70.1 (95% CI=64.4 - 76.3) for TT-treated patients, compared to 62.9 (95% CI=58.8 - 67.3) for chemotherapy only. Conclusions: CGP-guided precision therapy use is associated with significantly higher survival in a reflex testing population. A reflex protocol can overcome key barriers to the use and timing of genomic testing to improve access to these life-extending treatment modalities.

Long-term epilepsy outcome dynamics revealed by natural language processing of clinic notes.

Electronic medical records allow for retrospective clinical research with large patient cohorts. However, epilepsy outcomes are often contained in free text notes that are difficult to mine. We recently developed and validated novel natural language processing (NLP) algorithms to automatically extract key epilepsy outcome measures from clinic notes. In this study, we assessed the feasibility of extracting these measures to study the natural history of epilepsy at our center. We applied our previously validated NLP algorithms to extract seizure freedom, seizure frequency, and date of most recent seizure from outpatient visits at our epilepsy center from 2010 to 2022. We examined the dynamics of seizure outcomes over time using Markov model-based probability and Kaplan-Meier analyses. Performance of our algorithms on classifying seizure freedom was comparable to that of human reviewers (algorithm F1 = .88 vs. human annotator = .86). We extracted seizure outcome data from 55 630 clinic notes from 9510 unique patients written by 53 unique authors. Of these, 30% were classified as seizure-free since the last visit, 48% of non-seizure-free visits contained a quantifiable seizure frequency, and 47% of all visits contained the date of most recent seizure occurrence. Among patients with at least five visits, the probabilities of seizure freedom at the next visit ranged from 12% to 80% in patients having seizures or seizure-free at the prior three visits, respectively. Only 25% of patients who were seizure-free for 6 months remained seizure-free after 10 years. Our findings demonstrate that epilepsy outcome measures can be extracted accurately from unstructured clinical note text using NLP. At our tertiary center, the disease course often followed a remitting and relapsing pattern. This method represents a powerful new tool for clinical research with many potential uses and extensions to other clinical questions.

Radio galaxy zoo EMU: towards a semantic radio galaxy morphology taxonomy

ABSTRACT We present a novel natural language processing (NLP) approach to deriving plain English descriptors for science cases otherwise restricted by obfuscating technical terminology. We address the limitations of common radio galaxy morphology classifications by applying this approach. We experimentally derive a set of semantic tags for the Radio Galaxy Zoo EMU (Evolutionary Map of the Universe) project and the wider astronomical community. We collect 8486 plain English annotations of radio galaxy morphology, from which we derive a taxonomy of tags. The tags are plain English. The result is an extensible framework, which is more flexible, more easily communicated, and more sensitive to rare feature combinations, which are indescribable using the current framework of radio astronomy classifications.

Automated Approach for Digitalizing Scope of Work Requirements to Support Contract Management

Scope of work requirements (SOWRs) specify the contractual obligations for the contractor to fulfill. Managing the information is crucial to the success of a construction project. This process is, however, often challenging because requirements are typically voluminous and written in an unstructured paper-based format. The current state-of-the-art models are mostly applicable to quantitative constraints for use in automated code compliance checking of design features. This study proposes a novel natural language processing (NLP) framework capable of digitalizing nonquantitative natural language SOWRs outlined in construction contracts. The model includes a comprehensive set of semantic and syntactic rules of linguistic features for extracting contractual work information including actors, actions, objects, constraints, tasks, and obligation. The framework was evaluated on a large textual corpus of provisions of design-build highway contracts. The model yields an impressive precision and recall of more than 93% and 87%, respectively. The proposed system is expected to help the project planner quickly develop an electronic database of contractual work requirements, enabling significantly improved efficiency in project planning and verification.

Public Awareness and Sentiment Analysis of COVID-Related Discussions Using BERT-Based Infoveillance

Understanding different aspects of public concerns and sentiments during large health emergencies, such as the COVID-19 pandemic, is essential for public health agencies to develop effective communication strategies, deliver up-to-date and accurate health information, and mitigate potential impacts of emerging misinformation. Current infoveillance systems generally focus on discussion intensity (i.e., number of relevant posts) as an approximation of public awareness, while largely ignoring the rich and diverse information in texts with granular information of varying public concerns and sentiments. In this study, we address this grand challenge by developing a novel natural language processing (NLP) infoveillance workflow based on bidirectional encoder representation from transformers (BERT). We first used a smaller COVID-19 tweet sample to develop a content classification and sentiment analysis model using COVID-Twitter-BERT. The classification accuracy was between 0.77 and 0.88 across the five identified topics. In the sentiment analysis with a three-class classification task (positive/negative/neutral), BERT achieved decent accuracy, 0.7. We then applied the content topic and sentiment classifiers to a much larger dataset with more than 4 million tweets in a 15-month period. We specifically analyzed non-pharmaceutical intervention (NPI) and social issue content topics. There were significant differences in terms of public awareness and sentiment towards the overall COVID-19, NPI, and social issue content topics across time and space. In addition, key events were also identified to associate with abrupt sentiment changes towards NPIs and social issues. This novel NLP-based AI workflow can be readily adopted for real-time granular content topic and sentiment infoveillance beyond the health context.

Feasibility and acceptability testing of CommSense: A novel communication technology to enhance health equity in clinician-patient interactions.

Quality patient-clinician communication is paramount to achieving safe and compassionate healthcare, but evaluating communication performance during real clinical encounters is challenging. Technology offers novel opportunities to provide clinicians with actionable feedback to enhance their communication skills. This pilot study evaluated the acceptability and feasibility of CommSense, a novel natural language processing (NLP) application designed to record and extract key metrics of communication performance and provide real-time feedback to clinicians. Metrics of communication performance were established from a review of the literature and technical feasibility verified. CommSense was deployed on a wearable (smartwatch), and participants were recruited from an academic medical center to test the technology. Participants completed a survey about their experience; results were exported to SPSS (v.28.0) for descriptive analysis. Forty (n = 40) healthcare participants (nursing students, medical students, nurses, and physicians) pilot tested CommSense. Over 90% of participants "strongly agreed" or "agreed" that CommSense could improve compassionate communication (n = 38, 95%) and help healthcare organizations deliver high-quality care (n = 39, 97.5%). Most participants (n = 37, 92.5%) "strongly agreed" or "agreed" they would be willing to use CommSense in the future; 100% (n = 40) "strongly agreed" or "agreed" they were interested in seeing information analyzed by CommSense about their communication performance. Metrics of most interest were medical jargon, interruptions, and speech dominance. Participants perceived significant benefits of CommSense to track and improve communication skills. Future work will deploy CommSense in the clinical setting with a more diverse group of participants, validate data fidelity, and explore optimal ways to share data analyzed by CommSense with end-users.

Identifying essential nonprofits with a novel NLP Method

AbstractIn this research note, we propose a classification method for identifying whether a 501c3 nonprofit organization is considered essential for economic recovery. During the first few months of the COVID‐19 pandemic, many nonprofit organizations experienced negative financial effects from the economic recession. While these nonprofits saw increased demand for their services, the weakness in the overall economy led to a decline in donations. Fiscal assistance by local, state, and federal government to essential organizations was a critical element to an economic recovery, and governments needed to prioritize aid to the most essential organizations first. By identifying essential nonprofit organizations in advance, these organizations could quickly and efficiently receive financial assistance. Using descriptive text data provided by Ohio nonprofit organizations in their IRS tax filings, we propose a novel natural language processing (NLP) technique to measure the degree of “essentialness” to a nonprofit's work. We show that our model offers an improvement to the classification system known as the National Taxonomy of Exempt Organizations (NTEE). Our machine learning model is also compared to an independent evaluation of a nonprofit's essentialness produced by human researchers.

Natural Language Processing for Automated Classification of Qualitative Data From Interviews of Patients With Cancer

ObjectivesThis study sought to explore the use of novel natural language processing (NLP) methods for classifying unstructured, qualitative textual data from interviews of patients with cancer to identify patient-reported symptoms and impacts on quality of life. MethodsWe tested the ability of 4 NLP models to accurately classify text from interview transcripts as “symptom,” “quality of life impact,” and “other.” Interview data sets from patients with hepatocellular carcinoma (HCC) (n = 25), biliary tract cancer (BTC) (n = 23), and gastric cancer (n = 24) were used. Models were cross-validated with transcript subsets designated for training, validation, and testing. Multiclass classification performance of the 4 models was evaluated at paragraph and sentence level using the HCC testing data set and analyzed by the one-versus-rest technique quantified by the receiver operating characteristic area under the curve (ROC AUC) score. ResultsNLP models accurately classified multiclass text from patient interviews. The Bidirectional Encoder Representations from Transformers model generally outperformed all other models at paragraph and sentence level. The highest predictive performance of the Bidirectional Encoder Representations from Transformers model was observed using the HCC data set to train and BTC data set to test (mean ROC AUC, 0.940 [SD 0.028]), with similarly high predictive performance using balanced and imbalanced training data sets from BTC and gastric cancer populations. ConclusionsNLP models were accurate in predicting multiclass classification of text from interviews of patients with cancer, with most surpassing 0.9 ROC AUC at paragraph level. NLP may be a useful tool for scaling up processing of patient interviews in clinical studies and, thus, could serve to facilitate patient input into drug development and improving patient care.

INDEPROP: Information-Preserving De-propagandization of News Articles (Student Abstract)

We propose INDEPROP, a novel Natural Language Processing (NLP) application for combating online disinformation by mitigating propaganda from news articles. INDEPROP (Information-Preserving De-propagandization) involves fine-grained propaganda detection and its removal while maintaining document level coherence, grammatical correctness and most importantly, preserving the news articles’ information content. We curate the first large-scale dataset of its kind consisting of around 1M tokens. We also propose a set of automatic evaluation metrics for the same and observe its high correlation with human judgment. Furthermore, we show that fine-tuning the existing propaganda detection systems on our dataset considerably improves their generalization to the test set.

P109 Early diagnosis of inflammatory arthritis (IA) using machine learning analysis of GP referral letters and blood tests to improve pre-hospital referral triage

Abstract Background/Aims The diagnosis of inflammatory arthritis (IA) from referral letters is challenging. We aimed to develop a model integrating GP referral letter data and pre-referral blood test results (BTR) to identify clusters of patient phenotypes (characteristics) that predict at referral that patients will have any of three types of IA (rheumatoid arthritis, seronegative inflammatory arthritis and psoriatic arthritis) requiring early DMARD treatment. Methods The anonymised text of original GP referral letters of patients on our departmental DMARD monitoring database with diagnoses of IA was compared with referral letters of patients with other inflammatory rheumatological conditions (OICs) entered into the same database over the same period. We developed and used novel natural language processing (NLP) methods based on bidirectional encoder representations from transformers (BERT) to identify patients with IA and OICs for triage purposes. Results We have analysed 867 OIC patients and 267 IJD letters using our NLP methods. Data augmentation was used to address imbalance and small data challenges in this real-world application. Our method using only GP referral letters achieved overall accuracy (using confirmed diagnosis as the outcome) of 80% in identifying IJD and OCIs, reaching 86% sensitivity and 84% precision (reproducibility) at detecting OICs, compared with 71% sensitivity and 68% precision in detecting IA patients. Conclusion Our NLP methods alone identified OIC with &amp;gt;80% sensitivity and precision. In data science terms our NLP based method thus has high sensitivity for identifying a wide variety of OICs likely to need DMARD treatment. Our methodology may have great practical value if GP referral letters can be analysed at source. For high specificity and precision very large amounts of data are required to train the NLP model; we had many more OIC letters than IA letters available in analysable formats from other databases in our department. Analysable text of referral letters from primary care is currently difficult to obtain and performing NLP on photographs of referral letters in electronic patient records (EPRs) is a major challenge. These significant bottlenecks in NHS data systems must be removed to allow direct access and analysis of GP referral letters in the compatible format. BTR will be easier to analyse in the EPRs. Combined analysis of pre-referral NLP and BTR may increase the efficiency of automated triage of GP referrals with suspected early IA. Machine learning analysis of GP referrals potentially allows triage of the right patient into the right clinic first time, assisting demand management and capacity in stretched rheumatology clinics and improving patient experience by ensuring the right clinic appointment. Further work is underway to improve the precision of the system with a view to embedding this into our advanced Advice and Guidance triage process. Disclosure A. Bradlow: None. B. Wang: None. W. Li: None. E. Bazuaye: None. A.T.Y. Chan: Member of speakers’ bureau; UCB, Novartis, Sanofi, AbbVie, Celgene and Janssen.

Novel approach by natural language processing for COVID-19 knowledge discovery

BackgroundThe impact of COVID-19 on public health has mandated an ‘all hands on deck’ scientific response. The current clinical study and basic research on COVID-19 are mainly based on existing publications or our knowledge of coronavirus. However, efficiently retrieval of accurate, relevant knowledge on COVID-19 can pose significant challenges for researchers. MethodsTo improve quality in accessing important literature findings, we developed a novel natural language processing (NLP) method to automatically recognize the associations among potential targeted host organ systems, associated clinical manifestations, and pathways. We further validated these associations through clinician experts' evaluations and prioritize candidate drug targets through bioinformatics network analysis. ResultsWe found that the angiotensin-converting enzyme 2 (ACE2), a receptor that SARS-CoV-2 required for cell entry, is associated with cardiovascular and endocrine organ system and diseases. Furthermore, we found SARS-CoV-2 is associated with some important pathways such as IL-6, TNF-alpha, and IL-1 beta-induced dyslipidemia, which are related to inflammation, lipogenesis, and oxidative stress mechanisms, suggesting potential drug candidates. ConclusionWe prioritized the list of therapeutic targets involved in antiviral and immune modulating drugs for experimental validation, rendering it valuable during public health crises marked by stresses on clinical and research capacity. Our automatic intelligence pipeline also contributes to other novel and emerging disease management and treatments in the future.

Ankle- and Toe-Brachial Index for Peripheral Artery Disease Identification: Unlocking Clinical Data Through Novel Methods.

Despite its high prevalence and clinical impact, research on peripheral artery disease (PAD) remains limited due to poor accuracy of billing codes. Ankle-brachial index (ABI) and toe-brachial index can be used to identify PAD patients with high accuracy within electronic health records. We developed a novel natural language processing (NLP) algorithm for extracting ABI and toe-brachial index values and laterality (right or left) from ABI reports. A random sample of 800 reports from 94 Veterans Affairs facilities during 2015 to 2017 was selected and annotated by clinical experts. We trained the NLP system using random forest models and optimized it through sequential iterations of 10-fold cross-validation and error analysis on 600 test reports and evaluated its final performance on a separate set of 200 reports. We also assessed the accuracy of NLP-extracted ABI and toe-brachial index values for identifying patients with PAD in a separate cohort undergoing ABI testing. The NLP system had an overall precision (positive predictive value) of 0.85, recall (sensitivity) of 0.93, and F1 measure (accuracy) of 0.89 to correctly identify ABI/toe-brachial index values and laterality. Among 261 patients with ABI testing (49% PAD), the NLP system achieved a positive predictive value of 92.3%, sensitivity of 83.1%, and specificity of 93.1% to identify PAD when compared with a structured chart review. The above findings were consistent in a range of sensitivity analysis. We successfully developed and validated an NLP system for identifying patients with PAD within the Veterans Affairs electronic health record. Our findings have broad implications for PAD research and quality improvement.

Discovering the same job ads expressed with the different sentences by using hybrid clustering algorithms

Text mining studies on job ads have become widespread in recent years to determine the qualifications required for each position. It can be said that the researches made for Turkish are limited while a large resource pool is encountered for the English language. Kariyer.Net is the biggest company for the job ads in Turkey and 99% of the ads are Turkish. Therefore, there is a necessity to develop novel Natural Language Processing (NLP) models in Turkish for analysis of this big database. In this study, the job ads of Kariyer.Net have been analyzed, and by using a hybrid clustering algorithm, the hidden associations in this dataset as the big data have been discovered. Firstly, all ads in the form of HTML codes have been transformed into regular sentences by the means of extracting HTML codes to inner texts. Then, these inner texts containing the core ads have been converted into the sub ads by traditional methods. After these NLP steps, hybrid clustering algorithms have been used and the same ads expressed with the different sentences could be managed to be detected. For the analysis, 57 positions about Information Technology sectors with 6,897 ad texts have been focused on. As a result, it can be claimed that the clusters obtained contain useful outcomes and the model proposed can be used to discover common and unique ads for each position.