Deep Learning-based Natural Language Processing Research Articles

Using deep learning-based natural language processing to identify reasons for statin nonuse in patients with atherosclerotic cardiovascular disease

BackgroundStatins conclusively decrease mortality in atherosclerotic cardiovascular disease (ASCVD), the leading cause of death worldwide, and are strongly recommended by guidelines. However, real-world statin utilization and persistence are low, resulting in excess mortality. Identifying reasons for statin nonuse at scale across health systems is crucial to developing targeted interventions to improve statin use.MethodsWe developed and validated deep learning-based natural language processing (NLP) approaches (Clinical Bidirectional Encoder Representations from Transformers [BERT]) to classify statin nonuse and reasons for statin nonuse using unstructured electronic health records (EHRs) from a diverse healthcare system.ResultsWe present data from a cohort of 56,530 ASCVD patients, among whom 21,508 (38%) lack guideline-directed statin prescriptions and statins listed as allergies in structured EHR portions. Of these 21,508 patients without prescriptions, only 3,929 (18%) have any discussion of statin use or nonuse in EHR documentation. The NLP classifiers identify statin nonuse with an area under the curve (AUC) of 0.94 (95% CI 0.93–0.96) and reasons for nonuse with a weighted-average AUC of 0.88 (95% CI 0.86–0.91) when evaluated against manual expert chart review in a held-out test set. Clinical BERT identifies key patient-level reasons (side-effects, patient preference) and clinician-level reasons (guideline-discordant practices) for statin nonuse, including differences by type of ASCVD and patient race/ethnicity.ConclusionsOur deep learning NLP classifiers can identify crucial gaps in statin nonuse and reasons for nonuse in high-risk populations to support education, clinical decision support, and potential pathways for health systems to address ASCVD treatment gaps.

잠재 관광객의 효율적 의사결정 촉진을 위한 온라인 여행사의 호텔 추천 시스템에 관한 연구

This study proposes a new recommendation algorithm to facilitate decision-making by successfully predicting the preferences of potential tourists to hotels in Online Travel Agency (OTA) service. To this end, we developed a new hotel recommendation algorithm based on deep sequential interaction embedding. Specifically, we used Doc2Vec, a deep learning-based natural language processing technology, to learn expression vectors taking into account the interaction order of tourists and items, and then input them into MLP and GMF to learn the nonlinearity and linearity of interactions. Experiments with one real-world dataset demonstrate that our algorithm outperforms other recent recommendation algorithms with a performance improvement of 13.23% in hotel tourism. From the experimental results, the usefulness of the recommender system in OTA and the extension to other tourism fields were discussed, and theoretical and practical implications were presented.

Report Users' Perceived Sentiments of Key Audit Matters and Firm Performance: Evidence from a Deep Learning-Based Natural Language Processing Approach

ABSTRACTWe investigate the associations between the sentiment report users perceive in key audit matters (KAMs) and current and future firm performance. We also investigate the validity of the bidirectional encoder representations from transformers (BERT) model for automatically extracting KAM sentiment in Taiwanese listed firms' audit reports. Positive associations between KAM sentiment and current and next-year firm performances, measured by Tobin's Q, ROA, and ROE, are discovered based on a two-year sample of 1,606 firm–year observations, including manually labeled sentiment data in 2017 and BERT-extracted sentiment data in 2018. However, the evidence of the positive association between KAM sentiment and current firm market performance (Tobin's Q) is weaker in 2017 than in 2018 statistically. Our results suggest that KAM sentiment reflects future firm performance and support the application of the BERT deep learning approach for textual mining. This study has implications for regulators, practitioners, and academics.JEL Classifications: D83; L25; M42.

Deep-ADCA: Development and Validation of Deep Learning Model for Automated Diagnosis Code Assignment Using Clinical Notes in Electronic Medical Records.

Currently, the International Classification of Diseases (ICD) codes are being used to improve clinical, financial, and administrative performance. Inaccurate ICD coding can lower the quality of care, and delay or prevent reimbursement. However, selecting the appropriate ICD code from a patient’s clinical history is time-consuming and requires expert knowledge. The rapid spread of electronic medical records (EMRs) has generated a large amount of clinical data and provides an opportunity to predict ICD codes using deep learning models. The main objective of this study was to use a deep learning-based natural language processing (NLP) model to accurately predict ICD-10 codes, which could help providers to make better clinical decisions and improve their level of service. We retrospectively collected clinical notes from five outpatient departments (OPD) from one university teaching hospital between January 2016 and December 2016. We applied NLP techniques, including global vectors, word to vectors, and embedding techniques to process the data. The dataset was split into two independent training and testing datasets consisting of 90% and 10% of the entire dataset, respectively. A convolutional neural network (CNN) model was developed, and the performance was measured using the precision, recall, and F-score. A total of 21,953 medical records were collected from 5016 patients. The performance of the CNN model for the five different departments was clinically satisfactory (Precision: 0.50~0.69 and recall: 0.78~0.91). However, the CNN model achieved the best performance for the cardiology department, with a precision of 69%, a recall of 89% and an F-score of 78%. The CNN model for predicting ICD-10 codes provides an opportunity to improve the quality of care. Implementing this model in real-world clinical settings could reduce the manual coding workload, enhance the efficiency of clinical coding, and support physicians in making better clinical decisions.

Deep learning-based NLP data pipeline for EHR-scanned document information extraction.

ObjectiveScanned documents in electronic health records (EHR) have been a challenge for decades, and are expected to stay in the foreseeable future. Current approaches for processing include image preprocessing, optical character recognition (OCR), and natural language processing (NLP). However, there is limited work evaluating the interaction of image preprocessing methods, NLP models, and document layout.Materials and MethodsWe evaluated 2 key indicators for sleep apnea, Apnea hypopnea index (AHI) and oxygen saturation (SaO2), from 955 scanned sleep study reports. Image preprocessing methods include gray-scaling, dilating, eroding, and contrast. OCR was implemented with Tesseract. Seven traditional machine learning models and 3 deep learning models were evaluated. We also evaluated combinations of image preprocessing methods, and 2 deep learning architectures (with and without structured input providing document layout information), with the goal of optimizing end-to-end performance.ResultsOur proposed method using ClinicalBERT reached an AUROC of 0.9743 and document accuracy of 94.76% for AHI, and an AUROC of 0.9523 and document accuracy of 91.61% for SaO2.DiscussionThere are multiple, inter-related steps to extract meaningful information from scanned reports. While it would be infeasible to experiment with all possible option combinations, we experimented with several of the most critical steps for information extraction, including image processing and NLP. Given that scanned documents will likely be part of healthcare for years to come, it is critical to develop NLP systems to extract key information from this data.ConclusionWe demonstrated the proper use of image preprocessing and document layout could be beneficial to scanned document processing.

딥러닝 기반 스마트관광 추천 알고리즘 개발 연구 : 새로운 OTA 레스토랑 추천 시스템의 제안

The purpose of this study is to propose a smart tourism restaurant recommendation algorithm based on deep sequential interaction embedding to successfully predict the preferred choices of tourists and promote their decision-making. We use Doc2Vec, a deep learning-based natural language processing technology, to learn the representation vectors of tourists and items by considering their interactions’ sequence, and then input them into multi-layer perceptron and matrix factorization that make up a deep artificial neural network to learn nonlinearity and linearity of interactions. Finally, we predict the possibility of tourists preferring restaurants and recommend top-N restaurants for each user. Indeed, the results of comparing the TripAdvior and MovieLens data sets demonstrated that our proposed recommendation algorithm (DSESTRR) significantly improved the recommendation accuracy compared to other algorithms. Based on the above results, the usefulness of DSESTRR in the tourism field was discussed, and theoretical and practical implications were proposed.

Classifying social determinants of health from unstructured electronic health records using deep learning-based natural language processing

ObjectiveSocial determinants of health (SDOH) are non-medical factors that can profoundly impact patient health outcomes. However, SDOH are rarely available in structured electronic health record (EHR) data such as diagnosis codes, and more commonly found in unstructured narrative clinical notes. Hence, identifying social context from unstructured EHR data has become increasingly important. Yet, previous work on using natural language processing to automate extraction of SDOH from text (a) usually focuses on an ad hoc selection of SDOH, and (b) does not use the latest advances in deep learning. Our objective was to advance automatic extraction of SDOH from clinical text by (a) systematically creating a set of SDOH based on standard biomedical and psychiatric ontologies, and (b) training state-of-the-art deep neural networks to extract mentions of these SDOH from clinical notes. DesignA retrospective cohort study. Setting and participantsData were extracted from the Medical Information Mart for Intensive Care (MIMIC-III) database. The corpus comprised 3,504 social related sentences from 2,670 clinical notes. MethodsWe developed a framework for automated classification of multiple SDOH categories. Our dataset comprised narrative clinical notes under the “Social Work” category in the MIMIC-III Clinical Database. Using standard terminologies, SNOMED-CT and DSM-IV, we systematically curated a set of 13 SDOH categories and created annotation guidelines for these. After manually annotating the 3,504 sentences, we developed and tested three deep neural network (DNN) architectures – convolutional neural network (CNN), long short-term memory (LSTM) network, and the Bidirectional Encoder Representations from Transformers (BERT) – for automated detection of eight SDOH categories. We also compared these DNNs to three baselines models: (1) cTAKES, as well as (2) L2-regularized logistic regression and (3) random forests on bags-of-words. Model evaluation metrics included micro- and macro- F1, and area under the receiver operating characteristic curve (AUC). ResultsAll three DNN models accurately classified all SDOH categories (minimum micro-F1 = 0.632, minimum macro-AUC = 0.854). Compared to the CNN and LSTM, BERT performed best in most key metrics (micro-F1 = 0.690, macro-AUC = 0.907). The BERT model most effectively identified the “occupational” category (F1 = 0.774, AUC = 0.965) and least effectively identified the “non-SDOH” category (F = 0.491, AUC = 0.788). BERT outperformed cTAKES in distinguishing social vs non-social sentences (BERT F1 = 0.87 vs. cTAKES F1 = 0.06), and outperformed logistic regression (micro-F1 = 0.649, macro-AUC = 0.696) and random forest (micro-F1 = 0.502, macro-AUC = 0.523) trained on bag-of-words. ConclusionsOur study framework with DNN models demonstrated improved performance for efficiently identifying a systematic range of SDOH categories from clinical notes in the EHR. Improved identification of patient SDOH may further improve healthcare outcomes.

Performances of depression detection through deep learning-based natural language processing to mandarin chinese medical records: Comparison between civilian and military populations

Objective: A certain portion of patients with depression is under-diagnosed and has attracted the attention in the field of natural language processing (NLP). In this study, we intended to explore the feasibility of transferring unstructured textual records into a screening tool to early detect depression. Methods: We recruited 22,355 medical records in Mandarin traditional Chinese from the psychiatry emergency department of a military psychiatry center from 2004 to 2019. We preprocessed all the context of present illness histories as corpus and the presence of clinical diagnoses of depression as an outcome. A state-of-the-art NLP model was developed based on a pretrained bidirectional encoder representation from transformers (BERT) model along with several convolutional neural network (CNN) and trained by the training set (80% of original data) of total samples (BERTgeneral) and of civilian samples (BERTcivilian) and of military samples (BERTmilitary) independently. The receiver operating characteristic (ROC) and area under curve (AUC) of three trained models were compared for predicting depression for the test dataset (20% of original data) of general and specific samples. Results: The experimental results demonstrated excellent performance of BERTgeneral for general samples (AUC = 0.93, sensitivity = 0.817, specificity = 0.920 for optimal cut-off point) and civilian sample (AUC = 0.91, sensitivity = 0.851, specificity = 0.851 for optimal cut-off point). BERTgeneral showed a significant underperformance of for military samples (AUC = 0.79, sensitivity = 0.712, specificity = 0.732, p < 0.05 for optimal cut-off point). That of BERTmilitary was slight higher (AUC = 0.82, sensitivity = 0.708, specificity = 0.786 for optimal cut-off point) for military samples. Conclusion: This study showed the feasibility of applying deep learning technique as a depression-detection assistant tool in Mandarin Chinese medical records. However, the subjects' specific situation, e.g., military status, is warranted for further investigation.

Machine learning and deep learning-based Natural Language Processing for auto-vetting the appropriateness of Lumbar Spine Magnetic Resonance Imaging Referrals

Manual vetting of radiology referrals is an essential daily task to ensure the appropriateness of the received referrals. Such tasks require sufficient clinical experience and may challenge the radiology staff. With the emerging of artificial intelligence (AI) technology and advancement in natural language processing NLP, most of the available machine learning-based NLP models targeted research cohort building and healthcare quality. Other healthcare management tasks such as auto-vetting radiology referrals have not been adequately encoded. Furthermore, challenges, including class imbalance and lack of direct comparison with humans, are yet to be investigated sufficiently. In this study, a set of machine learning and deep learning models were developed for auto-vetting of lumbar spine magnetic resonance imaging LSMRI referrals as indicated or not indicated for scanning using referrals from two hospitals. The impact of applying one of the text augmentation techniques on the models' performance has been investigated. In addition, the performance of four different feature extraction techniques has been critically analyzed. Moreover, a comparison has been conducted between the developed models with two expert radiologists who were not involved in establishing the gold standard labels using an unseen dataset. The results show that the models’ performances significantly improved with the augmented data, with an increase in F1 scores ranging from 1% to 8%. Support vector machine with bag of words achieved the highest AUC reaching 0.99. Convolutional neural network model achieved the second-highest model with AUC = 0.97. All models outperformed the two expert radiologists when comparisons were conducted on the unseen dataset.

Predicting aspect-based sentiment using deep learning and information visualization: The impact of COVID-19 on the airline industry

• The main purpose of this study is to analyze airline reviews and understand consumer satisfaction before and during COVID-19 pandemic. • This study applies deep learning-based natural language processing, word embeddings such as BERT, and visual analytics to learn linguistic features and aspect-level ratings. • Our model outperforms traditional machine learning methods and thus can be used to predict unrated aspect ratings. • Our data and visual analyses reveal that both review and aspect ratings drop significantly during the pandemic. • This study complements the existing research methods and extends the use of data-driven and visual analytics approaches to better understand customer satisfaction within the context of the COVID-19 pandemic. This study investigates customer satisfaction through aspect-level sentiment analysis and visual analytics. We collected and examined the flight reviews on TripAdvisor from January 2016 to August 2020 to gauge the impact of COVID-19 on passenger travel sentiment in several aspects. Till now, information systems, management, and tourism research have paid little attention to the use of deep learning and word embedding techniques, such as bidirectional encoder representations from transformers, especially for aspect-level sentiment analysis. This paper aims to identify perceived aspect-based sentiments and predict unrated sentiments for various categories to address this research gap. Ultimately, this study complements existing sentiment analysis methods and extends the use of data-driven and visual analytics approaches to better understand customer satisfaction in the airline industry and within the context of the COVID-19. Our proposed method outperforms baseline comparisons and therefore contributes to the theoretical and managerial literature.

BERT-Based Natural Language Processing of Drug Labeling Documents: A Case Study for Classifying Drug-Induced Liver Injury Risk.

Background & Aims: The United States Food and Drug Administration (FDA) regulates a broad range of consumer products, which account for about 25% of the United States market. The FDA regulatory activities often involve producing and reading of a large number of documents, which is time consuming and labor intensive. To support regulatory science at FDA, we evaluated artificial intelligence (AI)-based natural language processing (NLP) of regulatory documents for text classification and compared deep learning-based models with a conventional keywords-based model. Methods: FDA drug labeling documents were used as a representative regulatory data source to classify drug-induced liver injury (DILI) risk by employing the state-of-the-art language model BERT. The resulting NLP-DILI classification model was statistically validated with both internal and external validation procedures and applied to the labeling data from the European Medicines Agency (EMA) for cross-agency application. Results: The NLP-DILI model developed using FDA labeling documents and evaluated by cross-validations in this study showed remarkable performance in DILI classification with a recall of 1 and a precision of 0.78. When cross-agency data were used to validate the model, the performance remained comparable, demonstrating that the model was portable across agencies. Results also suggested that the model was able to capture the semantic meanings of sentences in drug labeling. Conclusion: Deep learning-based NLP models performed well in DILI classification of drug labeling documents and learned the meanings of complex text in drug labeling. This proof-of-concept work demonstrated that using AI technologies to assist regulatory activities is a promising approach to modernize and advance regulatory science.

Learning and critiquing pairwise activity relationships for schedule quality control via deep learning-based natural language processing

In construction, schedule mistakes causing delays beyond substantial completion dates cost contractors expensive liquidated damages. Hence, several industry guidelines, such as the DCMA's 14 point assessment, define schedule quality and offer systematic methods for ensuring it. These guidelines list “logic” as an essential control metric, and they require planners to ensure their schedules are free of missing or wrong logical dependencies. Checking the logic requires extensive construction domain knowledge, and planners perform it entirely manually as there are no available software solutions that support it. This paper offers a novel machine learning-based solution that learns construction scheduling domain knowledge from existing records completely automatically and applies it to validate the logic in input schedules achieving an F1 score of 88.3%. Furthermore, we tailor our method to use the learned knowledge to schedule a list of unordered activities. The details of the method, experimental results, benefits, and limitations are discussed.

A STUDY ON THE SENTIMENT ANALYSIS (POSITIVE, NEGATIVE) OF WORDS APPEARING IN KYRGYZ NEWS BY APPLYING THE DEEP LEARNING-BASED NLP (NATURAL LANGUAGE PROCESSING) TECHNIQUES FOR STUDENTS PRACTICE

This study is theoretical on the sentiment analysis field of deep learning-based natural language processing, which is the world's advanced technology, namely data collection and preprocessing stage, tokenizing stage, Sentiment Dictionary construction stage, positive and negative word extraction stage through sentiment analysis, deep learning introduces major contents and related technologies such as model configuration, execution stage, and data visualization stage. In addition, speech processing technology performed in the data collection stage, STT (Speech to Text) and TTS (Text to Speech) technology will be introduced. In this study, a program was written using various open sources (libraries) including ’keras’ 2.0 version used in the deep learning natural language processing field of the python language base. This study is executed to help students who are studying deep learning natural language processing.

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

In radiology, natural language processing (NLP) allows the extraction of valuable information from radiology reports. It can be used for various downstream tasks such as quality improvement, epidemiological research, and monitoring guideline adherence. Class imbalance, variation in dataset size, variation in report complexity, and algorithm type all influence NLP performance but have not yet been systematically and interrelatedly evaluated. In this study, we investigate these factors on the performance of four types [a fully connected neural network (Dense), a long short-term memory recurrent neural network (LSTM), a convolutional neural network (CNN), and a Bidirectional Encoder Representations from Transformers (BERT)] of deep learning-based NLP. Two datasets consisting of radiologist-annotated reports of both trauma radiographs (n = 2469) and chest radiographs and computer tomography (CT) studies (n = 2255) were split into training sets (80%) and testing sets (20%). The training data was used as a source to train all four model types in 84 experiments (Fracture-data) and 45 experiments (Chest-data) with variation in size and prevalence. The performance was evaluated on sensitivity, specificity, positive predictive value, negative predictive value, area under the curve, and F score. After the NLP of radiology reports, all four model-architectures demonstrated high performance with metrics up to > 0.90. CNN, LSTM, and Dense were outperformed by the BERT algorithm because of its stable results despite variation in training size and prevalence. Awareness of variation in prevalence is warranted because it impacts sensitivity and specificity in opposite directions.

Deep learning-based natural language processing in ophthalmology: applications, challenges and future directions.

Artificial intelligence (AI) is the fourth industrial revolution in mankind's history. Natural language processing (NLP) is a type of AI that transforms human language, to one that computers can interpret and process. NLP is still in the formative stages of development in healthcare, with promising applications and potential challenges in its applications. This review provides an overview of AI-based NLP, its applications in healthcare and ophthalmology, next-generation use case, as well as potential challenges in deployment. The integration of AI-based NLP systems into existing clinical care shows considerable promise in disease screening, risk stratification, and treatment monitoring, amongst others. Stakeholder collaboration, greater public acceptance, and advancing technologies will continue to shape the NLP landscape in healthcare and ophthalmology. Healthcare has always endeavored to be patient centric and personalized. For AI-based NLP systems to become an eventual reality in larger-scale applications, it is pertinent for key stakeholders to collaborate and address potential challenges in application. Ultimately, these would enable more equitable and generalizable use of NLP systems for the betterment of healthcare and society.

Early prediction of diagnostic-related groups and estimation of hospital cost by processing clinical notes

As healthcare providers receive fixed amounts of reimbursement for given services under DRG (Diagnosis-Related Groups) payment, DRG codes are valuable for cost monitoring and resource allocation. However, coding is typically performed retrospectively post-discharge. We seek to predict DRGs and DRG-based case mix index (CMI) at early inpatient admission using routine clinical text to estimate hospital cost in an acute setting. We examined a deep learning-based natural language processing (NLP) model to automatically predict per-episode DRGs and corresponding cost-reflecting weights on two cohorts (paid under Medicare Severity (MS) DRG or All Patient Refined (APR) DRG), without human coding efforts. It achieved macro-averaged area under the receiver operating characteristic curve (AUC) scores of 0·871 (SD 0·011) on MS-DRG and 0·884 (0·003) on APR-DRG in fivefold cross-validation experiments on the first day of ICU admission. When extended to simulated patient populations to estimate average cost-reflecting weights, the model increased its accuracy over time and obtained absolute CMI error of 2·40 (1·07%) and 12·79% (2·31%), respectively on the first day. As the model could adapt to variations in admission time, cohort size, and requires no extra manual coding efforts, it shows potential to help estimating costs for active patients to support better operational decision-making in hospitals.

Phishing websites detection via CNN and multi-head self-attention on imbalanced datasets

Phishing websites belong to a social engineering attack where perpetrators fake legitimate websites to lure people to access so as to illegally acquire user’s identity, password, privacy and even properties. This attack imposes a great threat to people and becomes more and more severe. In order to identify phishing websites, many proposals have shown their merits. For example, the classical proposal CNN-LSTM received a very high precision by combining Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) together. However, despite CNN achieved great success in AI area, LSTM still exists the biases issue since it always treats the later features much more important than the former ones. In the meanwhile, as the self-attention mechanism can discover the text’s inner dependency relationships, it has been widely applied to various tasks of deep learning-based Natural Language Processing (NLP). If we treat a URL as a text string, this mechanism can learn comprehensive URL representations. In order to improve the accuracy for phishing websites detection further, in this paper, we propose a novel Convolutional Neural Network (CNN) with self-attention named self-attention CNN for phishing Uniform Resource Locators (URLs) identification. Specifically, self-attention CNN first leverages Generative Adversarial Network (GAN) to generate phishing URLs so as to balance the datasets of legitimate and phishing URLs. Then it utilizes CNN and multi-head self-attention to construct our new classifier which is comprised of four blocks, namely the input block, the attention block, the feature block and the output block. Finally, the trained classifier can give a high-accuracy result for an unknown website URL. Overall thorough experiments indicate that self-attention CNN achieves 95.6% accuracy, which outperforms CNN-LSTM, single CNN and single LSTM by 1.4%, 4.6% and 2.1% respectively.

Deep Learning-based Natural Language Processing Methods Comparison for Presumptive Detection of Cyberbullying in Social Networks

Due to TIC development in the last years, users have managed to satisfy many social experiences through several digital media like blogs, web and especially social networks. However, not all social media users have had good experiences with these media. Since there are malicious users that are able to cause negative psychological effects over people, this is called cyberbullying. For this reason, social networks such as Twitter are looking to implement models based on deep learning or machine learning capable of recognizing harassing comments on their platforms. However, most of these models are focused on the use of English language and there are very few models adapted for Spanish language. This is why, in this paper we propose the evaluation of an RNN+LSTM neural network, as well as a BERT model through sentiment analysis, to perform the detection of cyberbullying based on Spanish language for Ecuadorian accounts of the social network Twitter. The results obtained show a balance between execution time and accuracy obtained for the RNN + LSTM model. In addition, evaluations of comments that are not explicitly offensive show a better performance for the BERT model, which outperforms its counterpart by 20%.

Online critical review classification in response strategy and service provider rating: Algorithms from heuristic processing, sentiment analysis to deep learning

This research proposes and tests mechanisms for defining and identifying the critical online consumer reviews that firms could prioritize to optimize their online response strategies, while incorporating the latest artificial intelligence (AI) technology to deal with the overwhelming volume of information. Three sets of analytical tools are introduced: Heuristic Processing, Linguistic Feature Analysis, and Deep Learning-based Natural Language Processing (NLP), to extract review information. Twelve algorithms to classify critical reviews were developed accordingly and empirically tested for their effectiveness. Our econometric analysis of 110,146 online reviews from a chain operation in hospitality industry over seven years identifies six outstanding algorithms. Firm value rating, comment length, valence, and certain consumer emotions, in addition to past comment-response behavior, are found to be superior in predicting incoming review criticality. However, the service attributes such as urgency to reply and the feasibility of actions to take are not as informative.

Understanding spatial language in radiology: Representation framework, annotation, and spatial relation extraction from chest X-ray reports using deep learning.

Radiology reports contain a radiologist's interpretations of images, and these images frequently describe spatial relations. Important radiographic findings are mostly described in reference to an anatomical location through spatial prepositions. Such spatial relationships are also linked to various differential diagnoses and often described through uncertainty phrases. Structured representation of this clinically significant spatial information has the potential to be used in a variety of downstream clinical informatics applications. Our focus is to extract these spatial representations from the reports. For this, we first define a representation framework based on the Spatial Role Labeling (SpRL) scheme, which we refer to as Rad-SpRL. In Rad-SpRL, common radiological entities tied to spatial relations are encoded through four spatial roles: Trajector, Landmark, Diagnosis, and Hedge, all identified in relation to a spatial preposition (or Spatial Indicator). We annotated a total of 2,000 chest X-ray reports following Rad-SpRL. We then propose a deep learning-based natural language processing (NLP) method involving word and character-level encodings to first extract the Spatial Indicators followed by identifying the corresponding spatial roles. Specifically, we use a bidirectional long short-term memory (Bi-LSTM) conditional random field (CRF) neural network as the baseline model. Additionally, we incorporate contextualized word representations from pre-trained language models (BERT and XLNet) for extracting the spatial information. We evaluate both gold and predicted Spatial Indicators to extract the four types of spatial roles. The results are promising, with the highest average F1 measure for Spatial Indicator extraction being 91.29 (XLNet); the highest average overall F1 measure considering all the four spatial roles being 92.9 using gold Indicators (XLNet); and 85.6 using predicted Indicators (BERT pre-trained on MIMIC notes). The corpus is available in Mendeley at http://dx.doi.org/10.17632/yhb26hfz8n.1 and https://github.com/krobertslab/datasets/blob/master/Rad-SpRL.xml.