Natural Language Processing Program Research Articles

Improving dynamic prognostication of patients with hepatocellular carcinoma by node fusion algorithm equipped survival path mapping.

e13594 Background: Patients with hepatocellular carcinoma (HCC) always require routine surveillance and repeated treatment. Our previous work (1) developed a machine learning methodology called “Survival Path Mapping”, which converted the time series data into a cascading survival map to facilitate dynamic prognosis prediction and treatment planning. However, in previous work, the exponential growth of paths raised challenges of exhausting limited cohort and over-fitting risk. Novel algorithm enhancing data utilization in the Survival Path Mapping was in urgent need. Methods: A node fusion algorithm was designed. First, feature selection algorithm bifurcated each parent-nodes into temporary child-nodes, which constitutes temporary child-nodes set. Then, clustering was performed within temporary child-nodes set according to key features of each child-node. Finally, in each cluster, multiple child-nodes would be merged into one if there was no significant difference in survival. For nodes with bifurcation, a treatment recommendation algorithm was developed. A total of 19446 patients with newly diagnosed primary HCC at Sun Yat-sen University Cancer Center (SYSUCC) between 2008 and 2023 were retrospectively included as internal cohort (divided into internal train and holdout validation cohort); meanwhile 1135 patients from five hospitals were included as multicenter external validation cohort. Dedicated natural language processing program was developed to extract structured features from medical imaging reports. The time series data of patients were converted into data of time slices, with an interval of 3 months. Results: The original survival path mapping (raw-SP) consists of 463 different nodes, while the node fusion survival path mapping (fusion-SP) consisted of 84 nodes (81.8% decrease from raw-SP). The 3-year c-index of fusion-SP were 0.745 (95% CI 0.717-0.772), significantly better (p<0.001) than that of raw-SP: 0.725 (95% CI 0.696-0.755), in the internal holdout validation. Both raw-SP and fusion-SP significantly (p < 0.001) outperformed conventional BCLC and CNLC staging. In the multicenter external validation, the fusion-SP also demonstrated better performance than raw-SP and BCLC staging system. The recommended therapy calculated based fusion-SP system for patients receiving second-line or subsequent treatment had superior or non-inferior efficacy compared to BCLC guideline’s recommendation. Conclusions: The proposed node fusion algorithm solved exponential growth of paths in previous survival path algorithm, improving the accuracy and external validity of dynamic prognosis prediction for HCC patients. The fusion-SP model also retained its transparency and could be used to guide dynamic treatment planning, with its effectiveness in treatment guidance evaluated in prospective trials in the future. 1. Nat Commun 9, 2230; 2018.

AI Chatbots And Psychotherapy: A Match Made In Heaven?

Dear Editor, Artificial Intelligence (AI) is revolutionizing psychotherapy by combating its inaccessibility.1 AI chatbots and conversational agents are among the most promising integrations in psychiatry and psychology. By harnessing the power of machine learning and artificial intelligence (Figure I), AI chatbots and conversational agents can provide a level of interaction and support previously unattainable in the landscape of mental health. In this letter, I will explore some of their numerous benefits. Chatbots have evolved from early examples such as ELIZA, a programmable natural language processing program, and PARRY, a program that mimicked the behaviour of a human with schizophrenia, to modern ones such as Woebot, Tess, and Replika, which offer cognitive behavioural therapy, sessions, and self-understanding, respectively. Some chatbots such as Kaspar and Nao are also embodied in physical robots, which help children with autism spectrum disorder (ASD) learn social skills and facial recognition. AI chatbots can address the challenges of mental health care, such as the shortage of professionals2, especially in underdeveloped countries; the stigma of seeking help, which is prevalent in many cultures; the cost of therapy, which can be prohibitive for many people; and the limitations of human therapists, who can have variable attention spans, attention-to-detail, and memory capacities. AI chatbots can also offer benefits such as accessibility, anonymity, personalization, and effectiveness. Studies have shown that AI chatbots can improve the outcomes and experiences of mental health patients in various ways, such as enhancing the performance of those with ASD with robotic "therapists"1, increasing their disclosure of sensitive information to chatbots2, and reducing their distress after talking about their problems with conversational agents3. The benefits of AI chatbots for mental health are clear. However, they also face many challenges that limit their usability and reliability. They may lack the emotional skills of human therapists and may not be accountable for their diagnoses and recommendations. They may also not be based on sound mental health knowledge and evidence, as technology companies may not have expertise and experience in this field. Therefore, technology companies and mental health professionals must collaborate and adopt a rational approach to develop and evaluate chatbots. Only then can chatbots realize their full potential and complement existing mental health services safely and ethically.

Intelligent Educational Recommendation Platform with AI Chatbots

The objectives of this research were as follows. 1) Analyze the intelligent educational recommendation platform with AI Chatbots. 2) Design the architecture of the intelligent educational recommendation platform with AI Chatbots. 3) Develop the architecture of the intelligent educational recommendation platform with AI Chatbots. 4) Study the appropriateness of developing the intelligent educational recommendation platform with AI Chatbots. The sample used in the research was seven experts in information system development from various institutions in higher education. The architecture of the intelligent educational recommendation platform with AI Chatbots there is two main components: 1) Stakeholders consisting of system administrators and external users, and 2) The working process of the intelligent educational recommendation platform with AI Chatbots consists of four parts including natural language processing, dialog management, database and application programming interface (API), and response generation. Assessment of the appropriateness of the architecture of the intelligent educational recommendation platform with AI Chatbots found that 1) the architecture of the intelligent educational recommendation platform with AI Chatbots, overall at a high appropriated, 2) the architecture of the intelligent educational recommendation platform with AI Chatbots, an individual element at a high appropriated, and 3) the architecture of the intelligent educational recommendation platform with AI Chatbots, Integrated elements at a high appropriated. As described earlier, the architecture of the intelligent educational recommendation platform with AI Chatbots can be a guideline for developing with AI Chatbots in the future.

From concept to practice: a scoping review of the application of AI to aphasia diagnosis and management

Purpose Aphasia is an acquired communication disability resulting from impairments in language processing following brain injury, most commonly stroke. People with aphasia experience difficulties in all modalities of language that impact their quality of life. Therefore, researchers have investigated the use of Artificial Intelligence (AI) to deliver innovative solutions in Aphasia management and rehabilitation. Materials and methods We conducted a scoping review of the use of AI in aphasia research and rehabilitation to explore the evolution of AI applications to aphasia, the progression of technologies and applications. Furthermore, we aimed to identify gaps in the use of AI in Aphasia to highlight the potential areas where AI might add value. We analysed 77 studies to determine the research objectives, the history of AI techniques in Aphasia and their progression over time. Results Most of the studies focus on automated assessment using AI, with recent studies focusing on AI for therapy and personalised assistive systems. Starting from prototypes and simulations, the use of AI has progressed to include supervised machine learning, unsupervised machine learning, natural language processing, fuzzy rules, and genetic programming. Conclusion Considerable scope remains to align AI technology with aphasia rehabilitation to empower patient-centred, customised rehabilitation and enhanced self-management.

Depression and anxiety are associated with an increased risk of infection, revision, and reoperation following total hip or knee arthroplasty.

The aim of this study was to determine the prevalence of depressive and anxiety disorders prior to total hip (THA) and total knee arthroplasty (TKA) and to assess their impact on the rates of any infection, revision, or reoperation. Between January 2000 and March 2019, 21,469 primary and revision arthroplasties (10,011 THAs; 11,458 TKAs), which were undertaken in 15,504 patients at a single academic medical centre, were identified from a 27-county linked electronic medical record (EMR) system. Depressive and anxiety disorders were identified by diagnoses in the EMR or by using a natural language processing program with subsequent validation from review of the medical records. Patients with mental health diagnoses other than anxiety or depression were excluded. Depressive and/or anxiety disorders were common before THA and TKA, with a prevalence of 30% in those who underwent primary THA, 33% in those who underwent revision THA, 32% in those who underwent primary TKA, and 35% in those who underwent revision TKA. The presence of depressive or anxiety disorders was associated with a significantly increased risk of any infection (primary THA, hazard ratio (HR) 1.5; revision THA, HR 1.9; primary TKA, HR 1.6; revision TKA, HR 1.8), revision (THA, HR 1.7; TKA, HR 1.6), re-revision (THA, HR 2.0; TKA, HR 1.6), and reoperation (primary THA, HR 1.6; revision THA, HR 2.2; primary TKA, HR 1.4; revision TKA, HR 1.9; p < 0.03 for all). Patients with preoperative depressive and/or anxiety disorders were significantly less likely to report "much better" joint function after primary THA (78% vs 87%) and primary TKA (86% vs 90%) compared with those without these disorders at two years postoperatively (p < 0.001 for all). The presence of depressive or anxiety disorders prior to primary or revision THA and TKA is common, and associated with a significantly higher risk of infection, revision, reoperation, and dissatisfaction. This topic deserves further study, and surgeons may consider mental health optimization to be of similar importance to preoperative variables such as diabetic control, prior to arthroplasty.

Clinical Implementation of a Combined Artificial Intelligence and Natural Language Processing Quality Assurance Program for Pulmonary Nodule Detection in the Emergency Department Setting.

This quality assurance study assessed the implementation of a combined artificial intelligence (AI) and natural language processing (NLP) program for pulmonary nodule detection in the emergency department setting. The program was designed to function outside of normal reading workflows to minimize radiologist interruption. In all, 19,246 CT examinations including at least some portion of the lung anatomy performed in the emergent setting from October 1, 2021, to June 1, 2022, were processed by the combined AI-NLP program. The program used an AI algorithm trained on 6-mm to 30-mm pulmonary nodules to analyze CT images and an NLP to analyze radiological reports. Cases flagged as negative for pulmonary nodules by the NLP but positive by the AI algorithm were classified as suspected discrepancies. Discrepancies result in secondary review of examinations for possible addenda. Out of 19,246 CT examinations, 50 examinations (0.26%) resulted in secondary review, and 34 of 50 (68%) reviews resulted in addenda. Of the 34 addenda, 20 patients received instruction for new follow-up imaging. Median time to addendum was 11 hours. The majority of reviews and addenda resulted from missed pulmonary nodules on CT examinations of the abdomen and pelvis. A background quality assurance process using AI and NLP helped improve the detection of pulmonary nodules and resulted in increased numbers of patients receiving appropriate follow-up imaging recommendations. This was achieved without disrupting in-shift radiologist workflow or causing significant delays in patient follow for the diagnosed pulmonary nodule.

Developing a machine learning model to detect diagnostic uncertainty in clinical documentation.

Diagnostic uncertainty, when unrecognized or poorly communicated, can result in diagnostic error. However, diagnostic uncertainty is challenging to study due to a lack of validated identification methods. This study aims to identify distinct linguistic patterns associated with diagnostic uncertainty in clinical documentation. This case-control study compares the clinical documentation of hospitalized children who received a novel uncertain diagnosis (UD) diagnosis label during their admission to a set of matched controls. Linguistic analyses identified potential linguistic indicators (i.e., words or phrases) of diagnostic uncertainty that were then manually reviewed by a linguist and clinical experts to identify those most relevant to diagnostic uncertainty. A natural language processing program categorized medical terminology into semantic types (i.e., sign or symptom), from which we identified a subset of these semantic types that both categorized reliably and were relevant to diagnostic uncertainty. Finally, a competitive machine learning modeling strategy utilizing the linguistic indicators and semantic types compared different predictive models for identifying diagnostic uncertainty. Our cohort included 242 UD-labeled patients and 932 matched controls with a combination of 3070 clinical notes. The best-performing model was a random forest, utilizing a combination of linguistic indicators and semantic types, yielding a sensitivity of 89.4% and a positive predictive value of 96.7%. Expert labeling, natural language processing, and machine learning methods combined with human validation resulted in highly predictive models to detect diagnostic uncertainty in clinical documentation and represent a promising approach to detecting, studying, and ultimately mitigating diagnostic uncertainty in clinical practice.

77 Results of a Novel National Emergency Department Chief Complaint Database

77 Results of a Novel National Emergency Department Chief Complaint Database

Sustainable Air-Conditioning Systems Enabled by Artificial Intelligence: Research Status, Enterprise Patent Analysis, and Future Prospects

Artificial intelligence (AI) technologies have developed rapidly since 2000. Numerous academic papers have been published regarding energy efficiency improvements for air-conditioning systems. This study reviewed 12 review papers and selected 85 specific cases of applications of AI for HVAC energy usage reduction. In addition to academic studies, 31,221 patents related to HVAC energy-saving equipment filed by 11 companies were investigated. In order to analyze the large amount of data, this study developed a resource description framework (RDF) as an analysis tool. This tool was used with a natural language processing (NLP) program to compare the contents of academic papers and patents. With the automated analysis program, this study aimed to link academic research and corporate research and development, mainly the enterprise patent applications, to analyze the reasons why AI can effectively save energy. This represents a complete analysis of the current status of academic and industrial development. Six methods were identified to save energy effectively, including model-based predictive control (MPC), thermal comfort control, model-free predictive control, control optimization, multi-agent control (MAC), and knowledge-based system/rule set (KBS/RS)-based control. The energy savings of these methods were quantified to be 8.8–25.5%. These methods are widely covered by the examined corporate patent applications. After using NLP to retrieve patent keywords, the landscapes of enterprise patents were constructed and the future research directions were identified. It is concluded that 10 topics, including novel neural network designs, smartphone-assisted machine learning, and transfer learning, can be used to increase the energy-saving effects of AI and enable sustainable air-conditioning systems.

Transducer Cascades for Biological Literature-Based Discovery

G protein-coupled receptors (GPCRs) control the response of cells to many signals, and as such, are involved in most cellular processes. As membrane receptors, they are accessible at the surface of the cell. GPCRs are also the largest family of membrane receptors, with more than 800 representatives in mammal genomes. For this reason, they are ideal targets for drugs. Although about one third of approved drugs target GPCRs, only about 16% of GPCRs are targeted by drugs. One of the difficulties comes from the lack of knowledge on the intra-cellular events triggered by these molecules. In the last two decades, scientists have started mapping the signaling networks triggered by GPCRs. However, it soon appeared that the system is very complex, which led to the publication of more than 320,000 scientific papers. Clearly, a human cannot take into account such massive sources of information. These papers represent a mine of information about both ontological knowledge and experimental results related to GPCRs, which have to be exploited in order to build signaling networks. The ABLISS project aims at the automatic building of GPCRs networks using automated deductive reasoning, allowing to integrate all available data. Therefore, we processed the automatic extraction of network information from the literature using Natural Language Processing (NLP). We mainly focused on the experimental results about GPCRs reported in the scientific papers, as so far there is no source gathering all these experimental results. We designed a relational database in order to make them available to the scientific community later. After introducing the more general objectives of the ABLISS project, we describe the formalism in detail. We then explain the NLP program using the finite state methods (Unitex graph cascades) we implemented and discuss the extracted facts obtained. Finally, we present the design of the relational database that stores the facts extracted from the selected papers.

Sharing emotion while spectating video game play: Exploring Twitch users' emotional change after the outbreak of the COVID-19 pandemic

This paper examines how the COVID-19 pandemic associates with Twitch users' emotion, using natural language processing (NLP) as a method. Two comparable sets of text data were collected from Twitch internet relay chats (IRCs): one after the outbreak of the pandemic and another one before that. Positive emotion, negative emotion, and attitude to social interaction were tested by comparing the two text sets via a dictionary-based NLP program. Particularly regarding negative emotion, three negative emotions—anger, anxiety, and sadness—were measured given the nature of the pandemic. The results show that users' anger and anxiety significantly increased after the outbreak of the pandemic, while changes in sadness and positive emotion were not statistically significant. In terms of attitude to social interaction, users used significantly fewer “social” words after the outbreak of the pandemic than before. These findings were interpreted considering the nature of Twitch as a unique live mixed media platform, and how the COVID-19 pandemic is different from previous crisis events was discussed based on prior literature.

Automated Modeling of Clinical Narrative with High Definition Natural Language Processing Using Solor and Analysis Normal Form.

One important concept in informatics is data which meets the principles of Findability, Accessibility, Interoperability and Reusability (FAIR). Standards, such as terminologies (findability), assist with important tasks like interoperability, Natural Language Processing (NLP) (accessibility) and decision support (reusability). One terminology, Solor, integrates SNOMED CT, LOINC and RxNorm. We describe Solor, HL7 Analysis Normal Form (ANF), and their use with the high definition natural language processing (HD-NLP) program. We used HD-NLP to process 694 clinical narratives prior modeled by human experts into Solor and ANF. We compared HD-NLP output to the expert gold standard for 20% of the sample. Each clinical statement was judged "correct" if HD-NLP output matched ANF structure and Solor concepts, or "incorrect" if any ANF structure or Solor concepts were missing or incorrect. Judgements were summed to give totals for "correct" and "incorrect". 113 (80.7%) correct, 26 (18.6%) incorrect, and 1 error. Inter-rater reliability was 97.5% with Cohen's kappa of 0.948. The HD-NLP software provides useable complex standards-based representations for important clinical statements designed to drive CDS.

A Semiautomated Chart Review for Assessing the Development of Radiation Pneumonitis Using Natural Language Processing: Diagnostic Accuracy and Feasibility Study.

BackgroundHealth research frequently requires manual chart reviews to identify patients in a study-specific cohort and examine their clinical outcomes. Manual chart review is a labor-intensive process that requires significant time investment for clinical researchers.ObjectiveThis study aims to evaluate the feasibility and accuracy of an assisted chart review program, using an in-house rule-based text-extraction program written in Python, to identify patients who developed radiation pneumonitis (RP) after receiving curative radiotherapy.MethodsA retrospective manual chart review was completed for patients who received curative radiotherapy for stage 2-3 lung cancer from January 1, 2013 to December 31, 2015, at British Columbia Cancer, Kelowna Centre. In the manual chart review, RP diagnosis and grading were recorded using the Common Terminology Criteria for Adverse Events version 5.0. From the charts of 50 sample patients, a total of 1413 clinical documents were obtained for review from the electronic medical record system. The text-extraction program was built using the Natural Language Toolkit Python platform (and regular expressions, also known as RegEx). Python version 3.7.2 was used to run the text-extraction program. The output of the text-extraction program was a list of the full sentences containing the key terms, document IDs, and dates from which these sentences were extracted. The results from the manual review were used as the gold standard in this study, with which the results of the text-extraction program were compared.ResultsFifty percent (25/50) of the sample patients developed grade ≥1 RP; the natural language processing program was able to ascertain 92% (23/25) of these patients (sensitivity 0.92, 95% CI 0.74-0.99; specificity 0.36, 95% CI 0.18-0.57). Furthermore, the text-extraction program was able to correctly identify all 9 patients with grade ≥2 RP, which are patients with clinically significant symptoms (sensitivity 1.0, 95% CI 0.66-1.0; specificity 0.27, 95% CI 0.14-0.43). The program was useful for distinguishing patients with RP from those without RP. The text-extraction program in this study avoided unnecessary manual review of 22% (11/50) of the sample patients, as these patients were identified as grade 0 RP and would not require further manual review in subsequent studies.ConclusionsThis feasibility study showed that the text-extraction program was able to assist with the identification of patients who developed RP after curative radiotherapy. The program streamlines the manual chart review further by identifying the key sentences of interest. This work has the potential to improve future clinical research, as the text-extraction program shows promise in performing chart review in a more time-efficient manner, compared with the traditional labor-intensive manual chart review.

Derivation and Validation of Natural Language Processing Algorithms to Identify and Classify Venous Thrombotic Events from Lower Extremity Duplex Ultrasound Reports

Background: The precise anatomic location and extent of most venous thromboembolism (VTE) is not captured through diagnostic codes and is a major limitation to research. Correctly identifying the specific site of thrombosis, whether proximal or distal deep vein thrombosis (DVT) or superficial vein thrombosis (SVT) is critical.An accurate natural language processing (NLP) tool would make analysis of large datasets from electronic medical records possible and could be an significant improvement compared to analyses using the International Classification of Diseases codes. Using an open-source NLP tool, we evaluated existing algorithms and then created and validated new NLP algorithms to classify lower extremity thrombosis as not only a DVT or SVT, but also to identify more precise anatomic locations.Methods: A random sample of deidentified ultrasound reports were extracted from electronic medical records, manually reviewed, and classified as either positive or negative for a DVT or SVT (any chronicity). Reports with DVT were further classified into proximal (popliteal, femoral, deep femoral, common femoral, iliac veins, or vena cava) and distal (posterior tibial, anterior tibial, peroneal, soleal, or gastrocnemius veins) DVT. SVT was further classified into greater saphenous vein, small saphenous vein, or other site. Thrombosis near the junction of deep and superficial veins was only designated as involving specific vein segments if there was intraluminal thrombosis at that site. Initial sets of 100 ultrasound reports were used for derivation and reiterative testing of the algorithm, consisting of 50 that were positive for DVT/SVT and 50 that were negative for DVT/SVT. Text from radiology reports underwent cleaning to remove extraneous punctuation, text outside of the “findings” and “impression” section, and all text matches for “superficial femoral vein” were replaced with “femoral vein”. Using target phrases, the simple NLP tool either classified the reports as present or absent, signifying either positive or negative for thrombosis at the site of interest. After maximizing accuracy in the derivation cohorts, each NLP algorithm was tested in the complete, manually reviewed dataset. Sensitivity (Sn) and specificity (Sp) were calculated and confidence intervals were determined by the binomial exact method.Results: A total of 1206 ultrasounds were reviewed, among which there was 687 positives for DVT (503 had proximal DVT and 378 had distal DVT involvement). A total of 176 had SVT, 114 with involvement of the great saphenous vein (GSV) and 65 with involvement of the small saphenous vein (SSV). The previously published NLP (designed to identify DVT) had a poor Sn 45.0%, but reasonable Sp (91.0%) for DVT. Among the incorrect positive determinations (n=47), 33 were incorrectly positive due to presence of SVT (in the absence of DVT). Our newly developed NLP algorithm correctly identified DVT at any site (Sn: 96.2% 95% CI 94.5-97.5; 661/687 and Sp: 93.8% 95% CI 91.4-95.7; 487/519). Among ultrasounds positive for DVT, the NLP algorithm to determine proximal DVT site had a 96.0% Sn (95% CI 93.9-97.6; 483/503) and a 97.8% Sp (95% CI 94.5-99.4; 179/183). The algorithm for distal DVT had a 96.8% Sn (95% CI 94.5-98.4; 366/378) and a 96.1% Sp (95% CI 93.3-98.0; 296/308). The algorithm to identify SVT was also highly accurate (Sn 97.7%, 95% CI 94.3-99.4; 172/176 and Sp 95.5%, 95% CI 94.1-96.7; 984/1030). Among ultrasounds positive for SVT, the Sn to determine GSV site was 94.7% (95% CI 88.9-98.0; 108/114) and Sp was 95.1% (95% CI 86.3-99.0; 58/61). The algorithm to identify SSV site had a Sn of 98.5% (95% CI 91.7-99.9; 64/65) and a Sp of 100% (95% CI 96.7-100.0; 110/110).Conclusion: Using a previously published, open source, simple NLP program, we have improved the sensitivity and specificity for identification of DVT and created an algorithm to accurately identify SVT. Using a multifaceted analysis approach, we were able to accurately further subclassify the anatomic location of thrombosis on ultrasound reports. This tool and the developed algorithms will allow analysis of large data sets with minimal effort and great accuracy, capitalizing on the power of large electronic datasets to offer new insights on pathophysiology and clinical prognosis. [Display omitted] DisclosuresNo relevant conflicts of interest to declare.

Intentional Observational Clinical Research Design: Innovative Design for Complex Clinical Research Using Advanced Technology.

The growing use of robots in nursing and healthcare facilities has prompted increasing research on human–robot interactions. However, specific research designs that can guide researchers to conduct rigorous investigations on human–robot interactions are limited. This paper aims to discuss the development and application of a new research design—the Intentional Observational Clinical Research Design (IOCRD). Data sources to develop the IOCRD were derived from surveyed literature of the past decade, focusing on clinical nursing research and theories relating robotics to nursing and healthcare practice. The distinction between IOCRD and other research design is the simultaneous data generation collected using advanced technological devices, for example, the wireless Bonaly-light electrocardiogram (ECG) to track heart rate variability of research subjects, robot application programs on the iPad mini to control robot speech and gestures, and Natural Language Processing programs. Even though IOCRD was developed for human–robot research, there remain vast opportunities for its use in nursing practice and healthcare. With the unique feature of simultaneous data generation and analysis, an interdisciplinary collaborative research team is strongly suggested. The IOCRD is expected to contribute guidance for researchers in conducting clinical research related to robotics in nursing and healthcare.

Search-Based Algorithm With Scatter Search Strategy for Automated Test Case Generation of NLP Toolkit

Natural language processing (NLP), as a theory-motivated computational technique, has extensive applications. Automated test case generation based on path coverage, which is a popular structural testing activity, can automatically reveal logic defects that exist in NLP programs and can save testing consumption. NLP programs have many paths that can only be covered by specific input variables. This feature makes conventional search-based algorithm very difficult covering all possible paths in NLP programs. A strategy is required for improving the search ability of search-based algorithms. In this paper, we propose a scatter search strategy to automatically generate test cases for covering all possible paths of NLP programs. The scatter search strategy empowers search-based algorithms to explore all input variables and cover the paths that require specific input variables within a small amount of test cases. The experiment results show that the proposed scatter search strategy can quickly cover the paths, which requires specific input variables. Many test cases and running time consumptions will be saved when search-based algorithms combine with scatter search strategy.

Ontology-based metamorphic testing for chatbots

Modern-day demands for services often require an availability on a 24/7 basis as well as online accessibility around the globe. For this sake, personalized virtual assistants, called chatbots, are implemented. Such systems offer services, goods or information in natural language. These natural language processing (NLP) programs respond to the user in real time and offer an intuitive and simple interface to interact with. Advantages like these make them increasingly popular. Therefore, ensuring correct functionality of chatbots is of increasing importance. However, since different implementations and user behaviour result in unpredictable results, the chatbot’s input and output data are difficult to predict and classify as well. Under such circumstances, test cases can be inferred from the domain of possible inputs of a system under test (SUT). Ontologies are concepts used in AI to provide formal representations of knowledge for a specific domain. Such ontological models contain structured information that is used for test generation. On the other hand, testing of chatbots represents a challenge because of the absence of a test oracle. In this paper, both challenges are addressed by conceptualizing ontologies for input generation and output processing in form of a metamorphic testing approach. In this scenario, both concepts are applied for automated testing of chatbots. The approach is demonstrated on a real system from the tourism domain, thereby discussing the obtained results.

Testing the Use of Natural Language Processing Software and Content Analysis to Analyze Nursing Hand-off Text Data.

Natural language processing software programs are used primarily to mine both structured and unstructured data from the electronic health record and other healthcare databases. The mined data are used, for example, to identify vulnerable at-risk populations and predicting hospital associated infections and complications. Natural language processing programs are seldomly used in healthcare research to analyze the how providers are communicating essential patient information from one provider to another or how the language that is used impacts patient outcomes. In addition to analyzing how the message is being communicated, few studies have analyzed what is communicated during the exchange in terms of data, information, and knowledge. The analysis of the "how" and "what" of healthcare provider communication both written and verbal has the potential to decrease errors and improve patient outcomes. Here, we will discuss the feasibility of using an innovative within-methods triangulation data analysis to uncover the contextual and linguistic meaning of the nurse-to-nurse change-of-shift hand-off communication. The innovative within-methods triangulation data analysis uses a natural language processing software program and content analysis to analyze the nursing hand-off communication.

Automated analysis of lexical features in frontotemporal degeneration

We implemented an automated analysis of lexical aspects of semi-structured speech produced by healthy elderly controls (n = 37) and three patient groups with frontotemporal degeneration (FTD): behavioral variant FTD (n = 74), semantic variant primary progressive aphasia (svPPA, n = 42), and nonfluent/agrammatic PPA (naPPA, n = 22). Based on previous findings, we hypothesized that the three patient groups and controls would differ in the counts of part-of-speech (POS) categories and several lexical measures. With a natural language processing program, we automatically tagged POS categories of all words produced during a picture description task. We further counted the number of wh-words, and we rated nouns for abstractness, ambiguity, frequency, familiarity, and age of acquisition. We also computed the cross-entropy estimation, where low cross-entropy indicates high predictability, and lexical diversity for each description. We validated a subset of the POS data that were automatically tagged with the Google Universal POS scheme using gold-standard POS data tagged by a linguist, and we found that the POS categories from our automated methods were more than 90% accurate. For svPPA patients, we found fewer unique nouns than in naPPA and more pronouns and wh-words than in the other groups. We also found high abstractness, ambiguity, frequency, and familiarity for nouns and the lowest cross-entropy estimation among all groups. These measures were associated with cortical thinning in the left temporal lobe. In naPPA patients, we found increased speech errors and partial words compared to controls, and these impairments were associated with cortical thinning in the left middle frontal gyrus. bvFTD patients' adjective production was decreased compared to controls and was correlated with their apathy scores. Their adjective production was associated with cortical thinning in the dorsolateral frontal and orbitofrontal gyri. Our results demonstrate distinct language profiles in subgroups of FTD patients and validate our automated method of analyzing FTD patients' speech.

442 Frontal fibrosing alopecia: Utilization of the Lichen Planopilaris Activity Index to assess treatment outcomes

Frontal fibrosing alopecia (FFA) is a primary cicatricial alopecia which affects the frontotemporal scalp and face. The etiology of FFA is thought to be related to environmental exposures and a genetic susceptibility. FFA is considered to be a relatively new disease; there is a paucity of knowledge surrounding its pathogenesis, treatment and prognosis. Because FFA is histologically similar to lichen planopilaris (LPP), we chose to characterize long-term outcomes of our FFA patients using the LPP Activity Index (LPPAI), a validated scoring system introduced in 2010 which combines symptoms, signs and progression. We used this tool to assess longitudinally (0 and 12 months) the efficacy of treatments in our hair disease clinics. We completed a retrospective chart review of patients with a diagnosis of FFA seen by one investigator (MH) from 7/1/09-6/30/19. Clinical data and patient demographics were extracted from the electronic medical record Epic by the University of Minnesota Best Practices Integrated Informatics Core yielding 706 patients diagnosed with lichen planus, LPP and/or FFA. A further keyword search, with the terms frontal fibrosing alopecia or FFA, was performed using a natural language processing program, identifying 142 patients. This patient population was then limited to 24 patients due to availability of LPPAI data. A paired sample t-test was performed to evaluate differences between LPPAI scores at the baseline visit and the 12-month visit. Our analyses revealed that our patient population is predominantly female, white, and postmenopausal. Additionally, FFA patients achieved a statistically significant reduction in LPPAI score after 12 months of treatment; however, no patients were found to have a complete absence of symptoms or signs. In conclusion, the LPPAI is a valuable tool which can provide a standardized method to quantify clinical progression and response to therapy.