Natural Language Processing Systems Research Articles

Unveiling Text Representation with 'Bag of Words'

Techniques for natural language processing (NLP) have grown to be essential tools for deciphering and drawing insightful conclusions from massive volumes of text data. A thorough review of numerous natural language processing (NLP) techniques, including as tokenization, stemming, lemmatization, named entity recognition, sentiment analysis, and topic modelling, is provided in this abstract. These methods are essential for applications like sentiment analysis, machine translation, text assistant categorization, and information retrieval. Furthermore, the capabilities of NLP systems have been greatly improved by recent developments in deep learning, especially with models like BERT and GPT. This has allowed them to reach state-of-the-art performance in a variety of language understanding tasks. The difficulties and potential paths for future study in NLP, including managing ambiguity, comprehending context, and enhancing multilingual assistance, are also highlighted in this abstract. Using NLP tools to their full potential, researchers.

Artificial intelligibility: the role of gender in assigning humanness to natural language processing systems

ABSTRACT In place of ‘Artificial Intelligence’, this article proposes artificial intelligibility as the more accurate term for describing the mistaken assignment of humanness to non-living objects. Artificial Intelligibility is manifested when a user assumes an object has capacity to understand them simply because it is understandable to them. Relatively simplistic Natural Language Processing systems may perform genres of humanness in conversational interactions to the degree that they are imagined to be sentient, as ‘Artificial Intelligence’. However, decolonial scholars have observed that humanness developed as a mutable, sociogenic construct. Decolonial gender theorists have further derived the role played by heteronormativity and repronormativity in upholding this power. Equating the intelligible performance of a gendered genre of humanness with intelligence risks obfuscating this interplay while reinforcing eugenic stratifications of life. This article reframes behavioural measures of ‘intelligence’ to gendered ‘intelligibility’ to explore the role of gender in enabling entry into the symbolic order of humanness. It presents three key findings from my doctoral thesis to question how and why some gendered, NLP-incorporating devices can be imagined to have lives that matter within the same economy of value that renders some humans and animals killable.

Exploring noise-induced techniques to strengthen deep learning model resilience

AbstractIn artificial intelligence, combating overfitting and enhancing model generalization is crucial. This research explores innovative noise-induced regularization techniques, focusing on natural language processing tasks. Inspired by gradient noise and Dropout, this study investigates the interplay between controlled noise, model complexity, and overfitting prevention. Utilizing long short-term memory and bidirectional long short term memory architectures, this study examines the impact of noise-induced regularization on robustness to noisy input data. Through extensive experimentation, this study shows that introducing controlled noise improves model generalization, especially in language understanding. This contributes to the theoretical understanding of noise-induced regularization, advancing reliable and adaptable artificial intelligence systems for natural language processing.

Improving Robustness of Intent Detection Under Adversarial Attacks: A Geometric Constraint Perspective.

Deep neural networks (DNNs)-based natural language processing (NLP) systems are vulnerable to being fooled by adversarial examples presented in recent studies. Intent detection tasks in dialog systems are no exception, however, relatively few works have been attempted on the defense side. The combination of linear classifier and softmax is widely used in most defense methods for other NLP tasks. Unfortunately, it does not encourage the model to learn well-separated feature representations. Thus, it is easy to induce adversarial examples. In this article, we propose a simple, yet efficient defense method from the geometric constraint perspective. Specifically, we first propose an M-similarity metric to shrink variances of intraclass features. Intuitively, better geometric conditions of feature space can bring lower misclassification probability (MP). Therefore, we derive the optimal geometric constraints of anchors within each category from the overall MP (OMP) with theoretical guarantees. Due to the nonconvex characteristic of the optimal geometric condition, it is hard to satisfy the traditional optimization process. To this end, we regard such geometric constraints as manifold optimization processes in the Stiefel manifold, thus naturally avoiding the above challenges. Experimental results demonstrate that our method can significantly improve robustness compared with baselines, while retaining the excellent performance on normal examples.

Integrating predictive coding and a user-centric interface for enhanced auditing and quality in cancer registry data

Data curation for a hospital-based cancer registry heavily relies on the labor-intensive manual abstraction process by cancer registrars to identify cancer-related information from free-text electronic health records. To streamline this process, a natural language processing system incorporating a hybrid of deep learning-based and rule-based approaches for identifying lung cancer registry-related concepts, along with a symbolic expert system that generates registry coding based on weighted rules, was developed. The system is integrated with the hospital information system at a medical center to provide cancer registrars with a patient journey visualization platform. The embedded system offers a comprehensive view of patient reports annotated with significant registry concepts to facilitate the manual coding process and elevate overall quality. Extensive evaluations, including comparisons with state-of-the-art methods, were conducted using a lung cancer dataset comprising 1428 patients from the medical center. The experimental results illustrate the effectiveness of the developed system, consistently achieving F1-scores of 0.85 and 1.00 across 30 coding items. Registrar feedback highlights the system’s reliability as a tool for assisting and auditing the abstraction. By presenting key registry items along the timeline of a patient’s reports with accurate code predictions, the system improves the quality of registrar outcomes and reduces the labor resources and time required for data abstraction. Our study highlights advancements in cancer registry coding practices, demonstrating that the proposed hybrid weighted neural-symbolic cancer registry system is reliable and efficient for assisting cancer registrars in the coding workflow and contributing to clinical outcomes.

Token-modification adversarial attacks for natural language processing: A survey

Many adversarial attacks target natural language processing systems, most of which succeed through modifying the individual tokens of a document. Despite the apparent uniqueness of each of these attacks, fundamentally they are simply a distinct configuration of four components: a goal function, allowable transformations, a search method, and constraints. In this survey, we systematically present the different components used throughout the literature, using an attack-independent framework which allows for easy comparison and categorisation of components. Our work aims to serve as a comprehensive guide for newcomers to the field and to spark targeted research into refining the individual attack components.

Using Natural Language Processing to Identify Home Health Care Patients at Risk for Diagnosis of Alzheimer's Disease and Related Dementias.

This study aimed to: (1) validate a natural language processing (NLP) system developed for the home health care setting to identify signs and symptoms of Alzheimer's disease and related dementias (ADRD) documented in clinicians' free-text notes; (2) determine whether signs and symptoms detected via NLP help to identify patients at risk of a new ADRD diagnosis within four years after admission. This study applied NLP to a longitudinal dataset including medical record and Medicare claims data for 56,652 home health care patients and Cox proportional hazard models to the subset of 24,874 patients admitted without an ADRD diagnosis. Selected ADRD signs and symptoms were associated with increased risk of a new ADRD diagnosis during follow-up, including: motor issues; hoarding/cluttering; uncooperative behavior; delusions or hallucinations; mention of ADRD disease names; and caregiver stress. NLP can help to identify patients in need of ADRD-related evaluation and support services.

Exploration, detection, and mitigation: Unveiling gender bias in NLP

Natural Language Processing (NLP) systems have a mundane impact, yet they harbour either obvious or potential gender bias. The automation of decision-making in NLP models even exacerbates unfair treatment. In recent years, researchers have started to notice this issue and have made some approaches to detect and mitigate these biases, yet no consensus on the approaches exists. This paper discusses the interdisciplinary field of linguistics and computer sciences by presenting the most common gender bias categories and breaking them down with ethical and artificial intelligence approaches. Specific methods for detecting and minimizing bias are shown around biases present in raw data, annotator, model, and the linguistic gender system. In this paper, an overview of the hotspots and future perspectives of this research topic is presented. Limitations of some detection methods are pinpointed, providing novel insights into future research.

Ensemble pretrained language models to extract biomedical knowledge from literature.

The rapid expansion of biomedical literature necessitates automated techniques to discern relationships between biomedical concepts from extensive free text. Such techniques facilitate the development of detailed knowledge bases and highlight research deficiencies. The LitCoin Natural Language Processing (NLP) challenge, organized by the National Center for Advancing Translational Science, aims to evaluate such potential and provides a manually annotated corpus for methodology development and benchmarking. For the named entity recognition (NER) task, we utilized ensemble learning to merge predictions from three domain-specific models, namely BioBERT, PubMedBERT, and BioM-ELECTRA, devised a rule-driven detection method for cell line and taxonomy names and annotated 70 more abstracts as additional corpus. We further finetuned the T0pp model, with 11 billion parameters, to boost the performance on relation extraction and leveraged entites' location information (eg, title, background) to enhance novelty prediction performance in relation extraction (RE). Our pioneering NLP system designed for this challenge secured first place in Phase I-NER and second place in Phase II-relation extraction and novelty prediction, outpacing over 200 teams. We tested OpenAI ChatGPT 3.5 and ChatGPT 4 in a Zero-Shot setting using the same test set, revealing that our finetuned model considerably surpasses these broad-spectrum large language models. Our outcomes depict a robust NLP system excelling in NER and RE across various biomedical entities, emphasizing that task-specific models remain superior to generic large ones. Such insights are valuable for endeavors like knowledge graph development and hypothesis formulation in biomedical research.

Artificial Intelligence Methods in Natural Language Processing: A Comprehensive Review

The rapid evolution of Artificial Intelligence (AI) since its inception in the mid-20th century has significantly influenced the field of Natural Language Processing (NLP), transforming it from a rule-based system to a dynamic and adaptive model capable of understanding the complexities of human language. This paper aims to offer a comprehensive review of the various applications and methodologies of AI in NLP, serving as a detailed guide for future research and practical applications. In the early sections, the paper elucidates the indispensable role of AI in NLP, highlighting its transition from symbolic reasoning to a focus on machine learning and deep learning, and its extensive applications in sectors such as healthcare, transportation, and finance. It emphasizes the symbiotic relationship between AI and NLP, facilitated by platforms like AllenNLP, which aid in the development of advanced language understanding models. Further, the paper explores specific AI techniques employed in NLP, including machine learning, Naive Bayes, and Support Vector Machines, and identifies pressing challenges and avenues for future research. It delves into the applications of AI in NLP, showcasing its transformative potential in tasks such as machine translation, facilitated by deep learning methods, and the development of chatbots and virtual assistants that have revolutionized human-technology interaction. The paper also highlights other fields impacted by AI techniques, including text summarization, sentiment analysis, and named entity recognition, emphasizing the efficiency and accuracy brought about by the integration of AI in these areas. In conclusion, the paper summarizes the remarkable advancements and persistent challenges in NLP, such as language ambiguity and contextual understanding, and underscores the need for diverse and representative labeled data for training. Looking forward, it identifies promising research avenues including Explainable AI, Few-shot and Zero-shot Learning, and the integration of NLP with other data modalities, aiming for a holistic understanding of multimodal data. The paper calls for enhanced robustness and security in NLP systems, especially in sensitive applications like content moderation and fake news detection, to foster trust and reliability in AI technologies. It advocates for continual learning in NLP models to adapt over time without losing previously acquired knowledge, paving the way for a future where AI and NLP work synergistically to understand and generate human language more effectively and efficiently.

A Scientific Investigation of Reproduced Knowledge Based Approach in Worldwide Possibilities

This paper examines parts of man-made thinking, show, ramifications of modernized thinking, history, applications, headway and accomplishments. The field of reproduced insight (ML) is sufficiently vivacious that it is as of now loosening up at a speeding up pace, lying at the convergence of programming and encounters, and at the point of convergence of modernized thinking (man-made awareness) and information science. Late movement in ML has been driven both by the improvement of new learning calculations hypothesis, and by the diligent shoot in the accessibility of colossal extent of information (routinely suggested as "tremendous information") and irrelevant expense assessment. It is the science and arranging of making fast machines, particularly adroit PC programs. It is related with the overall undertaking of utilizing PCs to make heads or tails of human information, yet man-made thinking doesn't need to restrict itself to strategies that are regularly distinguishable. Subsequently, in this paper, we plan to surmise the scholarly improvement of simulated intelligence and ML in finance research, taking on a checking survey joined with an implanted audit to seek after and examine the administrations of these ideas. While no consensual meaning of Motorized thinking (mirrored information) exists, reproduced knowledge is generally portrayed as the assessment of calculations that think about figuring out, reason and development. Keywords: Artificial intelligence, Machine learning, Neural Networks, Natural Language Processing and Knowledge Base System , Virtual Redirection.

Evaluations of Large Language Models a Bibliometric analysis

The development of artificial intelligence (AI) and the increased curiosity about how large language models (LLMs) may maximize an organization's opportunities and the ethical implications of LLMs, such as the ability to generate human-like text, give rise to concerns regarding disinformation and fake news. As a result, it is crucial to develop evaluation benchmarks that take into account the social and ethical implications involved. The great challenges of LLMs lack awareness of their own limitations, yet they persist in producing responses to the best of their capabilities. This often results in seemingly plausible but ultimately incorrect answers, posing challenges to the implementation of reliable generative AI in industry. This paper aims to delve into the evaluation metrics of machine-learning models' performance, specifically focusing on LLM. Therefore, bibliometric analysis utilized to explore and analyze various techniques and methods used in evaluating large language models. Additionally, it sheds light on the specific areas of focus when evaluating these models. The results show that natural language processing systems, classification of information, and computational linguistics are some of the techniques used to evaluate large language models. This work paves the way for future investigations employing extensive language models.

Sarcasm detection methods based on machine learning and deep learning

Sarcasm is an expression of emotion, which often expresses criticism and disgust in a positive tone, and we see it everywhere on social networking platforms. Because of this nature, sarcasm can be challenging to process in Natural Language Processing (NLP) Systems. Failing to detect sarcasm correctly can negatively affect the results of natural language processing tasks such as text correct and sentiment analysis. In recent years, there are more and more studies on sarcasm detection, and the research methods are diverse. According to the survey, the methods based on machine learning (ML) and deep learning (DL) account for the majority. Therefore, this paper mainly summarizes the literature on sarcasm detection based on ML and DL methods in recent years. This paper first introduces the application of sarcasm, then expounds the general architecture of sarcasm detection and makes a distinction between machine learning and deep learning. Then the author investigates the relevant literature and compares the results. Finally, the challenges and research directions in this field are summarized. The purpose of this paper is to facilitate the follow-up research of sarcasm detection.

Using a Natural Language Processing Approach to Support Rapid Knowledge Acquisition.

Implementing artificial intelligence to extract insights from large, real-world clinical data sets can supplement and enhance knowledge management efforts for health sciences research and clinical care. At Vanderbilt University Medical Center (VUMC), the in-house developed Word Cloud natural language processing system extracts coded concepts from patient records in VUMC's electronic health record repository using the Unified Medical Language System terminology. Through this process, the Word Cloud extracts the most prominent concepts found in the clinical documentation of a specific patient or population. The Word Cloud provides added value for clinical care decision-making and research. This viewpoint paper describes a use case for how the VUMC Center for Knowledge Management leverages the condition-disease associations represented by the Word Cloud to aid in the knowledge generation needed to inform the interpretation of phenome-wide association studies.

Development of a Natural Language Processing System to Identify Clinical Documentation of Electronic Cigarette Use.

Electronic Nicotine Delivery Systems (ENDS) use has increased substantially in the United States since 2010. To date, there is limited evidence regarding the nature and extent of ENDS documentation in the clinical note. In this work we investigate the effectiveness of different approaches to identify a patient's documented ENDS use. We report on the development and validation of a natural language processing system to identify patients with explicit documentation of ENDS using a large national cohort of patients at the United States Department of Veterans Affairs.

Automatic Extraction of Skin and Soft Tissue Infection Status from Clinical Notes.

The reliable identification of skin and soft tissue infections (SSTIs) from electronic health records is important for a number of applications, including quality improvement, clinical guideline construction, and epidemiological analysis. However, in the United States, types of SSTIs (e.g. is the infection purulent or non-purulent?) are not captured reliably in structured clinical data. With this work, we trained and evaluated a rule-based clinical natural language processing system using 6,576 manually annotated clinical notes derived from the United States Veterans Health Administration (VA) with the goal of automatically extracting and classifying SSTI subtypes from clinical notes. The trained system achieved mention- and document-level performance metrics of the range 0.39 to 0.80 for mention level classification and 0.49 to 0.98 for document level classification.

Quranic Studies Evolution: A Bibliometric Analysis from 1880 to 2023

This analysis focuses on the study of the Quran, from 1880 to 2023 using data obtained from Scopus. Made accessible through a BibTeX file on GitHub. The examination covers aspects of research trends including the number of citations per year the influence of sources notable authors and their affiliations the countries of corresponding authors the production trends in different countries over time and a network illustrating how key concepts are interconnected. By conducting an analysis several significant findings were discovered. The data demonstrated an increase in citations per year indicating a growing importance and scholarly interest in Quranic Studies. Noteworthy authors such as Rippin A and Reynolds GS as prominent affiliations like the University of Malaya and International Islamic University Malaysia made substantial contributions to this field. The distribution of corresponding authors countries highlighted Malaysias, Indonesias and Irans roles. Furthermore when examining production trends over time for countries it became evident that contributions from the United Kingdom, USA,Malaysia,Iran and Indonesia had seen growth. Additionally the interconnectedness between concepts such as natural language processing systems, computational linguistics,semantics and information retrieval were emphasized in the cooccurrence network, within Quranic Studies. This, in depth analysis provides insights into the development significant contributors and intersections of themes in the study of the Quran. It establishes a foundation for research endeavors, in this area.

ChatGPT May Offer an Adequate Substitute for Informed Consent to Patients Prior to Total Knee Arthroplasty-Yet Caution Is Needed.

Prior to undergoing total knee arthroplasty (TKA), surgeons are often confronted with patients with numerous questions regarding the procedure and the recovery process. Due to limited staff resources and mounting individual workload, increased efficiency, e.g., using artificial intelligence (AI), is of increasing interest. We comprehensively evaluated ChatGPT's orthopedic responses using the DISCERN instrument. Three independent orthopedic surgeons rated the responses across various criteria. We found consistently high scores, predominantly exceeding a score of three out of five in almost all categories, indicative of the quality and accuracy of the information provided. Notably, the AI demonstrated proficiency in conveying precise and reliable information on orthopedic topics. However, a notable observation pertains to the generation of non-existing references for certain claims. This study underscores the significance of critically evaluating references provided by ChatGPT and emphasizes the necessity of cross-referencing information from established sources. Overall, the findings contribute valuable insights into the performance of ChatGPT in delivering accurate orthopedic information for patients in clinical use while shedding light on areas warranting further refinement. Future iterations of natural language processing systems may be able to replace, in part or in entirety, the preoperative interactions, thereby optimizing the efficiency, accessibility, and standardization of patient communication.

Bridging information gaps in menopause status classification through natural language processing.

To use natural language processing (NLP) of clinical notes to augment existing structured electronic health record (EHR) data for classification of a patient's menopausal status. A rule-based NLP system was designed to capture evidence of a patient's menopause status including dates of a patient's last menstrual period, reproductive surgeries, and postmenopause diagnosis as well as their use of birth control and menstrual interruptions. NLP-derived output was used in combination with structured EHR data to classify a patient's menopausal status. NLP processing and patient classification were performed on a cohort of 307512 female Veterans receiving healthcare at the US Department of Veterans Affairs (VA). NLP was validated at 99.6% precision. Including the NLP-derived data into a menopause phenotype increased the number of patients with data relevant to their menopausal status by 118%. Using structured codes alone, 81173 (27.0%) are able to be classified as postmenopausal or premenopausal. However, with the inclusion of NLP, this number increased 167804 (54.6%) patients. The premenopausal category grew by 532.7% with the inclusion of NLP data. By employing NLP, it became possible to identify documented data elements that predate VA care, originate outside VA networks, or have no corresponding structured field in the VA EHR that would be otherwise inaccessible for further analysis. NLP can be used to identify concepts relevant to a patient's menopausal status in clinical notes. Adding NLP-derived data to an algorithm classifying a patient's menopausal status significantly increases the number of patients classified using EHR data, ultimately enabling more detailed assessments of the impact of menopause on health outcomes.

Hospitalization, case fatality, comorbidities, and isolated pathogens of adult inpatients with pneumonia from 2013 to 2022: a real-world study in Guangzhou, China

BackgroundIn the context of increasing population aging, ongoing drug-resistant pathogens and the COVID-19 epidemic, the changes in the epidemiological and clinical characteristics of patients with pneumonia remain unclear. This study aimed to assess the trends in hospitalization, case fatality, comorbidities, and isolated pathogens of pneumonia-related adult inpatients in Guangzhou during the last decade.MethodsWe retrospectively enrolled hospitalized adults who had doctor-diagnosed pneumonia in the First Affiliated Hospital of Guangzhou Medical University from January 1, 2013 to December 31, 2022. A natural language processing system was applied to automatically extract the clinical data from electronic health records. We evaluated the proportion of pneumonia-related hospitalizations in total hospitalizations, pneumonia-related in-hospital case fatality, comorbidities, and species of isolated pathogens during the last decade. Binary logistic regression analysis was used to assess predictors for patients with prolonged length of stay (LOS).ResultsA total of 38,870 cases were finally included in this study, with 70% males, median age of 64 (53, 73) years and median LOS of 7.9 (5.1, 12.8) days. Although the number of pneumonia-related hospitalizations showed an upward trend, the proportion of pneumonia-related hospitalizations decreased from 199.6 per 1000 inpatients in 2013 to 123.4 per 1000 in 2021, and the case fatality decreased from 50.2 per 1000 in 2013 to 23.9 per 1000 in 2022 (all P < 0.05). The most common comorbidities were chronic obstructive pulmonary disease, lung malignancy, cardiovascular diseases and diabetes. The most common pathogens were Pseudomonas aeruginosa, Candida albicans, Acinetobacter baumannii, Stenotrophomonas maltophilia, Klebsiella pneumoniae, and Staphylococcus aureus. Glucocorticoid use during hospitalization (Odd Ratio [OR] = 1.86, 95% Confidence Interval (CI): 1.14–3.06), immunosuppressant use during hospitalization (OR = 1.99, 1.14–3.46), ICU admission (OR = 16.23, 95%CI: 11.25–23.83), receiving mechanical ventilation (OR = 3.58, 95%CI: 2.60–4.97), presence of other underlying diseases (OR = 1.54, 95%CI: 1.15–2.06), and elevated procalcitonin (OR = 1.61, 95%CI: 1.19–2.19) were identified as independent predictors for prolonged LOS.ConclusionThe proportion of pneumonia-related hospitalizations and the in-hospital case fatality showed downward trends during the last decade. Pneumonia inpatients were often complicated by chronic underlying diseases and isolated with gram-negative bacteria. ICU admission was a significant predictor for prolonged LOS in pneumonia inpatients.