Models Of Language Processing Research Articles

GPT-4 shows potential for identifying social anxiety from clinical interview data

While the potential of Artificial Intelligence (AI)—particularly Natural Language Processing (NLP) models—for detecting symptoms of depression from text has been vastly researched, only a few studies examine such potential for the detection of social anxiety symptoms. We investigated the ability of the large language model (LLM) GPT-4 to correctly infer social anxiety symptom strength from transcripts obtained from semi-structured interviews. N = 51 adult participants were recruited from a convenience sample of the German population. Participants filled in a self-report questionnaire on social anxiety symptoms (SPIN) prior to being interviewed on a secure online teleconference platform. Transcripts from these interviews were then evaluated by GPT-4. GPT-4 predictions were highly correlated (r = 0.79) with scores obtained on the social anxiety self-report measure. Following the cut-off conventions for this population, an F1 accuracy score of 0.84 could be obtained. Future research should examine whether these findings hold true in larger and more diverse datasets.

ODSBAHIA-PTBR: A Natural Language Processing Model to Support Sustainable Development Goals

Objective: The present study aims to propose an approach for the objective classification of texts in Portuguese in relation to the Sustainable Development Goals (SDGs) of Brazil's 2030 Agenda. Theoretical Framework: The study uses natural language processing (NLP) techniques with deep learning, using pre-trained models such as BERTimbau Base, DeBERTinha and Albertina. In addition, it considers the existing gaps in the literature regarding the classification of texts in Portuguese related to the 17 UN SDGs and also including three new SDGs proposed in the document Guide Agenda 2030: Integrating SDGs, Education and Society, prepared in 2020 in partnership between UnB and UNESP, SDGs 18 (Ethnic-Racial Equality), 19 (Art, Culture and Communication) and 20 (Rights of Indigenous Peoples and Traditional Communities). Method: La investigación es exploratoria, descriptiva y aplicada, con enfoque cuantitativo y procedimientos experimentales. Los modelos previamente entrenados se ajustaron al conjunto de datos de etiquetas múltiples creado específicamente para la tarea. La Base BERTimbau presentó el mejor rendimiento y se utilizó como base para la creación del modelo ODSBahia-PTBR, evaluado con métricas como precisión (82%), recuerdo (72%) y F1-Score (77%). Results and Discussion: El ODSBahia-PTBR logró una precisión del 95% al traducir y clasificar el conjunto de datos OSDG. Los resultados ponen de manifiesto la efectividad del modelo en la identificación y categorización de textos alineados con los ODS, siendo especialmente relevante para el seguimiento de las interseccionalidades entre los ODS propuestos. Research Implications: The SDGbahia-PTBR model has practical implications by offering an innovative tool for different stakeholders to monitor and analyze initiatives aligned with the SDGs, contributing to the evaluation and promotion of the 2030 Agenda. Originality/Value: This research is a pioneer in including SDGs 18, 19 and 20 in Portuguese-language text classifiers, offering an unprecedented and applicable approach to sustainable monitoring in Brazil and other Portuguese-speaking countries.

Evolving Strategies in Machine Learning: A Systematic Review of Concept Drift Detection

In this comprehensive literature review, we rigorously adhere to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines for our process and reporting. This review employs an innovative method integrating the advanced natural language processing model T5 (Text-to-Text Transfer Transformer) to enhance the accuracy and efficiency of screening and data extraction processes. We assess strategies for handling the concept drift in machine learning using high-impact publications from notable databases that were made accessible via the IEEE and Science Direct APIs. The chronological analysis covering the past two decades provides a historical perspective on methodological advancements, recognizing their strengths and weaknesses through citation metrics and rankings. This review aims to trace the growth and evolution of concept drift mitigation strategies and to provide a valuable resource that guides future research and deepens our understanding of this rapidly changing field. Key findings highlight the effectiveness of diverse methodologies such as drift detection methods, window-based methods, unsupervised statistical methods, and neural network techniques. However, challenges remain, particularly with imbalanced data, computational efficiency, and the application of concept drift detection to non-tabular data like images. This review aims to trace the growth and evolution of concept drift mitigation strategies and provide a valuable resource that guides future research and deepens our understanding of this rapidly changing field.

Decoding business analytics: discovering the hidden core through a novel taxonomy

PurposeThis study develops a comprehensive taxonomy of the business analytics (BA) discipline, uncovering its intellectual core and revealing its evolution over the past 12 years.Design/methodology/approachUsing stakeholder-driven identity formation theory, this study explores how organizational identity emerges through stakeholder negotiations. It investigates how top scholarly journals shape the BA discipline’s image and influence perceptions. High-quality articles from top journals listed by the Australian Business Deans Council are analyzed using latent Dirichlet allocation (LDA), a natural language processing and topic modeling method.FindingsThe study outlines key research areas identified as analytics methods, marketing, finance, operations and decision support analytics, along with 12 subareas. An analysis of the top 100 topics reveals prevalent research themes, showcasing the breadth of BA. A 12-year time-series review shows initial growth followed by maturation across most areas, except for decision support analytics, which maintained steady growth. These findings provide empirical evidence of BA’s development as a distinct discipline, highlighting its interdisciplinary nature and evolving research focus.Originality/valueThis study presents the first comprehensive, data-driven taxonomy of BA research, distilling the intellectual core into five key areas and 12 subareas, while identifying 100 supporting themes. It extends the stakeholders’ approach to identity development theory in the context of BA, providing empirical support for discussions on the field’s identity and diversity. The findings offer valuable insights for scholars, industries, managers and professionals, guiding curriculum development, research directions and practical applications of BA.

Integrating structured and unstructured data for predicting emergency severity: an association and predictive study using transformer-based natural language processing models

BackgroundEfficient triage in emergency departments (EDs) is critical for timely and appropriate care. Traditional triage systems primarily rely on structured data, but the increasing availability of unstructured data, such as clinical notes, presents an opportunity to enhance predictive models for assessing emergency severity and to explore associations between patient characteristics and severity outcomes. This study aimed to evaluate the effectiveness of combining structured and unstructured data to predict emergency severity more accurately.MethodsData from the 2021 National Hospital Ambulatory Medical Care Survey (NHAMCS) for adult ED patients were used. Emergency severity was categorized into urgent (scores 1–3) and non-urgent (scores 4–5) based on the Emergency Severity Index. Unstructured data, including chief complaints and reasons for visit, were processed using a Bidirectional Encoder Representations from Transformers (BERT) model. Structured data included patient demographics and clinical information. Four machine learning models—Logistic Regression, Random Forest, Gradient Boosting, and Extreme Gradient Boosting—were applied to three data configurations: structured data only, unstructured data only, and combined data. A mean probability model was also created by averaging the predicted probabilities from the structured and unstructured models.ResultsThe study included 8,716 adult patients, of whom 74.6% were classified as urgent. Association analysis revealed significant predictors of emergency severity, including older age (OR = 2.13 for patients 65 +), higher heart rate (OR = 1.56 for heart rates > 90 bpm), and specific chronic conditions such as chronic kidney disease (OR = 2.28) and coronary artery disease (OR = 2.55). Gradient Boosting with combined data demonstrated the highest performance, achieving an area under the curve (AUC) of 0.789, an accuracy of 0.726, and a precision of 0.892. The mean probability model also showed improvements over structured-only models.ConclusionsCombining structured and unstructured data improved the prediction of emergency severity in ED patients, highlighting the potential for enhanced triage systems. Integrating text data into predictive models can provide more accurate and nuanced severity assessments, improving resource allocation and patient outcomes. Further research should focus on real-time application and validation in diverse clinical settings.

Contextualized race and ethnicity annotations for clinical text from MIMIC-III

Observational health research often relies on accurate and complete race and ethnicity (RE) patient information, such as characterizing cohorts, assessing quality/performance metrics of hospitals and health systems, and identifying health disparities. While the electronic health record contains structured data such as accessible patient-level RE data, it is often missing, inaccurate, or lacking granular details. Natural language processing models can be trained to identify RE in clinical text which can supplement missing RE data in clinical data repositories. Here we describe the Contextualized Race and Ethnicity Annotations for Clinical Text (C-REACT) Dataset, which comprises 12,000 patients and 17,281 sentences from their clinical notes in the MIMIC-III dataset. Using these sentences, two sets of reference standard annotations for RE data are made available with annotation guidelines. The first set of annotations comprise highly granular information related to RE, such as preferred language and country of origin, while the second set contains RE labels annotated by physicians. This dataset can support health systems’ ability to use RE data to serve health equity goals.

Analyzing the Accessibility of Chatbots and Generative AI

This paper analyzes the availability of chatbots and generative artificial intelligence and highlights the important importance of inclusivity in technology design. Accessibility remains paramount in the digital environment, especially for people with disabilities. Legal frameworks such as the Americans with Disabilities Act, along with moral obligations, emphasize the need to have AI systems available. The current land of availability reveals significant gaps, including bias in verbal language processing models and a lack of guidance for diverse consumer needs. However, challenging situations remain, ranging from moral dilemmas around privacy and economic constraints to major deployments. Future developments such as multimodal artificial intelligence and emotive chatbots provide promising avenues for improving accessibility. Ultimately, setting and complying with accessibility requirements requires a collaborative effort between policymakers, developers, and users to ensure that AI generation is useful for all. This document advocates continuous improvement and rigorous monitoring of accessibility practices to promote inclusive virtual environments. Keywords Accessibility, Chatbots, Generative AI, Inclusive Design, Natural Language Processing.

A Multi-Architecture Approach for Offensive Language Identification Combining Classical Natural Language Processing and BERT-Variant Models

Offensive content is a complex and multifaceted form of harmful material that targets individuals or groups. In recent years, offensive language (OL) has become increasingly harmful, as it incites violence and intolerance. The automatic identification of OL on social networks is essential to curtail the spread of harmful content. We address this problem by developing an architecture to effectively respond to and mitigate the impact of offensive content on society. In this paper, we use the Davidson dataset containing 24,783 samples of tweets and proposed three different architectures for detecting OL on social media platforms. Our proposed approach involves concatenation of features (TF-IDF, Word2Vec, sentiments, and FKRA/FRE) and a baseline machine learning model for the classification. We explore the effectiveness of different dimensions of GloVe embeddings in conjunction with deep learning models for classifying OL. We also propose an architecture that utilizes advanced transformer models such as BERT, ALBERT, and ELECTRA for pre-processing and encoding, with 1D CNN and neural network layers serving as the classification components. We achieve the highest precision, recall, and F1 score, i.e., 0.89, 0.90, and 0.90, respectively, for both the “bert encased preprocess/1 + small bert/L4H512A8/1 + neural network layers” model and the “bert encased preprocess/1 + electra small/2 + cnn” architecture.

Harnessing AI to Illuminate the Soul of the Organization

Artificial intelligence and advanced analytics are emerging as promising new tools for gaining meaningful insights into organizational culture. Traditional qualitative methods for assessing culture are time-intensive and limited in scope, while AI offers data-driven, comprehensive approaches. This article reviews recent developments in using AI at both the macro and micro levels to enhance understanding of cultural dynamics. At a macro level, AI is powering new tools for conducting large-scale cultural assessments across multiple dimensions. More granularly, natural language processing, network analysis, and predictive modeling are illuminating sentiments, interactions, and influence within internal communications data. Examples from pioneering organizations demonstrate impactful applications of cultural insights derived from AI in areas like mergers and acquisitions, recruiting and onboarding, diversity initiatives, and efforts to nurture vibrant cultures. While the field is still evolving, innovative practitioners are beginning to harness AI’s potential to illuminate the complexity of organizational culture in compelling new ways.

Enhancing Campus Communication: Advanced ML Techniques for College Chatbots

This research explores the development and deployment of a college chatbot using advanced machine learning techniques to enhance communication and support within the campus community. The chatbot is designed to efficiently handle a variety of student inquiries by leveraging natural language processing and deep learning models. Through a detailed methodology, evaluation of metrics, and real-world testing, this study demonstrates how the chatbot enhances student engagement and administrative efficiency. Challenges encountered during development included managing ambiguous queries and ensuring data accuracy and relevance. Future work aims to expand the chatbot's capabilities with personalized interactions, voice-based queries, and multilingual support to better serve a diverse student body and further improve functionality. This research paper provides an analysis of designing, applying, and utilizing a college chatbot’s powered by advanced ML techniques. The functions demonstrate its prospective to rationalize campus communication, offering a scalable, efficient, and user-friendly solution to meet the changing needs of modern educational institutions.

How Does Social Support Detected Automatically in Discussion Forums Relate to Online Learning Burnout? The Moderating Role of Students’ Self-Regulated Learning

Engaging students in online discussion forums with social support holds significant potential for preventing and alleviating student burnout. However, the mechanisms by which different types of social support influence learning burnout remain poorly understood. Additionally, existing methods for detecting social support detection are limited in both practical application and theoretical advancement. This study addresses these gaps by developing a robust text classification model for social support and examining its effects on online learning burnout among learners with varying levels of self-regulated learning. We first developed a robust natural language processing model based on bidirectional encoder representations from transformers - bidirectional long short-term memory (BERT-Bi-LSTM) framework, trained on 11226 manually labeled posts from various course forums. This model was then applied to classify forum posts from an educational technology course over one semester. Multiple regression analysis revealed that informational support was negatively associated with two dimensions of learning burnout: emotional exhaustion and improper behavior, and emotional support was negatively correlated with emotional exhaustion and a low sense of achievement. Moreover, a moderating effect analysis indicated that self-regulated learning moderated the negative associations between informational support and improper behavior, as well as between emotional support and emotional exhaustion, with stronger effects observed among learners with lower self-regulated learning. These findings contribute to advancing automated content analysis of social support and provide actionable insights for mitigating student burnout through targeted social support.

An Automated Daily News Reports Generating Application Involving Keyword-Based News Scraping, Summarization, and Sentimental Analysis Leveraging NLP Models

In the era of information overload, obtaining relevant, concise, and sentiment-analyzed news content is essential for effective decision-making. This paper introduces an automated daily news reporting system that streamlines the process of collecting, summarizing, and analyzing the sentiment of news articles fetched based on the user's keyword. By leveraging stateof-the-art Natural Language Processing (NLP) models like GPT-3.5 for two-level summarization and BERTweet for sentiment analysis, the system provides users with concise, sentiment-labeled reports, enhancing their understanding of news trends and emotional tone. The architecture integrates data scraping, text extraction, and sentiment classification within a cloud-based Python microservice, supported by Flask. The system incorporates user localization options, allowing users to customize news retrieval by region and time preferences. Finalized reports are formatted into HTML and delivered directly to authenticated user emails through Gmail API, ensuring seamless and secure distribution. This research underscores the significance of automated news summarization and sentiment analysis in modern information retrieval. It provides a scalable and personalized solution that enables real-time synthesis of large volumes of news content, making it accessible and relevant for diverse audiences worldwide.

TimeGPT in load forecasting: A large time series model perspective

Machine learning models have made significant progress in load forecasting, but their forecast accuracy is limited in cases where historical load data is scarce. Inspired by the outstanding performance of large language models (LLMs) in computer vision and natural language processing, this paper aims to discuss the potential of large time series models in load forecasting with scarce historical data. Specifically, the large time series model is constructed as a time series generative pre-trained transformer (TimeGPT), which is trained on massive and diverse time series datasets consisting of 100 billion data points (e.g., finance, transportation, banking, web traffic, weather, energy, healthcare, etc.). Then, the scarce historical load data is used to fine-tune the TimeGPT, which helps it to adapt to the data distribution and characteristics associated with load forecasting. Simulation results show that TimeGPT outperforms the popular benchmarks for load forecasting on several real datasets with scarce training samples, particularly for short look-ahead times. However, it cannot be guaranteed that TimeGPT is always superior to benchmarks for load forecasting with scarce data, since the performance of TimeGPT may be affected by the distribution differences between the load data and the training data. In practical applications, operators can divide the historical data into a training set and a validation set, and then use the validation set loss to decide whether TimeGPT is the best choice for a specific dataset.

Advancements and Applications of Large Language Models in Natural Language Processing: A Comprehensive Review

Abstract. Large language models (LLMs) have revolutionized the field of natural language processing (NLP), demonstrating remarkable capabilities in understanding, generating, and manipulating human language. This comprehensive review explores the development, applications, optimizations, and challenges of LLMs. This paper begin by tracing the evolution of these models and their foundational architectures, such as the Transformer, GPT, and BERT. We then delve into the applications of LLMs in natural language understanding tasks, including sentiment analysis, named entity recognition, question answering, and text summarization, highlighting real-world use cases. Next, we examine the role of LLMs in natural language generation, covering areas such as content creation, language translation, personalized recommendations, and automated responses. We further discuss LLM applications in other NLP tasks like text style transfer, text correction, and language model pre-training. Subsequently, we explore techniques for optimizing and improving LLMs, including model compression, explainability, robustness, and security. Finally, we address the challenges posed by the significant computational requirements, sample inefficiency, and ethical considerations surrounding LLMs. We conclude by discussing potential future research directions, such as efficient architectures, few-shot learning, bias mitigation, and privacy-preserving techniques, which will shape the ongoing development and responsible deployment of LLMs in NLP.

Practical evasion attack against neural network-based macro-malware detection method

In recent years, various methods have been proposed to detect unknown malware using machine learning models. These models extract features from malware and classify them as benign or malicious. However, there have been reports of evasion attacks against machine learning-based malware detectors. Previous research has focused on these evasion attacks, particularly against models that detect Visual Basics for Applications (VBA) malware using natural language processing models such as Bag of Words (BoW) and Latent Semantic Indexing (LSI). However, these models rely on word frequency as a feature, overlooking the context, and their evaluation involved an equal number of benign and malicious samples, leaving their effectiveness in real-world scenarios unverified. To address these limitations, our study introduces a tokenizer that preserves word order during token conversion. We evaluated its accuracy using an imbalanced dataset and employed models such as recurrent neural networks (RNNs) and long short-term memory (LSTM) networks to understand the context of words. Its detection rate for malicious malware exceeds 0.8, indicating sufficient performance. Moreover, by incorporating words found only in benign samples as arguments in non-functional operations, we managed to reduce the evasion attack detection rate to as low as 0.89. Ultimately, we confirmed that the detection rate of malicious samples remained consistent in real-world conditions, demonstrating the effectiveness of our approach.

Latent mechanisms of language disorganization relate to specific dimensions of psychopathology

Comprehensible communication is critical for social functioning and well-being. In psychopathology, incoherent discourse is assumed to reflect disorganized thinking, which is classically linked to psychotic disorders. However, people do not express everything that comes to mind, rendering inferences from discourse to the underlying structure of thought challenging. Indeed, a range of psychopathologies are linked to self-reported disorganized thinking in the absence of language output incoherence. Here we combine natural language processing and computational modeling of free association to detail the relationship between disorganized thinking and language (in)coherence in a large sample of participants varying across different dimensions of psychopathology. Our approach allowed us to differentiate between disorganized thinking, disinhibited thought expression and deliberate creativity. We find evidence for both under-regulated and over-regulated disorganized thinking, which relate to two specific dimensions of psychopathology: self-reported eccentricity and suspiciousness. Broadly, these results underscore the theoretical progress afforded by analyzing latent dimensions underlying behavior and psychopathology.

Integrating machine learning and sentiment analysis in movie recommendation systems

The fast growth of the film business, along with an ever-increasing number of movie options, has highlighted the need for better recommendation algorithms. This study investigates the application of sentiment analysis in a movie recommendation system with the goal of improving the user experience. The importance of this sector stems from its ability to bridge the gap between user interests and the vast number of cinematic products, addressing individual emotional states and preferences. Researchers choose to generate movie recommendations based on the sentiments conveyed by viewers’ reviews of the movies. Sentiment-based movie recommendation system research employs techniques such as natural language processing and hybrid models with the goal of increasing user satisfaction. To this purpose, the integration of advanced machine learning algorithms such as cosine similarity, support vector machine, and Naive Bayes improves recommendation systems with sentiment analysis. Cosine similarity improves movie recommendations by recognizing minor user preferences, while support vector machines and Naive Bayes enhance sentiment analysis by offering a nuanced interpretation of textual attitudes. Through trials, the proposed system employs two public datasets for sentiment analysis, namely the TMDB5k dataset and the Reviews dataset, and makes predictions (positive, negative, or neutral) based on the content of the review through conducting sentiment analysis on the text using the Viscous Accretion Disk Evolution Resource (VADER) approach. The findings, based on users’ feedback, are more accurate and informative regarding movie quality, where SVM accuracy is 99.28% and Naïve Bayes accuracy is 96.60% when used with VADER sentiment analysis.

Beyond Content: A Trauma-Informed Framework for Academic Writing Evaluation

The paper proposes a novel framework for assessing student academic writing integrating an understanding of trauma into the evaluation process. This framework emphasises the importance of recognising linguistic markers in student writing that may indicate underlying psychological distress, such as post-traumatic stress disorder (PTSD). Traditional higher education academic assessments often overlook these markers, focusing on content quality and adherence to formal writing standards. This oversight could lead to missed opportunities for early intervention, particularly in educational settings where students do not openly disclose their mental health challenges. Building on trauma-informed pedagogy, this study explores how Natural Language Processing (NLP) models can be leveraged to identify linguistic markers associated with poor mental health, such as PTSD, anxiety, and depression, within students' academic writing. By analysing writing patterns like non-linear narrative structures, obsessive thoughts, and disjointed syntax, we argue that NLP can offer an essential tool for early detection of trauma-related challenges. Such markers are often overlooked in traditional grading systems, which prioritize form and rhetoric. A case study using student writing samples demonstrates how changes in rhetorical fluency and writing quality can correlate with a documented decline in mental health. The results of NLP analysis reveal a progressive decline in coherence, lexical diversity, and thematic focus, which align with known linguistic markers of trauma. These findings underscore the potential of NLP to serve as an early-warning system, alerting educators to the need for intervention and support. Despite the promise of these methods, current NLP models face limitations in linguistic diversity, reproducibility, and population bias. Therefore, we advocate for the development of more inclusive models built on ethical frameworks that consider the socio-rhetorical contexts of student writing. Additionally, large and secure datasets are required to ensure representativeness, with attention to student privacy concerns. Ultimately, this paper calls for higher education institutions to adopt trauma-sensitive evaluation frameworks that integrate academic and emotional well-being, ensuring more equitable and compassionate assessments.

FANE

In today's society, the continuous exchange of vast amounts of information, often irrelevant or misleading, highlights the need for greater awareness to distinguish between accurate and false information. Recognizing the reliability of information is critical to limiting the spread of fake news, a pervasive problem affecting various sectors, influencing public opinion, and shaping decisions in health care, politics, culture, and history. This paper proposes a methodology to assess the veracity of information, leveraging natural language processing (NLP) and probabilistic models to extract relevant features and predict the reliability of content. The features analyzed include semantic, syntactic, and social dimensions. The proposed methodology was tested using datasets that include social media news and comments captured during the lockdown due to COVID-19, providing relevant context for the analysis. Experimental validation of these different datasets yields promising results, demonstrating the effectiveness of the proposed approach.

Automated Multi-Dialect Speech Recognition using Stacked Attention-based Deep Learning with Natural Language Processing Model

Automated Multi-Dialect Speech Recognition using Stacked Attention-based Deep Learning with Natural Language Processing Model