Bidirectional Encoder Representations From Transformers Research Articles

Integrating structured and unstructured data for predicting emergency severity: an association and predictive study using transformer-based natural language processing models

BackgroundEfficient triage in emergency departments (EDs) is critical for timely and appropriate care. Traditional triage systems primarily rely on structured data, but the increasing availability of unstructured data, such as clinical notes, presents an opportunity to enhance predictive models for assessing emergency severity and to explore associations between patient characteristics and severity outcomes. This study aimed to evaluate the effectiveness of combining structured and unstructured data to predict emergency severity more accurately.MethodsData from the 2021 National Hospital Ambulatory Medical Care Survey (NHAMCS) for adult ED patients were used. Emergency severity was categorized into urgent (scores 1–3) and non-urgent (scores 4–5) based on the Emergency Severity Index. Unstructured data, including chief complaints and reasons for visit, were processed using a Bidirectional Encoder Representations from Transformers (BERT) model. Structured data included patient demographics and clinical information. Four machine learning models—Logistic Regression, Random Forest, Gradient Boosting, and Extreme Gradient Boosting—were applied to three data configurations: structured data only, unstructured data only, and combined data. A mean probability model was also created by averaging the predicted probabilities from the structured and unstructured models.ResultsThe study included 8,716 adult patients, of whom 74.6% were classified as urgent. Association analysis revealed significant predictors of emergency severity, including older age (OR = 2.13 for patients 65 +), higher heart rate (OR = 1.56 for heart rates > 90 bpm), and specific chronic conditions such as chronic kidney disease (OR = 2.28) and coronary artery disease (OR = 2.55). Gradient Boosting with combined data demonstrated the highest performance, achieving an area under the curve (AUC) of 0.789, an accuracy of 0.726, and a precision of 0.892. The mean probability model also showed improvements over structured-only models.ConclusionsCombining structured and unstructured data improved the prediction of emergency severity in ED patients, highlighting the potential for enhanced triage systems. Integrating text data into predictive models can provide more accurate and nuanced severity assessments, improving resource allocation and patient outcomes. Further research should focus on real-time application and validation in diverse clinical settings.

Customer Churn Prediction Approach Based on LLM Embeddings and Logistic Regression

Nowadays, predicting customer churn is essential for the success of any company. Loyal customers generate continuous revenue streams, resulting in long-term success and growth. Moreover, companies are increasingly prioritizing the retention of existing customers due to the higher costs associated with attracting new ones. Consequently, there has been a growing demand for advanced methods aimed at enhancing customer loyalty and satisfaction, as well as predicting churners. In our work, we focused on building a robust churn prediction model for the telecommunications industry based on large embeddings from large language models and logistic regression to accurately identify churners. We conducted extensive experiments using a range of embedding techniques, including OpenAI Text-embedding, Google Gemini Text Embedding, bidirectional encoder representations from transformers (BERT), Sentence-Transformers, Sent2vec, and Doc2vec, to extract meaningful features. Additionally, we tested various classifiers, including logistic regression, support vector machine, random forest, K-nearest neighbors, multilayer perceptron, naive Bayes, decision tree, and zero-shot classification, to build a robust model capable of making accurate predictions. The best-performing model in our experiments is the logistic regression classifier, which we trained using the extracted feature from the OpenAI Text-embedding-ada-002 model, achieving an accuracy of 89%. The proposed model demonstrates a high discriminative ability between churning and loyal customers.

Designing a novel framework of email spam detection using an improved heuristic algorithm and dual-scale feature fusion-based adaptive convolution neural network

ABSTRACT Nowadays, e-mail is becoming more prevalent among individuals. In recent days, it has been declared to be the least expensive and quickest mode of communicating. In recent years, e-mail spam has become a big problem, so the number of e-mail spam has also increased, and they are used for unethical and illegal conduct, scams, and theft. As a result, the main objective of the study is to detect e-mail spam using an adaptive deep-learning model with feature fusion. Initially, the input image and text are collected from benchmark datasets. Further, the raw text undergoes the pre-processing stage, and it is then followed by extracting the features using Bidirectional Encoder Representations from Transformers (BERT). Similarly, the Vision Transformer (ViT) is employed for extracting the features from raw images. Finally, these features are given in Dual Scale Feature Fusion-based Adaptive Convolutional Neural Network (DFF-ACNNet), and it is then fused for performing the classification in Convolutional Neural Network (CNN). Here, the hyper-parameters are tuned by using Modified Squid Game Optimizer (MSGO). Finally, the evaluation is done with various metrics to estimate the performance of the model and the accuracy of the developed system is 98.46, which is higher than the other conventional approaches.

Needle in a haystack: Harnessing AI in drug patent searches and prediction.

The classification codes granted by patent offices are useful instruments for simplifying the bewildering variety of patents in existence. They are singularly unhelpful, however, in locating a specific subgroup of patents such as that of drug-related pharmaceutical patents for which no classification codes exist. Taking advantage of advances in artificial intelligence and in natural language processing in particular, we offer a new method of identifying chemical drug-related patents in this article. The aim is primarily that of demonstrating how the proverbial needle in a haystack was identified, namely through leveraging the superb pattern-recognition abilities of the BERT (Bidirectional Encoder Representations from Transformers) algorithm. We build three different databases to train our algorithm and fine-tune its abilities to identify the patent group in question by exposing it to additional texts containing structures that are much more likely to be present in them, until we obtain the highest possible F1-score, combined with an accuracy of 94.40%. We also demonstrate some possible uses of the algorithm. Its application to the US patent office database enables the identification of potential chemical drug patents up to ten years before drug approval, whereas its application to the German patent office reveals the regional nature of drug R&D and patenting strategies. The hope is that both the method proposed and its applications will be further refined and expanded forthwith.

A hybrid BERT-CPSO model for multi-class depression detection using pure hindi and hinglish multimodal data on social media

Due to the psychological strain that depression causes, there has been a noticeable increase in the number of persons compromising their lives in recent years. Social media platforms provide researchers with an entirely novel viewpoint on identifying individuals who are depressed. Previous research on automatic learning models for depression detection revealed low detection accuracy and an absence of optimizing techniques that could enhance detection accuracy. Furthermore, there is no such dataset, and very little study has been done on the multimodal pure Hindi and code-mixed Hinglish language domains. In light of this, we developed a Hindi dataset and suggested reliable methods for depression detection based on multimodal data, i.e., text and images, using the Hindi and Hinglish languages. This study aims to accomplish three things: first, it will evaluate text data using an effective Bidirectional Encoder Representations from Transformers (BERT) approach and compare it with other transfer learning variants; second, it will analyze image data by suggesting a Convolutional Neural Network (CNN) optimized with a nature-inspired algorithm, namely Particle Swarm Optimization (PSO), or CPSO; and third, it will classify the multimodal data into depressive and non-depressive posts by suggesting a hybrid of the best-performing models on text and images, namely BERT-CPSO (BTCPSO). The results produced with the BERT model showed the best accuracy of 95% for text data, in contrast to RoBERTa, DistilBERT, and XLNet. Further, CPSO outperforms other Machine Learning (ML) and Deep Learning (DL) algorithms for image data with an accuracy of 95%. Additionally, comparing the proposed CPSO with a basic CNN revealed that integrating the PSO technique with CNN increased the model's accuracy in detecting depressed posts by 5%. In conclusion, hybrid BERT-CPSO outperforms other BERT combinations with ML and DL algorithms for multimodal data, achieving 97%, 95%, 98%, and 96%, respectively, in accuracy, recall, precision, and F1-scores. As a result, the findings of comparing the suggested technique with the earlier models show the effectiveness of the approach that has been provided and can help medical professionals diagnose depression with precision.

Machine learning algorithms for breast cancer analysis: performance and accuracy comparison

Breast cancer, a leading cause of cancer mortality among women, necessitates early detection to improve survival rates. Traditional diagnostics face accuracy and speed limitations, prompting this study to explore machine learning for enhanced diagnostics. We applied bidirectional encoder representations from transformers (BERT), long short-term memory (LSTM), naive Bayes, support vector machine (SVM), and random forest to the Breast Cancer Wisconsin dataset, implementing a thorough methodology involving data preprocessing, feature extraction, and model validation. BERT led in accuracy at 92.5%, showcasing advanced algorithms' potential in medical diagnostics, with random forest 90.6%, SVM 89.3%, LSTM 88.7%, and naive Bayes 85.2%; also showing promising results. The study underscores the importance of incorporating machine learning, especially BERT, into clinical decision-making, potentially revolutionizing breast cancer diagnostics by improving accuracy and efficiency. We recommend healthcare practitioners integrate these algorithms into their diagnostic processes. Future research should reeefine these algorithms and extend their application to enhance patient care further.

Sentiment-electroencephalogram fusion for efficient product review prediction using correlation-based deep learning neural network

<span lang="EN-US">Various techniques have been proposed and implemented in previous work for sentiment analysis prediction. However, achieving satisfactory quality of description and fault prediction remains a challenging task. To overcome these limitations, this study proposes an efficient prediction technique that utilizes sentiment analysis of product reviews and electroencephalogram (EEG) signals using correlation-based deep learning neural network (CDNN). The study employs two types of datasets: EEG signals and Amazon product reviews. During the pre-processing phase, EEG signals undergo normalization, while Amazon product reviews undergo tokenization, stop word removal, and weighting factor application to convert unstructured data into a structured format. Subsequently, the pre-processed EEG signals and reviews are analyzed to extract features like emotion, demographic information, personality traits, and sentiment. These features are then employed in sentiment analysis via an entropy-based deep-learning neural network. The proposed CDNN utilizes the grasshopper optimization algorithm (EGOA) to optimize hyperparameters for each layer. Comparative performance assessment against established methods like convolutional neural network (CNN), <a name="_Hlk167111148"></a>long short-term memory (LSTM), multiclass <a name="_Hlk175651957"></a>support vector machine (M-SVM), and bidirectional encoder representations from transformers (BERT) is conducted, and the results are evaluated. Experimental result reveal that the proposed system outperforms traditional approaches.</span>

Customer Sentiment Analysis for E-Commerce: A Hybrid approach using CNNs & BERT

As the online shopping industry continues to grow at a staggering rate, e-commerce channels have also begun the use of automated systems for sentiment analysis so as to be able to assess comprehension of user reviews. Over the past few years, machine learning (ML) has come of age with the introduction of deep learning models such as Convolutional Neural Networks (CNNs) and Bidirectional Encoder Representations from Transformers (BERT) which have greatly improved the levels of accuracy achieved in sentiment analysis. This paper aims to address this problem by proposing a hybrid model, based on networks of convolutional neural networks (CNNs) and BERT, for the purposes of e-commerce sentiment classification. Posting more plausible situations, academic findings, and perspectives for future research, we focus more on practical issues. When we compared traditional approaches with our hybrid, it became clear that the model performs better in many dimensions: accuracy, precision, and progressively recall

Lightweight advanced deep-learning models for stress detection on social media

Nowadays, stress reveals itself as a ubiquitous presence, manifesting in novel forms in our modern daily life. Indeed, digital platforms and social media collect various impressions, reactions, and feelings that could provide valuable real-time sentiment data. Nevertheless, understanding stress and mental states among people is difficult because it relies on self-reporting and detecting related expressions, statements, and articulations. In this paper, we consider extracting nuanced insights and stress expressions from Reddit and Twitter posts using lightweight advanced deep-learning methods and Bidirectional Encoder Representations from Transformers (BERT) embeddings. Our findings highlight the potency of transformer BERT models, whether utilized as embedding feature extractors or as text sentiment classifiers. Moreover, the proposed lightweight deep architectural models promoted the field of stress detection in social media, achieving high classification performance. Practically, the BERT Electra model reached 85.67% accuracy on the small Reddit dataset, while our Convolutional Neural Network (CNN) model obtained 97.62% on the large Twitter dataset. Our contributions are not only restricted to the scientific understanding of stress but also extend to the well-being of individuals and global mental health.

Scalable Transformer Accelerator with Variable Systolic Array for Multiple Models in Voice Assistant Applications

Transformer model is a type of deep learning model that has quickly become fundamental in natural language processing (NLP) and other machine learning tasks. Transformer hardware accelerators are usually designed for specific models, such as Bidirectional Encoder Representations from Transformers (BERT), and vision Transformer models, like the ViT. In this study, we propose a Scalable Transformer Accelerator Unit (STAU) for multiple models, enabling efficient handling of various Transformer models used in voice assistant applications. Variable Systolic Array (VSA) centralized design, along with control and data preprocessing in embedded processors, enables matrix operations of varying sizes. In addition, we propose an efficient variable structure and a row-wise data input method for natural language processing where the word count changes. The proposed scalable Transformer accelerator accelerates text summarization, audio processing, image search, and generative AI used in voice assistance.

Privacy-Enhanced Sentiment Analysis in Mental Health: Federated Learning with Data Obfuscation and Bidirectional Encoder Representations from Transformers

This research aims to find an optimal balance between privacy and performance in forecasting mental health sentiment. This paper investigates federated learning (FL) augmented with a novel data obfuscation (DO) technique, where synthetic data is used to "mask" real data points. Bidirectional Encoder Representations from Transformer (BERT) is used for sentiment analysis, forming a new framework, FL-BERT+DO, that addresses the privacy-performance trade-off. With FL, data remains decentralized, ensuring that user-sensitive information is retained on local devices rather than being shared with the FL server. The integration of BERT gives our system an enhanced feature of context sense-making from text conduct, and our model is extremely proficient in emotion categorization tasks. The experiments were performed on combined (real and replica synthetic) datasets containing emotions and showed significant enhancements compared to baseline methods. The proposed FL-BERT+DO framework shows the following metrics: prediction accuracy, 82.74%; precision, 83.30%; recall, 82.74%; F1-score, 82.80%. Further, we assessed its performance in the adversarial setup using membership inference and linkage attacks to ensure the privacy-preserved performance did not suffer deeply. It demonstrates that, even for large datasets, providing privacy-preserving prediction is possible and can significantly improve existing methods of addressing personal issues, like mental health support. Based on the results of our work, we can propose the development of secure decentralized learning systems that are capable of providing high accuracy of sentiment analysis and meeting strict privacy constraints.

Layer-Wise Learning Rate Optimization for Task-Dependent Fine-Tuning of Pre-Trained Models: An Evolutionary Approach

The superior performance of large-scale pre-trained models, such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT), has received increasing attention in both academic and industrial research and has become one of the current research hotspots. A pre-trained model refers to a model trained on large-scale unlabeled data, whose purpose is to learn general language representation or features for fine-tuning or transfer learning in subsequent tasks. After pre-training is complete, a small amount of labeled data can be used to fine-tune the model for a specific task or domain. This two-stage method of “pre-training+fine-tuning” has achieved advanced results in natural language processing (NLP) tasks. Despite widespread adoption, existing fixed fine-tuning schemes that adapt well to one NLP task may perform inconsistently on other NLP tasks given that different tasks have different latent semantic structures. In this article, we explore the effectiveness of automatic fine-tuning pattern search for layer-wise learning rates from an evolutionary optimization perspective. Our goal is to use evolutionary algorithms to search for better task-dependent fine-tuning patterns for specific NLP tasks than typical fixed fine-tuning patterns. Experimental results on two real-world language benchmarks and three advanced pre-training language models show the effectiveness and generality of the proposed framework.

Contextual Emotional Transformer-Based Model for Comment Analysis in Mental Health Case Prediction

Mental health (MH) assessment and prediction have become critical areas of focus in healthcare, leveraging developments in natural language processing (NLP). Recent advancements in machine learning have facilitated the exploration of predictive models for MH based on user-generated comments that overlooked the integration of emotional attention mechanisms. The methods often struggle with contextual nuances and emotional subtleties, leading to suboptimal predictions. The prevailing challenge lies in accurately understanding the emotional context embedded within textual comments, which is crucial for effective prediction and intervention. In this research, we introduce a novel approach employing contextual emotional transformer-based models (CETM) for comment analysis in MH case prediction. CETM leverages state-of-the-art transformer architectures enhanced with contextual embedding layers and emotional attention mechanisms for MH case prediction. By incorporating contextual information and emotional cues, CETM captures the underlying emotional states and MH indicators expressed in user comments. Through extensive experimentation and evaluation, both Roberta and bidirectional encoder representations from transformers (BERT) models exhibited enhanced accuracy, precision, recall, and F1 scores compared to their counterparts lacking emotional attention. Notably, the Roberta model attained a greater accuracy of 94.5% when matched to BERT’s 87.6% when emotional attention was employed. Hence, by incorporating emotional context into the predictive model, we achieved significant improvements, which offers promising avenues for more precise and personalized MH interventions.

Rumor detection using dual embeddings and text-based graph convolutional network

AbstractSocial media platforms like Twitter and Facebook have gradually become vital for communication and information exchange. However, this often leads to the spread of unreliable or false information, such as harmful rumors. Currently, graph convolutional networks (GCNs), particularly TextGCN, have shown promise in text classification tasks, including rumor detection. Their success is due to their ability to identify structural patterns in rumors and effectively use neighborhood information. We present a novel rumor detection model using TextGCN, which utilizes a word-document graph to represent rumor texts. This model uses dual embedding from two pre-trained transformer models: generative pre-trained transformers (GPT) and bidirectional encoder representations from transformers (BERT). These embeddings serve as node representations within the graph, enhancing rumor detection. Combining these deep neural networks effectively extracts significant contextual features from rumors. This graph undergoes convolution, and through graph-based learning, the model detects a rumor. We evaluated our model using publicly available rumor datasets, such as PHEME, Twitter15, and Twitter16. It achieved 88.64% accuracy on the PHEME dataset, surpassing similar models, and performed well on Twitter15 and Twitter16 with accuracies of 81.98% and 83.41%, respectively.

Differential Impacts of Environmental, Social, and Governance News Sentiment on Corporate Financial Performance in the Global Market: An Analysis of Dynamic Industries Using Advanced Natural Language Processing Models

This study examines how sentiment analysis of environmental, social, and governance (ESG) news affects the financial performance of companies in innovative sectors such as mobility, technology, and renewable energy. Using approximately 9828 general ESG articles from Google News and approximately 140,000 company-specific ESG articles, we performed term frequency-inverse document frequency (TF-IDF) analysis to identify key ESG-related terms and visualize their materiality across industries. We then applied models such as bidirectional encoder representations from transformers (BERT), the robustly optimized BERT pretraining approach (RoBERTa), and big bidirectional encoder representations from transformers (BigBird) for multiclass sentiment analysis, and distilled BERT (DistilBERT), a lite BERT (ALBERT), tiny BERT (TinyBERT), and efficiently learning an encoder that classifies token replacements accurately (ELECTRA) for positive and negative sentiment identification. Sentiment analysis results were correlated with profitability, cash flow, and stability indicators over a three-year period (2019–2021). ESG ratings from Morgan Stanley Capital International (MSCI), a prominent provider that evaluates companies’ sustainability practices, further enriched our analysis. The results suggest that sentiment impacts financial performance differently across industries; for example, positive sentiment correlates with financial success in mobility and renewable energy, while consumer goods often show positive sentiment even with low environmental ESG scores. The study highlights the need for industry-specific ESG strategies, especially in dynamic sectors, and suggests future research directions to improve the accuracy of ESG sentiment analysis.

Maritime piracy and armed robbery analysis in the Straits of Malacca and Singapore through the utilization of natural language processing

ABSTRACT Piracy and armed robbery have become serious problems in maritime economic activities, particularly in the Straits of Malacca and Singapore. These crimes occur due to various influential factors, necessitating analysis through incident reports. This study aims to identify the factors influencing piracy and armed robbery in these areas using natural language processing (NLP) technologies. Data were obtained from reports compiled by relevant organizations such as the Global Integrated Shipping Information System (GISIS) from the International Maritime Organization (IMO) and Annual Reports on Piracy and Armed Robbery from the Regional Cooperation Agreement on Combating Piracy and Armed Robbery against Ships in Asia (ReCAAP). These data were reconstructed through NLP and further analyzed using the Bidirectional Encoder Representations from Transformer (BERT) model derivative, BERTopic. Several topics were obtained from the analysis, suggesting key factors influencing the outcome of piracy and armed robbery incidents. These topics will serve as the basis for suggestions and solution provisions that can be effectively applied and utilized to further decrease the frequency and success rate of piracy and armed robbery in the future. The findings provide crucial insights for developing strategies to mitigate the impact of these criminal activities on maritime economic activities.

A spatio-temporal evolution analysis framework based on sentiment recognition for temple murals

Murals are important resources for carrying cultural heritage, historical evidence and artistic memory. The sentiment of a mural is the transmission of its inner thoughts, closely related to the region and dynasty to which the mural belongs. To explore the sentiment evolution patterns of temple murals, we construct a spatio-temporal evolution analysis framework based on sentiment recognition. This framework mainly includes feature extraction, sentiment recognition and sentiment evolution analysis. First, we extract the colour features, local features, global semantic features, patch features and structure relations to represent the visual features of temple murals. Second, the semantics of spatio-temporal attributes and titles of murals are extracted through the fine-tuned BERT (Bidirectional Encoder Representations from Transformers) to enhance the feature discrimination for sentiment recognition. Third, we introduce the SMOTE (Synthetic Minority Oversampling Technique) to reduce the influence of imbalanced data and select RF (random forest) as the optimal classifier. The F1 score of the fine-grained sentiment recognition model is up to 81.37%. Finally, we collect the temple murals and reveal the characteristics and patterns of sentiment evolution from the spatial, temporal and spatio-temporal perspectives.

Deep Learning-Based Identification of Immoral Posts on Social Media Using Fine-tuned Bert Model

The propagation of immoral content on social media poses substantial worries to online societal well-being and communication standards. While beneficial, traditional machine learning (ML) methods fall short of capturing the difficulty of textual and sequential data. This work reports this gap by suggesting a deep learning-based technique for detecting immoral posts on social media. The proposed model presents a fine-tuned Bidirectional Encoder representation from Transformers (BERT) with word embedding methods. Word2Vec and Global Vectors for Word Representation (GloVe) are employed to improve the identification of immoral posts on social media platforms to advance detection accuracy and strength. The incentive behind this study stems from the increasing demand for more sophisticated methods to struggle with damaging content. The proposed model is considered to capture the complicated patterns and semantic nuances in immoral posts by decreasing the dependence on manual feature engineering. The model is trained and assessed using benchmark datasets containing SARC and HatEval, which deliver a detailed set of labelled user-generated posts. The proposed model shows the best performance compared to traditional ML approaches. The fine-tuned Bert-based Word2Vec embeddings achieved a precision of 95.68%, recall of 96.85 %, and F1 scores of 96.26% on the SARC dataset. Fine-tuned Bert-based GloVe on the HatEval dataset achieved superior precision of 96.65, recall of 97.75, and F1-score of 97.20. The proposed results highlight the potential of the deep learning (DL) approach and fine-tuned BERT models, considerably refining the detection of unethical content on social networks.

Mitigating Bias in Large Language Models: A Multi-Task Training Approach Using BERT

Abstract. Large language models (LLMs), such as ChatGPT, have become essential tools due to their advanced natural language processing capabilities. However, these models, trained on extensive internet text, can inadvertently learn and propagate unwanted biases, impacting their outputs. This study addresses this issue by analyzing and mitigating such biases through a multi-task and multi-stage training approach. Utilizing the Winograd Bias (Winobias) dataset, the research fine-tunes the Bidirectional Encoder Representations from Transformers (BERT) model to reduce biased outputs. The approach includes an initial mask task to establish a general understanding and a subsequent cloze task to specifically target and mitigate biases. Results demonstrate a significant reduction in bias, with the original model showing approximately 90% certainty in biased outputs, whereas the de-biased model reduced this certainty to 55%. This study effectively showcases a method for bias reduction by modifying only a few parameters, emphasizing a practical approach to enhancing fairness and balance in LLMs used across various applications.

Exploring the Impact of Word2Vec Embeddings Across Neural Network Architectures for Sentiment Analysis

Abstract. Sentiment analysis is crucial for understanding public opinion, gauging customer satisfaction, and making informed business decisions based on the emotional tone of textual data. This study investigates the performance of different Word2Vec-based embedding strategies static, non-static, and multichannel for sentiment analysis across various neural network architectures, including Convolution Neural Networks (CNNs), Long Short-Term Memory (LSTM), and Gated Recurrent Units (GRUs). Despite the rise of advanced contextual embedding methods such as Bidirectional Encoder Representations from Transformers (BERT), Word to Vector (Word2Vec) retains its importance due to its simplicity and lower computational demands, making it ideal for use in settings with limited resources. The goal is to evaluate the impact of fine-tuning Word2Vec embeddings on the accuracy of sentiment classification. Using the Internet Movie Database (IMDb), this work finds that multichannel embeddings, which combine static and non-static representations, provide the best performance across most architectures, while static embeddings continue to deliver strong results in specific sequential models. These findings highlight the balance between efficiency and accuracy in traditional word embeddings, particularly when advanced models are not feasible.