Performance Of Text Classification Research Articles

MuTCELM: An optimal multi-TextCNN-based ensemble learning for text classification

Feature extraction plays a critical role in text classification, as it converts textual data into numerical representations suitable for machine learning models. A key challenge lies in effectively capturing both semantic and contextual information from text at various levels of granularity while avoiding overfitting. Prior methods have often demonstrated suboptimal performance, largely due to the limitations of the feature extraction techniques employed. To address these challenges, this study introduces Multi-TextCNN, an advanced feature extractor designed to capture essential textual information across multiple levels of granularity. Multi-TextCNN is integrated into a proposed classification model named MuTCELM, which aims to enhance text classification performance. The proposed MuTCELM leverages five distinct sub-classifiers, each designed to capture different linguistic features from the text data. These sub-classifiers are integrated into an ensemble framework, enhancing the overall model performance by combining their complementary strengths. Empirical results indicate that MuTCELM achieves average improvements across all datasets in accuracy, precision, recall, and F1-macro scores by 0.2584, 0.2546, 0.2668, and 0.2612, respectively, demonstrating significant performance gains over baseline models. These findings underscore the effectiveness of Multi-TextCNN in improving model performance relative to other ensemble methods. Further analysis reveals that the non-overlapping confidence intervals between MuTCELM and baseline models indicate statistically significant differences, suggesting that the observed performance improvements of MuTCELM are not attributable to random chance but are indeed statistically meaningful. This evidence indicates the robustness and superiority of MuTCELM across various languages and text classification tasks.

A Review of Prompt-Free Few-Shot Text Classification Methods

Text-based comments play a crucial role in providing feedback for various industries. However, effectively filtering and categorizing this feedback based on custom context-specific criteria requires sophisticated language modeling techniques. While traditional approaches have shown effectiveness, they often require a substantial amount of data to compensate for their modeling deficiencies. In this work, we focus on highlighting the performance and limitations of prompt-free few-shot text classification using open-source pre-trained sentence transformers. On the one hand, our research includes a comprehensive study across different benchmark datasets, encompassing 9 dimensions such as sentiment analysis, topic modeling, grammatical acceptance, and emotion classification. Also, we worked at making different experiences to test Prompt-Free Few-Shot Text Classification. On the other hand, we underline prompt-free few-shot classification limitations when the targeted criteria are complex. As an alternative approach, prompting an instruction-fine-tuned language model has demonstrated favorable outcomes, as proven by our application in the specific use case of “Identifying and extracting resolution results and actions from explanatory notes”, achieving an accuracy rate of 80%.

Rethinking of BERT sentence embedding for text classification

Text classification is a fundamental task in NLP that is used in several real-life tasks and applications. Large pre-trained language models such as BERT achieve state-of-the-art performance in several NLP tasks including text classification tasks. Although BERT boosts text classification performance, the common way of using it for classification lacks many aspects of its advantages. This work rethinks the way of using BERT final layer and hidden layers embeddings by proposing different aggregation architectures for text classification tasks such as sentiment analysis and sarcasm detection. This research also proposes different approaches for using BERT as a feature extractor without fine-tuning whose performance surpasses its fine-tuning counterpart. It also proposes promising multi-task learning aggregation architectures to improve the performance of the related classification problems. The experiments of the different architectures show that freezing BERT can outperform fine-tuning it for sentiment analysis. The experiments also show that multi-task learning while freezing BERT boosts the performance of yet hard tasks such as sarcasm detection. The best-performing models achieved new state-of-the-art performance on the ArSarcasm-v2 dataset for Arabic sarcasm detection and sentiment analysis. For multi-task learning and freezing BERT, a new SOTA F1-score of 64.41 was achieved for the sarcasm detection with a 3.47% improvement and near SOTA FPN of 75.78 for the sentiment classification. For single-task learning, a new SOTA FPN of 75.26 was achieved for the sentiment with a 1.81% improvement.

ADVANCED TECHNIQUES IN MULTI-LABEL TEXT CLASSIFICATION: INTEGRATION OF BETA ANT COLONY AND DEEP LEARNING APPROACHES

The rapid growth of unstructured textual data in various domains has necessitated the development of sophisticated techniques for multi-label text classification. Traditional methods often struggle with handling the complexity and interdependence of multiple labels, leading to suboptimal performance. This paper presents an advanced approach that integrates the Beta Ant Colony Optimization (BACO) algorithm with deep learning techniques for multi-label text classification. The BACO algorithm effectively explores the feature space and selects relevant features by leveraging pheromone trails, while the deep learning model captures intricate patterns and relationships within the textual data. The integration of these two methodologies aims to enhance the efficiency and accuracy of multi-label classification tasks, particularly in domains where label dependency is prominent. Empirical evaluations on benchmark datasets demonstrate that the proposed hybrid approach outperforms existing state-of-the-art techniques in terms of precision, recall, F1-score, and computational efficiency. The findings suggest that combining heuristic optimization algorithms with deep learning can significantly improve multi-label text classification performance, providing a robust solution for real-world applications.

From outputs to insights: a survey of rationalization approaches for explainable text classification.

Deep learning models have achieved state-of-the-art performance for text classification in the last two decades. However, this has come at the expense of models becoming less understandable, limiting their application scope in high-stakes domains. The increased interest in explainability has resulted in many proposed forms of explanation. Nevertheless, recent studies have shown that rationales, or language explanations, are more intuitive and human-understandable, especially for non-technical stakeholders. This survey provides an overview of the progress the community has achieved thus far in rationalization approaches for text classification. We first describe and compare techniques for producing extractive and abstractive rationales. Next, we present various rationale-annotated data sets that facilitate the training and evaluation of rationalization models. Then, we detail proxy-based and human-grounded metrics to evaluate machine-generated rationales. Finally, we outline current challenges and encourage directions for future work.

Contrastive learning based on linguistic knowledge and adaptive augmentation for text classification

Pre-trained language models based on contrastive learning have shown to be effective in text classification. Despite its great success, contrastive learning still has some limitations. First, external linguistic knowledge has been shown to improve the performance of pre-trained language models, but how to use it in contrastive learning is still unclear. Second, general contrastive learning generates training samples with fixed data augmentation during the whole training period, while different augmentation methods are suitable for different downstream tasks. Fixed data augmentation can lead to suboptimal settings. In this paper, we propose contrastive learning based on linguistic knowledge and adaptive augmentation, which can obtain high-quality sentence representations to improve the performance of text classification. Specifically, we construct word-level positive and negative sample pairs by WordNet and propose a novel word-level contrastive learning function to inject linguistic knowledge. Then we dynamically select the augmentation policy by alignment and uniformity. This adaptive augmentation policy can acquire more generalized sentence representations with little computational overhead. Experiments on multiple public datasets demonstrate that our method outperforms state-of-the-art methods.

ENHANCING MULTI-CLASS TEXT CLASSIFICATION WITH APRIORI-BASED FEATURE SELECTION

: In the field of Natural Language Processing, selecting the right features is crucial for reducing unnecessary model complexity, speeding up training, and improving the ability to generalize. However, the multi-class text classification problem makes it challenging for models to generalize well, which complicates feature selection. This paper investigates how feature selection impacts model performance for multi-class text classification, using a dataset of projects completed by TÜBİTAK TEYDEB between 2009 and 2022. The study employs LSTM, a deep learning method, to classify the projects into nine different industries based on various attributes. The paper proposes a new feature selection approach based on the Apriori algorithm, which reduces the number of attribute combinations considered and makes model training more efficient. Model performance is evaluated using metrics like accuracy, loss, validation scores, and test scores. The key findings are that feature selection significantly affects model performance, and different feature sets have varying impacts on performance.

Fine-grained and coarse-grained contrastive learning for text classification

Pre-trained language models based on contrastive learning have shown to be effective in text classification. Although its great success, contrastive learning still has shortcomings. First, most of the existing contrastive learning methods neglect to consider linguistic knowledge, which can improve the performance of contrastive learning. Second, hard negatives have been proven to improve the quality of visual representations. However, generating hard negatives still needs to be explored in the field of text classification. In this paper, we propose a novel fine-grained (word-level) and coarse-grained (sentence-level) contrastive learning that can obtain high-quality sentence representations to improve the performance of text classification. Specifically, we construct word-level positive and negative sample pairs by WordNet and perform fine-grained contrastive learning to inject linguistic knowledge. Then we utilize the mixup to generate hard negatives while eliminating false negatives and use coarse-grained contrastive learning to train the model. Experiments on multiple public datasets demonstrate that our method outperforms state-of-the-art methods.

Comparing human text classification performance and explainability with large language and machine learning models using eye-tracking

To understand the alignment between reasonings of humans and artificial intelligence (AI) models, this empirical study compared the human text classification performance and explainability with a traditional machine learning (ML) model and large language model (LLM). A domain-specific noisy textual dataset of 204 injury narratives had to be classified into 6 cause-of-injury codes. The narratives varied in terms of complexity and ease of categorization based on the distinctive nature of cause-of-injury code. The user study involved 51 participants whose eye-tracking data was recorded while they performed the text classification task. While the ML model was trained on 120,000 pre-labelled injury narratives, LLM and humans did not receive any specialized training. The explainability of different approaches was compared based on the top words they used for making classification decision. These words were identified using eye-tracking for humans, explainable AI approach LIME for ML model, and prompts for LLM. The classification performance of ML model was observed to be relatively better than zero-shot LLM and non-expert humans, overall, and particularly for narratives with high complexity and difficult categorization. The top-3 predictive words used by ML and LLM for classification agreed with humans to a greater extent as compared to later predictive words.

A novel method of detecting malware on Android mobile devices with explainable artificial intelligence

The increasing prevalence of malware targeting android mobile devices has raised significant concerns regarding user privacy and security. In response, effective methods for malware classification and detection are crucial to protect users from malicious applications. This paper presents an approach that leverages deep learning techniques and explainable artificial intelligence (XAI) for android mobile malware classification and detection. Convolutional neural networks (CNNs) are deep learning model that has shown impressive performance in several application areas, including image and text classification. In the context of android mobile malware, CNNs have shown promising results in capturing intricate patterns and features inherent in malware samples. By training these models on large datasets of benign and malicious applications, accurate classification can be achieved. To enhance transparency and interpretability, XAI techniques are integrated into the classification process. These techniques provide insights into the decision-making process of the deep learning models, enabling the identification of critical features and characteristics that contribute to the classification results. This research, by combining deep learning and XAI methods, presents a fresh strategy for identifying and categorizing Android malware. This research paper will focus on a fascinating CNN-based malware categorization technique.

OPTIMIZING LONG TEXT CLASSIFICATION PERFORMANCE THROUGH KEYWORD-BASED SENTENCE SELECTION: A CASE STUDY ON ONLINE NEWS CLASSIFICATION FOR INDONESIAN GDP GROWTH-RATE DETECTION

Efficiently managing lengthy textual data, particularly in online news, is crucial for enhancing the performance of long text classification. This study delves into innovative approaches to streamline the Gross Domestic Product (GDP) computation process by harnessing modern data analytics, Natural Language Processing (NLP), and online news sources. Leveraging online news data introduces real-time information, promising to improve the accuracy and timeliness of economic indicators like GDP. However, handling the complexity of extensive textual data poses a challenge, demanding advanced NLP techniques. This research shifts from traditional word-weight-based methods to keyword-based extractive summarization techniques to address this. These tailored approaches ensure that selected sentences align precisely with specific keywords relevant to the research case, such as GDP growth rate detection. The study emphasizes the necessity of adapting summarization methods to capture information in unique research contexts effectively. According to classification results, the implementation of sentence selection successfully demonstrated improved performance in terms of classification accuracy. Specifically, there was an average accuracy increase of 0.0226 for machine learning and 0.0164 for transfer learning models. Additionally, in terms of computational efficiency, sentence selection also accelerates processing time during hyperparameter tuning and fine-tuning, as observed using the same computational resources.

Harnessing large language models over transformer models for detecting Bengali depressive social media text: A comprehensive study

In an era where the silent struggle of underdiagnosed depression pervades globally, our research delves into the crucial link between mental health and social media. This work focuses on early detection of depression, particularly in extroverted social media users, using LLMs such as GPT 3.5, GPT 4 and our proposed GPT 3.5 fine-tuned model DepGPT, as well as advanced Deep learning models(LSTM, Bi-LSTM, GRU, BiGRU) and Transformer models(BERT, BanglaBERT, SahajBERT, BanglaBERT-Base). The study categorized Reddit and X datasets into “Depressive” and “Non-Depressive” segments, translated into Bengali by native speakers with expertise in mental health, resulting in the creation of the Bengali Social Media Depressive Dataset (BSMDD). Our work provides full architecture details for each model and a methodical way to assess their performance in Bengali depressive text categorization using zero-shot and few-shot learning techniques. Our work demonstrates the superiority of SahajBERT and Bi-LSTM with FastText embeddings in their respective domains also tackles explainability issues with transformer models and emphasizes the effectiveness of LLMs, especially DepGPT (GPT 3.5 fine-tuned), demonstrating flexibility and competence in a range of learning contexts. According to the experiment results, the proposed model, DepGPT, outperformed not only Alpaca Lora 7B in zero-shot and few-shot scenarios but also every other model, achieving a near-perfect accuracy of 0.9796 and an F1-score of 0.9804, high recall, and exceptional precision. Although competitive, GPT-3.5 Turbo and Alpaca Lora 7B show relatively poorer effectiveness in zero-shot and few-shot situations. The work emphasizes the effectiveness and flexibility of LLMs in a variety of linguistic circumstances, providing insightful information about the complex field of depression detection models.

SPContrastNet: A Self-Paced Contrastive Learning Model for Few-Shot Text Classification

Meta-learning has recently promoted few-shot text classification, which identifies target classes based on information transferred from source classes through a series of small tasks or episodes. Existing works constructing their meta-learner on Prototypical Networks need improvement in learning discriminative text representations between similar classes that may lead to conflicts in label prediction. The overfitting problems caused by a few training instances need to be adequately addressed. In addition, efficient episode sampling procedures that could enhance few-shot training should be utilized. To address the problems mentioned above, we first present a contrastive learning framework that simultaneously learns discriminative text representations via supervised contrastive learning while mitigating the overfitting problem via unsupervised contrastive regularization, and then we build an efficient self-paced episode sampling approach on top of it to include more difficult episodes as training progresses. Empirical results on eight few-shot text classification datasets show that our model outperforms the current state-of-the-art models. The extensive experimental analysis demonstrates that our supervised contrastive representation learning and unsupervised contrastive regularization techniques improve the performance of few-shot text classification. The episode-sampling analysis reveals that our self-paced sampling strategy improves training efficiency.

A Heterogeneous Directed Graph Attention Network for inductive text classification using multilevel semantic embeddings

In the current study, a novel network model is proposed for text classification based on Graph Attention Networks (GATs) and sentence-transformer embeddings. Most existing methods with a pretraining model as an input layer still treat words as the minimum processing unit. However, word embedding is not an efficient and appropriate solution when dealing with long texts containing many professional words. This study aims to design a model capable of handling text classification tasks at multilevel semantic segmentation. The main contribution of this study is that a novel GAT variant is designed using global nodes and Squeeze-and-Excitation Networks (SENet) to capture semantic information. Moreover, a novel unidirectional attention mechanism is introduced for our model to avoid the message passing of irrelevant noisy information within global nodes. The numerical results show that according to the characteristics of datasets, specific semantic information combinations can effectively improve the accuracy and performance of text classification. Without fine-tuning of the pretrained encoder, the new state-of-the-art performance is achieved on three benchmark datasets. In addition, a comprehensive analysis of the graph attention mechanism in the model for specific cases suggests that the unidirectional attention mechanism and the use of global nodes are key contributing factors to multilevel semantic fusion.

Comparative Analysis of XGBoost Performance for Text Classification with CPU Parallel and Non-Parallel Processing

This paper shows the findings of a study that looks at how CPU parallel processing changes the way Extreme Gradient Boosting (XGBoost) classifies text. XGBoost models can sort news stories into set groups faster and more accurately, with or without CPU parallelism. This is the main goal of the study. The Keras dataset is used to prepare the text so that the TF-IDF (Term Frequency-Inverse Document Frequency) features can be found. These features will then be used to train the XGBoost model. This is used to check out two different kinds of the XGBoost classifier. There is parallelism between one of them and not it in the other. How well the model works can be observed by how accurate it is. This includes both how long it takes to learn and estimate and how well predictions work. The models take very different amounts of time to compute, but they are all pretty close in terms of how accurate they are. Parallel processing on the CPU has made tasks proceed more rapidly, and XGBoost is now better at making the most of that speed to do its task. The purpose of the study is to show that parallel processing can speed up XGBoost models without affecting their accuracy. This is helpful for putting text into categories.

Comparing Hierarchical Approaches to Enhance Supervised Emotive Text Classification

The performance of emotive text classification using affective hierarchical schemes (e.g., WordNet-Affect) is often evaluated using the same traditional measures used to evaluate the performance of when a finite set of isolated classes are used. However, applying such measures means the full characteristics and structure of the emotive hierarchical scheme are not considered. Thus, the overall performance of emotive text classification using emotion hierarchical schemes is often inaccurately reported and may lead to ineffective information retrieval and decision making. This paper provides a comparative investigation into how methods used in hierarchical classification problems in other domains, which extend traditional evaluation metrics to consider the characteristics of the hierarchical classification scheme, can be applied and subsequently improve the classification of emotive texts. This study investigates the classification performance of three widely used classifiers, Naive Bayes, J48 Decision Tree, and SVM, following the application of the aforementioned methods. The results demonstrated that all the methods improved the emotion classification. However, the most notable improvement was recorded when a depth-based method was applied to both the testing and validation data, where the precision, recall, and F1-score were significantly improved by around 70 percentage points for each classifier.

Cigarette tasting Chinese text classification for low-resource scenarios

At present, the Chinese text field is facing challenges from low resource issues such as data scarcity and annotation difficulties. Moreover, in the domain of cigarette tasting, cigarette tasting texts tend to be colloquial, making it difficult to obtain valuable and high-quality tasting texts. Therefore, in this paper, we construct a cigarette tasting dataset (CT2023) and propose a novel Chinese text classification method based on ERNIE and Comparative Learning for Low-Resource scenarios (ECLLR). Firstly, to address the issues of limited vocabulary diversity and sparse features in cigarette tasting text, we utilize Term Frequency-Inverse Document Frequency (TF-IDF) to extract key terms, supplementing the discriminative features of the original text. Secondly, ERNIE is employed to obtain sentence-level vector embedding of the text. Finally, contrastive learning model is used to further refine the text after fusing the keyword features, thereby enhancing the performance of the proposed text classification model. Experiments on the CT2023 dataset demonstrate an accuracy rate of 96.33% for the proposed method, surpassing the baseline model by at least 11 percentage points, and showing good text classification performance. It is thus clear that the proposed approach can effectively provide recommendations and decision support for cigarette production processes in tobacco companies.

Machine learning for text document classification-efficient classification approach

<p>Numerous alternative methods for text classification have been created because of the increase in the amount of online text information available. The cosine similarity classifier is the most extensively utilized simple and efficient approach. It improves text classification performance. It is combined with estimated values provided by conventional classifiers such as Multinomial Naive Bayesian (MNB). Consequently, combining the similarity between a test document and a category with the estimated value for the category enhances the performance of the classifier. This approach provides a text document categorization method that is both efficient and effective. In addition, methods for determining the proper relationship between a set of words in a document and its document categorization is also obtained.</p>

A Class Specific Feature Selection Method for Improving the Performance of Text Classification

Recently, a significant amount of research work has been carried out in the field of feature selection. Although these methods help to increase the accuracy of the machine learning classification, the selected subset of features considers all the classes and may not select recommendable features for a particular class. The main goal of our paper is to propose a new class-specific feature selection algorithm that is capable of selecting an appropriate subset of features for each class. In this regard, we first perform class binarization and then select the best features for each class. During the feature selection process, we deal with class imbalance problems and redundancy elimination. The Weighted Average Voting Ensemble method is used for the final classification. Finally, we carry out experiments to compare our proposed feature selection approach with the existing popular feature selection methods. The results prove that our feature selection method outperforms the existing methods with an accuracy of more than 37%.

Enhanced text classification through an improved discrete laying chicken algorithm

AbstractThe exponential growth of digital text documents presents a significant challenge for text classification algorithms, as the vast number of words in each document can hinder their efficiency. Feature selection (FS) is a crucial technique that aims to eliminate irrelevant features and enhance classification accuracy. In this study, we propose an improved version of the discrete laying chicken algorithm (IDLCA) that utilizes noun‐based filtering to reduce the number of features and improve text classification performance. Although LCA is a newly proposed algorithm, it has not been systematically applied to discrete problems before. Our enhanced version of LCA employs different operators to improve both exploration and exploitation of this algorithm to find better solutions in discrete mode. To evaluate the effectiveness of the proposed method, we compared it with some conventional nature‐inspired feature selection methods using various learning models such as decision trees (DT), K‐nearest neighbor (KNN), Naive Bayes (NB), and support vector machine (SVM) on five benchmark datasets with three different evaluation metrics. The experimental results demonstrate the effectiveness of the proposed algorithm in comparison to the existing one. The code is available at https://github.com/m0javad/Improved-Discrete-Laying-Chicken-Algorithm.