Text Classification Techniques Research Articles

Comparative Analysis of Machine and Deep Learning Techniques for Text Classification with Emphasis on Data Preprocessing

Physician-written discharge medical notes include vital details regarding their patients' health. Numerous deep learning algorithms have shown effective in gleaning crucial insights from unstructured medical notes data, leading to potentially useful outcomes in the medical field. The goal of this research is to determine how different deep learning algorithms perform as models for text classification issues in long short term memory (LSTM). Titanic Disaster Dataset has been used for pre-processing is essential since there is a lot of unnecessary information in textual data. Next, clean up the data by eliminating duplicate rows and filling in the blanks. Besides traditional machine learning algorithms such as naive bayes (NB), gradient boosting (GB), and support vector machine (SVM), we use deep learning algorithms to classify data, including bidirectional – LSTM using Conditional Random Fields (CRFs). BiLSTM is the most precise model compared to other models and baseline research, with a classification accuracy of 98.5%.

On Embedding Implementations in Text Ranking and Classification Employing Graphs

This paper aims to enhance the Graphical Set-based model (GSB) for ranking and classification tasks by incorporating node and word embeddings. The model integrates a textual graph representation with a set-based model for information retrieval. Initially, each document in a collection is transformed into a graph representation. The proposed enhancement involves augmenting the edges of these graphs with embeddings, which can be pretrained or generated using Word2Vec and GloVe models. Additionally, an alternative aspect of our proposed model consists of the Node2Vec embedding technique, which is applied to a graph created at the collection level through the extension of the set-based model, providing edges based on the graph’s structural information. Core decomposition is utilized as a method for pruning the graph. As a byproduct of our information retrieval model, we explore text classification techniques based on our approach. Node2Vec embeddings are generated by our graphs and are applied in order to represent the different documents in our collections that have undergone various preprocessing methods. We compare the graph-based embeddings with the Doc2Vec and Word2Vec representations to elaborate on whether our approach can be implemented on topic classification problems. For that reason, we then train popular classifiers on the document embeddings obtained from each model.

Text Classification on Customer Feedback: A Systematic Literatures Review

User feedback in text classification serves multiple purposes, including refining models, enhancing datasets, adapting to user preferences, identifying emerging topics, and evaluating system performance, all of which contribute to the creation of more effective and user-centric classification systems. Many text classification techniques, including data mining, machine learning, and deep learning approaches, have been employed in previous literature, each making significant contributions to the field. This paper aims to contribute by guiding researchers seeking commonly used classification techniques and evaluation metrics in text processing. Additionally, it identifies the classification technique that generates higher accuracy and works as a basis for researchers to synthesize studies within their respective fields. PRISMA methodology is adapted to systematically review 28 current literatures on text classification on user feedback. The results obtained are guided by four research questions; paper distribution year, dataset source and size; evaluation metric and model accuracy. The review has shown that support vector machines (SVM) are frequently employed and consistently achieve high levels of accuracy as high as 97.17% with various datasets used. The future direction of this work could explore models that integrate sentiment analysis and natural language understanding to more accurately capture nuanced user opinions and preferences.

A Review of Metaheuristic Optimization Techniques in Text Classification

Metaheuristic algorithms, inspired by natural phenomena and human-based strategies, offer versatile approaches to navigate diverse search spaces and adapt to dynamic environments. These algorithms, including evolutionary algorithms, swarm intelligence, bio-inspired methods, human-based approaches, and plant-inspired techniques, have found applications across diverse domains such as engineering, finance, healthcare, logistics, and telecommunications. In the text classification domain, metaheuristic techniques have emerged as powerful tools to enhance the accuracy, efficiency, and robustness of classification systems. By optimizing feature subsets, fine-tuning model parameters, and addressing challenges such as feature selection, dimensionality reduction, class imbalance, and noisy data, metaheuristic algorithms provide flexible solutions that adapt to various text datasets and tasks. This review paper comprehensively explores recent advancements in metaheuristic applications in text classification across six categories. From evolutionary-based methods to swarm-based approaches, bio-inspired techniques to physics/chemistry-based strategies, human-based methods to plant-based algorithms, researchers have leveraged diverse metaheuristic techniques to push the boundaries of text classification. Through a systematic analysis of recent research studies, this review provides insights into the strengths, limitations, and future directions of metaheuristic optimization in the context of text classification.

Text Classification in Natural Language Processing

This paper presents an overview of text classification techniques, focusing on the pre-processing steps, feature extraction methods, and model selection strategies employed in the process. Algorithms such as Naive Bayes, Support Vector Machines (SVM), logistic regression, and neural networks are used. Furthermore, recent advancements in deep learning models for text classification, including Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are used. Comprehensive understanding of text classification methodologies in NLP and insights into current trends and challenges in the field are mentioned

Harnessing the Power of Beta Scoring in Deep Active Learning for Multi-Label Text Classification

Within the scope of natural language processing, the domain of multi-label text classification is uniquely challenging due to its expansive and uneven label distribution. The complexity deepens due to the demand for an extensive set of annotated data for training an advanced deep learning model, especially in specialized fields where the labeling task can be labor-intensive and often requires domain-specific knowledge. Addressing these challenges, our study introduces a novel deep active learning strategy, capitalizing on the Beta family of proper scoring rules within the Expected Loss Reduction framework. It computes the expected increase in scores using the Beta Scoring Rules, which are then transformed into sample vector representations. These vector representations guide the diverse selection of informative sample, directly linking this process to the model's expected proper score. Comprehensive evaluations across both synthetic and real datasets reveal our method's capability to often outperform established acquisition techniques in multi-label text classification, presenting encouraging outcomes across various architectural and dataset scenarios.

Enhancing accident cause analysis through text classification and accident causation theory: A case study of coal mine gas explosion accidents

Accident analysis is a crucial aspect of accident prevention, and natural language processing (NLP) techniques can efficiently be applied to analyze the causes of accidents. However, existing analysis methods primarily rely on text clustering and lack the application of accident causation theories, leading to a lack of specific accident cause analysis results. Combining text classification techniques with accident causation theories is an effective approach to address this issue. In this study, we integrated text classification techniques with accident causation theories and utilized coal mine gas explosion accidents as an example. We constructed a corpus, trained a BERT model, and evaluated its performance to obtain a text classification model for accident cause analysis. The results indicated that the BERT model-based text classification algorithm had an accuracy and macro-average F1 value of 0.9878 and 0.7792, respectively, significantly outperforming the control model. The application of this approach demonstrated that combining accident causation theories with text classification techniques for accident cause analysis can improve the efficiency of accident analysis while ensuring the richness of details in analyzing accident causes. By efficiently analyzing a large number of accident cases, this approach can provide a data foundation for data-driven accident prevention and technical support for integrated accident prevention.

Living review framework for better policy design and management of hazardous waste in Australia

The significant increase in hazardous waste generation in Australia has led to the discussion over the incorporation of artificial intelligence into the hazardous waste management system. Recent studies explored the potential applications of artificial intelligence in various processes of managing waste. However, no study has examined the use of text mining in the hazardous waste management sector for the purpose of informing policymakers. This study developed a living review framework which applied supervised text classification and text mining techniques to extract knowledge using the domain literature data between 2022 and 2023. The framework employed statistical classification models trained using iterative training and the best model XGBoost achieved an F1 score of 0.87. Using a small set of 126 manually labelled global articles, XGBoost automatically predicted the labels of 678 Australian articles with high confidence. Then, keyword extraction and unsupervised topic modelling with Latent Dirichlet Allocation (LDA) were performed. Results indicated that there were 2 main research themes in Australian literature: (1) the key waste streams and (2) the resource recovery and recycling of waste. The implication of this framework would benefit the policymakers, researchers, and hazardous waste management organisations by serving as a real time guideline of the current key waste streams and research themes in the literature which allow robust knowledge to be applied to waste management and highlight where the gap in research remains.

Decoding Digital Discourse: Artificial Intelligence – powered Cyberbullying Detection

Social media platforms have fundamentally transformed how people share information and communicate. While they offer significant benefits, they also pose challenges, such as the increasing prevalence of cyberbullying. While many studies have emphasized the accuracy of text classification techniques for detecting cyberbullying, this research explores the potential of automating not just the detection but also the reporting of harmful posts. We developed a Support Vector Machine model using WEKA, designed to identify cyberbullying statements in the English language. This model yielded an accuracy of 57% with a kappa score of 0.2094. After developing the model, we extracted public posts from Twitter and applied text preprocessing methods, including cleaning and tokenization. These preprocessed data were then transformed into a Bag-of- Words (BoW) representation. When a post is identified as cyberbullying by our model, a comprehensive report is generated detailing the author's name, post content, and the timestamp. This innovative method holds promise for the timely detection of malicious content, offering social media platform administrators an efficient tool for prompt intervention.

Breaking news: Unveiling a new dataset for Portuguese news classification and comparative analysis of approaches.

Every day thousands of news are published on the web and filtering tools can be used to extract knowledge on specific topics. The categorization of news into a predefined set of topics is a subject widely studied in the literature, however, most works are restricted to documents in English. In this work, we make two contributions. First, we introduce a Portuguese news dataset collected from WikiNews an open-source media that provide news from different sources. Since there is a lack of datasets for Portuguese, and an existing one is from a single news channel, we aim to introduce a dataset from different news channels. The availability of comprehensive datasets plays a key role in advancing research. Second, we compare different architectures for Portuguese news classification, exploring different text representations (BoW, TF-IDF, Embedding) and classification techniques (SVM, CNN, DJINN, BERT) for documents in Portuguese, covering classical methods and current technologies. We show the trade-off between accuracy and training time for this application. We aim to show the capabilities of available algorithms and the challenges faced in the area.

Contextualizing injury severity from occupational accident reports using an optimized deep learning prediction model.

This study introduced a novel approach for predicting occupational injury severity by leveraging deep learning-based text classification techniques to analyze unstructured narratives. Unlike conventional methods that rely on structured data, our approach recognizes the richness of information within injury narrative descriptions with the aim of extracting valuable insights for improved occupational injury severity assessment. Natural language processing (NLP) techniques were harnessed to preprocess the occupational injury narratives obtained from the US Occupational Safety and Health Administration (OSHA) from January 2015 to June 2023. The methodology involved meticulous preprocessing of textual narratives to standardize text and eliminate noise, followed by the innovative integration of Term Frequency-Inverse Document Frequency (TF-IDF) and Global Vector (GloVe) word embeddings for effective text representation. The proposed predictive model adopts a novel Bidirectional Long Short-Term Memory (Bi-LSTM) architecture and is further refined through model optimization, including random search hyperparameters and in-depth feature importance analysis. The optimized Bi-LSTM model has been compared and validated against other machine learning classifiers which are naïve Bayes, support vector machine, random forest, decision trees, and K-nearest neighbor. The proposed optimized Bi-LSTM models' superior predictability, boasted an accuracy of 0.95 for hospitalization and 0.98 for amputation cases with faster model processing times. Interestingly, the feature importance analysis revealed predictive keywords related to the causal factors of occupational injuries thereby providing valuable insights to enhance model interpretability. Our proposed optimized Bi-LSTM model offers safety and health practitioners an effective tool to empower workplace safety proactive measures, thereby contributing to business productivity and sustainability. This study lays the foundation for further exploration of predictive analytics in the occupational safety and health domain.

A Hybrid Deep Learning GRU based Approach for Text Classification using Word Embedding

Text categorization has become an increasingly important issue for businesses that handle massive volumes of data generated online, and it has found substantial use in the field of NLP. The capacity to group texts into separate categories is crucial for users to effectively retain and utilize important information. Our goal is to improve upon existing recurrent neural network (RNN) techniques for text classification by creating a deep learning strategy through our study. Raising the quality of the classifications made is the main difficulty in text classification, nevertheless, as the overall efficacy of text classification is often hampered by the data semantics' inadequate context sensitivity. Our study presents a unified approach to examine the effects of word embedding and the GRU on text classification to address this difficulty. In this study, we use the TREC standard dataset. RCNN has four convolution layers, four LSTM levels, and two GRU layers. RNN, on the other hand, has four GRU layers and four LSTM levels. One kind of recurrent neural network (RNN) that is well-known for its comprehension of sequential data is the gated recurrent unit (GRU). We found in our tests that words with comparable meanings are typically found near each other in embedding spaces. The trials' findings demonstrate that our hybrid GRU model is capable of efficiently picking up word usage patterns from the provided training set. Remember that the depth and breadth of the training data greatly influence the model's effectiveness. Our suggested method performs remarkably well when compared to other well-known recurrent algorithms such as RNN, MV-RNN, and LSTM on a single benchmark dataset. In comparison to the hybrid GRU's F-measure 0.952, the proposed model's F-measure is 0.982%. We compared the performance of the proposed method to that of the three most popular recurrent neural network designs at the moment RNNs, MV-RNNs, and LSTMs, and found that the new method achieved better results on two benchmark datasets, both in terms of accuracy and error rate.

Advancing Preauthorization Task in Healthcare: An Application of Deep Active Incremental Learning for Medical Text Classification

This study presents a novel approach to medical text classification using a deep active incremental learning model, aiming to improve the automation of the preauthorization process in medical health insurance. By automating decision-making for request approval or denial through text classification techniques, the primary focus is on real-time prediction, utilization of limited labeled data, and continuous model improvement. The proposed approach combines a Bidirectional Long Short-Term Memory (Bi-LSTM) neural network with active learning, using uncertainty sampling to facilitate expert-based sample selection and online learning for continuous updates. The proposed model demonstrates improved predictive accuracy over a baseline Long Short-Term Memory (LSTM) model. Through active learning iterations, the proposed model achieved a 4% improvement in balanced accuracy over 100 iterations, underscoring its efficiency in continuous refinement using limited labeled data.

Text Classification from positive and unlabeled examples using Support Vector Machine (SVM)

Support Vector Machines (SVMs) are a powerful machine learning algorithm that can be used for text classification. Traditional SVMs require both positive and negative examples to train the model. However, in many real-world scenarios, it can be difficult or expensive to obtain negative examples. This study explores the application of SVMs in textclassification when only positive and unlabeled examples are available. Theresults showed that the proposed approach achieved competitive performance compared to traditional supervised methods, even when trained on limited labeled examples. The utilization of SVC in the proposed approach is twofold. First, the SVC model is used to classify theunlabeled examples as positive or negative. Second, the SVC model is used to select the positive examples that are added to the training set. Thisiterative process of training and selecting examples helps to improve the classification accuracy of the SVM model. The proposed approach is a promising method for text classification when only positive and unlabeled examples are available. Theapproach is effective in achieving competitive performance compared to traditional supervised methods, even when trained on limited labeled examples. This work contributes to enhancing text classification techniques, particularly in situationswith resource constraints and challenging label acquisition. Keywords: Support Vector Machine(S VM), Text Classifications ,Text Mining, SVC, Supervised Methods

MinJoT: Multimodal infusion Joint Training for noise learning in text and multimodal classification problems

Amidst the critical role that high-quality labeled data plays in advancing machine learning, the persistence of noise within widely-used datasets remains a challenge. While noise learning has gained traction within machine learning, particularly in computer vision, its exploration in text and multimodal classification domains has lagged. Furthermore, a comprehensive comparison of noise learning techniques in text and multimodal classification has been lacking, partly due to variations in experimental noise settings across prior studies. Addressing these gaps, this research introduces a pioneering Multimodal Infusion Joint Training (MinJoT) framework featuring a novel co-regularized loss function that seamlessly integrates multimodal information during joint training. This framework notably excels in maintaining model robustness amidst noisy data environments. Adapting established noise learning methods from computer vision to text classification, the study conducts extensive experiments across five English and Chinese textual and multimodal datasets, involving four methods, five noise modes, and seven noise rates. Critically, this work challenges the implicit assumption that widely-used datasets are devoid of noise, revealing that these datasets indeed encompass noise levels ranging from 0.61% to 15.77% which is defined as intrinsic noise in this study. For the first time, the study investigates the impact of intrinsic noise on model performance, categorizing it into distinct levels of ambiguity. To facilitate accurate method comparison, a new dataset, Golden-Chnsenticorp (G-Chnsenticorp), is introduced, carefully crafted to be free of intrinsic noise. This research establishes the inaugural noise learning benchmark for text classification, while simultaneously pioneering the first noise learning framework tailored for multimodal sentiment classification. Through these contributions, the study advances the understanding of noise learning in text and multimodal contexts, providing a novel framework, benchmarks, and insights to propel the field forward.

A Deep Learning based Model using Review Associated Feature Extraction Approach for Sentiment Analysis

With the advancement of internet technologies, in the present days, online forums, social media platforms and e-commerce sites have made the product reviews process very easy. There are a lot of mobile applications, websites and forums where consumers used to share and circulate their opinions, experiences, ideas and views regarding products, brands and services. In consequence, online user reviews have become a deciding factor for many consumers prior to purchasing their selected items. The sentiment analysis is a technique to extract sentiments, feelings and insights from customer reviews and public texts. Therefore, plenty of businesses perform sentiment analysis in order to more thoroughly comprehend of their customer opinions and suggestions regarding their products and services. Furthermore, a number of scientific researchers also have a keen interest in classifying customer reviews into a set of labels employing text classification techniques. The objective of the this research work is to develop an approach to extract review associated features using Part-of-Speech (POS) tagging and design a CNN model to classify the reviews' sentiment as positive or negative. In this paper, an approach to extract review associated feature has been presented. Natural Language Processing (NLP) techniques are utilized for data preprocessing to remove uninformative data from reviews. Deep learning model CNN is used for sentiment classification and Amazon mobile reviews dataset is used for the experiment. The proposed model is experimentally evaluated and provides enhanced performance than other models also provides improved accuracy of 97.23% on Amazon mobile review dataset.

Text Study of Reader Magazine in the Context of Big Data

Abstract Exploring the textual emotional value of the Reader is to help readers understand the Reader’s embedded emotions in an all-around way. In this paper, two text analysis methods, latent semantic analysis and probabilistic latent semantic analysis are described, starting from the analysis model of text research, and the automatic text classification technique is illustrated. The principles of support vector machines are explained, the optimal decision function selection solution is performed using kernel functions, and a PLSA-SVM text analysis model is also constructed using the PLSA method jointly with SVM. The structural features of Reader magazine were analyzed and explained, and the PLSA-SVM text analysis model was used to analyze the data on the narrative features of Reader magazine. In terms of narrative themes, the main narrative themes of Reader magazine were distributed as affection and love, which increased by 16.84% and 17.42% from 2012 to 2021, respectively. In terms of narrative perspectives, the proportion of first-person and third-person narrative perspectives is comparable, increasing by 9.03% and 12.3% from 2012 to 2021, respectively. Thus, in the context of big data, the PLSA-SVM text analysis model can be used to analyze the text narrative features of Reader magazine effectively, and Reader magazine can use the analysis to optimize further the text content and issue texts that are attuned to readers’ emotions.

ChatAgri: Exploring potentials of ChatGPT on cross-linguistic agricultural text classification

In the era of sustainable smart agriculture, a vast amount of agricultural news text is posted online, accumulating significant agricultural knowledge. To efficiently access this knowledge, effective text classification techniques are urgently needed. Deep learning approaches, such as fine-tuning strategies on pre-trained language models (PLMs), have shown remarkable performance gains. Nonetheless, these methods face several complex challenges, including limited agricultural training data, poor domain transferability (especially across languages), and complex and expensive deployment of large models. Inspired by the success of recent ChatGPT models (e.g., GPT-3.5, GPT-4), this work explores the potential of applying ChatGPT in the field of agricultural informatization. Various crucial factors, such as prompt construction, answer parsing, and different ChatGPT variants, are thoroughly investigated to maximize its capabilities. A preliminary comparative study is conducted, comparing ChatGPT with PLMs-based fine-tuning methods and PLMs-based prompt-tuning methods. Empirical results demonstrate that ChatGPT effectively addresses the mentioned research challenges and bottlenecks, making it an ideal solution for agricultural text classification. Moreover, ChatGPT achieves comparable performance to existing PLM-based fine-tuning methods, even without fine-tuning on agricultural data samples. We hope this preliminary study could inspire the emergence of a general-purpose AI paradigm for agricultural text processing.

Machine Learning-Based Text Classification Comparison: Turkish Language Context

The growth in textual data associated with the increased usage of online services and the simplicity of having access to these data has resulted in a rise in the number of text classification research papers. Text classification has a significant influence on several domains such as news categorization, the detection of spam content, and sentiment analysis. The classification of Turkish text is the focus of this work since only a few studies have been conducted in this context. We utilize data obtained from customers’ inquiries that come to an institution to evaluate the proposed techniques. Classes are assigned to such inquiries specified in the institution’s internal procedures. The Support Vector Machine, Naïve Bayes, Long Term-Short Memory, Random Forest, and Logistic Regression algorithms were used to classify the data. The performance of the various techniques was then analyzed after and before data preparation, and the results were compared. The Long Term-Short Memory technique demonstrated superior effectiveness in terms of accuracy, achieving an 84% accuracy rate, surpassing the best accuracy record of traditional techniques, which was 78% accuracy for the Support Vector Machine technique. The techniques performed better once the number of categories in the dataset was reduced. Moreover, the findings show that data preparation and coherence between the classes’ number and the number of training sets are significant variables influencing the techniques’ performance. The findings of this study and the text classification technique utilized may be applied to data in dialects other than Turkish.

A comprehensive dual-layer architecture for phishing and spam email detection

The widespread use of email communication for sharing personal, professional, and financial data has rendered it vulnerable to cyber-attacks. Detecting untrustworthy emails with minimal errors is crucial to counter these threats. This research paper focuses on classifying spam and phishing emails, commonly employed to steal confidential information by impersonating legitimate sources. The scale of such attacks is alarmingly high, causing substantial financial losses across sectors like banking, healthcare, technology, and other businesses. This research aims to classify both spam and phishing emails, addressing the limitations of existing studies that only focus on one type and consider either the email body or content for feature selection. This research incorporates features from both the email body and content during model training. The authors propose a novel approach that ensures highly accurate classification while effectively handling the common issue of data imbalance in email phishing and spam classification. The approach utilizes a dual-layer architecture, with each layer containing a trained or pre-trained model that classifies data instances into their respective classes. Layer 1 classifies the phishing class, while Layer 2 classifies the spam class. Building upon the proven effectiveness of deep learning techniques for text classification and analysis, the proposed architecture employs models like Artificial Neural Networks (ANN), Recurrent Neural Networks (RNN), and Convolutional Neural Networks (CNN). Experimental evaluation demonstrates the approach’s remarkable accuracy, recall, precision, and F1-score, achieving 99.51%, 99.68%, 99.5%, and 99.52%, respectively. This signifies its high efficacy in detecting and classifying malicious emails with minimal errors, thus holding great promise in enhancing system security against cyber-attacks in email communication.