Text Classification System Research Articles

Automatic TNM staging of colorectal cancer radiology reports using pre-trained language models

Background and Objective:Colorectal cancer is one of the major causes of cancer death worldwide. Essential for prognosis and treatment planning, TNM staging offers critical insights into the advancement of colorectal cancer. However, manual TNM staging from colon magnetic resonance imaging (MRI) is a laborious and error prone process. This study introduces an automated text classification system for TNM staging of colon MRI images in Spanish. Methods:A dataset of 1319 Spanish colon MRI reports was collected and manually labeled with TNM staging. In order to automate the task of TNM staging, a multimodal system was proposed. The system is based on RoBERTa language model pre-trained on a combination of biomedical and clinical Spanish language corpora and uses Natural Language Processing (NLP) techniques to extract relevant categorical and numerical features from MRI reports. Results:The performance of the system was evaluated using different metrics and the results obtained are very promising: the best performance among the proposed systems reached 0.7464, 0.8792 and 0.6776 of macro F1-score for T, N and M respectively. Conclusions:This study demonstrates the feasibility of using a language model for automatic TNM staging based on Spanish clinical reports of colorectal cancer patients. The proposed system can be a useful tool to improve the efficiency and accuracy of colorectal cancer diagnosis.

Automated classification of online reviews of otolaryngologists.

The study aimed to extract online comments of otolaryngologists in the 20 most populated cities in the United States from healthgrades.com, develop and validate a natural language processing (NLP) logistic regression algorithm for automated text classification of reviews into 10 categories, and compare 1- and 5-star reviews in directly-physician-related and non-physician-related categories. 1977 1-star and 12,682 5-star reviews were collected. The primary investigator manually categorized a training dataset of 324 1-star and 909 5-star reviews, while a validation subset of 100 5-star and 50 1-star reviews underwent dual manual categorization. Using scikit-learn, an NLP algorithm was trained and validated on the subsets, with F1 scores evaluating text classification accuracy against manual categorization. The algorithm was then applied to the entire dataset with comparison of review categorization according to 1- and 5-star reviews. F1 scores for NLP validation ranged between 0.71 and 0.97. Significant associations emerged between 1-star reviews and treatment plan, accessibility, wait time, office scheduling, billing, and facilities. 5-star reviews were associated with surgery/procedure, bedside manner, and staff/mid-levels. The study successfully validated an NLP text classification system for categorizing online physician reviews. Positive reviews were found to have an association with directly-physician related context. 1-star reviews were related to treatment plan, accessibility, wait time, office scheduling, billing, and facilities. This method of text classification effectively discerned the nuances of human-written text, providing valuable insights into online healthcare feedback that is scalable.Level of evidence: Level 3.

Exploring Text Classification Systems for Automatically Coding Historical Occupations and Causes of Death

ObjectivesText classification models can be used to automatically categorize occupations and causes of death within historical documents. It is important to classify/code these categories as different words or textual descriptions could refer to the same occupation or cause of death. Given the many historical documents that are becoming available for research, accurate classification systems can be valuable resources. ApproachWe explore different text classification techniques, from traditional machine learning to deep learning, and investigate methodologies that transform occupations and causes of death into a vectorial space and use these representations as features to train text classification systems. Our data come from IPUMS USA/International, and SCADR. ResultsHistorians have coded occupations and causes of death for some census collections (e.g., US, Canada), but not yet for others (e.g., Scotland). We train and evaluate our classification systems using data from the US and Canada and then deploy it on data from Scotland. We quantitatively measure the performance of the classification systems for historical documents that have codes available. Additionally, once we deploy the model to data that does not yet have codes, we qualitatively evaluate our results by engaging with historians working on those data. We report and discuss these results to understand where the models are performing well and where the models are underperforming. ConclusionsResults suggest that there is value in building and deploying these classification models. We recommend the use of such models in conjunction with engaging with domain experts.

Migrating birds optimization-based feature selection for text classification.

Text classification tasks, particularly those involving a large number of features, pose significant challenges in effective feature selection. This research introduces a novel methodology, MBO-NB, which integrates Migrating Birds Optimization (MBO) approach with naïve Bayes as an internal classifier to address these challenges. The motivation behind this study stems from the recognized limitations of existing techniques in efficiently handling extensive feature sets. Traditional approaches often fail to adequately streamline the feature selection process, resulting in suboptimal classification accuracy and increased computational overhead. In response to this need, our primary objective is to propose a scalable and effective solution that enhances both computational efficiency and classification accuracy in text classification systems. To achieve this objective, we preprocess raw data using the Information Gain algorithm, strategically reducing the feature count from an average of 62,221 to 2,089. Through extensive experiments, we demonstrate the superior effectiveness of MBO-NB in feature reduction compared to other existing techniques, resulting in significantly improved classification accuracy. Furthermore, the successful integration of naïve Bayes within MBO offers a comprehensive and well-rounded solution to the feature selection problem. In individual comparisons with Particle Swarm Optimization (PSO), MBO-NB consistently outperforms by an average of 6.9% across four setups. This research provides valuable insights into enhancing feature selection methods, thereby contributing to the advancement of text classification techniques. By offering a scalable and effective solution, MBO-NB addresses the pressing need for improved feature selection methods in text classification, thereby facilitating the development of more robust and efficient classification systems.

Hierarchical Sentiment Analysis Framework for Hate Speech Detection: Implementing Binary and Multiclass Classification Strategy

A significant challenge in automating hate speech detection on social media is distinguishing hate speech from regular and offensive language. These identify an essential category of content that web filters seek to remove. Only automated methods can manage this volume of daily data. To solve this problem, the community of Natural Language Processing is currently investigating different ways of hate speech detection. In addition to those, previous approaches (e.g., Convolutional Neural Networks, multi-channel BERT models, and lexical detection) have always achieved low precision without carefully treating other related tasks like sentiment analysis and emotion classification. They still like to group all messages with specific words in them as hate speech simply because those terms often appear alongside hateful rhetoric. In this research, our paper presented the hate speech text classification system model drawn upon deep learning and machine learning. In this paper, we propose a new multitask model integrated with shared emotional representations to detect hate speech across the English language. The Transformer-based model we used from Hugging Face and sentiment analysis helped us prevent false positives. Conclusion. We conclude that utilizing sentiment analysis and a Transformer-based trained model considerably improves hate speech detection across multiple datasets.

Natural Language Processing Pretraining Language Model for Computer Intelligent Recognition Technology

Computer intelligent recognition technology refers to the use of computer vision, Natural Language Processing (NLP), machine learning and other technologies to enable computers to recognize, analyze, understand and answer human language and behavior. The common applications of computer intelligent recognition technology include image recognition, NLP, face recognition, target tracking, and other fields. NLP is a field of computer science, which involves the interaction between computers and natural languages. NLP technology can be used to process, analyze and generate natural language data, such as text, voice and image. Common NLP technology applications include language translation, emotion analysis, text classification, speech recognition and question answering system. Language model is a machine learning model, which uses a large number of text data for training to learn language patterns and relationships in text data. Although the language model has made great progress in the past few years, it still faces some challenges, including: poor semantic understanding, confusion in multilingual processing, slow language processing and other shortcomings. Therefore, in order to optimize these shortcomings, this article would study the pre-training language model based on NLP technology, which aimed at using NLP technology to optimize and improve the performance of the language model, thus optimizing the computer intelligent recognition technology. The model had a higher language understanding ability and more accurate prediction ability. In addition, the model could learn language rules and structures by using a large number of corpus, so as to better understand natural language. Through experiments, it could be known that the data size and total computing time of the traditional Generative Pretrained Transformer-2 (GPT-2) language model were 10 GB and 97 hours respectively. The data size and total computing time of BERT (Bidirectional Encoder Representations from Transformer) were 12 GB and 86 hours respectively. The data size and total computing time of the pre-training language model based on NLP were 18 GB and 71 hours respectively. Obviously, the pre-training language model based on NLP had a larger data size and shorter computing time. The experimental results showed that the NLP technology could better optimize the language model and effectively improve its various capabilities. This article opened up a new development direction for computer intelligent recognition technology and provided excellent technical support for the development of language models.

Automatic text classification of prostate cancer malignancy scores in radiology reports using NLP models

This paper presents the implementation of two automated text classification systems for prostate cancer findings based on the PI-RADS criteria. Specifically, a traditional machine learning model using XGBoost and a language model-based approach using RoBERTa were employed. The study focused on Spanish-language radiological MRI prostate reports, which has not been explored before. The results demonstrate that the RoBERTa model outperforms the XGBoost model, although both achieve promising results. Furthermore, the best-performing system was integrated into the radiological company’s information systems as an API, operating in a real-world environment.Graphical abstract

Text Classification for News Article

Abstract: In the era of information overload, effective organization and categorization of news articles are essential for providing users with relevant and timely information. This project focuses on the development and implementation of a text classification system for news articles. The primary goal is to automatically categorize news articles into predefined topics or classes, enhancing the user experience by enabling efficient content discovery and navigation. The project begins with a comprehensive collection of a diverse dataset of news articles spanning various domains such as politics, sports, technology, entertainment, and more. Preprocessing techniques are employed to clean and tokenize the text, followed by feature extraction methods that capture meaningful patterns within the text data. Various machine learning algorithms, including but not limited to, Naive Bayes, Support Vector Machines, and neural networks, are explored and evaluated to determine the optimal model for text classification. To enhance the performance of the classification system, advanced techniques such as word embeddings and transfer learning are investigated. Word embeddings like Word2Vec, Fast Text, or Glove capture semantic relationships between words, improving the model's ability to understand context. Transfer learning, particularly using pre-trained language models like BERT or GPT-3, leverages large-scale language understanding, enabling the model to generalize better even on limited labeled data. The evaluation of the text classification models involves metrics like accuracy, precision, recall, and F1-score, ensuring a comprehensive understanding of their performance across different classes. Hyperparameter tuning and model optimization are conducted to achieve the best possible results.

Bidirectional Gated Recurrent Unit with Glove Embedding and Attention Mechanism for Movie Review Classification

During the current Internet era, enormous volume of structured and unstructured textual data is generated and exchanged online that made the text classification more crucial. With the massive amount of user-generated content on social media platforms, businesses and organizations can leverage sentiment analysis to understand customer opinions and attitudes towards the products, services, or brand. Text classification is a natural language processing task in which a machine learning model is trained to categorize the text into predefined classes or categories. Due to the advent of promising Deep Learning techniques, intelligent and accurate text classification system can be developed. In movie review classification, algorithms are used to automatically categorize movie reviews into positive, negative, or sometimes neutral sentiments based on the opinions expressed in the text. The goal is to automate the process of determining whether a review reflects a positive or negative sentiment, helping users quickly understand the overall reception of a movie. This work proposes an automated movie review classification system based on Bidirectional Gated Recurrent Unit encoder with attention module. This method encodes the words using GloVe word embedding technique. Context information is captured in an efficient manner and the encoding is more reliable. Bidirectional GRU layer captures long-term dependencies in sentences, followed by a custom Attention layer that assigns higher weights to key components. The attention mechanism allows the model to selectively focus on important parts of the input sequence, by dynamically determining attention weights based on task relevance. The model was individually trained and evaluated using popular benchmark IMDb dataset. Experimental studies have proved the better performance of proposed movie review classification system that obtained accuracy of 98%.

Detecting Textual Backdoor Attacks via Class Difference for Text Classification System

Detecting Textual Backdoor Attacks via Class Difference for Text Classification System

Text classification system based on LLM

Text classification system based on LLM

Analyzing Cognitive Load Associated with Manual Text Classification Task Using Eye Tracking

The accuracy of machine learning-based automated text classification systems, such as spam filters and search engine results, heavily depends on the quality of manual text classification. However, the cognitive demands of manual text classification tasks, particularly when dealing with challenging or difficult-to-comprehend texts, have not been extensively explored in previous studies. This research aims to address this gap by investigating the cognitive load associated with manual text classification tasks through analyzing eye tracking data. In this study, 30 participants performed manual text classification tasks while their ocular parameters were recorded using an eye tracker. The findings of this study revealed that ocular parameters recorded through eye tracking provided valuable insights into the cognitive load experienced during manual text classification tasks. Furthermore, it was observed that complex narratives led to higher cognitive load estimation. Moreover, native English-speaking participants exhibited lower cognitive load, compared to non-native English speakers.

Text Classification System Using Text Mining with XGBoost Method

Large data nowadays can be used for analysis; thus, it can obtain important/valuable knowledge in various domains. Text analysis can be carried out by utilizing text mining using computational methods so that knowledge extraction can be carried out on large text data, including processing related to unstructured text data, which is written in natural language. Classification in text mining is a type of work with the searching process for a set of models or functions that describe and differentiate text data classes with the aim that the model can be used to predict the class of an object (text data) whose class is unknown. Text mining was carried out in this research to analyze text data through the Text Classification System using a classification method, namely the XGBoost (eXtreme Gradient Boosting) Method. A text classification system was developed to classify text in the form of articles. The highest accuracy obtained from the test is 77%.

Automatic explanation of the classification of Spanish legal judgments in jurisdiction-dependent law categories with tree estimators

Automatic legal text classification systems have been proposed in the literature to address knowledge extraction from judgments and detect their aspects. However, most of these systems are black boxes even when their models are interpretable. This may raise concerns about their trustworthiness. Accordingly, this work contributes with a system combining Natural Language Processing (nlp) with Machine Learning (ml) to classify legal texts in an explainable manner. We analyze the features involved in the decision and the threshold bifurcation values of the decision paths of tree structures and present this information to the users in natural language. This is the first work on automatic analysis of legal texts combining nlp and ml along with Explainable Artificial Intelligence techniques to automatically make the models’ decisions understandable to end users. Furthermore, legal experts have validated our solution, and this knowledge has also been incorporated into the explanation process as “expert-in-the-loop” dictionaries. Experimental results on an annotated data set in law categories by jurisdiction demonstrate that our system yields competitive classification performance, with accuracy values well above 90%, and that its automatic explanations are easily understandable even to non-expert users.

Performance analysis of a novel hybrid deep learning approach in classification of quality-related English text

Text classification technique is advancing rapidly alongside AI technology, showing signs of maturity. Moreover, there are always many unrestricted constraints that text classification must deal with in practical settings. English text is indeed a significant component of textual data and a significant source of data for persons seeking data from other countries. This study enhances the text classification method currently in use using text classification depending upon English quality. By using an illustration of English quality-related text classification systems, the concept as well as execution of text classification systems is demonstrated, concluding the study on text classification algorithms. The main task of this article is to categorize, and analyze huge volumes of information in English text using technique of integrating qualitative. Therefore, the fundamental components of superior English compositions are attained using Lion Optimization Algorithm (LOA). Convolutional Neural Network-Long Short Term Memory (CNN-LSTM) is utilized to classify the obtained texts. If there is a lot of training input, the typical English text classification method can easily display flaws like ambiguous characteristic elements. In light of such issues, the research suggests a quality-related English text classification approach based on convolutional neural network in order to enhance the precision and adaptability of English text classification.

Hybrid approach for text categorization: A case study with Bangla news article

The incredible expansion of online texts due to the Internet has intensified and revived the interest of sorting, managing and categorising the documents into their respective domains. This shows the pressing need for automatic text categorization system to assign a document into its appropriate domain. In this article, the focus is on showcasing the effectiveness of a hybrid approach that works elegantly by combining text-based and graph-based features. The hybrid approach was applied on 14,373 Bangla articles with 57,22,569 tokens collected from various online news corpora covering nine categories. This article also presents the individual application of both the features to explicate how they generally work. For classification purposes, the feature sets were passed through the Bayesian classification methods which yield satisfactory results with 98.73% accuracy for Naïve Bayes Multinomial (NBM). Also, to test the robustness and language independency of the system, the experiments were performed on two popular English datasets as well.

Evaluating a New Auto-ML Approach for Sentiment Analysis and Intent Recognition Tasks

Automated Machine Learning (AutoML) is a research area that aims to help humans solve Machine Learning (ML) problems by automatically discovering good ML pipelines (algorithms and their hyperparameters for every stage of a machine learning process) for a given dataset. Since we have a combinatorial optimization problem for which it is impossible to evaluate all possible pipelines, most AutoML systems use a Genetic Algorithm (GA) or Bayesian Optimization (BO) to find a good solution. These systems usually evaluate the performance of the pipelines using the K-fold cross-validation method, for which the more pipelines are evaluated, the higher the chance of finding an overfitted solution. To avoid the aforementioned issue, we propose a system named Auto-ML System for Text Classification (ASTeC), that uses the Bootstrap Bias Corrected CV (BBC-CV) method to evaluate the performance of the pipelines. More specifically, the proposed system combines GA, BO, and BBC-CV to find a good ML pipeline for the text classification task. We evaluated our approach by comparing it with state-of-the-art systems: in the the Sentiment Analysis (SA) task, we compared our approach to TPOT (Tree-based Pipeline Optimization Tool) and Google Cloud AutoML service, and for the Intent Recognition (IR) task, we compared with TPOT and MLJAR AutoML. Concerning the data, we analysed seven public datasets from the SA domain and sixteen from the IR domain. Four out of those sixteen are composed by written English text, while all of the others are in Brazilian Portuguese. Statistical tests show that, in 21 out of 23 datasets, our system's performance is equivalent to or better than the others.

TUNING TRADITIONAL LANGUAGE PROCESSING APPROACHES FOR PASHTO TEXT CLASSIFICATION

Today text classification becomes critical task for concerned individuals for numerous purposes. Hence, several researches have been conducted to develop automatic text classification for national and international languages. However, the need for an automatic text categorization system for local languages is felt. The main aim of this study is to establish a Pashto automatic text classification system. In order to pursue this work, we built a Pashto corpus which is a collection of Pashto documents due to the unavailability of public datasets of Pashto text documents. Besides, this study compares several models containing both statistical and neural network machine learning techniques including Multilayer Perceptron (MLP), Support Vector Machine (SVM), K Nearest Neighbor (KNN), decision tree, gaussian naïve Bayes, multinomial naïve Bayes, random forest, and logistic regression to discover the most effective approach. Moreover, this investigation evaluates two different feature extraction methods including unigram, and Time Frequency Inverse Document Frequency (IFIDF). Subsequently, this research obtained average testing accuracy rate 94% using MLP classification algorithm and TFIDF feature extraction method in this context.

Tuning Language Processing Approaches for Pashto Texts Classification

Nowadays, text classification for different purposes becomes a basic task for concerned people. Hence, much research has been done to develop automatic text classification for the majority of national and international languages. However, the need for an automated text classification system for local languages is felt. The main purpose of this study is to establish a novel automatic classification system of Pashto text. In order to follow this up, we established a collection of Pashto documents and constructed the dataset. In addition, this study includes several models containing statistical techniques and neural network neural machine learning including DistilBERT-base-multilingual-cased, Multilayer Perceptron, Support Vector Machine, K Nearest Neighbor, decision tree, Gaussian naïve Bayes, multinomial naïve Bayes, random forest, and logistic regression to discover the most effective approach. Moreover, this investigation evaluates two different feature extraction methods including bag of words, and Term Frequency Inverse Document Frequency. Subsequently, this research obtained an average testing accuracy rate of 94% using the MLP classification algorithm and TFIDF feature extraction method in single label multi-class classification. Similarly, MLP+TFIDF with F1-measure of 0.81 showed the best result. Experiments on the use of pre-trained language representation models (such as DistilBERT) for classifying Pashto texts show that we need a specific tokenizer for a particular language to obtain reasonable results.

Detecting textual adversarial examples through text modification on text classification systems

Detecting textual adversarial examples through text modification on text classification systems