Automatic Text Classification Research Articles

Retraction Note: Automatic text classification using machine learning and optimization algorithms

Retraction Note: Automatic text classification using machine learning and optimization algorithms

Hazard Analysis for Massive Civil Aviation Safety Oversight Reports Using Text Classification and Topic Modeling

There are massive amounts of civil aviation safety oversight reports collected each year in the civil aviation of China. The narrative texts of these reports are typically short texts, recording the abnormal events detected during the safety oversight process. In the construction of an intelligent civil aviation safety oversight system, the automatic classification of safety oversight texts is a key and fundamental task. However, all safety oversight reports are currently analyzed and classified into categories by manual work, which is time consuming and labor intensive. In recent years, pre-trained language models have been applied to various text mining tasks and have proven to be effective. The aim of this paper is to apply text classification to the mining of these narrative texts and to show that text classification technology can be a critical element of the aviation safety oversight report analysis. In this paper, we propose a novel method for the classification of narrative texts in safety oversight reports. Through extensive experiments, we validated the effectiveness of all the proposed components. The experimental results demonstrate that our method outperforms existing methods on the self-built civil aviation safety oversight dataset. This study undertakes a thorough examination of the precision and associated outcomes of the dataset, thereby establishing a solid basis for furnishing valuable insights to enhance data quality and optimize information.

Detecting information from Twitter on landslide hazards in Italy using deep learning models

BackgroundMass media are a new and important source of information for any natural disaster, mass emergency, pandemic, economic or political event, or extreme weather event affecting one or more communities in a country. Several techniques have been developed for data mining in social media for many natural events, but few of them have been applied to the automatic extraction of landslide events. In this study, Twitter has been investigated to detect data about landslide events in Italian-language. The main aim is to obtain an automatic text classification on the basis of information about natural hazards. The text classification for landslide events in Italian-language has still not been applied to detect this type of natural hazard.ResultsOver 13,000 data were extracted within Twitter considering five keywords referring to landslide events. The dataset was classified manually, providing a solid base for applying deep learning. The combination of BERT + CNN has been chosen for text classification and two different pre-processing approaches and bert-model have been applied. BERT-multicase + CNN without preprocessing archived the highest values of accuracy, equal to 96% and AUC of 0.96.ConclusionsTwo advantages resulted from this studio: the Italian-language classified dataset for landslide events fills that present gap of analysing natural events using Twitter. BERT + CNN was trained to detect this information and proved to be an excellent classifier for the Italian language for landslide events.

Automatic text classification of prostate cancer malignancy scores in radiology reports using NLP models

This paper presents the implementation of two automated text classification systems for prostate cancer findings based on the PI-RADS criteria. Specifically, a traditional machine learning model using XGBoost and a language model-based approach using RoBERTa were employed. The study focused on Spanish-language radiological MRI prostate reports, which has not been explored before. The results demonstrate that the RoBERTa model outperforms the XGBoost model, although both achieve promising results. Furthermore, the best-performing system was integrated into the radiological company’s information systems as an API, operating in a real-world environment.Graphical abstract

Automatic text classification of drug-induced liver injury using document-term matrix and XGBoost.

Regulatory agencies generate a vast amount of textual data in the review process. For example, drug labeling serves as a valuable resource for regulatory agencies, such as U.S. Food and Drug Administration (FDA) and Europe Medical Agency (EMA), to communicate drug safety and effectiveness information to healthcare professionals and patients. Drug labeling also serves as a resource for pharmacovigilance and drug safety research. Automated text classification would significantly improve the analysis of drug labeling documents and conserve reviewer resources. We utilized artificial intelligence in this study to classify drug-induced liver injury (DILI)-related content from drug labeling documents based on FDA's DILIrank dataset. We employed text mining and XGBoost models and utilized the Preferred Terms of Medical queries for adverse event standards to simplify the elimination of common words and phrases while retaining medical standard terms for FDA and EMA drug label datasets. Then, we constructed a document term matrix using weights computed by Term Frequency-Inverse Document Frequency (TF-IDF) for each included word/term/token. The automatic text classification model exhibited robust performance in predicting DILI, achieving cross-validation AUC scores exceeding 0.90 for both drug labels from FDA and EMA and literature abstracts from the Critical Assessment of Massive Data Analysis (CAMDA). Moreover, the text mining and XGBoost functions demonstrated in this study can be applied to other text processing and classification tasks.

Daugiaklasių duomenų klasifikavimo metodų tyrimas

The article analyzes the impact of the chosen method of model application on the classification of multi-label texts written in the Lithuanian language. The article presents a study of mult-label data classification methods in Lithuanian, which includes an analysis of the accuracy of the application of data classification methods for the automatic classification of multiclass texts written in Lithuanian. The classification methods, evaluation criteria, their applicability and the principles of data preparation for classification are reviewed. After preparing the text data for classification tasks, 44 combinations of classifiers were formed for the study and classification was performed using 3 different methods of multi-label data classification: category detection, category membership and category combination detection. The results obtained are compared in terms of time and classification accuracy, identifying the best performing classifiers and identifying the differences and advantages of the classification methods used.

Development and application of financial statement filing robot based on RPA technology

The constant development and application of new technologies, such as big data, artificial intelligence and the mobile Internet, have profoundly changed the personal and professional spheres. Despite these advances, finance professionals are still faced with a multitude of routine, repetitive and error-prone tasks. At the same time, they are challenged by the shift to management accounting, resulting in reduced productivity. This paper addresses these issues by introducing a financial statement filing robot developed using Robotic Process Automation (RPA) technology. The application of this robot has been shown to provide superior efficiency and accuracy, reduce the heavy burden of routine tasks, and facilitate a smooth transition to management accounting practices. In addition, this research provides a valuable reference for the application and diffusion of RPA technology in the financial sector. Given the large amount of text data generated by financial processes, this paper proposes an automatic text categorization model. The effectiveness of the model is demonstrated as a response to address the challenges encountered in the consultation and archiving process. This contribution informs the development of text categorization robots tailored to the needs of finance professionals.

Performance, Energy Consumption and Costs: A Comparative Analysis of Automatic Text Classification Approaches in the Legal Domain

The common practice in Machine Learning research is to evaluate the top-performing models based on their performance. However, this often leads to overlooking other crucial aspects that should be given careful consideration. In some cases, the performance differences between various approaches may be insignificant, whereas factors like production costs, energy consumption, and carbon footprint should be taken into account. Large Language Models (LLMs) are widely used in academia and industry to address NLP problems. In this study, we present a comprehensive quantitative comparison between traditional approaches (SVM-based) and more recent approaches such as LLM (BERT family models) and generative models (GPT2 and LLAMA2), using the LexGLUE benchmark. Our evaluation takes into account not only performance parameters (standard indices), but also alternative measures such as timing, energy consumption and costs, which collectively contribute to the carbon footprint. To ensure a complete analysis, we separately considered the prototyping phase (which involves model selection through training-validation-test iterations) and the in-production phases. These phases follow distinct implementation procedures and require different resources. The results indicate that simpler algorithms often achieve performance levels similar to those of complex models (LLM and generative models), consuming much less energy and requiring fewer resources. These findings suggest that companies should consider additional considerations when choosing machine learning (ML) solutions. The analysis also demonstrates that it is increasingly necessary for the scientific world to also begin to consider aspects of energy consumption in model evaluations, in order to be able to give real meaning to the results obtained using standard metrics (Precision, Recall, F1 and so on).

Research on fault diagnosis of patient monitor based on text mining

The conventional fault diagnosis of patient monitors heavily relies on manual experience, resulting in low diagnostic efficiency and ineffective utilization of fault maintenance text data. To address these issues, this paper proposes an intelligent fault diagnosis method for patient monitors based on multi-feature text representation, improved bidirectional gate recurrent unit (BiGRU) and attention mechanism. Firstly, the fault text data was preprocessed, and the word vectors containing multiple linguistic features was generated by linguistically-motivated bidirectional encoder representation from Transformer. Then, the bidirectional fault features were extracted and weighted by the improved BiGRU and attention mechanism respectively. Finally, the weighted loss function is used to reduce the impact of class imbalance on the model. To validate the effectiveness of the proposed method, this paper uses the patient monitor fault dataset for verification, and the macro F1 value has achieved 91.11%. The results show that the model built in this study can realize the automatic classification of fault text, and may provide assistant decision support for the intelligent fault diagnosis of the patient monitor in the future.

Study of Text Patterns Found on Social Networks of Mental Health Reactions to COVID-19.

SARS-CoV-2 is an infectious disease caused by the coronavirus that was first reported in December 2019 in China and immediately spread around the world causing a pandemic, which has caused countless deaths and cases in global health. Mental health has not gone untouched by this pandemic; due to the lockdown and the vast amounts of information disseminated, the Panamanian population has begun to feel the collateral effects. We propose classifying tweets using a machine learning (ML) and deep learning (DL) approach and pattern search to make recommendations to the emotional and psychological reactions of the Panamanian population. Our study has been carried out with a corpus in spanish extracted from X for the automatic classification of texts, from which we have categorized, through the ML&DL approach, the tweets about Covid-19 in Panama, in order to know if the population has suffered any mental health effects. We can say that the ML models provide competitive results in terms of automatic identification of texts with an accuracy of 90%. X is a social network and an important information channel where you can explore, analyze and organize opinions to make better decisions. Text mining and patron search are a natural language processing (NLP) task that, using ML&DL algorithms, can integrate innovative strategies into information and communication technologies.

AUTOMATIC EVALUATION OF QUALITY OF EXAMS' QUESTIONS WRITTEN IN ARABIC LANGUAGE BASED ON BLOOM’S TAXONOMY: A SURVEY

Most countries of the world are making great efforts in educational reform processes, and they have established quality departments, which have played a major role in the education reform process in first world countries and also in some developing countries, as it is one of the main pillars of the new management model to keep pace with international and local changes. The studies in the field of Blooms taxonomy in Arabic Language is very rare. However, there is a number of rich different studies conducted in English language using Bloom taxonomy classification of questions. The written exams questions are the most important tool in use among the evaluation tools that teachers use in universities or schools, and that importance increases when the results of those exams indicate that there is a shortage or lack of production, or that there are shortcomings or weaknesses in the production. Previous studies dealing with research on types of teachers’ Questions and mainly the analysis of final exam questions, classroom questions or textbook questions. Also there is a number of studies have applied Bloom’s taxonomy to automatic text classification in English language. The materials being classified, features and classifiers used in these studies are summarized below

Convolutional Neural Network Algorithm–Based Novel Automatic Text Classification Framework for Construction Accident Reports

Convolutional Neural Network Algorithm–Based Novel Automatic Text Classification Framework for Construction Accident Reports

CNN-Bi-LSTM Model for MOOC Forum Post Classification

The discussion forum of the massive open online course (MOOC) is a platform for students to communicate with teachers, teaching assistants, and platform managers. It is one of the important factors related to course quality. A reasonable classification of discussion posts in the forum will help students better communicate and solve problems, so as to improve the quality of teaching. Aiming at the classification of discussion forum posts, this paper proposes a text classification model integrating convolutional neural networks (CNN) and bidirectional long-short-term memory (Bi-LSTM). Firstly, the user types and behavior characteristics are analyzed to build the taxonomy. The taxonomy includes three categories: course related, teacher related and platform related. Then, a text classification model is constructed based on CNN and Bi-LSTM. In order to verify the effectiveness of the proposed model, it is applied to the classification of 19285 discussion posts from the MOOC platform of icourse163.org. The overall classification accuracy of the proposed model is 93.6%, which is 12%, 10%, and 8% higher than traditional machine learning methods, CNN and Bi-LSTM, respectively. The model is used for automatic text classification in MOOC discussion forum, which can provide effective help and support for learners, teachers and platform managers, and improve the automation level of MOOC platform.

Optimizing Automatic Text Classification Approach in Adaptive Online Collaborative Discussion–A Perspective of Attention Mechanism-Based Bi-LSTM

A text semantic classification is an essential approach to recognising the verbal intention of online learners, empowering reliable understanding and inquiry for the regulations of knowledge construction amongst students. However, online learning is increasingly switching from static watching patterns to the collaborative discussion. The current deep learning models, such as CNN and RNN, are ineffective in classifying verbal content contextually. Moreover, the contribution of verbal elements to semantics is often considerably varied, requiring the attachment of weights to these elements to increase verbal recognition precision. The Bi-LSTM is considered to be an adaptive model to investigate semantic relations according to the context. Moreover, the attention mechanism in deep learning simulating human vision could assign weights to target texts effectively. This study proposed to construct a deep learning model combining Bi-LSTM and attention mechanism, in which Bi-LSTM obtained the verbal features and keywords, and the generated keywords were weighed in accordance with the attention mechanism. A total of 12,000 sentences generated in online collaborative discussion activities have been classified into six categories, namely statement, negotiation, question, management, emotion and others. Results showed that the classification accuracy of Attention-Bi-LSTM reached 81.50%, which is higher than that of the baseline Bi-LSTM model. This study theoretically uncovers the features of collaborative discussion of onliners and practically provides an effective approach to automatic behaviour analysis in an online context.

Adapting Feature Selection Algorithms for the Classification of Chinese Texts

Text classification has been highlighted as the key process to organize online texts for better communication in the Digital Media Age. Text classification establishes classification rules based on text features, so the accuracy of feature selection is the basis of text classification. Facing fast-increasing Chinese electronic documents in the digital environment, scholars have accumulated quite a few algorithms for the feature selection for the automatic classification of Chinese texts in recent years. However, discussion about how to adapt existing feature selection algorithms for various types of Chinese texts is still inadequate. To address this, this study proposes three improved feature selection algorithms and tests their performance on different types of Chinese texts. These include an enhanced CHI square with mutual information (MI) algorithm, which simultaneously introduces word frequency and term adjustment (CHMI); a term frequency–CHI square (TF–CHI) algorithm, which enhances weight calculation; and a term frequency–inverse document frequency (TF–IDF) algorithm enhanced with the extreme gradient boosting (XGBoost) algorithm, which improves the algorithm’s ability of word filtering (TF–XGBoost). This study randomly chooses 3000 texts from six different categories of the Sogou news corpus to obtain the confusion matrix and evaluate the performance of the new algorithms with precision and the F1-score. Experimental comparisons are conducted on support vector machine (SVM) and naive Bayes (NB) classifiers. The experimental results demonstrate that the feature selection algorithms proposed in this paper improve performance across various news corpora, although the best feature selection schemes for each type of corpus are different. Further studies of the application of the improved feature selection methods in other languages and the improvement in classifiers are suggested.

Text classification by CEFR levels using machine learning methods and BERT language model

This paper presents a study of the problem of automatic classification of short coherent texts (essays) in English according to the levels of the international CEFR scale. Determining the level of text in natural language is an important component of assessing students knowledge, including checking open tasks in e-learning systems. To solve this problem, vector text models were considered based on stylometric numerical features of the character, word, sentence structure levels. The classification of the obtained vectors was carried out by standard machine learning classifiers. The article presents the results of the three most successful ones: Support Vector Classifier, Stochastic Gradient Descent Classifier, LogisticRegression. Precision, recall and F-score served as quality measures. Two open text corpora, CEFR Levelled English Texts and BEA-2019, were chosen for the experiments. The best classification results for six CEFR levels and sublevels from A1 to C2 were shown by the Support Vector Classifier with F-score 67 % for the CEFR Levelled English Texts. This approach was compared with the application of the BERT language model (six different variants). The best model, bert-base-cased, provided the F-score value of 69 %. The analysis of classification errors showed that most of them are between neighboring levels, which is quite understandable from the point of view of the domain. In addition, the quality of classification strongly depended on the text corpus, that demonstrated a significant difference in F-scores during application of the same text models for different corpora. In general, the obtained results showed the effectiveness of automatic text level detection and the possibility of its practical application.

Text Study of Reader Magazine in the Context of Big Data

Abstract Exploring the textual emotional value of the Reader is to help readers understand the Reader’s embedded emotions in an all-around way. In this paper, two text analysis methods, latent semantic analysis and probabilistic latent semantic analysis are described, starting from the analysis model of text research, and the automatic text classification technique is illustrated. The principles of support vector machines are explained, the optimal decision function selection solution is performed using kernel functions, and a PLSA-SVM text analysis model is also constructed using the PLSA method jointly with SVM. The structural features of Reader magazine were analyzed and explained, and the PLSA-SVM text analysis model was used to analyze the data on the narrative features of Reader magazine. In terms of narrative themes, the main narrative themes of Reader magazine were distributed as affection and love, which increased by 16.84% and 17.42% from 2012 to 2021, respectively. In terms of narrative perspectives, the proportion of first-person and third-person narrative perspectives is comparable, increasing by 9.03% and 12.3% from 2012 to 2021, respectively. Thus, in the context of big data, the PLSA-SVM text analysis model can be used to analyze the text narrative features of Reader magazine effectively, and Reader magazine can use the analysis to optimize further the text content and issue texts that are attuned to readers’ emotions.

Integrating Text Classification into Topic Discovery Using Semantic Embedding Models

Topic discovery involves identifying the main ideas within large volumes of textual data. It indicates recurring topics in documents, providing an overview of the text. Current topic discovery models receive the text, with or without pre-processing, including stop word removal, text cleaning, and normalization (lowercase conversion). A topic discovery process that receives general domain text with or without processing generates general topics. General topics do not offer detailed overviews of the input text, and manual text categorization is tedious and time-consuming. Extracting topics from text with an automatic classification task is necessary to generate specific topics enriched with top words that maintain semantic relationships among them. Therefore, this paper presents an approach that integrates text classification for topic discovery from large amounts of English textual data, such as 20-Newsgroups and Reuters Corpora. We rely on integrating automatic text classification before the topic discovery process to obtain specific topics for each class with relevant semantic relationships between top words. Text classification performs a word analysis that makes up a document to decide what class or category to identify; then, the proposed integration provides latent and specific topics depicted by top words with high coherence from each obtained class. Text classification accomplishes this with a convolutional neural network (CNN), incorporating an embedding model based on semantic relationships. Topic discovery over categorized text is realized with latent Dirichlet analysis (LDA), probabilistic latent semantic analysis (PLSA), and latent semantic analysis (LSA) algorithms. An evaluation process for topic discovery over categorized text was performed based on the normalized topic coherence metric. The 20-Newsgroups corpus was classified, and twenty topics with the ten top words were identified for each class. The normalized topic coherence obtained was 0.1723 with LDA, 0.1622 with LSA, and 0.1716 with PLSA. The Reuters Corpus was also classified, and twenty and fifty topics were identified. A normalized topic coherence of 0.1441 was achieved when applying the LDA algorithm, obtaining 20 topics for each class; with LSA, the coherence was 0.1360, and with PLSA, it was 0.1436.

A Comparative Survey of Instance Selection Methods applied to Non-Neural and Transformer-Based Text Classification

Progress in natural language processing has been dictated by the rule of more : more data, more computing power, more complexity, best exemplified by deep learning Transformers. However, training (or fine-tuning) large dense models for specific applications usually requires significant amounts of computing resources. One way to ameliorate this problem is through data engineering instead of the algorithmic or hardware perspectives. Our focus here is an under-investigated data engineering technique, with enormous potential in the current scenario – Instance Selection (IS) (a.k.a. Selective Sampling, Prototype Selection). The IS goal is to reduce the training set size by removing noisy or redundant instances while maintaining or improving the effectiveness (accuracy) of the trained models and reducing the training process cost. We survey classical and recent state-of-the-art IS techniques and provide a scientifically sound comparison of IS methods applied to an essential natural language processing task—Automatic Text Classification (ATC). IS methods have been normally applied to small tabular datasets and have not been systematically compared in ATC. We consider several neural and non-neural state-of-the-art ATC solutions and many datasets. We answer several research questions based on tradeoffs induced by a tripod (training set reduction, effectiveness, and efficiency). Our answers reveal an enormous unfulfilled potential for IS solutions. Specially, we show that in 12 out of 19 datasets, specific IS methods—namely, Condensed Nearest Neighbor, Local Set-based Smoother, and Local Set Border Selector—can reduce the size of the training set without effectiveness losses. Furthermore, in the case of fine-tuning the Transformer methods, the IS methods reduce the amount of data needed, without losing effectiveness and with considerable training-time gains.

Pediatric Injury Surveillance From Uncoded Emergency Department Admission Records in Italy: Machine Learning-Based Text-Mining Approach.

Unintentional injury is the leading cause of death in young children. Emergency department (ED) diagnoses are a useful source of information for injury epidemiological surveillance purposes. However, ED data collection systems often use free-text fields to report patient diagnoses. Machine learning techniques (MLTs) are powerful tools for automatic text classification. The MLT system is useful to improve injury surveillance by speeding up the manual free-text coding tasks of ED diagnoses. This research aims to develop a tool for automatic free-text classification of ED diagnoses to automatically identify injury cases. The automatic classification system also serves for epidemiological purposes to identify the burden of pediatric injuries in Padua, a large province in the Veneto region in the Northeast Italy. The study includes 283,468 pediatric admissions between 2007 and 2018 to the Padova University Hospital ED, a large referral center in Northern Italy. Each record reports a diagnosis by free text. The records are standard tools for reporting patient diagnoses. An expert pediatrician manually classified a randomly extracted sample of approximately 40,000 diagnoses. This study sample served as the gold standard to train an MLT classifier. After preprocessing, a document-term matrix was created. The machine learning classifiers, including decision tree, random forest, gradient boosting method (GBM), and support vector machine (SVM), were tuned by 4-fold cross-validation. The injury diagnoses were classified into 3 hierarchical classification tasks, as follows: injury versus noninjury (task A), intentional versus unintentional injury (task B), and type of unintentional injury (task C), according to the World Health Organization classification of injuries. The SVM classifier achieved the highest performance accuracy (94.14%) in classifying injury versus noninjury cases (task A). The GBM method produced the best results (92% accuracy) for the unintentional and intentional injury classification task (task B). The highest accuracy for the unintentional injury subclassification (task C) was achieved by the SVM classifier. The SVM, random forest, and GBM algorithms performed similarly against the gold standard across different tasks. This study shows that MLTs are promising techniques for improving epidemiological surveillance, allowing for the automatic classification of pediatric ED free-text diagnoses. The MLTs revealed a suitable classification performance, especially for general injuries and intentional injury classification. This automatic classification could facilitate the epidemiological surveillance of pediatric injuries by also reducing the health professionals' efforts in manually classifying diagnoses for research purposes.