Learning For Text Classification Research Articles

Domain knowledge-powered attention for air traffic management hazardous events classification

The automated classification of hazardous events in Air Traffic Management (ATM) involves concise texts with specialized terminology. The problem is currently addressed by machine learning text classification methods; however, some methods overlook domain knowledge, while others require it excessively. This research aims to identify a process for integrating low-cost, text-based domain knowledge into classification to address the issue of domain shift. To achieve this, the Wide and Deep Bidirectional Encoder Representations from Transformers (WD-BERT) model is proposed, featuring a unique knowledge-powered attention mechanism. WD-BERT's Wide Attention module assesses the compatibility between input text and domain knowledge to determine class probabilities, while its Deep Attention module extracts contextual features guided by domain knowledge. Additionally, a text mining method is employed to extract domain knowledge from ATM regulation texts. The model is trained and evaluated using a dataset derived from the China ATM Hazard Source Database. It achieves a multi-label classification accuracy of 80.24% and an F1-micro score of 0.88, outperforming comparative models with state-of-the-art performance. Breakdown analyses of the model indicate that the carefully designed attention mechanism endows WD-BERT with the ability to integrate domain knowledge, allowing it to excel in contexts involving complex sentences, multiple terms, and domain shifts. The innovation of this study lies in the proposed domain knowledge-powered attention mechanism, which allows simply organized domain knowledge to effectively guide ATM hazardous events classification, alleviating the impact of domain shift. This method can also be applied to other vertical domains without knowledge graphs.

Supervised Contrast Learning Text Classification Model Based on Data Quality Augmentation

Token-level data augmentation generates text samples by modifying the words of the sentences. However, data that are not easily classified can negatively affect the model. In particular, not considering the role of keywords when performing random augmentation operations on samples may lead to the generation of low-quality supplementary samples. Therefore, we propose a supervised contrast learning text classification model based on data quality augmentation. First, dynamic training is used to screen high-quality datasets containing beneficial information for model training. The selected data is then augmented with data based on important words with tag information. To obtain a better text representation to serve the downstream classification task, we employ a standard supervised contrast loss to train the model. Finally, we conduct experiments on five text classification datasets to validate the effectiveness of our model. In addition, ablation experiments are conducted to verify the impact of each module on classification.

Gender Disparity of Dyskinesia in Patients with Parkinson’s Disease: Deep Learning Text Classification Applied to the AAN Axon Registry (S2.006)

Gender Disparity of Dyskinesia in Patients with Parkinson’s Disease: Deep Learning Text Classification Applied to the AAN Axon Registry (S2.006)

Novel GCN Model Using Dense Connection and Attention Mechanism for Text Classification

Convolutional Neural Network (CNN) or Recurrent Neural Network (RNN) based text classification algorithms currently in use can successfully extract local textual features but disregard global data. Due to its ability to understand complex text structures and maintain global information, Graph Neural Network (GNN) has demonstrated considerable promise in text classification. However, most of the GNN text classification models in use presently are typically shallow, unable to capture long-distance node information and reflect the various scale features of the text (such as words, phrases, etc.). All of which will negatively impact the performance of the final classification. A novel Graph Convolutional Neural Network (GCN) with dense connections and an attention mechanism for text classification is proposed to address these constraints. By increasing the depth of GCN, the densely connected graph convolutional network (DC-GCN) gathers information about distant nodes. The DC-GCN multiplexes the small-scale features of shallow layers and produces different scale features through dense connections. To combine features and determine their relative importance, an attention mechanism is finally added. Experiment results on four benchmark datasets demonstrate that our model’s classification accuracy greatly outpaces that of the conventional deep learning text classification model. Our model performs exceptionally well when compared to other text categorization GCN algorithms.

Enhancing Emotion Detection with Adversarial Transfer Learning in Text Classification

Enhancing Emotion Detection with Adversarial Transfer Learning in Text Classification

Negative Economic Sentiment Index Based on Finnish News Titles

We construct an index for measuring negative economic sentiment in Finland by using news titles collected from the Finnish broadcasting company Yle's archive. Our approach uses supervised machine learning text classification for detecting news titles featuring negative economic sentiment, and the monthly aggregated proportional frequencies of those titles are then used for defining the index. We find a negative correlation between our index and the consumer confidence index by Statistics Finland, and more remarkably, our index seems to lead the consumer confidence index, somewhat, by one month. We also show that our index correlates positively with Finnish stock market volatility. In addition, based on a simple VAR model, we examine how certain macro variables respond to changes in economic sentiment and show that our index could prove helpful in assessing the current and near-future state of the Finnish economy.

Spatio-Temporal Relevance Classification from Geographic Texts Using Deep Learning

The growing proliferation of geographic information presents a substantial challenge to the traditional framework of a geographic information analysis and service. The dynamic integration and representation of geographic knowledge, such as triples, with spatio-temporal information play a crucial role in constructing a comprehensive spatio-temporal knowledge graph and facilitating the effective utilization of spatio-temporal big data for knowledge-driven service applications. The existing knowledge graph (or geographic knowledge graph) takes spatio-temporal as the attribute of entity, ignoring the role of spatio-temporal information for accurate retrieval of entity objects and adaptive expression of entity objects. This study approaches the correlation between geographic knowledge and spatio-temporal information as a text classification problem, with the aim of addressing the challenge of establishing meaningful connections among spatio-temporal data using advanced deep learning techniques. Specifically, we leverage Wikipedia as a valuable data source for collecting and filtering geographic texts. The Open Information Extraction (OpenIE) tool is employed to extract triples from each sentence, followed by manual annotation of the sentences’ spatio-temporal relevance. This process leads to the formation of quadruples (time relevance/space relevance) or quintuples (spatio-temporal relevance). Subsequently, a comprehensive spatio-temporal classification dataset is constructed for experiment verification. Ten prominent deep learning text classification models are then utilized to conduct experiments covering various aspects of time, space, and spatio-temporal relationships. The experimental results demonstrate that the Bidirectional Encoder Representations from Transformer-Region-based Convolutional Neural Network (BERT-RCNN) model exhibits the highest performance among the evaluated models. Overall, this study establishes a foundation for future knowledge extraction endeavors.

SENTIMENT LABELING AND TEXT CLASSIFICATION MACHINE LEARNING FOR WHATSAPP GROUP

The use of WhatsApp Group (WAG) for communication is increasing nowadays. WAG communication data can be analyzed from various perspectives. However, this data is imported in the form of unstructured text files. The aim of this research is to explore the potential use of the SentiwordNet lexicon for labeling the positive, negative, or neutral sentiment of WAG data from "Alumni94" and training and testing it with machine learning text classification models. The training and testing were conducted on six models, namely Random Forest, Decision Tree, Logistic Regression, K-Nearest Neighbors (KNN), Linear Support Vector Machine (SVM), and Artificial Neural Network. The labeling results indicate that neutral sentiment is the majority with 7588 samples, followed by 324 negative and 1617 positive samples. Among all the models, Random Forest showed better precision and recall, i.e., 83% and 64%. On the other hand, Decision Tree had slightly lower precision and recall, i.e., 80% and 66%, but exhibited a better f-measure of 71%. The accuracy evaluation results of the Random Forest and Decision Tree models showed significant performance compared to others, achieving an accuracy of 89% in classifying new messages. This research demonstrates the potential use of the SentiwordNet lexicon and machine learning in sentiment analysis of WAG data using the Random Forest and Decision Tree models

Performance analysis of sentiment classification based on deep learning methods

When it comes to natural language processing, the textual differentiation task is one of the classical and important research problems. Recently, the deep learning model has increasingly become one of the main methods to solve text classification problems. Common deep learning text classification models are convolutional neural networks (CNN), recurrent neural networks (RNN), the BERT model. For comparing the manifestation of various deep learning models in textual differentiation tasks horizontally, the thesis tests the classification accuracy of different deep learning models under the same experimental configuration. The experimental results show that using pre-trained word vectors helps to improve the classification accuracy of deep learning models. In addition, the reasonable design of more complex and larger deep learning models is helpful in enhancing the study capability of the specific model on text data. The experimental results indicate that the text classification model using pre-trained word vectors could gain higher accuracy than the model without pre-trained word vectors. In addition, in the comparison experiment of feedforward neural network (FNN), CNN, RNN and BERT model, BERT model performs best, and the text classification accuracy reaches 0.9232. Compared with a 1-layer FNN, the accuracy rate is increased by about 16%.

Using automated analysis to assess middle school students' competence with scientific argumentation

AbstractArgumentation is fundamental to science education, both as a prominent feature of scientific reasoning and as an effective mode of learning—a perspective reflected in contemporary frameworks and standards. The successful implementation of argumentation in school science, however, requires a paradigm shift in science assessment from the measurement of knowledge and understanding to the measurement of performance and knowledge in use. Performance tasks requiring argumentation must capture the many ways students can construct and evaluate arguments in science, yet such tasks are both expensive and resource‐intensive to score. In this study we explore how machine learning text classification techniques can be applied to develop efficient, valid, and accurate constructed‐response measures of students' competency with written scientific argumentation that are aligned with a validated argumentation learning progression. Data come from 933 middle school students in the San Francisco Bay Area and are based on three sets of argumentation items in three different science contexts. The findings demonstrate that we have been able to develop computer scoring models that can achieve substantial to almost perfect agreement between human‐assigned and computer‐predicted scores. Model performance was slightly weaker for harder items targeting higher levels of the learning progression, largely due to the linguistic complexity of these responses and the sparsity of higher‐level responses in the training data set. Comparing the efficacy of different scoring approaches revealed that breaking down students' arguments into multiple components (e.g., the presence of an accurate claim or providing sufficient evidence), developing computer models for each component, and combining scores from these analytic components into a holistic score produced better results than holistic scoring approaches. However, this analytical approach was found to be differentially biased when scoring responses from English learners (EL) students as compared to responses from non‐EL students on some items. Differences in the severity between human and computer scores for EL between these approaches are explored, and potential sources of bias in automated scoring are discussed.

A deep learning approach of financial distress recognition combining text

<abstract><p>The financial distress of listed companies not only harms the interests of internal managers and employees but also brings considerable risks to external investors and other stakeholders. Therefore, it is crucial to construct an efficient financial distress prediction model. However, most existing studies use financial indicators or text features without contextual information to predict financial distress and fail to extract critical details disclosed in Chinese long texts for research. This research introduces an attention mechanism into the deep learning text classification model to deal with the classification of Chinese long text sequences. We combine the financial data and management discussion and analysis Chinese text data in the annual reports of 1642 listed companies in China from 2017 to 2020 in the model and compare the effects of the data on different models. The empirical results show that the performance of deep learning models in financial distress prediction overcomes traditional machine learning models. The addition of the attention mechanism improved the effectiveness of the deep learning model in financial distress prediction. Among the models constructed in this study, the Bi-LSTM+Attention model achieves the best performance in financial distress prediction.</p></abstract>

Interpretable machine learning text classification for clinical computed tomography reports – a case study of temporal bone fracture

BackgroundMachine learning (ML) has demonstrated success in classifying patients’ diagnostic outcomes in free-text clinical notes. However, due to the machine learning model's complexity, interpreting the mechanism behind classification results remains difficult. MethodsWe investigated interpretable representations of text-based machine learning classification models. We created machine learning models to classify temporal bone fractures based on 164 temporal bone computed tomography (CT) text reports. We adopted the XGBoost, Support Vector Machine, Logistic Regression, and Random Forest algorithms. To interpret models, we used two major methodologies: (1) We calculated the average word frequency score (WFS) for keywords. The word frequency score shows the frequency gap between positively and negatively classified cases. (2) We used Local Interpretable Model-Agnostic Explanations (LIME) to show the word-level contribution to bone fracture classification. ResultsIn temporal bone fracture classification, the random forest model achieved an average F1-score of 0.93. WFS revealed a difference in keyword usage between fracture and non-fracture cases. Additionally, LIME visualized the keywords' contributions to the classification results. The evaluation of LIME-based interpretation achieved the highest interpreting accuracy of 0.97. ConclusionThe interpretable text explainer can improve physicians' understanding of machine learning predictions. By providing simple visualization, our model can increase the trust of computerized models. Our model supports more transparent computerized decision-making in clinical settings.

Retracted: Novel Multirole-Oriented Deep Learning Text Classification Model

Retracted: Novel Multirole-Oriented Deep Learning Text Classification Model

Social Media-Based Surveillance Systems for Healthcare using Machine Learning

One of the most popular domains that have caught the attention of researchers is real-time surveillance in the health and informatics segment. Many initiatives have been discovered due to this real-time surveillance surrounding public health informatics. Real-time surveillance in the health and informatics field has used the information from social media to predict the outbreak of diseases as well as to look after the diseases. There is no doubt in the fact that the availability of the data from social media in the recent past, especially the data from Twitter, has offered the researchers real-time syndromic surveillance in making quick analyses and conclusions in investigating the disease outbreak. The paper will get to know about the recent work of machine learning trends and text classification that has been utilized by the surveillance system by using the data from social media in the field of healthcare. Apart from this, the paper has also discussed the various limitations and challenges by taking into account the future direction that can be considered in this domain further.

Design of Shared Internet of Things System for English Translation Teaching Using Deep Learning Text Classification

The purpose is to adapt to the current social development and promote the English translation teaching reform. Based on the theories of deep learning (DL), text classification (TC), and the Internet of Things (IoT), this work analyzes the current situation of English translation teaching. Additionally, 100 text categories are selected from the English text corpus of Northwestern Polytechnic University as the research objects. The data are classified by the DL-based TC method and analyzed by introducing the simulated annealing algorithm. Finally, the storage and security performance of the shared IoT system are described. The results show that the proposed TC method can overcome the performance loss caused by the function extraction method, greatly reducing the training time and function space. The storage and security performance of the shared IoT system to encrypt English text will increase with the number of model iterations. Therefore, this work designs the English translation teaching-oriented shared IoT system using a DL-based TC. The finding plays an important role in subsequent English translation and enriching the theory of IoT.

An Intelligent system for the categorization of question time official documents of the Italian Chamber of Deputies

ABSTRACT In this work, we present an intelligent system for the automatic categorization of political documents, specifically the documents containing the parliamentary questions collected during the weekly Question Times at the Chamber of Deputies of the Italian Republic. The proposed intelligent system leverages text classification models to perform the document categorization. The system is aimed at supporting and facilitating the research activities of political science scholars, who deal with comparative and longitudinal analysis of thousands of documents. To select the best classification models for our specific task, several classical machine learning and deep learning-based text classification models have been experimentally compared.

Novel Multirole-Oriented Deep Learning Text Classification Model

In order to improve the analysis of multiple roles in novels, this article applies the deep learning text classification model to the analysis of novel roles. Moreover, in this article, the scale space formed by multiple text images of the same size is called an octave, and the text image size of adjacent groups is halved to construct a Gaussian pyramid. In text classification, this article uses the argument and amplitude information to form a direction gradient histogram and takes the argument corresponding to the largest peak as a main direction of the key point. Finally, this article constructs an intelligent analysis model. The research results show that the deep learning text classification model for multiple roles in novels proposed in this article has good effects on role analysis and text classification. For film and television scripts, the classification analysis of analysis texts and the contrast creation have very good application help.

Global Trends in Cancer Nanotechnology: A Qualitative Scientific Mapping Using Content-Based and Bibliometric Features for Machine Learning Text Classification.

Simple SummaryThis study is a new way of providing potential opportunities for prevention, diagnosis, and therapy to investigate the comprehensive trends in cancer nanotechnology research. This paper applied the qualitative method of bibliometric analysis on cancer nanotechnology using the PubMed database during the years 2000–2021. It mined nearly 50,000 papers published in multiple reputed journals. The impact of our findings is significant, which focuses on hybrid medical models and content-based and bibliometric features for machine learning models in cancer detection, diagnosis, imaging, and therapy related to cancer nanotechnology in the world. We mainly identified and classified the top and significant keywords, countries, authors, affiliations, and research areas representing the documents in the top 100 journals in cancer nanotechnology, which will help researchers explore more powerful anticancer nanomedicines in the next five to ten years.This study presents a new way to investigate comprehensive trends in cancer nanotechnology research in different countries, institutions, and journals providing critical insights to prevention, diagnosis, and therapy. This paper applied the qualitative method of bibliometric analysis on cancer nanotechnology using the PubMed database during the years 2000–2021. Inspired by hybrid medical models and content-based and bibliometric features for machine learning models, our results show cancer nanotechnology studies have expanded exponentially since 2010. The highest production of articles in cancer nanotechnology is mainly from US institutions, with several countries, notably the USA, China, the UK, India, and Iran as concentrated focal points as centers of cancer nanotechnology research, especially in the last five years. The analysis shows the greatest overlap between nanotechnology and DNA, RNA, iron oxide or mesoporous silica, breast cancer, and cancer diagnosis and cancer treatment. Moreover, more than 50% of the information related to the keywords, authors, institutions, journals, and countries are considerably investigated in the form of publications from the top 100 journals. This study has the potential to provide past and current lines of research that can unmask comprehensive trends in cancer nanotechnology, key research topics, or the most productive countries and authors in the field.

Lost in Transduction: Transductive Transfer Learning in Text Classification

Obtaining high-quality labelled data for training a classifier in a new application domain is often costly. Transfer Learning (a.k.a. “Inductive Transfer”) tries to alleviate these costs by transferring, to the “target” domain of interest, knowledge available from a different “source” domain. In transfer learning the lack of labelled information from the target domain is compensated by the availability at training time of a set of unlabelled examples from the target distribution. Transductive Transfer Learning denotes the transfer learning setting in which the only set of target documents that we are interested in classifying is known and available at training time. Although this definition is indeed in line with Vapnik’s original definition of “transduction”, current terminology in the field is confused. In this article, we discuss how the term “transduction” has been misused in the transfer learning literature, and propose a clarification consistent with the original characterization of this term given by Vapnik. We go on to observe that the above terminology misuse has brought about misleading experimental comparisons, with inductive transfer learning methods that have been incorrectly compared with transductive transfer learning methods. We then, give empirical evidence that the difference in performance between the inductive version and the transductive version of a transfer learning method can indeed be statistically significant (i.e., that knowing at training time the only data one needs to classify indeed gives an advantage). Our clarification allows a reassessment of the field, and of the relative merits of the major, state-of-the-art algorithms for transfer learning in text classification.

News Text Classification Method and Simulation Based on the Hybrid Deep Learning Model

This paper uses the database as the data source, using bibliometrics and visual analysis methods, to statistically analyze the relevant documents published in the field of text classification in the past ten years, to clarify the development context and research status of the text classification field, and to predict the research in the field of text classification priorities and research frontiers. Based on the in-depth study of the background, research status, related theories, and developments of online news text classification, this article analyzes the annual publication trend, subject distribution, journal distribution, institution distribution, author distribution, highly cited literature analysis, and research hotspots. Forefront and other aspects clarify the development context and research status of the text classification field and provide a theoretical reference for the further development of the text classification field. Then, on the basis of systematic research on text classification, deep learning, and news text classification theories, a deep learning-based network news text classification model is constructed, and the function of each module is introduced in detail, which will help the future news text classification of application and improvement provide theoretical basis. On the basis of the predecessors, this article separately studied and improved the neural network model based on the convolutional neural network, cyclic neural network, and attention mechanism and merged the three models into one model, which can obtain local associated features and contextual features and highlight the role of keywords. Finally, experiments are used to verify the effectiveness of the model proposed in this paper and compared with traditional text classification to prove the superiority of the network news text classification based on deep learning proposed in this paper. This article aims to study the internal connection between news comments and the number of votes received by news comments, and through the proposed model, the number of votes for news comments can be predicted.