Performance Of Text Classification Research Articles

A comparative study of automated legal text classification using random forests and deep learning

Automated legal text classification is a prominent research topic in the legal field. It lays the foundation for building an intelligent legal system. Current literature focuses on international legal texts, such as Chinese cases, European cases, and Australian cases. Little attention is paid to text classification for U.S. legal texts. Deep learning has been applied to improving text classification performance. Its effectiveness needs further exploration in domains such as the legal field. This paper investigates legal text classification with a large collection of labeled U.S. case documents through comparing the effectiveness of different text classification techniques. We propose a machine learning algorithm using domain concepts as features and random forests as the classifier. Our experiment results on 30,000 full U.S. case documents in 50 categories demonstrated that our approach significantly outperforms a deep learning system built on multiple pre-trained word embeddings and deep neural networks. In addition, applying only the top 400 domain concepts as features for building the random forests could achieve the best performance. This study provides a reference to select machine learning techniques for building high-performance text classification systems in the legal domain or other fields.

Mining patterns of autonomous vehicle crashes involving vulnerable road users to understand the associated factors

Autonomous or automated vehicles (AVs) have the potential to improve traffic safety by eliminating majority of human errors. As the interest in AV deployment increases, there is an increasing need to assess and understand the expected implications of AVs on traffic safety. Until recently, most of the literature has been based on either survey questionnaires, simulation analysis, virtual reality, or simulation to assess the safety benefits of AVs. Although few studies have used AV crash data, vulnerable road users (VRUs) have not been a topic of interest. Therefore, this study uses crash narratives from four-year (2017–2020) of AV crash data collected from California to explore the direct and indirect involvement of VRUs. The study applied text network and compared the text classification performance of four classifiers - Support Vector Machine (SVM), Naïve Bayes (NB), Random Forest (RF), and Neural Network (NN) and associated performance metrics to attain the objective. It was found that out of 252 crashes, VRUs were, directly and indirectly, involved in 23 and 12 crashes, respectively. Among VRUs, bicyclists and scooterists are more likely to be involved in the AV crashes directly, and bicyclists are likely to be at fault, while pedestrians appear more in the indirectly involvements. Further, crashes that involve VRUs indirectly are likely to occur when the AVs are in autonomous mode and are slightly involved minor damages on the rear bumper than the ones that directly involve VRUs. Additionally, feature importance from the best performing classifiers (RF and NN) revealed that crosswalks, intersections, traffic signals, movements of AVs (turning, slowing down, stopping) are the key predictors of the VRUs-AV related crashes. These findings can be helpful to AV operators and city planners.

Graph Fusion Network for Text Classification

Text classification is an important and classical problem in natural language processing. Recently, Graph Neural Networks (GNNs) have been widely applied in text classification and achieved outstanding performance. Despite the success of GNNs on text classification, existing methods are still limited in two main aspects. On the one hand, transductive methods cannot easily adapt to new documents. Since transductive methods incorporate all documents into their text graph, they need to reconstruct the whole graph and retrain their system from scratch when new documents come. However, this is not applicable to real-world situations. On the other hand, many state-of-the-art algorithms ignore the quality of text graphs, which may lead to sub-optimal performance. To address these problems, we propose a Graph Fusion Network (GFN), which can overcome these limitations and boost text classification performance. In detail, in the graph construction stage, we build homogeneous text graphs with word nodes, which makes the learning system capable of making inference on new documents without rebuilding the whole text graph. Then, we propose to transform external knowledge into structural information and integrate different views of text graphs to capture more structural information. In the graph reasoning stage, we divide the process into three steps: graph learning, graph convolution, and graph fusion. In the graph learning step, we adopt a graph learning layer to further adapt text graphs. In the graph fusion step, we design a multi-head fusion module to integrate different opinions. Experimental results on five benchmarks demonstrate the superiority of our proposed method.

Perbandingan Metode TF-ABS dan TF-IDF Pada Klasifikasi Teks Helpdesk Menggunakan K-Nearest Neighbor

Distribution of tickets to the destination unit is a very important function in the helpdesk application, but the process of distributing tickets manually by admin officers has drawbacks, namely ticket distribution errors can occur and increase ticket completion time if the number of tickets is large. Helpdesk text classification becomes important to automatically distribute tickets to the appropriate destination units in a short time. This study was conducted to compare the performance of helpdesk text classification at the Directorate General of State Assets of the Ministry of Finance using the K-Nearest Neighbor (KNN) method with the TF-ABS and TF-IDF weighting methods. The research was conducted by collecting complaint documents, preprocessing, word weighting, feature reduction, classification, and testing. Classification using KNN with parameters n_neighbor (k) namely k=1, k=3, k=5, k=7, k=9, k=11, k=13, k=15, k=17, and k=19 to classify 10,537 helpdesk texts into 8 categories. The test uses a confusion matrix based on the accuracy value and score-f1. The test results show that the TF-ABS weighting method is better than TF-IDF with the highest accuracy value of 90.04% at 15% and k=3.

An improved supervised term weighting scheme for text representation and classification

Term weighting scheme has significant effects on the text classification performance. The main reason is that in text classification tasks, term weighting scheme determines the way in which texts are represented in the vector space model. Currently, term frequency-inverse document frequency is the most widely utilized term weighting scheme but it does not use the available category information of the training texts. Taking this resource of category information (or category factor) into account in the study, an improved supervised term weighting method for representing text is developed, which combines a new measure of information namely cumulative residual entropy and the proportional distortion function. To verify the text classification performance of our proposed scheme, we conducted an extensive experimental comparison of proposed scheme with existing schemes on two corpora (i.e., Reuters-21578 and 20 Newsgroups datasets) with different characteristics. Results explicitly show that our proposed scheme can obtain significantly better effect for text classification than others. Specifically, when linear support vector machine classifier is run, performances were improved to 0.972 and 0.833 (micro-F1) on Reuters-21578 dataset and 20 Newsgroups dataset, respectively.

Sentiment Classification of News Text Data Using Intelligent Model

Text sentiment classification is a fundamental sub-area in natural language processing. The sentiment classification algorithm is highly domain-dependent. For example, the phrase “traffic jam” expresses negative sentiment in the sentence “I was stuck in a traffic jam on the elevated for 2 h.” But in the domain of transportation, the phrase “traffic jam” in the sentence “Bread and water are essential terms in traffic jams” is without any sentiment. The most common method is to use the domain-specific data samples to classify the text in this domain. However, text sentiment analysis based on machine learning relies on sufficient labeled training data. Aiming at the problem of sentiment classification of news text data with insufficient label news data and the domain adaptation of text sentiment classifiers, an intelligent model, i.e., transfer learning discriminative dictionary learning algorithm (TLDDL) is proposed for cross-domain text sentiment classification. Based on the framework of dictionary learning, the samples from the different domains are projected into a subspace, and a domain-invariant dictionary is built to connect two different domains. To improve the discriminative performance of the proposed algorithm, the discrimination information preserved term and principal component analysis (PCA) term are combined into the objective function. The experiments are performed on three public text datasets. The experimental results show that the proposed algorithm improves the sentiment classification performance of texts in the target domain.

A Novel Classification Model SA-MPCNN for Power Equipment Defect Text

The text classification of power equipment defect is of great significance to equipment health condition evaluation and power equipment maintenance decisions. Most of the existing classification methods do not sufficiently consider the semantic relation between words in the same sentence and cannot extract deep semantic features. To tackle those problems, this article proposes a novel classification method by combining the self-attention mechanism and multi-channel pyramid convolution neural networks. We utilize the bidirectional gated recurrent unit to model the text sequence and, on this basis, improve self-attention layer to dot multiplication on the forward and backward features to obtain the global attention score. Thereby, effective features are enhanced, invalid features are weakened, and important text representation vectors are obtained. To solve the problem that the shallow network structure cannot extract deep semantic features, we design a multi-channel pyramid convolution network, which first extracts deep text features from the channels of different windows and then fuses the text features of each channel. By comparing with the state-of-the-art methods, the model in this article has better performance in text classification of power equipment defects.

GL-GCN: Global and Local Dependency Guided Graph Convolutional Networks for aspect-based sentiment classification

Aspect-based sentiment classification, which aims at identifying the sentiment polarity of a sentence towards the specified aspect, has become a crucial task for sentiment analysis. Existing methods have proposed effective models and achieved satisfactory results, but they mainly focus on exploiting local structure information of a given sentence, such as locality, sequentiality or syntactical dependency constraints within the sentence. Recently, some research works, which utilizes global dependency information, has attracted increasing interest and significantly boosts the performance of text classification. In this paper, we simultaneously introduce both global structure information and local structure information into the task of aspect-based sentiment classification, and propose a novel aspect-based sentiment classification approach, i.e., Global and Local Dependency Guided Graph Convolutional Networks (GL-GCN). In particular, we exploit the syntactic dependency structure as well as sentence sequential information (e.g., the output of BiLSTM) to mine the local structure information of a sentence. On the other hand, we construct a word-document graph using the entire corpus to reveal the global dependency information between words. In addition, an attention mechanism is leveraged to effectively fuse both global and local dependency structure signals. Extensive experiments are conducted on five benchmark datasets in terms of both Accuracy and F1-Score, and the results illustrate that our proposed framework outperforms state-of-the-art methods for aspect-based sentiment classification. The model is implemented using PyTorch and is trained on GPU GeForce GTX 2080 Ti.

Enhancing text classification performance by preprocessing misspelled words in Indonesian language

Supervised learning using shallow machine learning methods is still a popular method in processing text, despite the rapidly advancing sector of unsupervised methodologies using deep learning. Supervised text classification for application user feedback sentiments in Indonesian Language is one of the applications which is quite popular in both the research community and industry. However, due to the nature of shallow machine learning approaches, various text preprocessing techniques are required to clean the input data. This research aims to implement and evaluate the role of Levenshtein distance algorithm in detecting and preprocessing misspelled words in Indonesian language, before the text data is then used to train a user feedback sentiment classification model using multinomial Naïve Bayes. This research experimented with various evaluation scenarios, and found that preprocessing misspelled words in Indonesian language using the Levenshtein distance algorithm could be useful and showed a promising 8.2% increase on the accuracy of the model’s ability to classify user feedback text according to their sentiments.

CrowdTC: Crowd-powered Learning for Text Classification

Text classification is a fundamental task in content analysis. Nowadays, deep learning has demonstrated promising performance in text classification compared with shallow models. However, almost all the existing models do not take advantage of the wisdom of human beings to help text classification. Human beings are more intelligent and capable than machine learning models in terms of understanding and capturing the implicit semantic information from text. In this article, we try to take guidance from human beings to classify text. We propose Crowd-powered learning for Text Classification (CrowdTC for short). We design and post the questions on a crowdsourcing platform to extract keywords in text. Sampling and clustering techniques are utilized to reduce the cost of crowdsourcing. Also, we present an attention-based neural network and a hybrid neural network to incorporate the extracted keywords as human guidance into deep neural networks. Extensive experiments on public datasets confirm that CrowdTC improves the text classification accuracy of neural networks by using the crowd-powered keyword guidance.

Text classification using improved bidirectional transformer

AbstractText data have an important place in our daily life. A huge amount of text data is generated everyday. As a result, automation becomes necessary to handle these large text data. Recently, we are witnessing important developments with the adaptation of new approaches in text processing. Attention mechanisms and transformers are emerging as methods with significant potential for text processing. In this study, we introduced a bidirectional transformer (BiTransformer) constructed using two transformer encoder blocks that utilize bidirectional position encoding to take into account the forward and backward position information of text data. We also created models to evaluate the contribution of attention mechanisms to the classification process. Four models, including long short term memory, attention, transformer, and BiTransformer, were used to conduct experiments on a large Turkish text dataset consisting of 30 categories. The effect of using pretrained embedding on models was also investigated. Experimental results show that the classification models using transformer and attention give promising results compared with classical deep learning models. We observed that the BiTransformer we proposed showed superior performance in text classification.

Text Mining in Cybersecurity

The growth of data volume has changed cybersecurity activities, demanding a higher level of automation. In this new cybersecurity landscape, text mining emerged as an alternative to improve the efficiency of the activities involving unstructured data. This article proposes a Systematic Literature Review ( SLR ) to present the application of text mining in the cybersecurity domain. Using a systematic protocol, we identified 2,196 studies, out of which 83 were summarized. As a contribution, we propose a taxonomy to demonstrate the different activities in the cybersecurity domain supported by text mining. We also detail the strategies evaluated in the application of text mining tasks and the use of neural networks to support activities involving unstructured data. The work also discusses text classification performance aiming its application in real-world solutions. The SLR also highlights open gaps for future research, such as the analysis of non-English content and the intensification in the usage of neural networks.

Hybrid Chinese text classification model based on pretraining model

In order to solve the problems of fuzzy boundary of Chinese words, polysemy of one word and the inability of traditional models to reflect the importance of each word in the text, a hybrid text classification model based on Ernie CNN and bilstm attention (mecba) is proposed. Firstly, Ernie model is used to generate the corresponding word vector, which can retain rich semantic information and enhance the semantic representation of words. Then, the word vector is input into CNN to extract local information, and the context feature is extracted by bilstm attention, and the output of the two is spliced; Finally, softmax classifier is used for classification. The experimental results show that the model is better than CNN, bilstm and other classification models, and can effectively improve the performance of Chinese text classification.

A Novel Optimized Convolutional Neural Network Based On Attention Pooling for Text Classification

Convolutional neural network (CNN) is widely used in Natural Language Processing (NLP) and has achieved relatively good results. Since the convolutional layer of CNN can be initialized by encoding important semantic features, CNN can extract important semantic features in the convolutional layer. The pooling layer of CNN uses the feature of max-pooling to filter the features, but it does not consider the key information in the sentence and the context semantic information in the text. Therefore, a novel neural network algorithm combining with the attention mechanism and the pooling layer is proposed in this paper to make the model pay attention to the keywords in the sentence and automatically maintain the most meaningful text message, thereby improving the performance of text classification. Our paper uses important information features to initialize the convolution filter so that the model can notice the important semantic features. The resulting feature is enhanced by combining the attention mechanism in the pooling layer, so that the model can maintain the important information features of the text. Experiments show that the proposed model achieves excellent results for multiple text classification tasks, including sentiment classification and topic classification.

Identifying Emotion on Indonesian Tweets using Convolutional Neural Networks

especially with the advancement of deep learning methods for text classification. Despite some effort to identify emotion on Indonesian tweets, its performance evaluation results have not achieved acceptable numbers. To solve this problem, this paper implements a classification model using a convolutional neural network (CNN), which has demonstrated expected performance in text classification. To easily compare with the previous research, this classification is performed on the same dataset, which consists of 4,403 tweets in Indonesian that were labeled using five different emotion classes: anger, fear, joy, love, and sadness. The performance evaluation results achieve the precision, recall, and F1-score at respectively 90.1%, 90.3%, and 90.2%, while the highest accuracy achieves 89.8%. These results outperform previous research that classifies the same classification on the same dataset.

Transfer Learning for Classifying Spanish and English Text by Clinical Specialties.

Transfer learning has demonstrated its potential in natural language processing tasks, where models have been pre-trained on large corpora and then tuned to specific tasks. We applied pre-trained transfer models to a Spanish biomedical document classification task. The main goal is to analyze the performance of text classification by clinical specialties using state-of-the-art language models for Spanish, and compared them with the results using corresponding models in English and with the most important pre-trained model for the biomedical domain. The outcomes present interesting perspectives on the performance of language models that are pre-trained for a particular domain. In particular, we found that BioBERT achieved better results on Spanish texts translated into English than the general domain model in Spanish and the state-of-the-art multilingual model.

History-based attention in Seq2Seq model for multi-label text classification

Multi-label text classification is an important yet challenging task in natural language processing. It is more complex than single-label text classification in that the labels tend to be correlated. To capture this complex correlations, sequence to sequence model has been widely applied, and achieved impressing performance for multi-label text classification. It encodes each document as contextual representations, and then decodes them to generate labels one by one. At each time step, the decoder usually adopts the attention mechanism to highlight important contextual representations to predict a related label, which has been proved to be effective. Nevertheless, the traditional attention approaches only utilize a hidden state to explore such contextual representations, which may result in prediction errors, or omit several trivial labels.To tackle this problem, in this paper, we propose “history-based attention”, which takes history information into consideration, to effectively explore informative representations for labels’ predictions in multi-label text classification. Our approach consists of two parts: history-based context attention and history-based label attention. History-based context attention considers historical weight trends to highlight important context words, which is helpful to predict trivial labels. History-based label attention explores historical labels to alleviate the error propagation problem. We conduct experiments on two popular text datasets (i.e., Arxiv Academic Paper Dataset and Reuters Corpus Volume I), it is demonstrated that the history-based attention mechanism could boost the performance to a certain extent, and the proposed method consistently outperforms highly competitive approaches.

A compact multi-pattern encoding descriptor for texture classification

Binary pattern family is considered as a powerful tool for visual texture classification. Most popular methods improve the classification performance by multi-feature fusion. However, many sub-features are redundant and low-discriminative and the classification system has high computational complexity and unsatisfactory results. To handle above problems, this paper proposes a compact multi-pattern encoding descriptor for visual texture classification. First, we develop local extremum patterns and local center pattern to represent the neighborhood intensity changes. Then, we design a compact encoding scheme to encode local maximum, minimum and center patterns into a three-bit binary code, named MMC pattern. Finally, a compact multi-pattern encoding descriptor is proposed by combining the traditional local sign pattern and MMC pattern. Experimental results on five representative texture databases demonstrate that our method achieves the state-of-the-art texture classification performance.

Ensemble approach for web page classification

Over the decades World Wide Web has become abundance source of distributed web content repository hyper-linked with diverse information domains. Performance of search engines in locating the information is exemplary but still there is inadequacy in search engines for focused crawling of web content. Web Page Classification being pivotal for information retrieval and management task plays imperative role for natural language processing in creating classified web document repositories and building indexed web directories. The conventional machine learning approaches extract the desired features from web pages in order to classify them whereas deep leaning algorithms learns the covet features as the network goes deeper and deeper. Transfer learning based Pre-trained models such as BERT attains impressive performance for text classification. In this study, we evaluate the effectiveness of adopting pre-trained model BERT for the task of classifying web pages into different categories. In this paper, we proposed an ensemble approach for web page classification by learning contextual representation using pre-trained bidirectional BERT and then applying deep Inception modelling with Residual connections for fine-tunes the target task by utilizing parallel multi-scale semantics. Experimental evaluation exhibit that proposed ensemble model outperforms benchmark baselines and achieve better performance in contrast to other transfer learning approaches evaluated on the web page classification task for different classification datasets.

New Weighted Mean-Based Patterns for Texture Analysis and Classification

ABSTRACT This paper presents anovel method for texture classification and biometric authentication based on a descriptor called the weighted mean-based patterns (WEM). The proposed descriptor has been developed for extracting texture features from a large dataset of hand images, which has been created by the authors. The method uses the distinctive features of finger knuckle print (FKP) for hand image retrieval, which can be used in biometric identity recognition systems. The proposed method also includes a feature selection step for eliminating less important patterns and a weighted distance measure for quantifying the similarity of images. The method uses the support vector machine (SVM) for the classification stage. The proposed method has been tested on the FKP image dataset to evaluate the image retrieval performance, and also on Brodatz, Vistex, and Stex datasets to evaluate the performance for texture classification. Higher performance of the proposed method is demonstrated through comparison with other methods. The proposed method is shown to be sufficiently precise for a variety of applications, including identity recognition and classification.