Feature Selection In Text Categorization Research Articles

Feature selection for sentiment analysis using hybrid multiobjective evolutionary algorithm

As the volume of data continues to grow, the significance of text classification is on the rise. This vast amount of data majorly exists in the form of texts. Effective data preparation is essential to extract sentiment data from this vast amount of text, as irrelevant and redundant information can impede valuable insights. Feature selection is an important step in the data preparation phase as it eliminates irrelevant and insignificant features from the huge features set. There exist a large body of work related to feature selection for image processing but limited research is done for text data. While some studies recognize the significance of feature selection in text classification, but there is still need for more efficient sentiment analysis models that optimize feature selection and reduce computational. This manuscript aims to bridge these gaps by introducing a hybrid multi-objective evolutionary algorithm as a feature selection mechanism, combining the power of multiple objectives and evolutionary processes. The approach combines two feature selection techniques within a binary classification model: a filter method, Information Gain (IG), and an evolutionary wrapper method, Binary Multi-Objective Grey Wolf Optimizer (BMOGWO). Experimental evaluations are conducted across six diverse datasets. It achieves a reduction of over 90 percent in feature size while improving accuracy by nearly nine percent. These results showcase the model’s efficiency in terms of computational time and its efficacy in terms of higher classification accuracy which improves sentiment analysis performance. This improvement can be beneficial for various applications, including recommendation systems, reviews analysis, and public opinion observation. However, it’s crucial to acknowledge certain limitations of this study. These encompass the need for broader classifier evaluation, and scalability considerations with larger datasets. These identified limitations serve as directions for future research and the enhancement of the proposed approach.

A novel redistribution-based feature selection for text classification

Text classification is the process of automatically categorizing text documents into predefined labels, gaining increasing importance with the growing volume of data. In the vector space model, terms within documents must be quantified to be used with classifiers. The abundance of unique terms associated with documents can lead to unwieldy vectors. To address this issue, feature selection, reducing the number of terms by removing irrelevant terms, is a common solution. In this paper, a novel feature selection method called Amount of ReDistribution to Establish Neutrality, ARDEN, is proposed for the task of text classification. ARDEN, is designed with statistical distance perspective by measuring the distance of a term to its neutral counterpart, which represents the least distinguishing term having uniform document frequencies across all classes. ARDEN introduces a method to measure the related distance, providing insights into the degree of deviation of the term from neutrality, thus capturing its distinguishing power. ARDEN is experimentally tested against state-of-the-art feature selection methods. The results suggest that it is a competitive feature selection method, exhibiting superior performance in terms of summary statistics, μ and δ′. Furthermore, ARDEN is designed for multi-class text classification problems, which does not require a globalization function. Additionally, the proposed method is compared against Wasserstein statistical distance metric, and clearly outperforms it. Finally, we also propose δ′ evaluation criterion in order to facilitate the interpretation of experimental outcomes across multiple feature sizes.

A COPRAS-based Approach to Multi-Label Feature Selection for Text Classification

In this article, we present a novel approach for text classification feature selection using a pseudo-relation matrix constructed with ridge regression. This approach involves generating a matrix that captures the relationship between features and target variables and then using that matrix to create a multi-criteria decision making (MCDM) problem. We then utilize the Complex Proportional Assessment (COPRAS) method to rank features in order of importance. To assess the performance of our proposed algorithm, we conduct tests on ten real-world text datasets and compare our results to the existing methods. Our findings indicate that the proposed approach outperforms other methods, and we provide statistical significance to support our claims. Our proposed algorithm provides a viable answer for researchers looking for efficient and effective feature selection methods for text data analysis, which can lead to a better understanding and interpretation of the underlying text.

Feature selection in text classification: Identifying spurious words with causal inference methods

As has been scrutinized by many, non-causal model may contain spurious correlations that act like shortcuts during the prediction phase, undermining cross-domain accuracy. This can be caused by biased training data that contains spurious words with neutral meanings yet can induce the model to predict wrongly. Based on this assumption, we propose a series of methods to detect these spurious words before feeding the model with the training data. We used advanced causal inference methods which are arising novas in recent studies, such as propensity score matching and inverse propensity score weighting to facilitate the feature selection before training. We experimented with multiple approaches to estimate propensity scores and got profound improvements. We further experimented with BERT model to evaluate the effectiveness of feature selection and find that the model performance with in-domain and out-of-domain testing samples is boosted after we remove the spurious words detected by our methods in the training data.

Utilizing Artificial Bee Colony Algorithm as Feature Selection Method in Arabic Text Classification

A huge amount of crucial information is contained in documents. The vast increase in the number of E-documents available for user access makes the utilization of automated text classification essential. Classifying or arranging documents into predefined groups is called Text classification. Feature selection (FS) is needed for minimizing the dimensionality of high-dimensional data and extracting only the features that are most pertinent to a particular task. One of the widely used algorithms for feature selection in text classification is the Evolutionary algorithm. In this paper, the filter method chi-square and the Artificial Bee Colony (ABC) algorithm were both used as FS methods. The chi-square method is a useful technique for reducing the number of features and removing those that are superfluous or redundant. The ABC technique considers the chi-square method's chosen features as viable solutions (food sources). The ABC algorithm searches for the most efficient selection of features that increase classification performance. Support Vector Machine and Naïve Bayes classifiers were used as a fitness function for the ABC algorithm. The experiment results demonstrated that the proposed feature selection method was able of decreasing the number of features by approximately 89.5%, and 94%, respectively when NB and SVM were used as fitness functions in comparison to the original dataset, while also enhancing classification performance

An Approach for Movie Review Classification in Turkish

Abstract Web 2.0 has given to all people the right to become a representative of a huge cast of informal media. The importance of this power is getting more evident everyday. Every social media actor can influence the rest of the world by one’s own opinions, feelings, and thoughts generously shared on multiple media. This information belonging to various fields of life can be very handy and be used to one’s advantage, gaining precious experience. One of the greatest problems that this poses is the huge number of data spread everywhere, which are difficult to process as row data per se. Social media and general sentiment text analysis is of much valuable use, accomplishing the task extracting pure gold out of raw mineral. The key point of this investigation is to characterize new reviews automatically. To start with, features selected out of all the word roots appearing in the comments were used to train the system according to known machine learning algorithms. Next, critical words determining positive or negative sense were extracted. Another strategy was attempted eliminating common terms and dealing only with the significant class-determining words to build vocabulary with them. Aparts from linear approach, vector based feature sets were prepared out all or some of the features. The outcomes acquired were analyzed and compared leading to important conclusions, emphasizing the importance of feature selection in text classification.

Feature Selection for Text Classification Using Machine Learning Approaches

In the present scenario, millions of internet users are contributing a huge amount of data in the form of unstructured text documents. In text classification, the high dimensional feature space, noise and irrelevant information of unstructured text documents are reducing the accuracy of text classifier. The feature selection scheme is adopted to address the high dimensional feature space problem of text classification. In this proposed research, a feature selection method based on the term frequency distribution measure is deployed. We have used the Naive Bayes and SVM classifiers with two benchmark datasets (WebKB and BBC). The experimental outcome confirms that the proposed feature selection method has a better classification accuracy when compared with other feature selection techniques.

Robustness and Predictive Performance of Homogeneous Ensemble Feature Selection in Text Classification

The use of ensemble paradigm with classifiers is a proven approach that involves combining the outcomes of several classifiers. It has recently been extrapolated to feature selection methods to find the most relevant features. Earlier, ensemble feature selection has been used in high dimensional, low sample size datasets like bioinformatics. To one's knowledge there is no such endeavor in the text classification domain. In this work, the ensemble feature selection using data perturbation in the text classification domain has been used with an aim to enhance predictability and stability. This approach involves application of the same feature selector to different perturbed versions of training data, obtaining different ranks for a feature. Previous works focus only on one of the metrics, that is, stability or accuracy. In this work, a combined framework is adopted that assesses both the predictability and stability of the feature selection method by using feature selection ensemble. This approach has been explored on univariate and multivariate feature selectors, using two rank aggregators.

An application of MOGW optimization for feature selection in text classification

Due to extensive web applications, sentiment classification (SC) has become a relevant issue of interest among text mining experts. The extensive online reviews prevent the application of effective models to be used in companies and in the decision making of individuals. Pre-processing greatly contributes in sentiment classification. The traditional bag-of-words approaches do not record multiple relationships among words. In this study, emphasis is on the pre-processing stage and data reduction techniques, which would make a big difference in sentiment classification efficiency. To classify opinions, a multi-objective-grey wolf-optimization algorithm is proposed where the two objectives aim for decreasing the error of Naive Bayes and K-nearest neighbour classifiers and a neural network as the final classifier. In evaluating this proposed framework, three datasets are applied. By obtaining 95.76% precision, 95.75% accuracy, 95.99% recall, and 95.82% f-measure, it is evident that this framework outperforms its counterparts.

The effects of globalisation techniques on feature selection for text classification

Text classification (TC) is very important and critical task in the 21th century as there exist high volume of electronic data on the Internet. In TC, textual data are characterised by a huge number of highly sparse features/terms. A typical TC consists of many steps and one of the most important steps is undoubtedly feature selection (FS). In this study, we have comprehensively investigated the effects of various globalisation techniques on local feature selection (LFS) methods using datasets with different characteristics such as multi-class unbalanced (MCU), multi-class balanced (MCB), binary-class unbalanced (BCU) and binary-class balanced (BCB). The globalisation techniques used in this study are summation (SUM), weighted-sum (AVG), and maximum (MAX). To investigate the effect of globalisation techniques, we used three LFS methods named as Discriminative Feature Selection (DFSS), odds ratio (OR) and chi-square (CHI2). In the experiments, we have utilised four different benchmark datasets named as Reuters-21578, 20Newsgroup., Enron1, and Polarity in addition to Support Vector Machines (SVM) and Decision Tree (DT) classifiers. According to the experimental results, the most successful globalisation technique is AVG while all situations are taken into account. The experimental results indicate that DFSS method is more successful than OR and CHI2 methods on datasets with MCU and MCB characteristics. However, CHI2 method seems more accurate than OR and DFSS methods on datasets with BCU and BCB characteristics. Also, SVM classifier performed better than DT classifier in most cases.

Generalized Term Similarity for Feature Selection in Text Classification Using Quadratic Programming

The rapid growth of Internet technologies has led to an enormous increase in the number of electronic documents used worldwide. To organize and manage big data for unstructured documents effectively and efficiently, text categorization has been employed in recent decades. To conduct text categorization tasks, documents are usually represented using the bag-of-words model, owing to its simplicity. In this representation for text classification, feature selection becomes an essential method because all terms in the vocabulary induce enormous feature space corresponding to the documents. In this paper, we propose a new feature selection method that considers term similarity to avoid the selection of redundant terms. Term similarity is measured using a general method such as mutual information, and serves as a second measure in feature selection in addition to term ranking. To consider balance of term ranking and term similarity for feature selection, we use a quadratic programming-based numerical optimization approach. Experimental results demonstrate that considering term similarity is effective and has higher accuracy than conventional methods.

Utility-based feature selection for text classification

Feature selection is a significant step before a classification task used to reduce excessive computational costs and enhance classification performance. This paper illustrates a novel feature selection method based on the concept of utility that is grounded in economics theory. In particular, we focus on a utility-based feature selection method for enhancing text classification. Different from existing feature selection methods, the proposed method selects discriminative semantic terms according to how authors utilize terms to express the main ideas in textual documents, i.e., the “utility of terms,” a criteria that can be used to measure the usefulness of terms on expressing authors’ main ideas. To our best knowledge, our work represents the successful research on the leveraging economics theory for developing a semantically rich feature selection method to improve text classification. Our empirical tests based on six UCI benchmark datasets confirm that the proposed method often outperforms other state-of-the-art feature selection methods in text classification. Moreover, our method provides an economics explanation of term weighting for information retrieval and semantic information acquisition in textual documents.

Supervised Hebb rule based feature selection for text classification

Text documents usually contain high dimensional non-discriminative (irrelevant and noisy) terms which lead to steep computational costs and poor learning performance of text classification. One of the effective solutions for this problem is feature selection which aims to identify discriminative terms from text data. This paper proposes a method termed “Hebb rule based feature selection (HRFS)”. HRFS is based on supervised Hebb rule and assumes that terms and classes are neurons and select terms under the assumption that a term is discriminative if it keeps “exciting” the corresponding classes. This assumption can be explained as “a term is highly correlated with a class if it is able to keep “exciting” the class according to the original Hebb postulate. Six benchmarking datasets are used to compare HRFS with other seven feature selection methods. Experimental results indicate that HRFS is effective to achieve better performance than the compared methods. HRFS can identify discriminative terms in the view of synapse between neurons. Moreover, HRFS is also efficient because it can be described in the view of matrix operation to decrease complexity of feature selection.

Feature selection for text classification: A review

Big multimedia data is heterogeneous in essence, that is, the data may be a mixture of video, audio, text, and images. This is due to the prevalence of novel applications in recent years, such as social media, video sharing, and location based services (LBS), etc. In many multimedia applications, for example, video/image tagging and multimedia recommendation, text classification techniques have been used extensively to facilitate multimedia data processing. In this paper, we give a comprehensive review on feature selection techniques for text classification. We begin by introducing some popular representation schemes for documents, and similarity measures used in text classification. Then, we review the most popular text classifiers, including Nearest Neighbor (NN) method, Naive Bayes (NB), Support Vector Machine (SVM), Decision Tree (DT), and Neural Networks. Next, we survey four feature selection models, namely the filter, wrapper, embedded and hybrid, discussing pros and cons of the state-of-the-art feature selection approaches. Finally, we conclude the paper and give a brief introduction to some interesting feature selection work that does not belong to the four models.

An efficient automatic multiple objectives optimization feature selection strategy for internet text classification

Research on feature selection in text classification is usually limited to propose various techniques to select a set of features with highest scores based on different metrics. The selected features are usually determined by using a separate validation dataset with a fixed threshold. Obviously, it may not generalize well to new data as the best number for selected features is various on different datasets. In this paper, we first conduct a deep analysis, and find that simply extracting the features based on the score calculated by a metric may not always be the best strategy as it may turn many documents into zero length, which make them not suitable for training. We then model the feature selection process as a multiple objectives optimization problem to gain the best number of selected features rationally and automatically. In addition, as the optimization process costs a lot of resources, we design a parallel algorithm to improve the running time using dynamic programming. Extensive experiments are performed on several popular datasets, and the results indicate that our proposed approach is effective and feasible.

An alternative framework for univariate filter based feature selection for text categorization

In this paper, we introduce an alternative framework for selecting a most relevant subset of the original set of features for the purpose of text categorization. Given a feature set and a local feature evaluation function (such as chi-square measure, mutual information etc.,) the proposed framework ranks the features in groups instead of ranking individual features. A group of features with rth rank is more powerful than the group of features with (r+1)th rank. Each group is made up of a subset of features which are supposed to be capable of discriminating every class from every other class. The added advantage of the proposed framework is that it automatically eliminates the redundant features while selecting features without requirement of study of features in combination. Further the proposed framework also helps in handling overlapping classes effectively through selection of low ranked yet powerful features. An extensive experimentation has been conducted on three benchmarking datasets using four different local feature evaluation functions with Support Vector Machine and Naïve Bayes classifiers to bring out the effectiveness of the proposed framework over the respective conventional counterparts.

A modified multi objective heuristic for effective feature selection in text classification

Text categorization is the process of sorting text documents into one or more predefined categories or classes of similar documents. Differences in the results of such categorization arise from the feature set chosen to base the association of a given document with a given category. This process is challenging mainly because there can be large number of discriminating words which render many of the current algorithms unable to complete this. For most of these tasks there exist both relevant as well as irrelevant features. The objective here is to bring about a text classification on the basis of the features selected and also pre-processing to bring down the dimensionality and increase the accuracy of classification of the feature vector. Here the most commonly used methods are meta-heuristic algorithms in order to facilitate selection. Artificial fish swarm algorithm (AFSA) takes the underlying intelligence of the behaviour of fish swarming to combat the problems of optimization as well as the combinatorial problems. This method has been greatly successful in diverse applications but does suffer from certain limitations like not having multiplicity. Therefore, a modification has been proposed to AFSA which is MAFSA that has a crossover in its operation in order to bring about an improvement in the text classification selection. SVM or Support Vector Machine, Adaboost classifiers and naive bayes are all used here. MAFSA has proved itself to be superior to AFSA in terms of precision and also the selected feature numbers.

Feature selection based on a normalized difference measure for text classification

The goal of feature selection in text classification is to choose highly distinguishing features for improving the performance of a classifier. The well-known text classification feature selection metric named balanced accuracy measure (ACC2) (Forman, 2003) evaluates a term by taking the difference of its document frequency in the positive class (also known as true positives) and its document frequency in the negative class (also known as false positives). This however results in assigning equal ranks to terms having equal difference, ignoring their relative document frequencies in the classes. In this paper we propose a new feature ranking (FR) metric, called normalized difference measure (NDM), which takes into account the relative document frequencies. The performance of NDM is investigated against seven well known feature ranking metrics including odds ratio (OR), chi squared (CHI), information gain (IG), distinguishing feature selector (DFS), gini index (GINI) ,balanced accuracy measure (ACC2) and Poisson ratio (POIS) on seven datasets namely WebACE(WAP,K1a,K1b), Reuters (RE0, RE1),spam email dataset and 20 newsgroups using the multinomial naive Bayes (MNB) and supports vector machines (SVM) classifiers. Our results show that the NDM metric outperforms the seven metrics in 66% cases in terms of macro-F1 measure and in 51% cases in terms of micro F1 measure in our experimental trials on these datasets.

FEATURE SELECTION USING SINGULAR VALUE DECOMPOSITION AND ORTHOGONAL CENTROID FEATURE SELECTION FOR TEXT CLASSIFICATION

Text mining is a narrow research field of data mining, which focuses on discovering new information from text document collections, mainly by using techniques from data mining, machine learning, natural language processing and information retrieval. Text classification is the process of analyzing text content and then giving decision whether this text can belong to one group, many groups or it does not belong to the text group which is defined before. On over the world, there have been many effective researches on this problem, especially on texts in English. However, there have been few researches on Vietnamese texts. Moreover, these researching results and applications are still limited partly due to the typical characteristics of Vietnamese language in term of words and sentences and there are many words with many meanings in many different contexts. Text classification problem is the one with many featues, thus to improve the effectiveness of text classification is the aim of may researchers. In this research, the author constructs two methods of feature selection: singular value decomposition and optimal orthogonal centroid feature selection in text classification with high efficiency of calculation proven on English text document and now they are proven on Vietnamese text document. There are many classification techniques, but we implemented on the learning machine algorithms support vector machines. This method has been proven to be effective for text classification problems. With the technique of feature selection singular value decomposition and optimal orthogonal centroid feature selection, the implementing result higher than that of traditional method.

Feature Extraction or Feature Selection for Text Classification: A Case Study on Phishing Email Detection

Dimensionality reduction is generally performed when high dimensional data like text are classified. This can be done either by using feature extraction techniques or by using feature selection techniques. This paper analyses which dimension reduction technique is better for classifying text data like emails. Email classification is difficult due to its high dimensional sparse features that affect the generalization performance of classifiers. In phishing email detection, dimensionality reduction techniques are used to keep the most instructive and discriminative features from a collection of emails, consists of both phishing and legitimate, for better detection. Two feature selection techniques - Chi-Square and Information Gain Ratio and two feature extraction techniques - Principal Component Analysis and Latent Semantic Analysis are used for the analysis. It is found that feature extraction techniques offer better performance for the classification, give stable classification results with the different number of features chosen, and robustly keep the performance over time.