Text Classification Problem Research Articles

Evolutionary Design of Problem-Adapted Image Descriptors for Texture Classification

Effective texture classification requires image descriptors capable of efficiently detecting, extracting, and describing the most relevant information in the images, so that, for instance, different texture classes can be distinguished despite image distortions such as varying illuminations, viewpoints, scales, and rotations. Designing such an image descriptor is a challenging task that typically involves the intervention of human experts. In this paper, a general method to automatically design effective image descriptors is proposed. Our method is based on grammatical evolution and, using a set of example images from a texture classification problem and a classification algorithm as inputs, generates problem-adapted image descriptors that achieve very competitive classification results. Our method is tested on five well-known texture data sets with different number of classes and image distortions to prove its effectiveness and robustness. Our classification results are statistically compared against those obtained by means of six popular hand-crafted texture descriptors in the state of the art. This statistical analysis shows that our evolutionarily designed descriptors outperform most of those designed by human experts.

Automatic classification of Lithuanian parliament bills

Quantitative methods are becoming more and more important in political science. However, they are not applicable without computers and computer based systems. In this paper we apply natural language technologies, mainly text classification, to categorise bills of the Lithuanian parliament into the predefined groups for further use in voting analysis and in other text analytic tasks. As only the titles of bills were used, in general it can be claimed that the problem of short text classification, which is poorly explored in consideration with the Lithuanian language, is addressed in this study.

On the Distinction of Subjectivity and Objectivity of Emotions in Texts

Emotion classification in texts is an instance of the text classification problem. It therefore could apply some existing text classifiers by considering each emotion as a label of the text. However, most of recent works does not differentiate the subjectivity and objectivity of the same emotion in the text. This paper firstly builds some datasets whose labels are emotion, in which the subject and object of the same emotion are considered as two separated labels. Secondly, this paper evaluates some existing classifiers via some scenarios on the built datasets. The results are then discussed on some difficulties of these kinds of problem.

Chinese Short Text Categorization Based on Semi-Supervised Learning

Most of the text on the Internet is unlabelled with the rapid development of the Internet, and it is difficult for us to classify the unlabelled text accurately under the condition of insufficient labelled samples. Sei-supervised learning is a method, which combines the labelled samples with the unlabelled samples, can solve the problem in a better way. AdaBoost is one of the most representative algorithm of boosting algorithms, and this paper used the improved decision tree to be weak classifiers of the AdaBoost. Based on this, this paper devised a boosting algorithm which was based on semi-supervised learning and the improved decision tree. The algorithm is devoted to solving the problem of the Chinese short text categorization under the condition of insufficient labelled samples. Experiments show that the algorithm can effectively improve the performance of the Chinese short text categorization on balanced and imbalanced data sets.

A Selective Multiple Instance Transfer Learning Method for Text Categorization Problems

Multiple instance learning (MIL) is a generalization of supervised learning which attempts to learn a distinctive classifier from bags of instances. This paper addresses the problem of the transfer learning-based multiple instance method for text categorization problem. To provide a safe transfer of knowledge from a source task to a target task, this paper proposes a new approach, called selective multiple instance transfer learning (SMITL), which selects the case that the multiple instance transfer learning will work in step one, and then builds a multiple instance transfer learning classifier in step two. Specifically, in the first step, we measure whether the source task and the target task are related or not by investigating the similarity of the positive features of both tasks. In the second step, we construct a transfer learning-based multiple instance method to transfer knowledge from a source task to a target task if both tasks are found to be related in the first step. Our proposed approach explicitly addresses the problem of safe transfer of knowledge for multiple instance learning on the text classification problem. Extensive experiments have shown that SMITL can determine whether the two tasks are related for most data sets, and outperforms classic multiple instance learning methods.

Vertical Ensemble Co-Training for Text Classification

High-quality, labeled data is essential for successfully applying machine learning methods to real-world text classification problems. However, in many cases, the amount of labeled data is very small compared to that of the unlabeled, and labeling additional samples could be expensive and time consuming. Co-training algorithms, which make use of unlabeled data to improve classification, have proven to be very effective in such cases. Generally, co-training algorithms work by using two classifiers, trained on two different views of the data, to label large amounts of unlabeled data. Doing so can help minimize the human effort required for labeling new data, as well as improve classification performance. In this article, we propose an ensemble-based co-training approach that uses an ensemble of classifiers from different training iterations to improve labeling accuracy. This approach, which we call vertical ensemble , incurs almost no additional computational cost. Experiments conducted on six textual datasets show a significant improvement of over 45% in AUC compared with the original co-training algorithm.

How to use negative class information for Naive Bayes classification

The Naive Bayes (NB) classifier is a popular classifier for text classification problems due to its simple, flexible framework and its reasonable performance. In this paper, we present how to effectively utilize negative class information to improve NB classification. As opposed to information retrieval, supervised learning based text classification already obtains class information, a negative class as well as a positive class, from a labeled training dataset. Since the negative class can also provide significant information to improve the NB classifier, the negative class information is applied to the NB classifier through two phases of indexing and class prediction tasks. As a result, the new classifier using the negative class information consistently performs better than the traditional multinomial NB classifier.

Automatic classification of RDoC positive valence severity with a neural network

ObjectiveOur objective was to develop a machine learning-based system to determine the severity of Positive Valance symptoms for a patient, based on information included in their initial psychiatric evaluation. Severity was rated on an ordinal scale of 0–3 as follows: 0 (absent=no symptoms), 1 (mild=modest significance), 2 (moderate=requires treatment), 3 (severe=causes substantial impairment) by experts. Materials and methodsWe treated the task of assigning Positive Valence severity as a text classification problem. During development, we experimented with regularized multinomial logistic regression classifiers, gradient boosted trees, and feedforward, fully-connected neural networks. We found both regularization and feature selection via mutual information to be very important in preventing models from overfitting the data. Our best configuration was a neural network with three fully connected hidden layers with rectified linear unit activations. ResultsOur best performing system achieved a score of 77.86%. The evaluation metric is an inverse normalization of the Mean Absolute Error presented as a percentage number between 0 and 100, where 100 means the highest performance. Error analysis showed that 90% of the system errors involved neighboring severity categories. ConclusionMachine learning text classification techniques with feature selection can be trained to recognize broad differences in Positive Valence symptom severity with a modest amount of training data (in this case 600 documents, 167 of which were unannotated). An increase in the amount of annotated data can increase accuracy of symptom severity classification by several percentage points. Additional features and/or a larger training corpus may further improve accuracy.

Classification of text documents based on score level fusion approach

Text document classification is a well known theme in the field of the information retrieval and text mining. Selection of most desired features in the text document plays a vital role in classification problem. This research article addresses the problem of text classification by considering Sentence–Vector Space Model (S-VSM) and Unigram representation models for the text document. An enhanced S-VSM model will be considered for the constructive representation of text documents. A neural network based representation for text documents is proposed for effective capturing of semantic information of the text data. Two different classifiers are designed based on the two different representation models of the text documents. Score level fusion is applied on two proposed models to find out the overall accuracy of the proposed model. Key contributions of the paper are an enhanced S-VSM model, an interval valued representation model for the proposed S-VSM approach. A word level representation model for semantic information preserving of the text document and score level fusion approach.

Intention Classification for Retrieval of Health Questions

Healthcare professionals have edited many health questions (HQs) and their answers for healthcare consumers on the Internet. The HQs provide both readable and reliable health information, and hence retrieval of those HQs that are relevant to a given question is essential for health education and promotion through the Internet. However, retrieval of relevant HQs needs to be based on the recognition of the intention of each HQ, which is difficult to be done by predefining syntactic and semantic rules. We thus model the intention recognition problem as a text classification problem, and develop two techniques to improve a learning-based text classifier for the problem. The two techniques improve the classifier by location-based and area-based feature weightings, respectively. Experimental results show that, the two techniques can work together to significantly improve a Support Vector Machine classifier in both the recognition of HQ intentions and the retrieval of relevant HQs. URL: http://ijkcdt.net/xml/09914/09914.pdf

Modelling of spatial causality among distinctive properties of an image using conditional random field for image classification

ABSTRACTIn this paper, we proposed an ordered patch-based method using conditional random field (CRF) in order to encode local properties and their spatial relationship in the images to address texture classification, face recognition and scene classification problems. Typical image classification approaches classify images without considering spatial causality among distinctive properties of an image to represent it in the feature space. In this method first, each image is encoded as a sequence of ordered patches including local properties. Second, the sequence of these ordered patches is modelled as a probabilistic feature vector using CRF to model spatial relationship of these local properties; and finally, image classification is performed on such probabilistic image representation. Experimental results on several standard image datasets indicate that the proposed method outperforms some of existing image classification methods.

Feature Selection for Natural Language Call Routing Based on Self-Adaptive Genetic Algorithm

The text classification problem for natural language call routing was considered in the paper. Seven different term weighting methods were applied. As dimensionality reduction methods, the feature selection based on self-adaptive GA is considered. k-NN, linear SVM and ANN were used as classification algorithms. The tasks of the research are the following: perform research of text classification for natural language call routing with different term weighting methods and classification algorithms and investigate the feature selection method based on self-adaptive GA. The numerical results showed that the most effective term weighting is TRR. The most effective classification algorithm is ANN. Feature selection with self-adaptive GA provides improvement of classification effectiveness and significant dimensionality reduction with all term weighting methods and with all classification algorithms.

KerMinSVM for imbalanced datasets with a case study on arabic comics classification

Many studies have been performed to classify large-sized text documents using different classifiers, ranging from simple distance classifiers such as K-Nearest-Neighbor (KNN) to more advanced classifiers such as Support Vector Machines. Traditional approaches fail when a short text is encountered due to sparsity resulting from a limited number of words. Another common problem in text classification is class imbalance (CI). CI occurs when one class of the data contains most of the samples while the other class contains only a few. Standard classifiers, when applied to imbalanced data, result in high accuracy for the majority class and low accuracy for the minority one. We were motivated to propose a novel framework for classifying the content of Arabic comics; therefore, we propose KerMinSVM, a kernel extension of our previously proposed MinSVM coupled with a new dimensionality featuring a reduction scheme based on word root frequency ratios (WRFR). KerMinSVM was tested on multiple imbalanced benchmark datasets, and the results were verified using three measures: accuracy, F-measure, and statistical analysis. WRFR was applied to the manual construction of the Arabic comic text dataset to detect strong content in children's comic books. Test results revealed that our proposed framework outperformed most of the methods for imbalanced datasets and short text classification.

Multi-label text classification using optimised feature sets

Multi-label text classification is the process of assigning multi-labels to an instance. A significant aspect of the text classification problem is the high dimensionality of the data which hinders the performance of the classifier. Hence, feature selection plays a significant role in classification process that removes the irrelevant data. In this paper, wrapper-based hybrid artificial bee colony and bacterial foraging optimisation (HABBFO) approach has been proposed to select the most appropriate feature subset for prediction. Initially, pre-processing such as tokenisation, stop word removal and stemming has been performed to extract the features (words). Experiments are conducted on the benchmark dataset and the results show that the proposed approach achieves better performance compared to the other feature selection techniques.

Analysis of Influence of Different Relations Types on the Quality of Thesaurus Application to Text Classification Problems

The main purpose of the article is to analyze how effectively different types of thesaurus relations can be used for solutions of text classification tasks. The basis of the study is an automatically generated thesaurus of a subject area, that contains three types of relations: synonymous, hierarchical and associative. To generate the thesaurus the authors use a hybrid method based on several linguistic and statistical algorithms for extraction of semantic relations. The method allows to create a thesaurus with a sufficiently large number of terms and relations among them. The authors consider two problems: topical text classification and sentiment classification of large newspaper articles. To solve them, the authors developed two approaches that complement standard algorithms with a procedure that take into account thesaurus relations to determine semantic features of texts. The approach to topical classification includes the standard unsupervised BM25 algorithm and the procedure, that take into account synonymous and hierarchical relations of the thesaurus of the subject area. The approach to sentiment classification consists of two steps. At the first step, a thesaurus is created, whose terms weight polarities are calculated depending on the term occurrences in the training set or on the weights of related thesaurus terms. At the second step, the thesaurus is used to compute the features of words from texts and to classify texts by the algorithm SVM or Naive Bayes. In experiments with text corpora BBCSport, Reuters, PubMed and the corpus of articles about American immigrants, the authors varied the types of thesaurus relations that are involved in the classification and the degree of their use. The results of the experiments make it possible to evaluate the efficiency of the application of thesaurus relations for classification of raw texts and to determine under what conditions certain relationships affect more or less. In particular, the most useful thesaurus connections are synonymous and hierarchical, as they provide a better quality of classification.

EFFICIENCY ASSESSMENT OF EUCLIDEAN AND MAKHALANOBIS DISTANCES FOR SOLVING A MAJOR TEXT CLASSIFICATION PROBLEM

Objectives The aim is to compare the efficiency of using the Euclidean and Mahalanobis metrics to solve the problem of determining the category of potential text recipients. The relevance of the task is determined by the need to develop a means of identifying the recipients of electronic documents. This has been complicated with the introduction of age restrictions on the content of Internet webpages and text resources. Moreover, there has been little coverage of this issue in the works of Russian researchers. Method A comparison of the relative efficiencies of using Euclid and Mahalanobis distances was carried out within the framework of the implementation of an intelligent system for text automatic classification based on the age category of their recipients. Results The main approaches to establishing proximity measures of objects represented as sets of classification characteristics are discussed and the choice of Euclidean and Mahalanobis metrics for numerical comparison of classification results is justified. A description of the sample texts and characteristics of category designations are given for a computational experiment. The computational experiment was carried out using texts included in the National Corpus of the Russian language. Conclusion The computational experiment allows the most effective method for solving the problem of determining the age category of potential text recipients to be selected. The results of the experiment showed the possibility of using Euclidean and Mahalanobis metrics for solving text classification problems; the preference for using Mahalanobis metrics for estimating distances by objects represented by correlated features was also confirmed. The presented comparison of the relative efficiencies of Euclid and Mahalanobis distances was carried out within the framework of the implementation of an intelligent system for automatic text classification based on the age category of their recipients.

Turning from TF-IDF to TF-IGM for term weighting in text classification

Massive textual data management and mining usually rely on automatic text classification technology. Term weighting is a basic problem in text classification and directly affects the classification accuracy. Since the traditional TF-IDF (term frequency & inverse document frequency) is not fully effective for text classification, various alternatives have been proposed by researchers. In this paper we make comparative studies on different term weighting schemes and propose a new term weighting scheme, TF-IGM (term frequency & inverse gravity moment), as well as its variants. TF-IGM incorporates a new statistical model to precisely measure the class distinguishing power of a term. Particularly, it makes full use of the fine-grained term distribution across different classes of text. The effectiveness of TF-IGM is validated by extensive experiments of text classification using SVM (support vector machine) and kNN (k nearest neighbors) classifiers on three commonly used corpora. The experimental results show that TF-IGM outperforms the famous TF-IDF and the state-of-the-art supervised term weighting schemes. In addition, some new findings different from previous studies are obtained and analyzed in depth in the paper.

Improving the text classification using clustering and a novel HMM to reduce the dimensionality

In text classification problems, the representation of a document has a strong impact on the performance of learning systems. The high dimensionality of the classical structured representations can lead to burdensome computations due to the great size of real-world data. Consequently, there is a need for reducing the quantity of handled information to improve the classification process. In this paper, we propose a method to reduce the dimensionality of a classical text representation based on a clustering technique to group documents, and a previously developed Hidden Markov Model to represent them. We have applied tests with the k-NN and SVM classifiers on the OHSUMED and TREC benchmark text corpora using the proposed dimensionality reduction technique. The experimental results obtained are very satisfactory compared to commonly used techniques like InfoGain and the statistical tests performed demonstrate the suitability of the proposed technique for the preprocessing step in a text classification task.

TEXT CLASSIFICATION USING MODIFIED MULTI CLASS ASSOCIATION RULE

This paper presents text classification using a modified Multi Class Association Rule Method. The method is based on Associative Classification which combines classification with association rule discovery. Although previous work proved that Associative Classification produces better classification accuracy compared to typical classifiers, the study on applying Associative Classification to solve text classification problem are limited due to the common problem of high dimensionality of text data and this will consequently results in exponential number of generated classification rules. To overcome this problem the modified Multi-Class Association Rule Method was enhanced in two stages. In stage one the frequent pattern are represented using a proposed vertical data format to reduce the text dimensionality problem and in stage two the generated rule was pruned using a proposed Partial Rule Match to reduce the number of generated rules. The proposed method was tested on a text classification problem and the result shows that it performed better than the existing method in terms of classification accuracy and number of generated rules.

Influence of normalization and color space to color texture classification

Color texture classification has recently attracted significant attention due to its multiple applications. The color texture images depend on the texture surface and its albedo, the illumination, the camera and its viewing position. A key problem to get an acceptable performance is the ambient illumination, which can vary the perceived structures in the surface. Given a color texture classification problem, it would be desirable to know which is the best approach to solve the problem making the minimal assumptions about the illumination conditions. The present work does an exhaustive evaluation of the state-of-the-art color texture classification methods, considering 5 different color spaces, 12 normalization methods to achieve illumination invariances, 19 texture feature vectors and 23 pure color feature vectors. Our experiments allow to conclude that parallel approaches are better than integrative approaches for color texture classification achieving the first positions in the Friedman ranking. Multiresolution Local Binary Patterns (MLBP) are the best intensity texture features, followed by wavelet and Gabor filters combined with luminance–chrominance color spaces (Lab and Lab2000HL), and for pure color classification the best are First Order Statistics (FOS) calculated in RGB color space. For intensity texture features, the learning methods work better on the four smallest datasets, although they could not be tested in other four bigger datasets due to its huge computational cost, nor with color texture classification. Normalization and color spaces slightly increase the average accuracy of color texture classification, although the differences achieved using normalization are not statistically significant in a paired T-Test. Lab2000HL and RGB are the best color spaces, but the former is the slowest one. Regarding elapsed time, the best vector features MLBP for intensity texture, Daub4 (Daubechies filters using mean and variance statistics) for color texture and FOS, for pure color are nearly the fastest or are in the middle interval of all the tested methods.