Automatic Text Classification Research Articles

A new Centroid-Based Classification model for text categorization

The automatic text categorization technique has gained significant attention among researchers because of the increasing availability of online text information. Therefore, many different learning approaches have been designed in the text categorization field. Among them, the widely used method is the Centroid-Based Classifier (CBC) due to its theoretical simplicity and computational efficiency. However, the classification accuracy of CBC greatly depends on the data distribution. Thus it leads to a misfit model and also has poor classification performance when the data distribution is highly skewed. In this paper, a new classification model named as Gravitation Model (GM) is proposed to solve the class-imbalanced classification problem. In the training phase, each class is weighted by a mass factor, which can be learned from the training data, to indicate data distribution of the corresponding class. In the testing phase, a new document will be assigned to a particular class with the max gravitational force. The performance comparisons with CBC and its variants based on the results of experiments conducted on twelve real datasets show that the proposed gravitation model consistently outperforms CBC together with the Class-Feature-Centroid Classifier (CFC). Also, it obtains the classification accuracy competitive to the DragPushing (DP) method while it maintains a more stable performance. Thus, the proposed gravitation model is proved to be less over-fitting and has higher learning ability than CBC model.

IMPROVED INFORMATION FILTERING FOR TWITTER BASED ON THE SEMANTIC KNOWLEDGE

As the outgrowth of the internet as well as the social networks like twitter, Facebook user may get flooded out of raw information. Twitter message is short and may not contain enough contextual information, so traditional clustering method which utilizes the traditional method like “Bag-of-words” accept some restriction. To overwhelm with this trouble, we offered an automatic text classification process. Some social networking sites have imposed limits on the no of character for the users like twitter imposed limit of only 140 characters to the users to post any tweet. To classify these kinds of tweets is a tedious task or even impossible to classify. Short text is hard to sort out due to the lack of semantic information, therefore in this research paper, a novel approach is presented that incorporate the semantic database and utilized it to elicit the necessary features to separate the short text. Experimental results indicate that the proposed approach effectively classify the incoming tweets into predefined categories such as ‘News’, ‘Events’, ‘Personal message’ and ‘Deals’.

An automatic classification of text documents based on correlative association of words

Training speed of the classifier without degrading its predictive capability is an important concern in text classification. Feature selection plays a key role in this context. It selects a subset of most informative words (terms) from the set of all words. The correlative association of words towards the classes increases an incertitude for the words to represent a class. The representative words of a class are either of positive or negative nature. The standard feature selection methods, viz. Mutual Information (MI), Information Gain (IG), Discriminating Feature Selection (DFS) and Chi Square (CHI), do not consider positive and negative nature of the words that affects the performance of the classifiers. To address this issue, this paper presents a novel feature selection method named Correlative Association Score (CAS). It combines the strength, mutual information, and strong association of the words to determine their positive and negative nature for a class. CAS selects a few (k) informative words from the set of all words (m). These informative words generate a set of N-grams of length 1-3. Finally, the standard Apriori algorithm ensembles the power of CAS and CHI to select the top most, b informative N-grams, where b is a number set by an empirical evaluation. Multinomial Naive Bayes (MNB) and Linear Support Vector Machine (LSVM) classifiers evaluate the performance of the selected N-Grams. Four standard text data sets, viz. Webkb, 20Newsgroup, Ohsumed10, and Ohsumed23 are used for experimental analysis. Two standard performance measures named Macro_F1 and Micro_F1 show a significant improvement in the results using proposed CAS method.

An Empirical Study of Clustering Algorithms to extract Knowledge from PubMed Articles

Extraction of useful information from biomedical literature is one of the thrust for the world nowadays due to availability of almost articles on the web in electronic form. Information retrieval (IR) from biomedical literature is finding useful patterns from the unstructured text corpus that satisfies information. In this paper intelligent text analysis is carried out on PubMed articles related to influenza virus. In this context, various algorithms are discussed to reveal the information from PubMed articles, like year wise count of articles containing influenza virus related terms (viz. H1N1, H5N1, and H7N1 etc.), countries with their publication count, which tells about the outbreaks of the diseases in these countries. The articles may be grouped by searching the keyword “influenza virus strain” pattern with the help of regular expressions. Automatic text categorization is another challenging issue for text mining. We applied k-means, fuzzy C-means, and fuzzy C-shell algorithm for automatic categorization of text articles. The association between words based on their co-occurrence is computed which further helps to categorize the documents based on their co-occurrences. The basic k-means clustering algorithm is first applied to cluster the documents, and then to handle the fuzzy nature of words which may belong to more than one cluster, fuzzy c-means clustering is applied to form more accurate clusters. As Fuzzy c-means method clusters the documents which are in linear spaces but not in the circle, spherical, or ellipsoidal spaces. A new method is proposed here, which considers the clusters of documents in the radius of the circle.

Automatic text classification algorithm based on Gauss improved convolutional neural network

The traditional KNN query is a kind of algorithm with good stability and accuracy performance. However, when the sample size is too large, the computational efficiency of the algorithm is affected greatly. Therefore, a kind of parallel MKNN text classification algorithm based on clustering center text series has been proposed. Firstly, the effective dimensionality reduction of similarity calculation amount of the algorithm is realized based on the clustering center, and the original large-scale document samples are replaced with a relatively small number of clustering sample centers to realize improvement of the KNN query process. Secondly, MapReduce parallel framework is used to meet real-time demand of large-scale text classification and calculation combined with features of text classification, and to effectively overcome slow speed of the KNN query process and ensure accuracy of text classification as higher as possible. Finally, the classification speed of proposed algorithm can be effectively improved under the premise of ensuring sufficient accuracy through comparison in experiment of text classification accuracy and algorithmic efficiency with the similar single-threaded algorithm.

Using bipartite heterogeneous networks to speed up inductive semi-supervised learning and improve automatic text categorization

Due to the volume of texts available in digital form, the organization, management and knowledge extraction are laborious and frequently impossible to be handled. To automatically cope with these tasks, usually classification models are generated through supervised learning techniques. Unfortunately, this type of learning usually demands a huge human effort to label large volume of texts to build accurate classification models. Since collecting unlabeled texts is easy and inexpensive in several domains, the generation of classification models through inductive semi-supervised learning has been highlighted in recent years. Inductive semi-supervised learning allows to build a classification model using labeled and unlabeled texts. In this scenario, the goal is to augment the set of labeled documents with unlabeled documents to better discriminate class patterns. Hence, fewer texts must be previously labeled. However, semi-supervised learning algorithms that consider texts represented in a vector space model usually obtain unsatisfactory classification performances and are surpassed by semi-supervised learning algorithms that consider texts represented in a network. Nevertheless, despite the classification performances, effective approaches based on networks are generated through the similarities among documents and the classification of a new document are also based on the computation of similarities. This implies to set parameters and compute similarities to both generation the networks and classification of new documents. This approach is not feasible to generate fast responses and consequently to classify a huge volume of texts. In this article, we propose an approach to induce a classification model through semi-supervised learning considering text collections represented by bipartite heterogeneous networks. Bipartite networks are easily and quickly generated, leading to classification performance equivalent or better than other approaches based on network or vector space model and allows a fast classification of new documents. The results presented in this article demonstrate that the proposed approach is able to (i) speed up semi-supervised learning, (ii) speed up the classification of new documents and (iii) surpass classification performance of other existing inductive semi-supervised learning techniques.

Classification by compression: Application of information-theory methods for the identification of themes of scientific texts

A method for automatic classification of scientific texts based on data compression is proposed. The method is implemented and investigated based on the data from an archive of scientific texts (arXiv.org) and in the CyberLeninka scientific electronic library (CyberLeninka.ru). Experiments showed that the method correctly identified the themes of scientific texts with a probability of 75–95%; its accuracy depends on the quality of the original data.

Semi-supervised learning in large scale text categorization

The rapid development of the Internet brings a variety of original information including text information, audio information, etc. However, it is difficult to find the most useful knowledge rapidly and accurately because of its huge number. Automatic text classification technology based on machine learning can classify a large number of natural language documents into the corresponding subject categories according to its correct semantics. It is helpful to grasp the text information directly. By learning from a set of hand-labeled documents, we obtain the traditional supervised classifier for text categorization (TC). However, labeling all data by human is labor intensive and time consuming. To solve this problem, some scholars proposed a semi-supervised learning method to train classifier, but it is unfeasible for various kinds and great number of Web data since it still needs a part of hand-labeled data. In 2012, Li et al. invented a fully automatic categorization approach for text (FACT) based on supervised learning, where no manual labeling efforts are required. But automatically labeling all data can bring noise into experiment and cause the fact that the result cannot meet the accuracy requirement. We put forward a new idea that part of data with high accuracy can be automatically tagged based on the semantic of category name, then a semi-supervised way is taken to train classifier with both labeled and unlabeled data, and ultimately a precise classification of massive text data can be achieved. The empirical experiments show that the method outperforms the supervised support vector machine (SVM) in terms of both F1 performance and classification accuracy in most cases. It proves the effectiveness of the semi-supervised algorithm in automatic TC.

Information-Theoretic method for classification of texts

We consider a method for automatic (i.e., unmanned) text classification based on methods of universal source coding (or “data compression”). We show that under certain restrictions the proposed method is consistent, i.e., the classification error tends to zero with increasing text lengths. As an example of practical use of the method we consider the classification problem for scientific texts (research papers, books, etc.). The proposed method is experimentally shown to be highly efficient.

Modified frequency-based term weighting schemes for text classification

With the rapid growth of textual content on the Internet, automatic text categorization is a comparatively more effective solution in information organization and knowledge management. Feature selection, one of the basic phases in statistical-based text categorization, crucially depends on the term weighting methods In order to improve the performance of text categorization, this paper proposes four modified frequency-based term weighting schemes namely; mTF, mTFIDF, TFmIDF, and mTFmIDF. The proposed term weighting schemes take the amount of missing terms into account calculating the weight of existing terms. The proposed schemes show the highest performance for a SVM classifier with a micro-average F1 classification performance value of 97%. Moreover, benchmarking results on Reuters-21578, 20Newsgroups, and WebKB text-classification datasets, using different classifying algorithms such as SVM and KNN show that the proposed schemes mTF, mTFIDF, and mTFmIDF outperform other weighting schemes such as TF, TFIDF, and Entropy. Additionally, the statistical significance tests show a significant enhancement of the classification performance based on the modified schemes.

Predicting the Relevance of Social Media Posts Based on Linguistic Features and Journalistic Criteria

An overwhelming quantity of messages is posted in social networks every minute. To make the utilization of these platforms more productive, it is imperative to filter out information that is irrelevant to the general audience, such as private messages, personal opinions or well-known facts. This work is focused on the automatic classification of public social text according to its potential relevance, from a journalistic point of view, hopefully improving the overall experience of using a social network. Our experiments were based on a set of posts with several criteria, including the journalistic relevance, assessed by human judges. To predict the latter, we rely exclusively on linguistic features, extracted by Natural Language Processing tools, regardless the author of the message and its profile information. In our first approach, different classifiers and feature engineering methods were used to predict relevance directly from the selected features. In a second approach, relevance was predicted indirectly, based on an ensemble of classifiers for other key criteria when defining relevance—controversy, interestingness, meaningfulness, novelty, reliability and scope—also in the dataset. The first approach achieved a F 1-score of 0.76 and an Area under the ROC curve (AUC) of 0.63. But the best results were achieved by the second approach, with the best learned model achieving a F 1-score of 0.84 with an AUC of 0.78. This confirmed that journalistic relevance can indeed be predicted by the combination of the selected criteria, and that linguistic features can be exploited to classify the latter.

Automatic Emergency Information Classification Based on Rules and Maximum Entropy Model

Text classification is the basic work of researching on the evolution of emergencies. While, in emergency information management systems, most of the emergency information is currently expressed in a text form and always manually classified, which might result in poor efficiency and low accuracy. To deal with it, this paper proposes an automatic emergency text classification method based on rules and Maximum Entropy Model (MEM). First, according to the Public Safety Triangular Framework (PSTF) theory, an emergency information constitution mechanism is established. Then, based on it, the corresponding feature words base is built and is used for constructing the Event Rules Base (ERB). ERB acts as classification basis and is input into MEM for training to generate the emergency text classifier. Experiment results on real data demonstrate that the proposed classification method is able to yield good performance.

Variable Global Feature Selection Scheme for automatic classification of text documents

The feature selection is important to speed up the process of Automatic Text Document Classification (ATDC). At present, the most common method for discriminating feature selection is based on Global Filter-based Feature Selection Scheme (GFSS). The GFSS assigns a score to each feature based on its discriminating power and selects the top-N features from the feature set, where N is an empirically determined number. As a result, it may be possible that the features of a few classes are discarded either partially or completely. The Improved Global Feature Selection Scheme (IGFSS) solves this issue by selecting an equal number of representative features from all the classes. However, it suffers in dealing with an unbalanced dataset having large number of classes. The distribution of features in these classes are highly variable. In this case, if an equal number of features are chosen from each class, it may exclude some important features from the class containing a higher number of features. To overcome this problem, we propose a novel Variable Global Feature Selection Scheme (VGFSS) to select a variable number of features from each class based on the distribution of terms in the classes. It ensures that, a minimum number of terms are selected from each class. The numerical results on benchmark datasets show the effectiveness of the proposed algorithm VGFSS over classical information science methods and IGFSS.

Automatic ICD-10 multi-class classification of cause of death from plaintext autopsy reports through expert-driven feature selection.

ObjectivesWidespread implementation of electronic databases has improved the accessibility of plaintext clinical information for supplementary use. Numerous machine learning techniques, such as supervised machine learning approaches or ontology-based approaches, have been employed to obtain useful information from plaintext clinical data. This study proposes an automatic multi-class classification system to predict accident-related causes of death from plaintext autopsy reports through expert-driven feature selection with supervised automatic text classification decision models.MethodsAccident-related autopsy reports were obtained from one of the largest hospital in Kuala Lumpur. These reports belong to nine different accident-related causes of death. Master feature vector was prepared by extracting features from the collected autopsy reports by using unigram with lexical categorization. This master feature vector was used to detect cause of death [according to internal classification of disease version 10 (ICD-10) classification system] through five automated feature selection schemes, proposed expert-driven approach, five subset sizes of features, and five machine learning classifiers. Model performance was evaluated using precisionM, recallM, F-measureM, accuracy, and area under ROC curve. Four baselines were used to compare the results with the proposed system.ResultsRandom forest and J48 decision models parameterized using expert-driven feature selection yielded the highest evaluation measure approaching (85% to 90%) for most metrics by using a feature subset size of 30. The proposed system also showed approximately 14% to 16% improvement in the overall accuracy compared with the existing techniques and four baselines.ConclusionThe proposed system is feasible and practical to use for automatic classification of ICD-10-related cause of death from autopsy reports. The proposed system assists pathologists to accurately and rapidly determine underlying cause of death based on autopsy findings. Furthermore, the proposed expert-driven feature selection approach and the findings are generally applicable to other kinds of plaintext clinical reports.

Text feature selection algorithm based on Chi-square rank correlation factorization

Feather Selection is an effective method to reduce the dimension of text feature. The existing feature selection methods usually use empirical estimation methods when determining the scale of the feature selection. These methods have achieved good results in some specific corpora. However, it is not easy to promote the further generalization of automatic text categorization due to the insufficient theoretical basis. Therefore, a text feature selection algorithm based on Chi-square rank correlation factorization is proposed based on the comprehensive consideration of the whole and local distribution of text features. Under the condition that the algorithm does not need any prior knowledge, the feature weights are portrayed and the feature selection is completed, fully reflects the characteristics of the probability distribution.

Automatic Categorization of PubMed microRNA Target Abstracts Based on Text Classification Techniques

This is a first study attempting to suggest an automatic categorization of microRNA articles from PubMed. In this study, text classification techniques using binary representations were applied on the abstract section of the articles. The PubMed articles related to microRNA targets were regarded as the positive class whereas documents retrieved using different criteria are used as a negative class. The results show that with a careful choice of the negative class, the PubMed articles about microRNA targets can be accurately distinguished. Moreover, we showed the robustness of the automatic text classification by building models not just from the top keywords but also from a combination of other keywords.

EFFICIENCY ASSESSMENT OF EUCLIDEAN AND MAKHALANOBIS DISTANCES FOR SOLVING A MAJOR TEXT CLASSIFICATION PROBLEM

Objectives The aim is to compare the efficiency of using the Euclidean and Mahalanobis metrics to solve the problem of determining the category of potential text recipients. The relevance of the task is determined by the need to develop a means of identifying the recipients of electronic documents. This has been complicated with the introduction of age restrictions on the content of Internet webpages and text resources. Moreover, there has been little coverage of this issue in the works of Russian researchers. Method A comparison of the relative efficiencies of using Euclid and Mahalanobis distances was carried out within the framework of the implementation of an intelligent system for text automatic classification based on the age category of their recipients. Results The main approaches to establishing proximity measures of objects represented as sets of classification characteristics are discussed and the choice of Euclidean and Mahalanobis metrics for numerical comparison of classification results is justified. A description of the sample texts and characteristics of category designations are given for a computational experiment. The computational experiment was carried out using texts included in the National Corpus of the Russian language. Conclusion The computational experiment allows the most effective method for solving the problem of determining the age category of potential text recipients to be selected. The results of the experiment showed the possibility of using Euclidean and Mahalanobis metrics for solving text classification problems; the preference for using Mahalanobis metrics for estimating distances by objects represented by correlated features was also confirmed. The presented comparison of the relative efficiencies of Euclid and Mahalanobis distances was carried out within the framework of the implementation of an intelligent system for automatic text classification based on the age category of their recipients.

借助GA-KNN及线性规划的基于本体的教学资源分类方案的研究

随着信息技术的高速发展，Internet上积累了大量的优秀的教学资源，人们开始利用网络这个学习平台来访问这些优秀资源，以达到自主学习的目的。为了能够快速访问到所需资源，必须对繁杂的资源进行分类。传统的人工分类难以完成这一工作，利用文本自动化分类技术，则可以实现对教学资源进行快速而有效的分类。本文将利用GA-KNN及线性规划来实现教育资源的自动化分类，从实验结果来看，它基本达到了预期的效果，大大提高了分类的精度。 With the rapid development of information technology, the Internet has accumulated a large number of outstanding teaching resources, and people begin to use the network learning platform to access these excellent resources in order to achieve the purpose of autonomous learning. In order to be able to quickly access to the required resources, we must classify multifarious resources. The traditional artificial classification is difficult to finish this job. Using text automatic classification technology, we can realize the rapid and effective classification of teaching resources. This article will use the GA-KNN and linear programming to achieve the automatic classification of education resources. From the experimental results, it achieves the expected effect, greatly improving the accuracy of classification.

Text Mining to Support Gene Ontology Curation and Vice Versa.

In this chapter, we explain how text mining can support the curation of molecular biology databases dealing with protein functions. We also show how curated data can play a disruptive role in the developments of text mining methods. We review a decade of efforts to improve the automatic assignment of Gene Ontology (GO) descriptors, the reference ontology for the characterization of genes and gene products. To illustrate the high potential of this approach, we compare the performances of an automatic text categorizer and show a large improvement of +225 % in both precision and recall on benchmarked data. We argue that automatic text categorization functions can ultimately be embedded into a Question-Answering (QA) system to answer questions related to protein functions. Because GO descriptors can be relatively long and specific, traditional QA systems cannot answer such questions. A new type of QA system, so-called Deep QA which uses machine learning methods trained with curated contents, is thus emerging. Finally, future advances of text mining instruments are directly dependent on the availability of high-quality annotated contents at every curation step. Databases workflows must start recording explicitly all the data they curate and ideally also some of the data they do not curate.

Literary Detective Work on the Computer. Michael P. Oakes.

Michael Oakes is a reader in Computational Linguistics at the University of Wolverhampton and an expert in statistical and corpus-based methods for language research. This book integrates, in textbook style, recent and old research about the application of quantitative methods to the automatic analysis of authorship and author profiles on the basis of properties of the text, a discipline often called computational stylometry. In addition, other language-related ‘detective work’, such as decipherment of old scripts and plagiarism detection are discussed as well. The currently dominant approach in computational stylometry is based on automatic text categorization, the field of computer science that also brought us spam filtering. As an example, for author identification, this approach would entail (1) defining a representation of text in terms of (mainly) linguistic properties, (2) training a model using statistical or machine learning methods on the basis of such representations of texts with known authorship, and (3) applying the learned model to unknown texts to decide authorship. This approach is based on the belief that when the linguistic properties used to represent a text are well chosen, the quantitative methods in (2) can learn how individual authors differ in style: author style would then be an idiosyncratic combination of preferences in the use of the linguistic properties, modeled in the text representations. Authorship methods based on some form of similarity or overlap between linguistic properties of different texts fit this scheme as well, as an instance of nearest-neighbor classification. Apart from this supervised learning, unsupervised techniques are used as well, clustering the texts after step (1) automatically into groups with similar stylistic properties.