Chinese Text Classification Research Articles

A Fast Algorithm for Chinese Text Categorization Based on Key Tree

To solving Chinese text categorization, a fast algorithm is proposed. The basic idea of the algorithm is: first constructs a weighted value of keywords dictionary which is constructed in key tree, then using the Hash function and the principle of giving priority for long term matching to mapping the strings in documentations to the dictionary. After that, calculate the sum of weights of the keywords which has been matched successfully. Finally take the maximum for the result of the classification. The algorithm can avoid the difficulty of Chinese word segmentation and its influence on accuracy of result. Theoretical analysis and experimental results indicate that the accuracy and the time efficiency of the algorithm is higher, whose comprehensive performance reaches to the level of current major technology.

A logistic regression-based smoothing method for Chinese text categorization

Automatic Chinese text classification is an important and a well-known technology in the field of machine learning. The first step for solving Chinese text categorization problems is to tokenize the Chinese words from a sequence of non-segmented sentences. However, previous literatures often employ a Chinese word tokenizer that was trained with different sources and then perform the conventional text classification approaches. However, these taggers are not perfect and often provide incorrect word boundary information. In this paper, we propose an N-gram-based language model which takes word relations into account for Chinese text categorization without Chinese word tokenizer. To prevent from out-of-vocabulary, we also propose a novel smoothing approach based on logistic regression to improve accuracy. The experimental result shows that our approach outperforms traditional methods at least 11% on micro-average F-measure.

Chinese Text Classification with a KNN Classifier Using an Adjusted Feature Weighting Method

KNN algorithm is used for Chinese text classification in this paper. First, TF-IDF is chosen as the feature weighting method. To the characteristics of corpus used in this paper, TF-IDF is adjusted to a new method. At last, experimental result shows the accuracy of KNN text classifier can be improved with the adjusted feature weighting method.

Application of TSVM Incremental Learning in Web Text Categorization

Based on the research of the current situation in text categorization, this paper has drawn an inductive conclusion on the key technology of text category, carried out an exploration on the theory of transductive support vector machine (TSVM) as well as the categorizing process of incremental learning, established web Chinese text categorization model on the basis of TSVM incremental learning, and explained the learning process of incremental learning in Chinese text categorization. Experimental studies show that incremental learning has significant effect on improving the categorizing performance.

Chinese text classification by the Naïve Bayes Classifier and the associative classifier with multiple confidence threshold values

Each type of classifier has its own advantages as well as certain shortcomings. In this paper, we take the advantages of the associative classifier and the Naïve Bayes Classifier to make up the shortcomings of each other, thus improving the accuracy of text classification. We will classify the training cases with the Naïve Bayes Classifier and set different confidence threshold values for different class association rules (CARs) to different classes by the obtained classification accuracy rate of the Naïve Bayes Classifier to the classes. Since the accuracy rates of all selected CARs of the class are higher than that obtained by the Naïve Bayes Classifier, we could further optimize the classification result through these selected CARs. Moreover, for those unclassified cases, we will classify them with the Naïve Bayes Classifier. The experimental results show that combining the advantages of these two different classifiers better classification result can be obtained than with a single classifier.

Chinese short text classification based on hyponymy relation

Concerning the short length and weak signal to describe the characteristics of short text,a framework of Chinese short-text classification was put forward by using hyponymy.In order to achieve the classification of the test text,the framework first used "Hownet" to determine the hyponymy between concept pairs in training text,thus determining the hyponymy between word pairs,and then the feature vectors of test text were extended.The experimental results show that short-text classification performance can be improved by using the hyponymy.

The Chinese Text Categorization System with Category Priorities

The process of text categorization involves some understanding of the content of the documents and/or some previous knowledge of the categories. For the content of the documents, we use a filtering measure for feature selection in our Chinese text categorization system. We modify the formula of Term Frequency-Inverse Document Frequency (TF-IDF) to strengthen important keywords’ weights and weaken unimportant keywords’ weights. For the knowledge of the categories, we use category priority to represent the relationship between two different categories. Consequently, the experimental results show that our method can effectively not only decrease noise text but also increase the accuracy rate and recall rate of text categorization.

N-grams based feature selection and text representation for Chinese Text Classification

In this paper, text representation and feature selection strategies for Chinese text classification based on n-grams are discussed. Two steps feature selection strategy is proposed which combines the preprocess within classes with the feature selection among classes. Four different feature selection methods and three text representation weights are compared by exhaustive experiments. Both C-SVC classifier and Naive bayes classifier are adopted to assess the results. All experiments are performed on Chinese corpus TanCorpV1.0 which includes more than 14,000 texts divided in 12 classes. Our experiments concern: (1) the performance comparison among different feature selection strategies: absolute text frequency, relative text frequency, absolute n-gram frequency and relative n-gram frequency; (2) the comparison of the sparseness and feature correlation in the “text by feature” matrices produced by four feature selection methods; (3) the performance comparison among three term weights: 0/1 logical value, n-gr...

Method for Chinese short text classification based on feature extension

In this paper,based on the characteristics that short texts describe weak signals,a method based on feature extension(STCFE) was introduced to classify Chinese short texts.In this method,the correlation rules between feature items of training set and testing set were mined by FP-Growth algorithm,and then those rules were applied to extend the features of the testing set.Meanwhile,to classify Chinese short texts effectively,semantic information was introduced and the DEF term formula of words was improved in HowNet.Experimental results show that the proposed method performs well,and its Micro-F1 and Macro-F1 are higher than those of conventional approaches.

Non-Independent Term Selection for Chinese Text Categorization

Chinese text categorization differs from English text categorization due to its much larger term set (of words or character n-grams), which results in very slow training and working of modern high-performance classifiers. This study assumes that this high-dimensionality problem is related to the redundancy in the term set, which cannot be solved by traditional term selection methods. A greedy algorithm framework named “non-independent term selection” is presented, which reduces the redundancy according to string-level correlations. Several preliminary implementations of this idea are demonstrated. Experiment results show that a good tradeoff can be reached between the performance and the size of the term set.

N-grams based feature selection and text representation for Chinese Text Classification

N-grams based feature selection and text representation for Chinese Text Classification

Technology for Chinese text categorization based on reverse matching algorithm

Concerning Chinese text categorization,a reverse matching algorithm was proposed.The basic idea was to construct a weighted value of classification subject terms list firstly,then use

The Chinese text categorization system with association rule and category priority

The process of text categorization involves some understanding of the content of the documents and/or some previous knowledge of the categories. For the content of the documents, we use the filtering measure for feature selection in our Chinese text categorization system. We modify the formula of TFIDF to strengthen important keywords' weights and weaken unimportant keywords' weights. For the knowledge of the categories, we use association rules to improve the precision of text classification and use category priority to represent the relationship between two different categories. Consequently, the experimental results show that our method can effectively not only decrease noise text but also increase the ratio of precision and recall of text categorization.

Machine learning for Asian language text classification

PurposeThe purpose of this research is to compare several machine learning techniques on the task of Asian language text classification, such as Chinese and Japanese where no word boundary information is available in written text. The paper advocates a simple language modeling based approach for this task.Design/methodology/approachNaïve Bayes, maximum entropy model, support vector machines, and language modeling approaches were implemented and were applied to Chinese and Japanese text classification. To investigate the influence of word segmentation, different word segmentation approaches were investigated and applied to Chinese text. A segmentation‐based approach was compared with the non‐segmentation‐based approach.FindingsThere were two findings: the experiments show that statistical language modeling can significantly outperform standard techniques, given the same set of features; and it was found that classification with word level features normally yields improved classification performance, but that classification performance is not monotonically related to segmentation accuracy. In particular, classification performance may initially improve with increased segmentation accuracy, but eventually classification performance stops improving, and can in fact even decrease, after a certain level of segmentation accuracy.Practical implicationsApply the findings to real web text classification is ongoing work.Originality/valueThe paper is very relevant to Chinese and Japanese information processing, e.g. webpage classification, web search.