A Text Feature Selection Method Based on the Small World Algorithm

Abstract

The methods of feature selections have a profound influence on the performance of text categorization. Due to the deficiency of the mathematical models, traditional feature selection methods are less likely to get remarkable improvement. Small world algorithm is a kind of swarm intelligence optimization algorithm to solve the optimization problem. As feature selection is the nature of the optimal combination problem of discrete space, in this paper we attempted to improve the effect of feature selection by making full use of the small world algorithm. We used the English R8 single label corpus of Reuters 21578 classic corpus and the data for English text classification experiment, while in the Chinese text classification experiment we chose 3600 texts from Chinese Fudan corpus and halved them for building training and testing sets. By using KNN and SVM classifier respectively in the candidate dimensions of 300,450,600, the results revealed that the small world algorithm method optimized the keywords selected and its effect was superior to the traditional feature selection methods.