A feature selection for Korean web document clustering

Abstract

This paper is a comparative study of feature selection methods for Korean Web documents clustering. First, we focused on how the term feature and the co-link of Web documents affect clustering performance. We clustered Web documents by native term feature, co-link and both, and compared the output results with the originally allocated category. And we selected term features for each category using X/sup 2/, information gain (IG), and mutual information (MI) from training documents, and applied these features to other experimental documents. In addition we suggested a new method named max feature selection, which selects terms that have the maximum count for a category in each experimental document, and applied X/sup 2/ (or MI or IG) values to each term instead of term frequency of documents, and clustered them. In the results, X/sup 2/ shows a better performance than IG or MI, but the difference appears to be slight. But when we applied the max feature selection method, the clustering performance improved notably. Max feature selection is a simple but effective means of feature space reduction and shows powerful performance for Korean Web document clustering.