Improving Text Classification using Local Latent Semantic Indexing

Abstract

Latent semantic indexing (LSI) has been shown to be extremely useful in information retrieval, but it is not an optimal representation for text classification. It always drops the text classification performance when being applied to the whole training set (global LSI) because this completely unsupervised method ignores class discrimination while only concentrating on representation. Some methods have been proposed to improve the classification by utilizing class discrimination information. However, their performance improvements over original term vectors are still very limited. In this paper, we propose a new method called local relevancy weighted LSI to improve text classification by performing a separate single value decomposition (SVD) on the transformed region of each class. Experimental results show that our method is much better than global and traditional methods on classification within a much smaller dimension.