Clustering news articles using efficient similarity measure and N-grams

Abstract

The rapid progress of information technology and web makes it easier to store huge amount of collected textual information, e.g., blogs, news articles, e-mail messages, reviews and forum postings. The growing size of textual dataset with high-dimensions and natural language pose a big challenge making it hard for such information to be categorised efficiently. Document clustering is an automatic unsupervised machine learning technique that aimed at grouping related set of items into clusters or subsets. The target is creating clusters with high internal coherence, but different from each other substantially. This paper presents a new document clustering technique using N-grams and efficient similarity measure known as 'improved sqrt-cosine similarity measure'. Comprehensive experiments are conducted to evaluate our proposed clustering technique and compared with an existing method. The results of the experiments show that our proposed clustering technique outperforms the existing techniques.