Abstract
Similarity measures have long been utilized in information retrieval and machine learning domains for multi-purposes including text retrieval, text clustering, text summarization, plagiarism detection, and several other text-processing applications. However, the problem with these measures is that, until recently, there has never been one single measure recorded to be highly effective and efficient at the same time. Thus, the quest for an efficient and effective similarity measure is still an open-ended challenge. This study, in consequence, introduces a new highly-effective and time-efficient similarity measure for text clustering and classification. Furthermore, the study aims to provide a comprehensive scrutinization for seven of the most widely used similarity measures, mainly concerning their effectiveness and efficiency. Using the K-nearest neighbor algorithm (KNN) for classification, the K-means algorithm for clustering, and the bag of word (BoW) model for feature selection, all similarity measures are carefully examined in detail. The experimental evaluation has been made on two of the most popular datasets, namely, Reuters-21 and Web-KB. The obtained results confirm that the proposed set theory-based similarity measure (STB-SM), as a pre-eminent measure, outweighs all state-of-art measures significantly with regards to both effectiveness and efficiency.
Highlights
In information retrieval and machine learning, a good number of techniques utilize the similarity/distance measures to perform many different tasks [1]
The following tables contain the results of each similarity measure which were averaged on each Number of Features (NF) over all K values in the range [1...120] as drawn in the appendix
Conclusions and future work Using the bag of words (BoW) model, K-nearest neighbor algorithm (KNN) classifier, and K-means algorithm, in the context of text classification and clustering, this paper introduces a new similarity measure that is based on the set theory mechanism and is named the set theory-based similarity measure (STB-SM)
Summary
In information retrieval and machine learning, a good number of techniques utilize the similarity/distance measures to perform many different tasks [1]. Clustering and classification are the most widely-used techniques for the task of knowledge discovery within the scientific fields [2,3,4,5,6,7,8,9,10]. Text classification and clustering have long been vital research areas of information retrieval (IR). While text classification is the process of classifying the text/document into its actual class by utilizing a similarity measure and a proper classifier. The enhancement of classification performance has still been the main task for researchers in the text mining field
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call