Abstract
Feature selection that aims to determine and select the distinctive terms representing a best document is one of the most important steps of classification. With the feature selection, dimension of document vectors are reduced and consequently duration of the process is shortened. In this study, feature selection methods were studied in terms of dimension reduction rates, classification success rates, and dimension reduction-classification success relation. As classifiers, kNN (k-Nearest Neighbors) and SVM (Support Vector Machines) were used. 5 standard (Odds Ratio-OR, Mutual Information-MI, Information Gain-IG, Chi-Square-CHI and Document Frequency-DF), 2 combined (Union of Feature Selections-UFS and Correlation of Union of Feature Selections-CUFS) and 1 new (Sum of Term Frequency-STF) feature selection methods were tested. The application was performed by selecting 100 to 1000 terms (with an increment of 100 terms) from each class. It was seen that kNN produces much better results than SVM. STF was found out to be the most successful feature selection considering the average values in both datasets. It was also found out that CUFS, a combined model, is the one that reduces the dimension the most, accordingly, it was seen that CUFS classify the documents more successfully with less terms and in short period compared to many of the standard methods.
Highlights
Text based data amounts reached enormous sizes on the web as a result of increasing number of computers, tablets and smart phones and their widespread use. This fact resulting from widespread use of technology caused changes in people’s habits
One of the instances of this is the topic of this paper which is news portals
This study aims to analyze the effects of feature selection methods on text classification
Summary
Text based data amounts reached enormous sizes on the web as a result of increasing number of computers, tablets and smart phones and their widespread use. This fact resulting from widespread use of technology caused changes in people’s habits. Columnists are one of the features that readers follow mostly on a news portal. A columnist may refer to various topics, in other words, more than one topic, write about a topic outside his area of interest and even title of his article may not be consistent with the content, being incoherent. Classification of an article in terms of its topic is important in order to give information about its content to readers
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: International Journal of Advanced Computer Science and Applications
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.