An intelligent web-page classifier with fair feature-subset selection

Abstract

The explosion of on-line information has given rise to many manually constructed topic hierarchies (such as Yahoo!!). But with the current growth rate in the amount of information, manual classification in topic hierarchies results in an immense information bottleneck. Therefore, developing an automatic classifier is an urgent need. However, the classifiers suffer from the enormous dimensionality, since the dimensionality is determined by the number of distinct keywords in a document corpus. More seriously, most classifiers are either working slowly or they are constructed subjectively without learning ability. In this paper, we address these problems with a fair feature Subset selection algorithm and an adaptive fuzzy learning network (AFLN) for classification. The fair feature subset selection algorithm is used to reduce the enormous dimensionality. It not only gives fair treatment to each category but also has ability to identify useful features, including both positive and negative features. On the other hand, the AFLN provides extremely fast training and testing and, more importantly, it has the ability to learn the human knowledge. Experimental results show that our proposed fair feature subset selection algorithm is effective in recognizing useful keywords for classification. It indeed can be used to reduce a surprising number of dimensions in classification models. Besides, experimental results also show the adaptive fuzzy learning network for classification with high-speed classification and high accuracy rate.