An intelligent web-page classifier with fair feature-subset selection

Abstract

The explosion of on-line information has given rise to many manually constructed topic hierarchies (such as Yahoo!!). But with the current growth rate in the amount of information, manual classification in topic hierarchies results in an immense information bottleneck. Therefore, developing an automatic classifier is an urgent need. However, classifiers suffer from enormous dimensionality, since the dimensionality is determined by the number of distinct keywords in a document corpus. More seriously, most classifiers are either working slowly or they are constructed subjectively without any learning ability. In this paper, we address these problems with a fair feature-subset selection (FFSS) algorithm and an adaptive fuzzy learning network (AFLN) for classification. The FFSS algorithm is used to reduce the enormous dimensionality. It not only gives fair treatment to each category but also has ability to identify useful features, including both positive and negative features. On the other hand, the AFLN provides extremely fast learning ability to model the uncertain behavior for classification so as to correct the fuzzy matrix automatically. Experimental results show that both FFSS algorithm and the AFLN lead to a significant improvement in document classification, compared to alternative approaches.