Effective Topic Distillation with Key Resource Pre-selection

Abstract

AbstractTopic distillation aims at finding key resources which are high-quality pages for certain topics. With analysis in non-content features of key resources, a pre-selection method is introduced in topic distillation research. A decision tree is constructed to locate key resource pages using query-independent non-content features including in-degree, document length, URL-type and two new features we found out involving site’s self-link structure analysis. Although the result page set contains only about 20% pages of the whole collection, it covers more than 70% of key resources. Furthermore, information retrieval on this page set makes more than 60% improvement with respect to that on all pages. These results were achieved using TREC 2002 web track topic distillation task for training and TREC 2003 corresponding task for testing. It shows an effective way of getting better performance in topic distillation with a dataset significantly smaller in size.KeywordsDocument LengthText RetrievalAnchor TextPage LengthNIST Special PublicationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.