Log-based indexing to improve web site search

Abstract

Despite the success of global search engines, web site search is still problematic in its retrieval accuracy. In this paper, we try to improve the performance of the site search by combining a new source of evidence from web server logs. We propose a novel approach of using server log analysis to extract terms to build the web page index. Then, this log-based index is combined with the text-based and anchor-based index to provide a more complete view on the page content. Experiments have shown that it could improve the effectiveness of the web site search significantly.