SiteHelper: a localized agent that helps incremental exploration of the World Wide Web

Abstract

The World Wide Web (the Web for short) is rapidly becoming an information flood as it continues to grow exponentially. This causes difficulty for users to find relevant pieces of information on the Web. Search engines and robots (spiders) are two popular techniques developed to address this problem. Search engines are indexing facilities over searchable databases. As the Web continues to expand, search engines are becoming redundant because of the large number of Web pages they return for a single search. Robots are similar to search engines; rather than indexing the Web, they traverse (“walk through”) the Web, analyzing and storing relevant documents. The main drawback of these robots is their high demand on network resources that results in networks being overloaded. This paper proposes an alternate way in assisting users in finding information on the Web. Since the Web is made up of many Web servers, instead of searching all the Web servers, we propose that each server does its own housekeeping. A software agent named SiteHelper is designed to act as a housekeeper for the Web server and as a helper for a Web user to find relevant information at a particular site. In order to assist the Web user finding relevant information at the local site, SiteHelper interactively and incrementally learns about the Web user's areas of interest and aids them accordingly. To provide such intelligent capabilities, SiteHelper deploys enhanced HCV with incremental learning facilities as its learning and inference engines.