Mobile Agents for Distributed and Heterogeneous Information Retrieval

Abstract

The heterogeneous, distributed and voluminous nature of many government and corporate data sources impose severe constraints on meeting the diverse requirements of users who analyze the data. Additionally, communication bandwidth limitations, time constraints, and multiple data formats impose further restrictions on users of these distributed data sources. In this paper, we present an Agent-based Complex QUerying and Information Retrieval Engine (ACQUIRE) for large, heterogeneous, and distributed data sources. ACQUIRE acts as a softbot or interface agent by presenting users with a view of a single, unified, homogenous data source, against which users can pose high-level declarative queries. ACQUIRE translates each such user query into a set of sub-queries by employing a combination of planning and traditional database query optimization techniques. ACQUIRE then spawns a set of mobile agents corresponding to these sub-queries, which in turn retrieve the data from various distributed data sources by dynamically optimizing the retrieval strategy as it is carried out. These mobile agents carry with them data-processing code that can be executed at the remote site, thus reducing the size of data returned by the agent. When all mobile agents have returned, ACQUIRE filters and merges the retrieved data and presents the results to the user. While the system is still very much a work in progress, current validation experiments on simulated NASA Distributed Active Archive Centers (DAACs) have demonstrated that complex queries can be effectively decomposed and retrieved by this approach.