Framework for query optimization in distributed statistical databases

Abstract

Recently, there has been a growing interest in statistical database (SDB) research. When SDBs are dispersed among computing facilities at various sites (e.g., in health-care networks) an additional dimension is added to the already difficult problems faced by the SDB designer. A distributed statistical database management system (DS-DBMS) consists of micro data (i.e., raw data) and macro data (i.e., aggregated objects called summary tables), which can be considered essentially as aggregated views of the raw data in a special format. The first part of the paper gives an overview of a model for the representation of both raw data (micro data) and summary tables (macro data). The model is an extension of the relational model (so that existing distributed database systems can be exploited). Most of the first part is devoted to defining operations on macro data sets. Based on these operations, a set of equivalent relational operations is described, as one of the main objectives in defining the micro and macro data sets, and the operations on them, has been to use as much as possible the capabilities that are already offered by most relational DBMSs. The second part of the paper deals with one of the important aspects of performance in a DS-DBMS, namely, the efficient processing of queries. This is heavily influenced by the performance of query optimizers. However, to provide query optimization in a DS-DBMS, special issues are raised that manifest themselves in different scenarios. Some of the important issues and problems raised are discussed and solutions proposed. In addition, a set of transformations on macro operations (similar to those in relational algebra) are introduced, which can be used for optimizing queries in DS-DBMSs.