Cardinality estimation in numeric on-line databases

Abstract

Numeric on-line databases (NDBs) have become essential in information retrieval (IR). NDBs differ from traditional bibliographic databases (BDBs) with respect to their content, structural complexity, data manipulation capabilities and the complexity of the user interfaces and user charging schemes. Recent trends in user charging for all on-line IR are toward charging for the information actually retrieved from the database rather than for the connect-time. However, the viability of such charging schemes depends on the user's possibilities of estimating the charges in advance, during the query negotiating phase. The problem of advance charge estimation in NDBs is addressed in this paper. In order to solve this problem, a method for estimating the numbers of data items satisfying the query expressions, i.e. the query cardinalities, is required. In this paper, an approach for advance estimation of query charges is developed and, based on it, a systematic and general method for query cardinality estimation is defined. The approach and the method are based on the well-known relational data model (RDM). The method is adapted to the new application area, user charge estimation in NDBs, and provides several improvements with respect to earlier cardinality estimation methods. Based on the method, several types of user charges can be estimated in advance. Tools based on the method are necessary components of query interfaces to NDBs if non-connect-timebased charging is used. The approach and the method are directly applicable to RDM-based NDBs.