Efficient query processing and optimization in SQL using compressed bitmap indexing for set predicates

Abstract

In today's complex world requires state-of-the-art data analysis over massive data sets. In data warehousing and OLAP applications, scalar-level predicates of set in SQL become highly inadequate which needs to support set-level comparison semantics, i.e., comparing a group of tuples with set of values. Complex queries composed by scalar-level operations are challenging for database engine to optimize, which results in costly evaluation. Bitmap indexing provides an important database capability to accelerate queries. Few database systems have implemented these indexes because of the difficulties of modifying fundamental assumptions in the low-level design of a database system. Bitmap index built one bitmap vector for each attribute value is gaining popularity in both column-oriented and row-oriented databases. It requires less space than the raw data provides opportunities for more efficient query processing. In this paper, we studied the property of bitmap index and developed a very effective bitmap pruning strategy for processing queries. Such index-pruning-based approach eliminates the need of scanning and processing the entire data set and thus speeds up the query processing significantly. Our approach is much more efficient than existing algorithms commonly used in row-oriented and column oriented databases.