APPLE: a new compression scheme for bitmap indexes

Abstract

Compressed bitmap indexes are increasingly used in databases and search engines. By exploiting bit-level parallelism and bitwise operations, e.g. AND/OR operations, they can significantly accelerate the development of many areas. The Word Aligned Hybrid (WAH) bitmap compression scheme using run-length encoding (RLE), is commonly recognized as the most efficient scheme in terms of CPU-performance. This paper presents a new form of compressed bitmap indexes named Adaptive Partitioned Position List Encoding (APPLE), which uses packed position lists for compression. For experiments, we compare it with Huffman encoding, and two enhanced variants of WAH : Concise and COMPAX. Our empirical results show this scheme achieves significant improvement.