Bitmap Vectors Research Articles

HyBiX: A novel encoding bitmap index for space- and time-efficient query processing

A bitmap-based index is an effective and efficient indexing method for answering selective queries in a read- only environment. It offers improved query execution time by applying low-cost Boolean operators on the index directly, before accessing raw data. A drawback of the bitmap index is that index size increases with the cardinality of indexed attributes, which additionally has an impact on a query execution time. This impact is related to an increase of query execution time due to the scanning of bitmap vectors to answer the queries. In this paper, we propose a new encoding bitmap index, called the HyBiX bitmap index. The HyBiX bitmap index was experimentally compared to existing encoding bitmap indexes in terms of space requirement, query execution time, and space and time trade-off for equality and range queries. As experimental results, the HyBiX bitmap index can reduce space requirements with high cardinality attributes with satisfactory execution times for both equality and range queries. The performance of the HyBiX bitmap index provides the second-best results for equality queries and the first-best for range queries in terms of space and time trade-off.

Optimizing Range Query Processing for Dual Bitmap Index

Bitmap-based indexes are known to be the most effective indexing method for retrieving and answering selective queries in a read-only environment. Various types of encoding bitmap indexes significantly improve query time efficiency by utilizing fast Boolean operations directly on the index before retrieving the raw data. In particular, the dual bitmap index improves the performance of equality queries in terms of the space vs. time trade-off. However, the performance of range queries is unsatisfactory. In this paper, an optimizing algorithm is proposed to improve the range query processing for the dual bitmap index. The results of the experiment conducted show that the proposed algorithm, called Dual-simRQ, reduces the number of bitmap vectors scanned and the Boolean operations performed, which impacts the overall performance for range query processing.

Evaluating Aggregate Functions of Iceberg Query Using Priority Based Bitmap Indexing Strategy

Aggregate function and iceberg queries are important and common in many applications of data warehouse because users are generally interested in looking for variance or unusual patterns. Normally, the nature of the queries to be executed on data warehouse are the queries with aggregate function followed by having clause, these type of queries are known as iceberg query. Especially to have efficient techniques for processing aggregate function of iceberg query is very important because their processing cost is much higher than that of the other basic relational operations such as SELECT and PROJECT. Presently available iceberg query processing techniques faces the problem of empty bitwise AND,OR XOR operation and requires more I/O access and time.To overcome these problems proposed research provides efficient algorithm to execute iceberg queries using priority based bitmap indexing strategy. Priority based approach consider bitmap vector to be executed as per the priority.Intermediate results are evaluated to find probability of result.Fruitless operations are identified and skipped in advance which help to reduce I/O access and time.Time and iteration required to process query is reduced [45-50] % compare to previous strategy. Experimental result proves the superiority of priorty based approach compare to previous bitmap processing approach.

Tracking Pointer and Look Ahead Matching Strategy to Evaluate Iceberg Driven Query

Iceberg driven query is important and common in many applications of data mining and data warehousing. Main property of iceberg driven query is it extracts small set of data from huge database. It works on aggregation function followed by GROUP BY and HAVING clause. Due to involvement of aggregation function execution of iceberg driven query becomes tedious and complex work. Main objective of this research is to improve the performance of iceberg driven query by reducing the time, number of iteration and I/O access required to execute it. Currently available iceberg driven query processing technique faces the problems of empty bitwise AND, OR and XOR operation. Because of these problems they require more time and I/O access to execute query. To overcome above problems this research proposes tracking pointer and look ahead matching strategy to evaluate iceberg driven query. Tracking pointer will initiate the evaluation process as per the priority of vector. Look ahead matching strategy help to identify probable vector instead of generating one by one till the end of vector list. This strategy decides the probability of bitmap vector to be executed. Thus in advance it identifies and avoids unnecessary operations to be performed on bitmap vector. Our experimental result shows that time and number of iteration required to evaluate iceberg driven query using proposed approach is reduced [40 to 50] % even though data size increases. Thus we prove the effectiveness and efficiency of proposed approach to process iceberg driven query.

An Algorithm to Evaluate Iceberg Query using Compacted Bitmap Vector

The data storing and retrieving are playing a major role in the data clustering and data warehousing techniques. The effectiveness of a data retrieving method depends upon the data specific queries for retrieving the data from the database. Iceberg query is a unique class of aggregation query, which computes aggregate values above a given threshold. Many data mining queries are basically ice berg queries. The major part taken into the consideration about the AND operation in the iceberg queries. The reduced number of AND operation increases the effectiveness of the iceberg query. In this work, an efficient iceberg query evaluation process is proposed by reducing the bitwise AND operations needed to find the item pairs. In the proposed approach two solutions are introduced to reduce the bitwise AND operations. Randomly identifying 'N' 1-bit positions instead of first 1-bit position and reducing the zero bit values from the Most Significant Side so that the bit map vector will be reduced in such a way, the bitwise operations needed is reduced. The experimentation is conducted on two datasets in order to evaluate the performance of the proposed iceberg query evaluation algorithm. In the case of retail datasets, the time required for processing 100000 tuples is 864 milliseconds. On the other hand, the same for synthetic dataset, T10I4D100K, is 910 milliseconds.

MiNT-OLAP cluster: minimizing network transmission cost in OLAP cluster for main memory analytical database

Powerful storage, high performance and scalability are the most important issues for analytical databases. These three factors interact with each other, for example, powerful storage needs less scalability but higher performance, high performance means less consumption of indexes and other materializations for storage and fewer processing nodes, larger scale relieves stress on powerful storage and the high performance processing engine. Some analytical databases (ParAccel, Teradata) bind their performance with advanced hardware supports, some (Asterdata, Greenplum) rely on the high scalability framework of MapReduce, some (MonetDB, Sybase IQ, Vertica) highlight performance on processing engine and storage engine. All these approaches can be integrated into an storage-performance-scalability (S-P-S) model, and future large scale analytical processing can be built on moderate clusters to minimize expensive hardware dependency. The most important thing is a simple software framework is fundamental to maintain pace with the development of hardware technologies. In this paper, we propose a schema-aware on-line analytical processing (OLAP) model with deep optimization from native features of the star or snowflake schema. The OLAP model divides the whole process into several stages, each stage pipes its output to the next stage, we minimize the size of output data in each stage, whether in central processing or clustered processing. We extend this mechanism to cluster processing using two major techniques, one is using NetMemory as a broadcasting protocol based dimension mirror synchronizing buffer, the other is predicate-vector based DDTA-OLAP cluster model which can minimize the data dependency of star-join using bitmap vectors. Our OLAP model aims to minimize network transmission cost (MiNT in short) for OLAP clusters and support a scalable but simple distributed storagemodel for large scale clustering processing. Finally, the experimental results show the speedup and scalability performance.

Efficient Iceberg Query Evaluation Using Compressed Bitmap Index

Decision support and knowledge discovery systems often compute aggregate values of interesting attributes by processing a huge amount of data in very large databases and/or warehouses. In particular, iceberg query is a special type of aggregation query that computes aggregate values above a user-provided threshold. Usually, only a small number of results will satisfy the threshold constraint. Yet, the results often carry very important and valuable business insights. Because of the small result set, iceberg queries offer many opportunities for deep query optimization. However, most existing iceberg query processing algorithms do not take advantage of the small-result-set property and rely heavily on the tuple-scan-based approach. This incurs intensive disk accesses and computation, resulting in long processing time especially when data size is large. Bitmap index, which builds one bitmap vector for each attribute value, is gaining popularity in both column-oriented and row-oriented databases in recent years. It occupies less space than the raw data and gives opportunities for more efficient query processing. In this paper, we exploited the property of bitmap index and developed a very effective bitmap pruning strategy for processing iceberg queries. Our index-pruning-based approach eliminates the need of scanning and processing the entire data set (table) and thus speeds up the iceberg query processing significantly. Experiments show that our approach is much more efficient than existing algorithms commonly used in row-oriented and column-oriented databases.

Toward In-Band Self-Organization in Energy-Efficient MAC Protocols for Sensor Networks

This paper presents a self-organizing medium access control (MAC) protocol framework for distributed sensor networks with arbitrary mesh topologies. The novelty of the proposed In-band self-organized MAC (ISOMAC) protocol lies in its in-band control mechanism for exchanging time-division multiple access (TDMA) slot information with distributed MAC scheduling. A fixed-length bitmap vector is used in each packet header for exchanging relative slot timing information across immediate and up to two-hop neighbors. It is shown that, by avoiding explicit timing information exchange, ISOMAC can work without networkwide time synchronization, which can be prohibitive for severely cost-constrained sensor nodes in very large networks. A slot-clustering effect, caused by in-band bitmap constraints, enables ISOMAC to offer better spatial channel reuse compared to traditional distributed TDMA protocols. ISOMAC employs a partial node wake-up and header-only transmission strategy to adjust energy expenditure based on the instantaneous nodal data rate. Both analytical and simulation models have been developed for characterizing the proposed protocol. Results demonstrate that, with in-band bitmap vectors of moderate length, ISOMAC converges reasonably quickly, that is, approximately within a four to eight TDMA frame duration. Also, if the bitmap header duration is restricted within 10 percent of packet duration, then the energy penalty of the in-band information is quite negligible. It is also shown that ISOMAC can be implemented in the presence of network time synchronization, although its performance without synchronization is just marginally worse than that with synchronization.

Multi-resolution bitmap indexes for scientific data

The unique characteristics of scientific data and queries cause traditional indexing techniques to perform poorly on scientific workloads, occupy excessive space, or both. Refinements of bitmap indexes have been proposed previously as a solution to this problem. In this article, we describe the difficulties we encountered in deploying bitmap indexes with scientific data and queries from two real-world domains. In particular, previously proposed methods of binning, encoding, and compressing bitmap vectors either were quite slow for processing the large-range query conditions our scientists used, or required excessive storage space. Nor could the indexes easily be built or used on parallel platforms. In this article, we show how to solve these problems through the use of multi-resolution, parallelizable bitmap indexes, which support a fine-grained trade-off between storage requirements and query performance. Our experiments with large data sets from two scientific domains show that multi-resolution, parallelizable bitmap indexes occupy an acceptable amount of storage while improving range query performance by roughly a factor of 10, compared to a single-resolution bitmap index of reasonable size.