Bitmap Index Research Articles

SPIMulator: A Spintronic Processing-In-Memory Simulator for Racetracks

In-memory processing is becoming a popular method to alleviate the memory bottleneck of the von Neumann computing model. With the goal of improving both latency and energy cost associated with such in-memory processing, emerging non-volatile memory technologies, such as Spintronic magnetic memory, are of particular interest as they can provide a near-SRAM read/write performance and eliminate nearly all static energy without experiencing any endurance limitations. Spintronic Racetrack Memory (RM) further addresses density concerns of spin-transfer torque memory (STT-MRAM). Moreover, it has recently been demonstrated that portions of RM nanowires can function as a polymorphic gate, which can be leveraged to implement multi-operand bulk bitwise operations. With more complex control, they can also be leveraged to build arithmetic integer and floating point processing in memory (PIM) primitives. This paper proposes SPIMulator, a Spintronic PIM sim ulator that can simulate the storage and PIM architecture of executing PIM commands in Racetrack memory. SPIMulator functionally models the polymorphic gate properties recently proposed for Racetrack memory, which allows transverse access that determines the number of ‘1’s in a segment of each Racetrack nanowire. From this simulation, SPIMulator can report real-time performance statistics such as cycle count and energy. Thus, SPIMulator simulates the multi-operand bit-wise logic operations recently proposed and can be easily extended to implement new PIM operations as they are developed. Due to the functional nature of SPIMulator, it can serve as a programming environment that allows development of PIM-based codes for verification of new acceleration algorithms. We demonstrate the value of SPIMulator through the modeling and estimations of performance and energy consumption of a variety of example applications, including the Advanced Encryption Standard (AES) for encryption primarily based on logical and look-up operations; multiplication of matrices, a frequent requirement in scientific, signal processing, and machine learning algorithms; and bitmap indices a common search table employed for database lookups.

Toward Forward and Backward Private Dynamic Searchable Symmetric Encryption Supporting Data Deduplication and Conjunctive Queries

In Internet of Things (IoT) application scenarios such as intelligent logistics, secure data access control and sharing based on dynamic searchable symmetric encryption (DSSE) has become a research hotspot in recent years. DSSE is an encryption technology which gratifies the above requirements, while protecting the sensitive data during the operation. The existing DSSE schemes do not achieve data deduplication and conjunctive queries concurrently, and exist problems such as complex update process, inflexible query method and privacy disclosure, etc. For this purpose, the paper proposes dynamic searchable symmetric encryption for data deduplication and conjunctive queries (DSSE-DC), and constructs a concrete DSSE-DC scheme. Our scheme uses symmetric encryption with homomorphic addition and bitmap index to develop a secure and reliable search architecture, and updates through modular addition. Meanwhile, we add a deduplication mechanism to realize data deduplication which avoids the waste of cloud storage resources. Furthermore, we introduce the idea of inner product matching to achieve efficient conjunctive queries. The adaptive security of DSSE-DC scheme is proved in the random oracle model. Whilst our scheme satisfies forward and backward privacy. In comparison to the existing DSSE schemes for conjunctive queries, our scheme has advantage in update and search performances, which is applicable to IoT applications like intelligent logistics.

Towards Privacy Preserving in 6G Networks: Verifiable Searchable Symmetric Encryption Based on Blockchain

The blockchain-based searchable symmetric encryption (SSE) scheme allows the retrieval and verification of outsourced data on cloud servers in sixth generation (6G) networks while ensuring the privacy of data. However, existing schemes are challenging to comprehensively meet the requirements of 6G-based intelligent application systems for low latency, high security, and high reliability. To address these limitations, we present VSSE, a novel blockchain-based SSE scheme designed for 6G-based intelligent application systems. Our scheme constructs a state chain structure to resist file injection attacks, thereby ensuring forward privacy. Moreover, we execute the search and verification operations separately on the cloud server and blockchain, while introducing a bitmap index structure and message authentication code (MAC) technology to achieve efficient searching and dynamic verification. Notably, VSSE also includes access control functionality, permitting only authorized users to access relevant files. The combination of remarkable efficiency and strong security establishes our VSSE as an ideal solution suitable for implementation in G-based intelligent application systems.

SecBerg: Secure and Practical Iceberg Queries in Cloud

Secure queries are fundamental to data security, particularly in cloud databases. In data analytics, one of the common and practical queries is the iceberg query that can find aggregate values above a specified threshold. However, existing secure aggregate query schemes: i) are unable to support secure iceberg queries equipped with the HAVING clause; ii) only consider additive aggregate functions; and iii) suffer from performance issues due to the use of homomorphic encryption to encrypt databases. In this paper, we present a secure iceberg query scheme, SecBerg, to support both addition-based and comparison-based aggregate functions and ensure high efficiency and security simultaneously. To make it possible, we propose a secure bitmap index system to encode database values and pioneer the use of the arithmetic secret sharing technique to protect databases in the cloud environment. Furthermore, we carefully design efficient and secure protocols over arithmetic secret sharing to construct our SecBerg. Extensive evaluations are conducted, and the results indicate that SecBerg is significantly more efficient than the state-of-the-art relevant scheme in computational overhead and can attain orders of magnitude performance improvement at best.

Detecting Logic Bugs of Join Optimizations in DBMS

Generation-based testing techniques have shown their effectiveness in detecting logic bugs of DBMS, which are often caused by improper implementation of query optimizers. Nonetheless, existing generation-based debug tools are limited to single-table queries and there is a substantial research gap regarding multi-table queries with join operators. In this paper, we propose TQS, a novel testing framework targeted at detecting logic bugs derived by queries involving multi-table joins. Given a target DBMS, TQS achieves the goal with two key components: Data-guided Schema and Query Generation (DSG) and Knowledge-guided Query Space Exploration (KQE). DSG addresses the key challenge of multi-table query debugging: how to generate ground-truth (query, result) pairs for verification. It adopts the database normalization technique to generate a testing schema and maintains a bitmap index for result tracking. To improve debug efficiency, DSG also artificially inserts some noises into the generated data. To avoid repetitive query space search, KQE forms the problem as isomorphic graph set discovery and combines the graph embedding and weighted random walk for query generation. We evaluated TQS on four popular DBMSs: MySQL, MariaDB, TiDB and PolarDB. Experimental results show that TQS is effective in finding logic bugs of join optimization in database management systems. It successfully detected 115 bugs within 24 hours, including 31 bugs in MySQL, 30 in MariaDB, 31 in TiDB, and 23 in PolarDB respectively.

The application of Internet of things and Oracle database in the research of intelligent data management system

Aiming at the needs of general large-scale monitoring systems, this paper designs an intelligent data management system that can provide fast data query and relieve sudden data congestion through in-depth research on Oracle database and data division. To the data access request from the front desk, the system can respond quickly through the real-time data monitoring module and the online analysis software OLAP mode database, which has far-reaching significance for the development of the Internet of Things and related systems. The experimental results show that, compared with the traditional system, the same bitmap index only occupies about 1/30 of the original table, and the data size is reduced by more than 10 times. The experimental data verifies the characteristics of the system in this paper to strengthen the background data receiving and processing capabilities, and alleviate the problems such as the reduction of the system running rate and even the system paralysis caused by the sudden mass data.

An efficient iterative hybrid multilevel fast multipole algorithm‐ physical optics method for simulating antenna mounted on the electrically large platform

An efficient iterative hybrid multilevel fast multipole algorithm (MLFMA) and the physical optics (PO) is presented to efficiently analyze the large size antenna mounted on the electrically large platform, in which, the MLFMA is used to accelerate the impedance matrix–vector multiply procedure of method of moment (MoM) and the PO is employed to simulate the large platform. In this hybrid scheme, to overcome the computational cost required in the evaluation of the electrically large PO region contribution to the MLFMA region, the strategy of the efficient iterative MoM-PO (EI-MoM-PO) is employed. In addition, the electrically large PO region shadowing effect coefficient will be checked before the computation of PO current, then the coefficient will be stored in bit format by using the Bitmap index method which is general enough to reduce the CPU time and memory consumption. The combination of EI-MLFMA-PO and Bitmap assists us to construct a proposed method that can reduce the computation cost dramatically and obtain good accuracy. Several examples are demonstrated to validate and prove the accuracy and efficiency of our method compared with the hybrid MLFMA-PO method in FEKO.

Study on foot-type classification for young adults based on static and dynamic plantar pressure distribution

The application of an effective and reliable foot type classification method is very important for foot type judgment, injury risk assessment, and correction. Therefore, this paper mainly aims to propose a new foot type classification method for young people based on bitmap index (BI), compare it with the traditional footprint classification method, and put forward and analyze the factors affecting foot type classification. Thirty-one healthy volunteers were asked to perform two types tests in order to study the plantar pressure distribution with static and dynamic conditions, the first type is footprint test with full load of static, and the other type is plantar pressure distribution, which contains four different tests: no load, half load, and full load of static state, as well as dynamic plantar pressure distribution during process of walking. The Intraclass Correlation Coefficient results (ICC) were good reliable and reproducible for BI value with dynamic test (DT-BI value 0.738, 95% confidence interval [0.535, 0.848], [Formula: see text]) and full-load (FL-BI value 0.725 [0.281, 0.814], [Formula: see text], p < 0.001), obtained with individual measures and a two-way mixed-effects model. It can be seen from Kappa coefficient and density map that DT-BI has high classification accuracy. Classification of foot type based on bitmap index values showed good reliability in people with varying BMI, which can help clinicians and researchers segment the sample population to better distinguish between different foot types of activity, gait or treatment effects.

Faster Multidimensional Data Queries on Infrastructure Monitoring Systems

The analytics in online performance monitoring systems have often been limited due to the query performance of large scale multidimensional data. In this paper, we introduce a faster query approach using the bit-sliced index (BSI). Our study covers multidimensional grouping and preference top-k queries with the BSI, algorithms design, time complexity evaluation, and the query time comparison on a real-time production performance monitoring system. Our research work extended the BSI algorithms to cover attributes filtering and multidimensional grouping. We evaluated the query time with the single attribute, multiple attributes, feature filtering, and multidimensional grouping. To compare with the existing prior arts, we made a benchmarking comparison with the bitmap indexing, sequential scan, and collection streaming grouping. In the result of our experiments with large scale production data, the proposed BSI approach outperforms the existing prior arts: 3 times faster than the bitmap indexing approach on single attribute top-k queries, 10 times faster than the collection stream approach on the multidimensional grouping. While comparing with the baseline sequential scan approach, our proposed algorithm BSI approach outperforms the sequential scan approach with a factor of 10 on multiple attributes queries and a factor of 100 on single attribute queries. In the previous research, we had evaluated the BSI time complexity and space complexity on simulation data with various distributions, this research work further studied, evaluated, and concluded the BSI approach query performance with real production data.

Database Padding for Dynamic Symmetric Searchable Encryption

Dynamic symmetric searchable encryption (DSSE) that enables the search and update of encrypted databases outsourced to cloud servers has recently received widespread attention for leakage-abuse attacks against DSSE. In this paper, we propose a dynamic database padding method to mitigate the threat of data leakage during the update operation of outsourcing data. First, we introduce an outlier detection technology where bogus files are generated for padding according to the outlier factors, hiding the document information currently matching search keywords. Furthermore, we design a new index structure suitable for the padded database using the bitmap index to simplify the update operation of the encrypted index. Finally, we present an application scenario of the padding method and realize a forward and backward privacy DSSE scheme (named PDB-DSSE). The security analysis and simulation results show that our dynamic padding algorithm is suitable for DSSE scheme and PDB-DSSE scheme maintains the security and efficiency of the retrieval and update of the DSSE scheme.

Indexing structures for the PLS blockchain

This paper studies known indexing structures from a new point of view: minimisation of data exchange between an IoT device acting as a blockchain client and the blockchain server running a protocol suite that includes two Guy Fawkes protocols, PLS and SLVP. The PLS blockchain is not a cryptocurrency instrument; it is an immutable ledger offering guaranteed non-repudiation to low-power clients without use of public key crypto. The novelty of the situation is in the fact that every PLS client has to obtain a proof of absence in all blocks of the chain to which its counterparty does not contribute, and we show that it is possible without traversing the block’s Merkle tree. We obtain weight statistics of a leaf path on a sparse Merkle tree theoretically, as our ground case. Using the theory we quantify the communication cost of a client interacting with the blockchain. We show that large savings can be achieved by providing a bitmap index of the tree compressed using Tunstall’s method. We further show that even in the case of correlated access, as in two IoT devices posting messages for each other in consecutive blocks, it is possible to prevent compression degradation by re-randomising the IDs using a pseudorandom bijective function. We propose a low-cost function of this kind and evaluate its quality by simulation, using the avalanche criterion.

Top-k dominating queries on incomplete large dataset.

Top-k dominating (TKD) query is one of the methods to find the interesting objects by returning the k objects that dominate other objects in a given dataset. Incomplete datasets have missing values in uncertain dimensions, so it is difficult to obtain useful information with traditional data mining methods on complete data. BitMap Index Guided Algorithm (BIG) is a good choice for solving this problem. However, it is even harder to find top-k dominance objects on incomplete big data. When the dataset is too large, the requirements for the feasibility and performance of the algorithm will become very high. In this paper, we proposed an algorithm to apply MapReduce on the whole process with a pruning strategy, called Efficient Hadoop BitMap Index Guided Algorithm (EHBIG). This algorithm can realize TKD query on incomplete datasets through BitMap Index and use MapReduce architecture to make TKD query possible on large datasets. By using the pruning strategy, the runtime and memory usage are greatly reduced. What’s more, we also proposed an improved version of EHBIG (denoted as IEHBIG) which optimizes the whole algorithm flow. Our in-depth work in this article culminates with some experimental results that clearly show that our proposed algorithm can perform well on TKD query in an incomplete large dataset and shows great performance in a Hadoop computing cluster.

An Improved Framework for Bitmap Indexes and their Use in Data Warehouse Optimization

Compression technique is basically used to compress the size of table or reduce the storage area. Oracle already gives this feature for the table compression as well as for the index compression. when index is created on particular column of a table then it contain some space, which require some storage or disk space by this technique we can save our disk space because in industry the company have to purchase the disk space according to the size of the their data and pay according to their disk space. To utilize this disk space for useful records data rather than wasting it. In this paper used the data pump utility for the compression of Bitmap index and table. Data pump utility performed for the logical backups in database.in this paper implemented data pump for compression, to release the space and change the index pointing location. It will not release the space even after deletion of records. This is of special interest for the case to compress the bitmap index and table space along with the’S (Data Manipulation Language).

An Optimization of Bitmap Index Compression Technique in Bulk Data Movement Infrastructure

An index is an optional structure, associated with a table or clustered table that can sometimes speed up data access. By creating an index on one or more columns of a table, you gain the ability, in some cases, to retrieve a small set of randomly distributed rows from the table. This research aims to free the filled disk space after documents have been erased. We verify the current compression technologies and apply the data pump to the bitmap index compression. Bitmap is more effective for minimal unique value, according to the traditional wisdom index. However, it needs no Bitmap index at high cardinality or low cardinality by data pump. In this paper we recommend a pump function for space release in the database after documentation has been removed. This utility does not free disk space even though the elimination of records from the table is seen by the Bitmap Index. To free up the room and adjust the index point position, we have introduced data pump for compression. The data pump is commonly used in oracle database for logical backups. Finally, we study the bitmap index which is widely used for industrial purposes and address unanswered questions for the future.

An Efficient Indexing Scheme for Network Traffic Collection and Retrieval System

Historical network traffic retrieval, both at the packet and flow level, has been applied in many fields of network security, such as network traffic analysis and network forensics. To retrieve specific packets from a vast number of packet traces, it is an effective solution to build indexes for the query attributes. However, it brings challenges of storage consumption and construction time overhead for packet indexing. To address these challenges, we propose an efficient indexing scheme called IndexWM based on the wavelet matrix data structure for packet indexing. Moreover, we design a packet storage format based on the PcapNG format for our network traffic collection and retrieval system, which can speed up the extraction of index data from packet traces. Offline experiments on randomly generated network traffic and actual network traffic are performed to evaluate the performance of the proposed indexing scheme. We choose an open-source and widely used bitmap indexing scheme, FastBit, for comparison. Apart from the native bitmap compression method Word-Aligned Hybrid (WAH), we implement an efficient bitmap compression method Scope-Extended COMPAX (SECOMPAX) in FastBit for performance evaluation. The comparison results show that our scheme outperforms the selected bitmap indexing schemes in terms of time consumption, storage consumption and retrieval efficiency.

Exploring Means to Enhance the Efficiency of GPU Bitmap Index Query Processing

Once exotic, computational accelerators are now commonly available in many computing systems. Graphics processing units (GPUs) are perhaps the most frequently encountered computational accelerators. Recent work has shown that GPUs are beneficial when analyzing massive data sets. Specifically related to this study, it has been demonstrated that GPUs can significantly reduce the query processing time of database bitmap index queries. Bitmap indices are typically used for large, read-only data sets and are often compressed using some form of hybrid run-length compression. In this paper, we present three GPU algorithm enhancement strategies for executing queries of bitmap indices compressed using word aligned hybrid compression: (1) data structure reuse (2) metadata creation with various type alignment and (3) a preallocated memory pool. The data structure reuse greatly reduces the number of costly memory system calls. The use of metadata exploits the immutable nature of bitmaps to pre-calculate and store necessary intermediate processing results. This metadata reduces the number of required query-time processing steps. Preallocating a memory pool can reduce or entirely remove the overhead of memory operations during query processing. Our empirical study showed that performing a combination of these strategies can achieve 32.4times to 98.7times speedup over the current state-of-the-art implementation. Our study also showed that by using our enhancements, a common gaming GPU can achieve a 15.0times speedup over a more expensive high-end CPU.

Optimizing bitmap index encoding for high performance queries

SummaryMany sources such as historical archives, sensor readings, health systems, and machine records produce ever‐increasing but often unchanging data. These accumulating data create a need for faster processing. Bitmap index, which can take advantage of multi‐core and multiprocessor systems, is designed to process data that increase over time but do not change frequently. It has a well‐known advantage, especially in queries on data with low cardinality. However, bitmap index can handle high cardinality data efficiently because it can use its own compression algorithm. Bitmap index has many encoding schemes that affect query processing time. In this study, we developed an algorithm that improves query performance by using optimal encoding among bitmap encodings. With this optimization algorithm, we witnessed up to 40% performance increase in queries made with bitmap indexes created with different encodings. Furthermore, in comparison with a commonly used relational database, we found significant improvements in the number of query operations per second performed on optimized encoded bitmap indexes generated by the introduced algorithm.

Withdrawal Notice: Design and Analysis of Optimization and Tuning in Data Warehouses Using Bitmap Indexes

The article has been withdrawn at the request of the author of the journal Recent Advances in Computer Science and Communications due to incoherent content. Bentham Science apologizes to the readers of the journal for any inconvenience this may have caused. The Bentham Editorial Policy on Article Withdrawal can be found at https://benthamscience.com/editorial-policies-main.php Bentham Science Disclaimer: It is a condition of publication that manuscripts submitted to this journal have not been published and will not be simultaneously submitted or published elsewhere. Furthermore, any data, illustration, structure or table that has been published elsewhere must be reported, and copyright permission for reproduction must be obtained. Plagiarism is strictly forbidden, and by submitting the article for publication the authors agree that the publishers have the legal right to take appropriate action against the authors, if plagiarism or fabricated information is discovered. By submitting a manuscript, the authors agree that the copyright of their article is transferred to the publishers if and when the article is accepted for publication.

Parallel acceleration of CPU and GPU range queries over large data sets

Data management systems commonly use bitmap indices to increase the efficiency of querying scientific data. Bitmaps are usually highly compressible and can be queried directly using fast hardware-supported bitwise logical operations. The processing of bitmap queries is inherently parallel in structure, which suggests they could benefit from concurrent computer systems. In particular, bitmap-range queries offer a highly parallel computational problem, and the hardware features of graphics processing units (GPUs) offer an alluring platform for accelerating their execution.In this paper, we present four GPU algorithms and two CPU-based algorithms for the parallel execution of bitmap-range queries. We show that in 98.8% of our tests, using real and synthetic data, the GPU algorithms greatly outperform the parallel CPU algorithms. For these tests, the GPU algorithms provide up to 54.1 × speedup and an average speedup of 11.5× over the parallel CPU algorithms. In addition to enhancing performance, augmenting traditional bitmap query systems with GPUs to offload bitmap query processing allows the CPU to process other requests.

BitFun

Geospatial array DBMSs handle big georeferenced arrays. Due to the geospatial data peculiarities, many queries have tunable parameters with values not known in advance: users gradually tune them until they get a satisfactory result. This generates a series of queries with slightly different structures and very similar outputs. Modern array DBMSs spend the same efforts to answer each such query. BitFun provides novel bitmap indexing strategies to continuously re-index arrays during queries with similar mathematical functions. It can be up to 8x faster than computing the results from scratch. We describe BitFun and offer lessons on real-world geospatial data, related to real practical tasks. A lesson involves tuning a math function parameter while the rich web GUI details the indexing process and query execution. Conference attendees will appreciate BitFun approaches, its performance, and learn its internals via fascinating lessons.