Cracking in-memory database index: A case study for Adaptive Radix Tree index

Abstract

Indexes provide a method to access data in databases quickly. It can improve the response speed of subsequent queries by building a complete index in advance. However, it also leads to a huge overhead of the continuous updating during creating the index. An in-memory database usually has a higher query processing performance than disk databases and is more suitable for real-time query processing. Therefore, there is an urgent need to reduce the index creation and update cost for in-memory databases. Database cracking technology is currently recognized as an effective method to reduce the index initialization time. However, conventional cracking algorithms are focused on simple column data structure rather than those complex index structures for in-memory databases. In order to show the feasibility of in-memory database index cracking and promote to future more extensive research, this paper conducted a case study on the Adaptive Radix Tree (ART), a popular tree index structure of in-memory databases. On the basis of carefully examining the ART index construction overhead, an algorithm using auxiliary data structures to crack the ART index is proposed. This makes it possible to build up an ART index step by step with incessant queries, and hence avoids the poor instant availability of a complete index which is constructed once and for all, but is time consuming. Furthermore, updating a cracking ART index is considered as well. Extensive experiments show that the average initialization time of the ART cracker index is reduced by 75%, and the query response time gradually approaches the original ART algorithm with the coming queries.