Bulk Loading of the Secondary Index in LSM-Based Stores for Flash Memory

Abstract

AbstractNoSQL databases have gained much attention recently. Due to utilizing Log Structured Merge (LSM) tree, they support fast write throughput and fast lookups on primary keys. Nevertheless, the implementation of the secondary index for these databases is still a challenging task. Modern data storage technologies like: flash memory or phase change memory make the situation even more difficult. The most important problems of such memory types are: limited write endurance and asymmetry between write and read latency. These limitations affect both the index structure and index modification methods.In this paper, we propose a new bulk loading of the secondary index in LSM-based stores for flash memory. The bulk loading happens when many insert operations are performed in one batch. The method works on a new LSM tree variant optimized for flash memory called Flash Aware LSM (FA-LSM) tree. To reach the optimal performance, the method can be adapted to the changing workload. We conduct several experiments which confirm that our method outperforms the traditional LSM insert strategy by about 30% while preserving high search efficiency.KeywordsLSM treeFlash memoryNoSQL database