Abstract
Persistent key-value stores have been widely adopted as storage engines for modern IT infrastructures because they provide high performance with simple design principles. Moreover, many key-value stores commonly employ LSM-tree as their index structure due to its attractive features such as high write throughput and storage space efficiency. Unfortunately, LSM-tree has critical drawbacks in that it leads to write/read amplification problem. One of the prevalent solutions for remedying the write amplification problem is the tiering merge policy that reduces the number of rewrites by delaying merge operations. However, in spite of this advantage, the tiering merge policy may lead to a side-effect that induces high read amplification, increasing search/scan cost for upcoming read operations. In this paper, we concentrate on mitigating the high read amplification problem of the tiering merge policy, while maintaining its low write amplification. To achieve this, we propose a novel LSM-tree scheme, called Spatially Fragmented LSM-tree (SFM), which delays merge operations only for the non-read-intensive key-spaces. For this, SFM identifies the read intensity of each key-spaces by dynamically estimating their read/write hotness. We have implemented SFM based on PebblesDB and evaluated the performance benefits of our scheme under real-world workloads of Facebook. Experimental results clearly show that our scheme improves throughput by up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.67\times $ </tex-math></inline-formula> compared with the conventional schemes while maintaining low write amplification, and also indicate that its latency is lowered by up to 41.41% on average by mitigating the read amplification problem of the existing schemes, by up to 43.68%.
Highlights
Persistent key-value stores are widely adopted as a storage engine to efficiently handle large-scale data in modern IT infrastructures [1]–[11]
We propose a novel LSM-tree scheme, called Spatially Fragmented LSM-tree (SFM), which is for mitigating high read amplification of the tiering merge policy while maintaining low write amplification
Whenever write requests occur in the LSMtree-based key-value stores, the incoming write requests are logged into the Commit log files that reside on the underlying persistent storage so that the key-value stores can ensure data consistency from system crashes or sudden power failures [13], [26]
Summary
Persistent key-value stores are widely adopted as a storage engine to efficiently handle large-scale data in modern IT infrastructures [1]–[11]. Whenever write requests occur in the LSMtree-based key-value stores, the incoming write requests are logged into the Commit log files that reside on the underlying persistent storage so that the key-value stores can ensure data consistency from system crashes or sudden power failures [13], [26]. These write requests are temporarily absorbed into the in-memory write buffer called Memtable [27]. These flushed keyvalue pairs are managed differently, depending on the merge policy of the LSM-tree: leveling merge policy and tiering merge policy [14]
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
