Garbage Collection Performance Research Articles

Magma

We present Magma, a write-optimized high data density key-value storage engine used in the Couchbase NoSQL distributed document database. Today's write-heavy data-intensive applications like ad-serving, internet-of-things, messaging, and online gaming, generate massive amounts of data. As a result, the requirement for storing and retrieving large volumes of data has grown rapidly. Distributed databases that can scale out horizontally by adding more nodes can be used to serve the requirements of these internet-scale applications. To maintain a reasonable cost of ownership, we need to improve storage efficiency in handling large data volumes per node, such that we don't have to rely on adding more nodes. Our current generation storage engine, Couchstore is based on a log-structured append-only copy-on-write B+Tree architecture. To make substantial improvements to support higher data density and write throughput, we needed a storage engine architecture that lowers write amplification and avoids compaction operations that rewrite the whole database files periodically. We introduce Magma, a hybrid key-value storage engine that combines LSM Trees and a segmented log approach from log-structured file systems. We present a novel approach to performing garbage collection of stale document versions avoiding index lookup during log segment compaction. This is the key to achieving storage efficiency for Magma and eliminates the need for random I/Os during compaction. Magma offers significantly lower write amplification, scalable incremental compaction, and lower space amplification while not regressing the read amplification. Through the efficiency improvements, we improved the single machine data density supported by the Couchbase Server by 3.3x and lowered the memory requirement by 10x, thereby reducing the total cost of ownership up to 10x. Our evaluation results show that Magma outperforms Couchstore and RocksDB in write-heavy workloads.

Deduplication Adapted CaseDB for Edge Computing

Log-structured merge-tree (LSM-tree) family key-value stores are becoming the databases most in demand for big data systems. They provide an easy-to-implement interface, and they automatically perform garbage collection by applying a compaction procedure over the multilevel structure. CaseDB offers various advantages by reducing write amplification considerably, using a metadata compaction technique. However, it suffers from a space amplification problem in update-intensive workloads. As an implementation of the LSM-tree structure, CaseDB refuses to instantly perform deletes, but delays them for the compaction process, resulting in an increasing amount of deprecated data. This paper proposes a deduplication extended compaction method for CaseDB. It scans for duplicated keys within the compaction method and removes the old values. Experiment results show that the proposed technique offers various threshold values of deduplication for different balances between space amplification and write amplification.

Mitigating Synchronous I/O Overhead in File Systems on Open-Channel SSDs

Synchronous I/O has long been a design challenge in file systems. Although open-channel solid state drives (SSDs) provide better performance and endurance to file systems, they still suffer from synchronous I/Os due to the amplified writes and worse hot/cold data grouping. The reason lies in the controversy design choices between flash write and read/erase operations. While fine-grained logging improves performance and endurance in writes, it hurts indexing and data grouping efficiency in read and erase operations. In this article, we propose a flash-friendly data layout by introducing a built-in persistent staging layer to provide balanced read, write, and garbage collection performance. Based on this, we design a new flash file system (FS) named StageFS , which decouples the content and structure updates. Content updates are logically logged to the staging layer in a persistence-efficient way, which achieves better write performance and lower write amplification. The updated contents are reorganized into the normal data area for structure updates, with improved hot/cold grouping and in a page-level indexing way, which is more friendly to read and garbage collection operations. Evaluation results show that, compared to recent flash-friendly file system (F2FS), StageFS effectively improves performance by up to 211.4% and achieves low garbage collection overhead for workloads with frequent synchronization.

Orca: GC and type system co-design for actor languages

ORCA is a concurrent and parallel garbage collector for actor programs, which does not require any STW steps, or synchronization mechanisms, and that has been designed to support zero-copy message passing and sharing of mutable data. ORCA is part of a runtime for actor-based languages, which was co-designed with the Pony programming language, and in particular, with its data race free type system. By co-designing an actor language with its runtime, it was possible to exploit certain language properties in order to optimize performance of garbage collection. Namely, ORCA relies on the guarantees of absence of race conditions in order to avoid read/write barriers, and it leverages the actor message passing, for synchronization among actors. In this paper we briefly describe Pony and its type system. We use pseudo-code in order to introduce how ORCA allocates and deallocates objects, how it shares mutable data without requiring barriers upon data mutation, and how can immutability be used to further optimize garbage collection. Moreover, we discuss the advantages of co-designing an actor language with its runtime, and we demonstrate that ORCA can be implemented in a performant and scalable way through a set of micro-benchmarks, including a comparison with other well-known collectors.

Performance Evaluation of Dynamic Page Allocation Strategies in SSDs

Solid-state drives (SSDs) with tens of NAND flash chips and highly parallel architectures are widely used in enterprise and client storage systems. As any write operation in NAND flash is preceded by a slow erase operation, an out-of-place update mechanism is used to distribute writes through SSD storage space to postpone erase operations as far as possible. SSD controllers use a mapping table along with a specific allocation strategy to map logical host addresses to physical page addresses within storage space. The allocation strategy is further responsible for accelerating I/O operations through better striping of physical addresses over SSD parallel resources. Proposals already exist for using static logical-to-physical address mapping that does not balance the I/O traffic load within the SSD, and its efficiency highly depends on access patterns. A more balanced distribution of I/O operations is to alternate resource allocation in a round-robin manner irrespective of logical addresses. The number of resources that can be dynamically allocated in this fashion is defined as the degree of freedom, and to the best of our knowledge, there has been no research thus far to show what happens if different degrees of freedom are used in allocation strategy. This article explores the possibility of using dynamic resource allocation and identifies key design opportunities that it presents to improve SSD performance. Specifically, using steady-state analysis of SSDs, we show that dynamism helps to mitigate performance and endurance overheads of garbage collection. Our steady-state experiments indicate that midrange/high-end SSDs with dynamic allocation can provide I/O operations per second (IOPS) improvement of up to 3.3x/9.6x, response time improvement of up to 56%/32%, and about 88%/96% average reduction in the standard deviation of erase counts of NAND flash blocks.

Assessing the limits of program-specific garbage collection performance

We consider the ultimate limits of program-specific garbage collector performance for real programs. We first characterize the GC schedule optimization problem using Markov Decision Processes (MDPs). Based on this characterization, we develop a method of determining, for a given program run and heap size, an optimal schedule of collections for a non-generational collector. We further explore the limits of performance of a generational collector, where it is not feasible to search the space of schedules to prove optimality. Still, we show significant improvements with Least Squares Policy Iteration, a reinforcement learning technique for solving MDPs. We demonstrate that there is considerable promise to reduce garbage collection costs by developing program-specific collection policies.

Taurus

Many distributed workloads in today's data centers are written in managed languages such as Java or Ruby. Examples include big data frameworks such as Hadoop, data stores such as Cassandra or applications such as the SOLR search engine. These workloads typically run across many independent language runtime systems on different nodes. This setup represents a source of inefficiency, as these language runtime systems are unaware of each other. For example, they may perform Garbage Collection at times that are locally reasonable but not in a distributed setting. We address these problems by introducing the concept of a Holistic Runtime System that makes runtime-level decisions for the entire distributed application rather than locally. We then present Taurus, a Holistic Runtime System prototype. Taurus is a JVM drop-in replacement, requires almost no configuration and can run unmodified off-the-shelf Java applications. Taurus enforces user-defined coordination policies and provides a DSL for writing these policies. By applying Taurus to Garbage Collection, we demonstrate the potential of such a system and use it to explore coordination strategies for the runtime systems of real-world distributed applications, to improve application performance and address tail-latencies in latency-sensitive workloads.

Taurus

Many distributed workloads in today's data centers are written in managed languages such as Java or Ruby. Examples include big data frameworks such as Hadoop, data stores such as Cassandra or applications such as the SOLR search engine. These workloads typically run across many independent language runtime systems on different nodes. This setup represents a source of inefficiency, as these language runtime systems are unaware of each other. For example, they may perform Garbage Collection at times that are locally reasonable but not in a distributed setting. We address these problems by introducing the concept of a Holistic Runtime System that makes runtime-level decisions for the entire distributed application rather than locally. We then present Taurus, a Holistic Runtime System prototype. Taurus is a JVM drop-in replacement, requires almost no configuration and can run unmodified off-the-shelf Java applications. Taurus enforces user-defined coordination policies and provides a DSL for writing these policies. By applying Taurus to Garbage Collection, we demonstrate the potential of such a system and use it to explore coordination strategies for the runtime systems of real-world distributed applications, to improve application performance and address tail-latencies in latency-sensitive workloads.

Efficient Victim Block Selection for Flash Storage Devices

Motivated by the needs to enhance the performance of garbage collection in low-cost flash storage devices, we propose a victim block selection design to efficiently identify the blocks for erases and reclaim the space of invalid data without extensively scanning flash memory for the data status stored in the storage, so as to improve the garbage collection performance on reclaiming the space of invalid data. Moreover, this design could easily identify and reclaim the space released by file systems. Experiments based on benchmark traces show significant performance improvement of garbage collection with limited system overheads.

Dynamic garbage collection scheme based on past update times for NAND flash-based consumer electronics

NAND flash memory has become the dominant storage device for consumer electronics such as smart phones, portable personal computers, and digital cameras. Because NAND flash memory often introduces an out-of-place update scheme to solve its erase-before-write hardware constraint, these consumer electronics devices should perform garbage collection to reclaim garbage and obtain free space. However, existing garbage collection schemes usually suffer from the high garbage collection overhead and the high degree of wear leveling. In this paper, a dynamic garbage collection scheme based on past update times is proposed for NAND flash-based consumer electronics. The proposed scheme can achieve a balance between the garbage collection overhead and the degree of wear leveling. Experimental results show that the proposed scheme is better than existing garbage collection schemes in terms of the number of copy operations, the number of erase operations, the energy consumption, and the degree of wear leveling.

Efficient FTL-Aware Data Categorization and Identification Scheme for Flash Memory

Hot data identification in flash memory is of great interest because it significantly affects the garbage collection and wear-leveling performance. Presently, certain hot and cold data classification schemes based on Bloom filters (BFs) have been proposed. Although BFs are efficient in most cases, there is a significant trade-off between false positive rates, which are the result of hash value collisions and memory utilization. In this paper, we suggest a better data categorization mechanism that is based on a hashing technique called Fingerprinting by Random Polynomials with the aim of reducing false positive rates and achieving lower memory consumption compared to the BF-based schemes. We also introduce a new methodology for classifying write requests by linking the definition of hot and cold write requests to the flash memory software layer, the flash translation layer (FTL) characteristics. Our approach improves space utilization by representing each logical block number (lbn) by one counter in the hash table, and achieves an extremely low error rate by choosing the degree of the hash function based on the address space of the flash memory. In addition, we achieved lower false identification rates. We demonstrate the performance using mathematical analysis and trace-driven simulation.

Block-Based Multi-Version B$^+$-Tree for Flash-Based Embedded Database Systems

In this paper, we propose a novel multi-version B <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$^+$</tex> <mathgraphic fileref="chang-ieq2-2308213.gif" graphicformat="GIF"/></formula> -tree index structure, called block-based multi-version B <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$^+$</tex> <mathgraphic fileref="chang-ieq3-2308213.gif" graphicformat="GIF"/></formula> -tree ( BbMVBT), for indexing multi-versions of data items in an embedded multi-version database (EMVDB ) on flash memory. An EMVDB needs to support streams of update transactions and version-range queries to access different versions of data items maintained in the database. In BbMVBT, the index is divided into two levels. At the higher level, a multi-version index is maintained for keeping successive versions of each data item. These versions are allocated consecutively in a version block. At the lower level, a version array is used to search for a specific data version within a version block. With the reduced index structure of BbMVBT, the overhead for managing the index in processing update operations can be greatly reduced. At the same time, BbMVBT can also greatly reduce the number of accesses to the index in processing version-range queries. To ensure sufficient free blocks for creating version blocks for efficient execution of BbMVBT, in this paper, we also discuss how to perform garbage collection using the purging-range queries for reclaiming “old” versions of data items and their associated entries in the index nodes. Analysis of the performance of BbMVBT is presented and verified with performance studies using both synthetic and real workloads. The performance results illustrate that BbMVBT can significantly improve the read and write performance to the multi-version index as compared with MVBT even though the sizes of the version blocks are not large.

A technique to improve garbage collection performance for NAND flash-based storage systems

Garbage collection (GC) is a time-consuming operation incorporated in flash memory storage systems, and thus, it has a significant impact on the storage performance. Flash memory provides an internal copy-back operation that can perform page data transfer efficiently during GC. However, this operation is not widely accepted due to some limitations: bit errors can be propagated during the transfer between the pages without being detected, and the operation can be used for blocks within the same plane. In order to address these problems, the page data must be read to a buffer and then transmitted to another page. When this method is used, not only does a temporal overhead occur but also I/O bus utilization and power consumption by the two data transfer operations are increased. In this paper, we propose a technique for reducing the data transfer overhead of GC, using which devices transmit data directly when garbage is collected in a flash memory-based storage system in which several flash memory devices share a single I/O bus, while only the partial updated corrected data is updated when an error is detected. The proposed technique improves the performance of GC and reduces I/O bus utilization compared to the existing methods.

A comprehensive evaluation of object scanning techniques

At the heart of all garbage collectors lies the process of identifying and processing reference fields within an object. Despite its key role, and evidence of many different implementation approaches, to our knowledge no comprehensive quantitative study of this design space exists. The lack of such a study means that implementers must rely on `conventional wisdom', hearsay, and their own costly analysis. Starting with mechanisms described in the literature and a variety of permutations of these, we explore the impact of a number of dimensions including: a) the choice of data structure, b) levels of indirection from object to metadata, and c) specialization of scanning code. We perform a comprehensive examination of these tradeoffs on four different architectures using eighteen benchmarks and hardware performance counters. We inform the choice of mechanism with a detailed study of heap composition and object structure as seen by the garbage collector on these benchmarks. Our results show that choice of scanning mechanism is important. We find that a careful choice of scanning mechanism alone can improve garbage collection performance by 16% and total time by 2.5%, on average, over a well tuned baseline. We observe substantial variation in performance among architectures, and find that some mechanisms--particularly specialization, layout of reference fields in objects, and encoding metadata in object headers--yield consistent, significant advantages.

FeGC: An efficient garbage collection scheme for flash memory based storage systems

NAND flash memory is a promising storage media that provides low-power consumption, high density, high performance, and shock resistance. Due to these versatile features, NAND flash memory is anticipated to be used as storage in enterprise-scale systems as well as small embedded devices. However, unlike traditional hard disks, flash memory should perform garbage collection that consists of a series of erase operations. The erase operation is time-consuming and it usually degrades the performance of storage systems seriously. Moreover, the number of erase operations allowed to each flash memory block is limited. This paper presents a new garbage collection scheme for flash memory based storage systems that focuses on reducing garbage collection overhead, and improving the endurance of flash memory. The scheme also reduces the energy consumption of storage systems significantly. Trace-driven simulations show that the proposed scheme performs better than various existing garbage collection schemes in terms of the garbage collection time, the number of erase operations, the energy consumption, and the endurance of flash memory.

On Improving the Reliability and Performance of the YAFFS Flash File System

In this paper, we propose three techniques to improve the performance of YAFFS (Yet Another Flash File System), while enhancing the reliability of the system. Specifically, we first propose to manage metadata and user data separately on segregated blocks. This modification not only leads to the reduction of the mount time but also reduces the garbage collection time. Second, we tailor the wear-leveling to the segregated metadata and user data blocks. That is, worn out blocks between the segregated blocks are swapped, which leads to more evenly worn out blocks increasing the lifetime of the system. Finally, we devise an analytic model to predict the expected garbage collection time. By accurately predicting the garbage collection time, the system can perform garbage collection at more opportune times when the user's perceived performance may not be negatively affected. Performance evaluation results based on real implementations show that our modifications enhance performance and reliability without incurring additional overheads. Specifically, the YAFFS with our proposed techniques outperforms the original YAFFS by six times in terms of mount speed and five times in terms of benchmark performance, while reducing the average erase count of blocks by 14%.

Performance of greedy garbage collection in flash-based solid-state drives

In flash-based solid-state drives (SSD) and log-structured file systems, new data is written out-of-place, which over time exhausts the available free space. New free space is created by the garbage-collection process, which reclaims the space occupied by invalidated data. The write amplification, incurred because of the additional write operations performed by the garbage-collection mechanism is a critical factor that negatively affects the lifetime and endurance of SSDs. We develop two complementary theoretical models of the SSD operation for uniformly-distributed random small user writes: a Markov chain model, which is useful to explore the performance characteristics of small and medium-sized systems, and a second model that captures the behavior of large systems. The combination of both models allows us to comprehensively characterize the system operation and behavior. Results of theoretical and practical importance are analytically derived and confirmed by means of simulation. Our results demonstrate that (i) as the system occupancy increases, the write amplification increases; (ii) as the number of blocks increases, the write amplification decreases and approaches a lower bound; and (iii) as the number of pages contained in a block increases, the write amplification increases and approaches an upper bound. They also show that, for large systems, the number of free pages reclaimed by the greedy garbage-collection mechanism after each block recycling takes one of two successive values, which provides a quasi-deterministic performance guarantee.

Swap space management technique for portable consumer electronics with NAND flash memory

Flash memory has more capacity and a lower price. It makes flash memory more suitable for portable consumer electronics. Portable consumer electronics such as table PC and smart phones use NAND flash memory as a secondary storage because it has many attractive features such as small size, fast access speeds, and light weight. The portable consumer electronics with NAND flash memory exploit a demand paging to run applications, and also use a swapping to extend a limited main memory space. However, if the portable consumer electronics use NAND flash memory as swap space, it should perform garbage collection, which is a time-consuming operation. Therefore, in order to manage swap space efficiently, this work presents a novel garbage collection policy for the portable consumer electronics with a swap system. The proposed policy has three features important in NAND flash memory based swap systems: (1) long endurance of NAND flash memory, (2) quick garbage collection, and (3) low energy consumption.

FRASystem: fault tolerant system using agents in distributed computing systems

In this paper, we present a fault tolerant and recovery system called FRASystem (Fault Tolerant & Recovery Agent System) using multi-agent in distributed computing systems. Previous rollback-recovery protocols were dependent on an inherent communication and an underlying operating system, which caused a decline of computing performance. We propose a rollback-recovery protocol that works independently on an operating system and leads to an increasing portability and extensibility. We define four types of agents: (1) a recovery agent performs a rollback-recovery protocol after a failure, (2) an information agent constructs domain knowledge as a rule of fault tolerance and information during a failure-free operation, (3) a facilitator agent controls the communication between agents, (4) a garbage collection agent performs garbage collection of the useless fault tolerance information. Since agent failures may lead to inconsistent states of a system and a domino effect, we propose an agent recovery algorithm. A garbage collection protocol addresses the performance degradation caused by the increment of saved fault tolerance information in a stable storage. We implemented a prototype of FRASystem using Java and CORBA and experimented the proposed rollback-recovery protocol. The simulations results indicate that the performance of our protocol is better than previous rollback-recovery protocols which use independent checkpointing and pessimistic message logging without using agents. Our contributions are as follows: (1) this is the first rollback-recovery protocol using agents, (2) FRASystem is not dependent on an operating system, and (3) FRASystem provides a portability and extensibility.

Deterministic Service Guarantees for NAND Flash using Partial Block Cleaning

Current generation embedded systems are capable of running applications that were realm of desktop systems a few years ago. Along with sophisticated processors, affordable storage due to NAND flash continues to be one of the enabling technologies driving the proliferation of embedded systems. NAND flash has its idiosyncrasies (eg: bulk erase, wearleveling) which results in a nonlinear and unpredictable read/write access times. In case of application domains such as streaming multimedia and real-time systems, a deterministic read/write access time is desired during design time. We propose a novel NAND flash translation layer called GFTL that guarantees fixed upper bounds (worst case service rates) for reads and writes that are comparable to a theoretical ideal case. Such guarantees are made possible by eliminating sources of non-determinism in GFTL design and using partial block cleaning. GFTL performs garbage collection in partial steps by dividing the garbage collection of a single block into several chunks, thereby interleaving and hiding the garbage collection latency while servicing requests. Further, GFTL guarantees are independent of flash utilization, size or state. Along with theoretical bounds, benchmark results show the efficacy of our approach. Based on experiments, GFTL requires an additional 16% of total blocks for flash management. A proof for additional blocks required is provided for a general case. GFTL service guarantees can be calculated from flash specifications. Thus, with GFTL a designer can determine the service guarantees and size requirements apriori, during design time.