SWEEP: Gathering Hot Read/Write Data Together to Minimize Read Reclaims in SSDs

Driven by the demand for high performance and large capacity storage system of modern diverse data-intensive latency sensitive applications, Solid State Drives (SSD) built upon Triple Level Cell (TLC) flash memory have been playing an indispensable role in the storage systems of almost all computing platforms. While providing the benefit of huge storage capacity increase, TLC flash memory in the meantime has suffered from significantly degraded raw reliability and thus drastically decreased lifetime entailed as Program/Erase (P/E) cycles. In an SSD, once a flash block has endured a rated number of P/E cycles, it becomes worn out. Worn out blocks are locked in the grown bad block list and then excluded from any further usage till the end of the drive, decreasing the over-provisioning space of an SSD and resultantly slowing down performance. In this work, we suggest a novel Flash Translation Layer (FTL) called Graceful Degradation FTL (GD-FTL), which smartly salvages worn-out TLC blocks by using them as Single Level Cell (SLC) blocks. Doing so brings us two-fold advantages: (1) we can always maintain a good level of over-provisioning space, preventing sharp SSD performance degradation; (2) the lifetime of SSD can be significantly extended due to the huge difference in the P/E cycles between TLC and SLC blocks. Moreover, GD-FTL causes very minimal changes to the FTL design and requires negligible implementation overheads.

Flash memory based SSD (solid state Drive) receives a lot of attention recently. SSD is a semiconductor device which provides great advantages in terms of high-speed random reads, low power consumption, compact size, and shock resistance. Traditional storage systems and algorithms are designed for hard disk drive (HDD), they do not work well on SSD because of SSD's asymmetric read/write performances and unavoidable internal activities, such as garbage collection (GC). There is a great need to optimize current storage systems and algorithms to accelerate data access in SSD. This dissertation presents four methods to improve the performance of the storage system by exploiting the characteristics of SSD. GC is one of the critical overhead of any flash memory based SSD. GC slows down I/O performance and decreases endurance of SSD. This dissertation introduces two methods to minimize the negative impact of GC, “WARCIP: Write Amplification Reduction by Clustering I/O Pages" and “Thermo-GC: Reducing Write Amplification by Tagging Migrated Pages during Garbage Collection". WARCIP uses a clustering algorithm to minimize the rewrite interval variance of pages in a flash block. As a result, pages in a flash block tend to have a similar lifetime, minimizing valid page migrations during GC. The idea of Thermo-GC is to identify data's hotness during GC operations and group data that have similar lifetimes to the same block. Thermo-GC can minimize valid page movements and reduce GC cost through clustering valid pages based on their hotness. Experiment results show that both WARCIP and Thermo-GC can improve the performance of SSD and reduce data movements during GC, implying extended lifetimes of SSDs. SSD fits naturally as a cache between the system RAM and the hard disk drive due to its performance/cost characteristics. But traditional cache replacements are designed for the hard disk drive, which do not work well on SSD because of SSD's asymmetric read/write performances and wearing issues. In this dissertation we present a new cache management algorithm. The idea is not to cache data in SSD upon the first access. Instead, SSD caches when data are determined to be hot enough and warrant caching in the SSD. Data cached in the SSD is managed using an asymmetrical replacement policy for read/write by means of conservative promotion upon hits. The nonvolatile characteristic of SSD allows cached data persistent even after power failures or system crashes. So the system can benefit from a hot restart. Current researches

Redundant arrays of independent disk (RAID) is an efficient approach to relieve reliability sacrifice caused by aggressive scale-out of solid state drives (SSDs). Unfortunately, RAID is unfriendly to SSDs due to redundant parity write and data rebuilding. This paper proposes a wear aware RAID design for SSDs, called WARD , which: 1) adaptively organizes RAID stripes according to real-time interblock unbalanced wear for relieving high performance and storage overhead caused by parity data and 2) migrates blocks about to break in advance and leaves these blocks unused to reduce data rebuilding overhead. An efficient block wear detection scheme is employed to detect block wear during the whole lifetime of SSDs. Beginning with a large stripe width RAID instead of the redundant worst-case RAID, WARD reorganizes RAID stripes once wear blocks with high bit error rates come out. WARD divides the original stripe into several short width RAID stripes according to the number of wear blocks and separates all wear blocks into different stripes. This not only reduces parity redundancy but also provides high reliability to avoid more than RAID recoverable error-prone chunks remaining in one stripe. For high wear blocks tending to wear-out, data in them are migrated in advance and then the blocks are left unused, which efficiently avoids performance shock caused by data rebuilding. A reliability model considering interblock unbalanced wear is proposed and reveals that WARD provides a high and stable reliability and greatly prolongs the lifetime of SSDs. Comprehensive experiments based on an SSDsim derivative simulator are carried out and experiment results show that WARD considerably improves system performance compared to the worst-case RAID.

Solid State Drives (SSDs) based on NAND flash have been widely studied. Most related works focus on improving SSDs' performance. Many novel techniques have been proposed to achieve this goal. In this paper, we don't focus on any given technique, but explore several principles to design an SSD. Beforehand, we build an analytical model for SSDs by resorting to queueing theory. The model covers most characteristics of SSDs. So, it is able to emulate SSDs accurately. We analyze different workloads with the built model. Some interesting phenomena are revealed from the results. Based on these discoveries, we derive several advices to design an SSD. For example, we find that, decreasing write traffic not only extends SSD's lifetime, but also reduces read latency. Therefore, we suggest that, it's worthy to decrease write traffic at the cost of some computational overhead. Techniques such as compression and deduplication can be used to reduce write traffic, even though they bring in considerable overhead. Besides this example, there are other six principles as will be discussed.

SSDs(Solid State Drives) have many attractive features such as high performance, low power consumption, shock resistance, and low weight, so they replace HDDs to a certain extent. An SSD has FTL(Flash Translation Layer) which emulate block storage devices like HDDs. A garbage collection, one of major functions of FTL, effects highly on the performance and the lifetime of SSDs. However, there is no de facto standard for new garbage collection algorithms. To solve this problem, we propose trace driven offline optimal algorithms for garbage collection of FTL. The proposed algorithm always guarantees minimal number of erase operation. In addition, we verify our proposed algorithm using TPC trace.

The NVMe Zoned Namespace (ZNS) is a high-performance interface for flash-based solid-state drives (SSDs), which divides the logical address space into fixed-size and sequential-write zones. Meanwhile, ZNS SSDs eliminate in-device garbage collection (GC) by shifting the responsibility of GC to the host. However, the host-side GC of ZNS SSDs is not efficient. On the one hand, data migration during GC first moves data to the host buffer and then writes back the transferred data to the new location in the SSD, resulting in an unnecessary end-to-end transfer overhead. On the other hand, due to the pre-configured mapping between zones and blocks, GC incurs a large block-to-block rewrite overhead, i.e., even if most of the data in a block of the victim zone is valid, the valid data will still be rewritten to another block in the target zone. To address these issues, this article proposes a novel ZNS SSD design that features dynamic zone mapping, termed Brick-ZNS . Brick-ZNS implements two key functionalities: in-storage data migration and address remapping. New ZNS commands are first designed to realize in-storage data migration to avoid the end-to-end transfer overhead of GC while ensuring performance predictability. Then, a remapping strategy exploiting parallel physical blocks is proposed to reduce the large block-to-block rewrite overhead while ensuring zone-level access parallelism. The basic idea of the strategy is to directly remap the parallel physical blocks with a sufficient amount of valid data in the victim zone to the target zone, hence avoiding the large block-to-block rewrite overhead. Based on a full-stack SSD emulator, the evaluation results show that Brick-ZNS improves write throughput by 25% and SSD lifetime by 1.41×.

With the expansion in data volumes, disk arrays require high speed and large volumes to adapt the cloud computing environment. Solid State Drives (SSDs) now have the trend to substitute the traditional external storage because of their better performance than Hard Disk Drives (HDDs), so they are proper to be utilized in disk arrays. However, traditional I/O scheduler in RAID systems use uniform I/O distribution to balance the workload, which is not suitable for SSD-based RAID. It is because that typical SSD has asymmetric performance on read/write operations, which can result in unbalanced I/O accesses in an SSD array. Furthermore, an even I/O distribution can have an uneven write distribution, which leads to unbalanced lifetime of various SSDs and affects the lifetime of SSD arrays significantly. Obviously, this problem cannot be well addressed through internal wear leveling mechanisms in SSDs. To address the above problems, in this paper, we propose an effective global I/O scheduler called "EGS", which can optimize the I/O performance and prolong the lifetime of SSD-based RAID-5 arrays. According to different access time of read/write operations, we separate the read and write operations to give different weights, and make each disk has a same amount of window (consists of several read/write I/Os) simultaneously. On the other hand, to avoid unbalanced write distribution, we design an additional policy to prolong SSDs' lifetime. To demonstrate the effectiveness of EGS, we conduct several simulations via Disksim. The results show that, compared to traditional I/O scheduling algorithms, EGS saves average response time by up to 40%, and increases the lifetime of SSD array by approximately 45%.

Flash‐based key‐value (KV) caching has received increasing attention in recent years with the advantages of flash‐based solid state drives (SSDs) in capacity and cost. By caching most data in SSD, the caching system can eliminate lots of time‐consuming requests to back‐end data stores to provide low‐latency services. To adapt to the unique technical constraints of flash memory, flash‐based KV caching adopts a slab‐based log‐structured management scheme in which the slab is the basic storage unit, and uses a small memory space as a write buffer to eliminate small random writes to SSD for consistent performance and increased lifetime of SSD. However, we have observed that under update‐intensive workloads with strong temporal locality, the slab‐based management in flash‐based KV caching introduces substantial SSD write traffic because of indistinguishable SSD flushing of hot items in slabs, which shortens the SSD lifetime and degrades the performance with increased erase operations. In this article, we first analyze the SSD write traffic in the flash‐based KV caching, and then propose a novel slab popularity‐based storage management scheme‐SPOPB, to extend SSD lifetime and improve performance. Our scheme identifies hot items using a self‐adaptive threshold to reorganize and classify slabs with both the hotness and size of items. Then SPOPB filters and retains the popular slabs containing hot items in the write buffer with redesigned replacement policy to reduce the SSD write traffic. Our experiments show that our design can effectively reduce the SSD write traffic by 63.6%, the erase counts by 55.6%, and improve the performance by 42%.

As the semiconductor process is scaled down, the endurance of NAND flash memory greatly deteriorates. To overcome such a poor endurance characteristic and to provide a reasonable storage lifetime, system-level endurance enhancement techniques are rapidly adopted in recent NAND flash-based storage devices like solid-state drives (SSDs). In this paper, we propose an integrated lifetime management approach for SSDs. The proposed lifetime management technique combines several lifetime-enhancement schemes, including lossless compression, deduplication, and performance throttling, in an integrated fashion so that the lifetime of SSDs can be maximally extended. By selectively disabling less effective lifetime-enhancement schemes, the proposed technique achieves both high performance and high energy efficiency while meeting the required lifetime. Our evaluation results show that the proposed technique, over the SSDs with no lifetime management schemes, improves write performance by up to 55% and reduces energy consumption by up to 43% while satisfying a 5-year lifetime warranty.

Solid-state drives (SSDs) offer compelling advantages, such as high performance and low cost, over traditional hard disks. Thus, SSDs have been increasingly deployed in various systems, from datacenters to personal computers, in the past few years. However, the advance of the big data era and ever-increasing work- loads place a great burden on both I/O buses and SSD's internal operations, which hinders the further adoption of SSD in the market. Hence, there is a great need to eliminate performance bottlenecks on SSDs and SSD-based storage systems to make SSDs match the speed of high-performance computing. The studies introduced in this dissertation tackle the challenges mentioned above by fully exploiting SSD's internal characteristics.\nThe first study addresses the I/O bottleneck issue in SSD-based storage systems. The idea is to offload compute-intensive tasks to where data reside. A special engine for regular expression (regex) search is designed and augmented inside SSD in the study. The regex search engine processes the data while they are transferred from NAND ashes to host. A library including several useful functions is created to make the hardware engine usable by applications on the host side. For evaluation purposes, the hardware engine is implemented in an FPGA, and the overall design is evaluated in an NVMe-based SSD using real-world regex cases. Experimental results show the I/O bandwidth utilization is reduced by up to 97%, and CPU utilization is reduced by as much as 82% for large data sets.\nThe second study focuses on SSD's performance isolation in the multitenancy environment. It presents a method to ensure the quality of services (QoS), especially when thousands of tenants share the SSD. The idea is to resolve I/O con in ash memory by properly placing data. Moreover, various performance metrics (i.e., throughput/IOPS and latency) are directly incorporated as criteria when scheduling I/O transactions inside the SSD. The overall design is implemented in an SSD simulator and is verified using a vast of real-world I/O traces that serve as thousands of tenants. From the experiments, significant reductions in tail latencies can be seen, which indicates our design can greatly reduce performance violations.\nThe third study aims at ensuring steady I/O performance in SSDs. According to the study, the primary cause of performance fluctuations is garbage collection (GC). GC is a background process when SSD retrieves free spaces, which is necessary for SSDs to function well. However, GC not only induces performance overhead but also shortens SSD's lifespan due to ash memory erasure. To minimize GC-induced performance fluctuations, an adaptive GC strategy that evenly distributes GCs over time is proposed. The GC strategy is derived by formulating a stochastic optimization model that characterizes the nature of GC processes. Several factors are considered to make the algorithm practical. A thorough evaluation is presented, showing that the proposed method can significantly reduce performance fluctuations and increase SSD's lifespan because fewer GC operations are triggered.\nThe fourth study reduces write amplification and GC count by separating hot and cold pages. Since data in the same block have a similar lifespan, fewer data need to be migrated during GC. The key observation that motivates the study is that valid data in a block tend to have a similar lifespan as GC goes on. Thus, we can take advantage of the GC process by using blocks' lifespan to identify the data's hotness. The proposed method is evaluated in terms of write amplification, erasure count, I/O performance, and additional resources required. Results show that this approach decreases GC count, increases endurance, and reduces average I/O response time because fewer data need to be migrated during GC.

Solid state drives (SSDs) widely used in cyber-physical systems have to use block cleaning operations to reclaim storage space because of the out-of-place update characteristics. As these operations often take high time costs and do harm to flash lifetime because of write amplification, SSD system performance has been largely decreased. Garbage collection and wear leveling, as two typical block cleaning techniques, are invoked without enough communication between each other, which may induce one block repeatedly cleaned within a short period, leading to worse performance and lifetime of SSDs. This paper proposes an unified block cleaning method called UniBC by considering the two invoking conditions unitedly to reduce the cost of overall block cleaning. In details, UniBC makes garbage collection able to consider the wear degree of victim blocks by utilizing constrained victim block selection techniques and wear leveling be aware of the utilization degree of victim blocks. By exploiting UniBC, repetitive block cleaning operations can be avoided and overall system performance can be improved. Experimental results show that UniBC can improve 10% block cleaning performance on average.

Nowadays solid state drives (SSDs) are gaining popularity and are replacing magnetic hard disk drives (HDDs) in enterprise storage systems. As a result, extracting the maximum performance from SSDs is becoming crucial to deal with the increasing storage volume and performance needs. Active disks were introduced as a way to offload data-processing tasks from the host into disks freeing system resources and achieving better performance. In this work, we present an active SSD architecture called Self-Sorting SSD that targets to offload sorting operations which are commonly used in data-intensive and database environments and that require heavy data transfer. Processing sorting operations directly on the SSD reduces data transfer from/to the storage devices, increasing system performance and the lifetime of SSDs. Experiments on a real SSD platform reveal that our proposed architecture outperforms traditional external merge sort by up to 60.75%, reduces energy consumption by up to 58.86%, and eliminates all the data transfer overhead to compute sorted results.

Currently, the available I/O schedulers in Linux kernel have long been designed and optimized under the default assumption of underlying traditional rotating HDDs. Unsurprisingly, their performance would be sub optimal when working with emerging Solid State Drives(SSD), which exhibit widely disparate operational characteristics than HDDs. In this study, we develop a new SSD scheduler which takes SSDs' peculiarities into account in its design. In more detail, firstly, we partition the entire SSD space into several regions as basic scheduling units and issue requests to them simultaneously to exploit internally rich parallelism. Secondly, we give preference to read requests over writes to take advantage of the fact that reads are much faster than writes, avoiding excessive read-blocked by-write interference. Lastly, within each region's scheduling queue, we sort write requests before issuing them in anticipation of transforming random writes to sequential patterns to avoid harmful random writes. Our evaluation results of a variety of workloads have demonstrated that Regional Scheduler is capable of improving the performance by 17-32% compared with the best of the four kernel I/O schedulers, in the meantime, improving SSD's lifetime because of the successful transformation of random writes to sequential writes.

The authors of this paper derived solid state drive (SSD) life time distribution from physics-based life model considering the random nature of real world customer data usage and product inherent physical properties. The paper is focused on the following two cases: Case 1: When only field write duty cycle is treated as a random variable while assuming all other physical characteristics are non-random, it is found that the SSD life time follows . Reciprocal-Weibull distribution when field Write Duty Cycle follows Weibull distribution, . Reciprocal-Exponential distribution when field Write Duty Cycle follows Exponential distribution, . Lognormal distribution when field Write Duty Cycle follows Lognormal distribution, . Reciprocal-Normal Distribution when field Write Duty Cycle follows Normal distribution. The corresponding mathematical expressions for reliability, unreliability, hazard rate, MTTF, etc. are derived for each scenario accordingly. Case 2: In real world, SSD endurance rating is also a random variable due to part-to-part variance from material in-homogeneity and inherent defects from manufacturing process. Given the distributions of field customer write duty cycle (stress) and SSD endurance rating (strength), the distribution of lifetime random variable can be derived either analytically, if closed form solution exists, or numerically using Monte Carlo simulation if no closed form solution exists. This paper provides a special case where the analytic solution exists when both random variables follow Lognormal distribution. A numerical example is given to show the application of the models developed in this paper. The results derived in this paper will benefit the SSD industry in various aspects of product design, development, reliability testing and prediction, field return/failure estimation and warranty management.

Lifespan-based garbage collection to improve SSD's reliability and performance