Heterogeneous Storage Devices Research Articles

IOCost: Block Input–Output Control for Containers in Datacenters

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Resource isolation is a requirement in datacenter environments. However, our production experience in Meta’s large scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. This paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across Meta’s datacenters comprised of millions of machines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools.</i>

Adaptive Parameter Tuning for Constructing Storage Tiers in an Autonomous Distributed Storage System

We are developing an autonomous distributed storage system to make effective use of networked heterogeneous storage devices. It dynamically configures the storage tiers using effective performance values measured during operation, rather than static performance indicators of the storage device. Data blocks are transferred to a more appropriate storage tier according to their read/write tendencies. In the previous paper, the storage tiers were set using statically defined thresholds, but in this paper, we propose a method to set the thresholds adaptively. This method eliminates the need to adjust the runtime parameters for threshold setting according to the runtime environment. It has also been improved so that appropriate parameter values can be set for individual storage, rather than applying a single parameter value to the entire system. In the previous system, the administrator had to readjust the parameter values periodically to match the usage status of the storage device and the behavior of the process. However, in the proposed method, the parameter values are fine-tuned during operation to reduce the workload of the administrator. The simulation results show that the proposed method can reduce the execution time by up to about 30% compared to the previous method with optimum parameters configured statically.

FILM

As modern applications generate data at an unprecedented speed and often require the querying/analysis of data spanning a large duration, it is crucial to develop indexing techniques that cater to larger-than-memory databases, where data reside on heterogeneous storage devices (such as memory and disk), and support fast data insertion and query processing. In this paper, we propose FILM, a F ully learned I ndex for L arger-than- M emory databases. FILM is a learned tree structure that uses simple approximation models to index data spanning different storage devices. Compared with existing techniques for larger-than-memory databases, such as anti-caching, FILM allows for more efficient query processing at significantly lower main-memory overhead. FILM is also designed to effectively address one of the bottlenecks in existing methods for indexing larger-than-memory databases that is caused by data swapping between memory and disk. More specifically, updating the LRU (for Least Recently Used) structure employed by existing methods for cold data identification (determining the data to be evicted to disk when the available memory runs out) often incurs significant delay to query processing. FILM takes a drastically different approach by proposing an adaptive LRU structure and piggybacking its update onto query processing with minimal overhead. We thoroughly study the performance of FILM and its components on a variety of datasets and workloads, and the experimental results demonstrate its superiority in improving query processing performance and reducing index storage overhead (by orders of magnitudes) compared with applicable baselines.

Supporting SLA via Adaptive Mapping and Heterogeneous Storage Devices in Ceph

This paper proposes a new resource management scheme that supports SLA (Service-Level Agreement) in a bigdata distributed storage system. Basically, it makes use of two mapping modes, isolated mode and shared mode, in an adaptive manner. In specific, to ensure different QoS (Quality of Service) requirements among clients, it isolates storage devices so that urgent clients are not interfered by normal clients. When there is no urgent client, it switches to the shared mode so that normal clients can access all storage devices, thus achieving full performance. To provide this adaptability effectively, it devises two techniques, called logical cluster and normal inclusion. In addition, this paper explores how to exploit heterogeneous storage devices, HDDs (Hard Disk Drives) and SSDs (Solid State Drives), to support SLA. It examines two use cases and observes that separating data and metadata into different devices gives a positive impact on the performance per cost ratio. Real implementation-based evaluation results show that this proposal can satisfy the requirements of diverse clients and can provide better performance compared with a fixed mapping-based scheme.

Proofs of Physical Reliability for Cloud Storage Systems

Cloud service providers (CSPs) promise to reliably store repositories outsourced by clients. Unfortunately, once files have left the client's control, he has no means to verify their redundant storage. In this article, we develop Proof of Physical Reliability (PoPR) auditing mechanisms that prove that a CSP stores an outsourced repository across multiple physical storage nodes. A PoPR complements the existing proof-of-retrievability (PoR) and proof-of-data possession (PDP) methods that are concerned with file retrievability, but without any verification of the fault-tolerance to physical storage nodes failures. A PoPR goes beyond retrievability by verifying that a file is redundantly stored across multiple physical storage nodes according to a pre-agreed layout and can, therefore, survive node failures. The verification mechanism relies on a combination of storage integrity and timing tests on the simultaneous retrieval of a collection of file symbols from multiple storage nodes. Compared to the state-of-the-art, our approach accommodates CSPs with heterogeneous storage devices (hard disks, SSDs, etc.) and does not assume constant data processing nor network delays. Instead, it can operate under any delay variance, because it relies only on (loose) delay bounds. We analytically prove the security of our construction and experimentally validate its success in heterogeneous storage settings.

Single Failure Recovery Method for Erasure Coded Storage System with Heterogeneous Devices

As the demand of data reliability becomes more and more larger, most of today's storage systems adopt erasure codes to assure the data could be reconstructed when suffering from physical device failures. In order to fast recover the lost data from a single failure, recovery optimization methods have attracted a lot of attention in recent years. However, most of the existing optimization methods focus on homogeneous devices, ignoring the fact that the storage devices are usually heterogeneous. In this paper, we propose a new recovery optimization method named HSR (Heterogeneous Storage Recovery) method, which uses both loads and speed rate among physical devices as the optimization target, in order to further improve the recovery performance for heterogeneous devices. The experiment results show that, compared to existing popular recovery optimization methods, HSR method gains much higher recovery speed over heterogeneous storage devices.

Power Management of Heterogeneous Autonomous DC Microgrid System with EVs Using Modified SoC Reference Based DC Bus Voltage Control

Due to stochastic and intermittent nature of distributed generation (DG) sources, different types of nonlinear loads, different transmission lines and heterogeneous energy storage devices (ESDs) the system becomes complex and diverse in nature. The b

IR+: Removing parallel I/O interference of MPI programs via data replication over heterogeneous storage devices

Abstract I/O requests from parallel processes to a disk compete with each other for a single disk head to access data. The disk efficiency can be significantly reduced due to frequent disk head seeks. In this paper, we propose a scheme, named IR+, to eliminate I/O interference for a single MPI program on disks by taking advantage of optimized access patterns and heterogeneous storage devices (e.g., solid-state disks). It identifies segments of files that could be involved in the interfering accesses and replicates them to their respectively designated storage servers. Moreover, it can further improve I/O performance by storing replicas on SSDs when accessing the on-disk replicas cannot sufficiently address the issue. IR+ has been implemented in the MPI library for high portability on top of the Lustre parallel file system. Our experiments with representative benchmarks, such as NPB BTIO and GTS , show that it can significantly improve I/O performance on the production HPC systems.

Cooperative Multi-Agent Control of Heterogeneous Storage Devices Distributed in a DC Microgrid

This paper proposes a multi-agent control strategy to coordinate power sharing between heterogeneous energy storage devices distributed throughout a DC microgrid. Without requiring a central controller, the proposed control strategy extends the benefits offered by hybrid energy storage systems to DC microgrids with batteries and ultracapacitors spatially distributed at different levels of the power distribution hierarchy. The proposed control strategy has the following advantages. 1) The high frequency microgrid load is provided by the ultracapacitors. 2) The low frequency load is provided by batteries used for bulk energy storage during islanded mode, and the main grid during grid connected operation. 3) The ultracapacitor voltages are regulated at a desired reference. 4) State of charge balancing is provided between the batteries. 5) The energy storage systems cooperate based on neighbor-to-neighbor output feedback over a sparse communication network. The only communication requirement is a spanning tree from the ultracapacitor leaders and battery leaders to their respective followers. Simulations are presented demonstrating the performance of the proposed control strategy for a 380 VDC datacenter during grid connected and islanded operation.

Towards cost-effective and high-performance caching middleware for distributed systems

One performance bottleneck of distributed systems lies on the hard disk drive (HDD) whose single read/write head has physical limitations to support concurrent I/Os. Although the solid-state drive (SSD) has been introduced for years, HDDs are still dominant storage due to large capacity and low cost. This paper proposes a caching middleware that manages the underlying heterogeneous storage devices in order to allow distributed file systems to achieve both high performance and low cost. Specifically, we design and implement a user-level caching system that offers SSD-like performance at a cost similar to a HDD. We demonstrate how such a middleware improves the performance of distributed file systems, such as the HDFS. Experimental results show that the caching system delivers up to 7X higher throughput and 76X higher IOPS than Linux Ext4 file system, and accelerates HDFS by 28% on 32 nodes.

A Distributed Storage System with Dynamic Tiering for iSCSI Environment

We propose a distributed storage system which relocates data blocks autonomously among the storage nodes. In order to optimize I/O performance with slight administrative workload, our system is aimed at the realization of the following two functions; (i) Run-time construction of the storage tiers from the heterogeneous storage devices, and (ii) Automated block migration among the tiers. We also show a technique to reduce management traffic in iSCSI environment.

An Effective Cache Algorithm for Heterogeneous Storage Systems

Modern storage environment is commonly composed of heterogeneous storage devices. However, traditional cache algorithms exhibit performance degradation in heterogeneous storage systems because they were not designed to work with the diverse performance characteristics. In this paper, we present a new cache algorithm called HCM for heterogeneous storage systems. The HCM algorithm partitions the cache among the disks and adopts an effective scheme to balance the work across the disks. Furthermore, it applies benefit-cost analysis to choose the best allocation of cache block to improve the performance. Conducting simulations with a variety of traces and a wide range of cache size, our experiments show that HCM significantly outperforms the existing state-of-the-art storage-aware cache algorithms.

HStorage-DB

As storage systems become increasingly heterogeneous and complex, it adds burdens on DBAs, causing suboptimal performance even after a lot of human efforts have been made. In addition, existing monitoring-based storage management by access pattern detections has difficulties to handle workloads that are highly dynamic and concurrent. To achieve high performance by best utilizing heterogeneous storage devices, we have designed and implemented a heterogeneity-aware software framework for DBMS storage management called hStorage-DB, where semantic information that is critical for storage I/O is identified and passed to the storage manager. According to the collected semantic information, requests are classified into different types. Each type is assigned a proper QoS policy supported by the underlying storage system, so that every request will be served with a suitable storage device. With hStorage-DB, we can well utilize semantic information that cannot be detected through data access monitoring but is particularly important for a hybrid storage system. To show the effectiveness of hStorage-DB, we have implemented a system prototype that consists of an I/O request classification enabled DBMS, and a hybrid storage system that is organized into a two-level caching hierarchy. Our performance evaluation shows that hStorage-DB can automatically make proper decisions for data allocation in different storage devices and make substantial performance improvements in a cost-efficient way.

ARC-H: Adaptive replacement cache management for heterogeneous storage devices

Heterogeneous storage architectures combine the strengths of different storage devices in a synergistically useful fashion, and are increasingly being used in mobile storage systems. In this paper, we propose ARC-H, an adaptive cache replacement algorithm for heterogeneous storage systems consisting of a hard disk and a NAND flash memory. ARC-H employs a dynamically adaptive management policy based on ghost buffers and takes account of recency, I/O cost per device, and workload patterns in making cache replacement decisions. Realistic trace-driven simulations show that ARC-H reduces service time by up to 88% compared with existing caching algorithms with a 20Mb cache. ARC-H also reduces energy consumption by up to 81%.

LBM: a low-power buffer management policy for heterogeneous storage in mobile consumer devices

Flash memory has many attractive features such as small size, low-power consumption, shock resistance, and high performance. Due to these good features, flash memory has been widely used in the mobile consumer devices such as portable media players (PMPs) and smart phones. However, the cost of flash memory is still high to accommodate ever-growing mobile applications and multimedia contents. Using flash memory and mobile disk together as secondary storage is an alternative solution to provide large storage capacity with reasonable cost. As heterogeneous storage devices are used together, the system needs a buffer management policy that is aware of different I/O characteristics of buffered blocks depending on which devices they belong to. Specifically, power consumption rate of each storage device should be considered in the design of an efficient buffer management policy since battery limitation of mobile systems is important. This paper presents a new buffer management policy for mobile systems consisting of heterogeneous storage devices such as flash memory and mobile disk. By considering different power-consumption rates of each storage media as well as I/O operation type and reference potential of buffered blocks, the proposed policy reduces storage power consumption significantly and also improves I/O performances.

BroadScale: Efficient scaling of heterogeneous storage systems

Scalable storage architectures enable digital libraries and archives for the addition or removal of storage devices to increase storage capacity and bandwidth or retire older devices. Past work in this area have mainly focused on statically scaling homogeneous storage devices. However, heterogeneous devices are quickly being adopted for storage scaling since they are usually faster, larger, more widely available, and more cost-effective. We propose BroadScale, an algorithm based on Random Disk Labeling, to dynamically scale heterogeneous storage systems by distributing data objects according to their device weights. Assuming a random placement of objects across a group of heterogeneous storage devices, our optimization objectives when scaling are to ensure a uniform distribution of objects, redistribute a minimum number of objects, and maintain fast data access with low computational complexity. We show through experimentation that BroadScale achieves these requirements when scaling heterogeneous storage.