Heterogeneous Storage Systems Research Articles

Data Distribution for Heterogeneous Storage Systems

The exponential growth of data in many science and engineering domains poses significant challenges to storage systems. Data distribution is a critical component in large-scale distributed storage systems and plays a vital role in placing petabytes of data and beyond, among tens to hundreds of thousands of storage devices. Meantime, heterogeneous storage systems, such as those having devices with hard disk drives (HDDs) and storage class memories (SCMs), have become increasingly popular for massive data storage due to their distinct and complement characteristics. This paper presents a new data distribution algorithm called SUORA (Scalable and Uniform storage via Optimally-adaptive and Random number Addressing) specifically for heterogeneous devices to maximize the benefits of them. SUORA provides a fully symmetric, highly efficient methodology to distribute data across a hybrid and tiered storage cluster. It divides heterogeneous devices into different buckets and segments, and adopts pseudo-random functions to map data onto them with the balanced consideration of capacity, performance and life-time. By analyzing hotness and access patterns, SUORA gradually moves hot data from HDDs to SCMs to optimize the throughput, and moves cold data reversely for load balance. It combines data replication with migration to significantly reduce movement overhead while making data placement more adaptive to different workloads. Extensive evaluations on simulation and Sheepdog storage system show that, with considering distinct characteristics of various devices thoroughly, SUORA improves the overall performance efficiency of heterogeneous storage systems.

An adaptive read/write optimized algorithm for Ceph heterogeneous systems via performance prediction and multi-attribute decision making

The advent of the Big Data era has brought considerable challenges to storing and managing massive data. Moreover, distributed storage systems are critical to the pressure and storage capacity costs. The Ceph cloud storage system only selects data storage nodes based on node storage capacity. This node selection method results in load imbalance and limited storage scenarios in heterogeneous storage systems. Therefore, we add node heterogeneity, network state, and node load as performance weights to the CRUSH algorithm and optimize the performance of the Ceph system by improving load balancing. We designed a cloud storage system model based on Software Defined Network (SDN) technology. This system model can avoid the tedious configuration and significant measurement overhead required to obtain network status in traditional network architecture. Then we propose adaptive read and write optimization algorithms based on SDN technology. The Object Storage Device (OSD) is initially classified based on the Node Heterogeneous Resource Classification Strategy. Then the SDN technology is used to obtain network and load conditions in real-time and an OSD performance prediction model is built to obtain weights for performance impact factors. Finally, a mathematical model is proposed for multi-attribute decision making in conjunction with the OSD state and its prediction model. Furthermore, this model is addressed to optimize read and write performance adaptively. Compared with the original Ceph system, TOPSIS_PA improves the performance of reading operations by 36%; TOPSIS_CW and TOPSIS_PACW algorithms improve the elastic read performance by 23 to 60% and 36 to 85%, and the elastic write performance by 180 to 468% and 188 to 611%, respectively.

Decentralized Coordination and Stabilization of Hybrid Energy Storage Systems in DC Microgrids

Hybrid energy storage system (HESS) is an attractive solution to compensate power balance issues caused by intermittent renewable generations and pulsed power load in DC microgrids. The purpose of HESS is to ensure optimal usage of heterogeneous storage systems with different characteristics. In this context, power allocation for different energy storage units is a major concern. At the same time, the wide integration of power electronic converters in DC microgrids would possibly cause the constant power load instability issue. This paper proposes a composite model predictive control based decentralized dynamic power sharing strategy for HESS. First, a composite model predictive controller (MPC) is proposed for a system with a single ESS and constant power loads (CPLs). It consists of a baseline MPC for optimized transient performance and a sliding mode observer to estimate system disturbances. Then, a coordinated scheme is developed for HESS by using the proposed composite MPC with a virtual resistance droop controller for the battery system and with a virtual capacitance droop controller for the supercapacitor (SC) system. With the proposed scheme, the battery only supplies smooth power at steady state, while the SC compensates all the fast fluctuations. The proposed scheme achieves a decentralized dynamic power sharing and optimized transient performance under large variation of sources and loads. The proposed approach is verified by simulations and experiments.

HintStor: A Framework to Study I/O Hints in Heterogeneous Storage

To bridge the giant semantic gap between applications and modern storage systems, passing a piece of tiny and useful information, called I/O access hints, from upper layers to the storage layer may greatly improve application performance and ease data management in storage systems. This is especially true for heterogeneous storage systems that consist of multiple types of storage devices. Since ingesting external access hints will likely involve laborious modifications of legacy I/O stacks, it is very hard to evaluate the effect and take advantages of access hints. In this article, we design a generic and flexible framework, called HintStor, to quickly play with a set of I/O access hints and evaluate their impacts on heterogeneous storage systems. HintStor provides a new application/user-level interface, a file system plugin, and performs data management with a generic block storage data manager. We demonstrate the flexibility of HintStor by evaluating four types of access hints: file system data classification, stream ID, cloud prefetch, and I/O task scheduling on a Linux platform. The results show that HintStor can execute and evaluate various I/O access hints under different scenarios with minor modifications to the kernel and applications.

Potential of I/O Aware Workflows in Climate and Weather

The efficient, convenient, and robust execution of data-driven workflows and enhanced data management are essential for productivity in scientific computing. In HPC, the concerns of storage and computing are traditionally separated and optimised independently from each other and the needs of the end-to-end user. However, in complex workflows, this is becoming problematic. These problems are particularly acute in climate and weather workflows, which as well as becoming increasingly complex and exploiting deep storage hierarchies, can involve multiple data centres. The key contributions of this paper are: 1) A sketch of a vision for an integrated data-driven approach, with a discussion of the associated challenges and implications, and 2) An architecture and roadmap consistent with this vision that would allow a seamless integration into current climate and weather workflows as it utilises versions of existing tools (ESDM, Cylc, XIOS, and DDN’s IME). The vision proposed here is built on the belief that workflows composed of data, computing, and communication-intensive tasks should drive interfaces and hardware configurations to better support the programming models. When delivered, this work will increase the opportunity for smarter scheduling of computing by considering storage in heterogeneous storage systems. We illustrate the performance-impact on an example workload using a model built on measured performance data using ESDM at DKRZ.

Incremental Throughput Allocation of Heterogeneous Storage With No Disruptions in Dynamic Setting

Solid-state drives (SSDs) have been added into storage systems for improving their performance, which will bring the heterogeneity into the storage medium. The throughput is one of the essential resources in heterogeneous storage systems, and how to allocate the throughput plays a crucial role in user performance. There are many types of research on the throughput allocation of heterogeneous storage systems. However, the throughput allocation of heterogeneous storage is facing new challenges in a dynamic setting, where users are not present in the system simultaneously, and enter the system dynamically. Drawing on economic game-theory, researchers have proposed many methods to tackle dynamic throughput allocation issues for heterogeneous storages, cross out enjoying Sharing Incentive (SI), Envy Freeness (EF), and Pareto Optimality (PO). However, they either relax constraints of fairness property to cause the allocation with weak fairness or interrupt some users present in the system to give up a piece of their allocations for new users entering the system, which will degrade these donors’ performance. Moreover, all of existing methods will cause lower resource utilization due to constraints of users’ dominant share equality. In this article, we propose a dynamic throughout allocation method based on gradual increase (DAGI), which can adapt to various workloads to make a fair allocation with a maximum resource utilization. Without relaxing constraints of fairness properties, when new users enter the system, DAGI can make a dynamic allocation with strong fairness by appropriately postponing the allocation of surplus throughputs, so this can provide an opportunity that DAGI can guarantee the final allocation with strong fairness when allocating remaining throughputs after all users are present in the system. Meanwhile, DAGI can gradually increase user allocation without reduction, which will not interrupt any users present in the system. Furthermore, DAGI can conduct a dynamic throughput allocation based on users’ local bottleneck resources, which can adapt to various workloads of users to improve resource utilization. Extensive experiments are conducted to prove the effectiveness of DAGI. The experimental results show that DAGI can achieve higher resource utilization and performance than existing methods, and can satisfy desirable game-theoretic properties with guaranteeing the strong fairness. In addition, DAGI gradually increases the allocation of each user without interrupting any user to reduce its allocation to degrade its performance.

MaRe: Processing Big Data with application containers on Apache Spark.

BackgroundLife science is increasingly driven by Big Data analytics, and the MapReduce programming model has been proven successful for data-intensive analyses. However, current MapReduce frameworks offer poor support for reusing existing processing tools in bioinformatics pipelines. Furthermore, these frameworks do not have native support for application containers, which are becoming popular in scientific data processing.ResultsHere we present MaRe, an open source programming library that introduces support for Docker containers in Apache Spark. Apache Spark and Docker are the MapReduce framework and container engine that have collected the largest open source community; thus, MaRe provides interoperability with the cutting-edge software ecosystem. We demonstrate MaRe on 2 data-intensive applications in life science, showing ease of use and scalability.ConclusionsMaRe enables scalable data-intensive processing in life science with Apache Spark and application containers. When compared with current best practices, which involve the use of workflow systems, MaRe has the advantage of providing data locality, ingestion from heterogeneous storage systems, and interactive processing. MaRe is generally applicable and available as open source software.

PRS: A Pattern-Directed Replication Scheme for Heterogeneous Object-Based Storage

Data replication is a key technique to achieve high data availability, reliability, and optimized performance in distributed storage systems. In recent years, with emerged new storage devices, heterogeneous object-based storage systems, such as a storage system with a mix of hard disk drives, solid state drives, and other non-volatile memory devices have become increasingly attractive since they combine the merits of different storage devices to deliver better promises. However, existing data replication schemes do not well consider distinct characteristics of heterogeneous storage devices yet, which could lead to suboptimal performance. This article introduces a new data replication scheme called Pattern-directed Replication Scheme (PRS) to achieve efficient data replication for heterogeneous storage systems. Different from traditional schemes, the PRS selectively replicates data objects and distributes replicas to various storage devices based on their characteristics. It aggregates objects that have I/O correlation into object groups by calculating object distance and makes replication for grouped objects according to application's data access pattern identified. In addition, the PRS uses a pseudo random algorithm to optimize replica placement by considering the storage device performance and capacity features. We have evaluated the pattern-directed replication scheme with extensive tests in Sheepdog, a typical object-based storage system. The experimental results confirm that it is a highly efficient replication scheme for heterogeneous storage systems. For instance, the read performance was improved by 105 percent to nearly 10x compared with existing replication schemes.

Dynamic Throughput Allocation among Multiple Servers for Heterogeneous Storage System

Dynamic Throughput Allocation among Multiple Servers for Heterogeneous Storage System

Feasibility analysis of heterogeneous energy storage technology for cloud energy storage with distributed generation

Fluctuations of electricity prices in demand response schemes and intermittency of renewable energy supplies necessitate the adoption of energy storage in power systems. This paper considers the heterogeneous cloud energy storage (HCES) on cloud energy storage operator side. The goal of this approach is to lower the cost of energy storage by exploiting the different operating characteristics and economics of different battery energy storage technologies. The customers are screened from the type of storage facility that they have purchased and cloud energy storage (CES) operator can maximize the benefit if the complementarily of the storage technologies can be utilized. Distributed generation such as photo-voltaic (PV) generation has also been considered on both side consumer as well as CES operator side for supplying power to energy storage facilities to reduce the cost of charging power. Furthermore, the paper also presents an economic viability of heterogeneous storage system using investment analysis methods. Numerical simulations are carried out based on HCES technological characteristics such as self discharge, round trip charge-discharge cycle efficiency, depth of discharge (DOD) and cycles life time. There is significant changes in the economic performances using HCES when compared with single energy storage.

The ATLAS Data Management System Rucio: Supporting LHC Run-2 and beyond

With this contribution we present some recent developments made to Rucio, the data management system of the High-Energy Physics Experiment ATLAS. Already managing 300 Petabytes of both official and user data, Rucio has seen incremental improvements throughout LHC Run-2, and is currently laying the groundwork for HEP computing in the HL-LHC era. The focus of this contribution are (a) the automations that have been put in place such as data rebalancing or dynamic replication of user data, as well as their supporting infrastructures such as real-time networking metrics or transfer time predictions; (b) the flexible approach towards inclusion of heterogeneous storage systems, including object stores, while unifying the potential access paths using generally available tools and protocols; (c) machine learning approaches to help with transfer throughput estimation; and (d) the adoption of Rucio for two other experiments, AMS and Xenon1t. We conclude by presenting operational numbers and figures to quantify these improvements, and extrapolate the necessary changes and developments for future LHC runs.

Data Replica Placement Mechanism for Open Heterogeneous Storage Systems

Abstract: Many big data-oriented network storage platforms are open and heterogeneous, which means that the data placement mechanism has a great impact on the system performance. The data replica placement mechanism needs to consider the data availability, the load balance and the quality of service for open heterogeneous storage systems. However, current data placement mechanisms are mainly for simple homogeneous storage systems, not entirely suitable for heterogeneous storage systems. We proposed a novel data replica placement mechanism with the adjustable replica deployment strategy (ARDS) for open heterogeneous data storage systems, which considers the data availability, the data access frequency and the storage capacity. The mechanism takes the data availability as premise of the initialization number of replica, then adjusts the number of replica according to the data access frequency, and places replicas based on the node spare capacity, which effectively avoids the condition of load unbalance. We implemented simulation experiments based on OMNet++. The experimental results show that the ARDS-based mechanism improves the system performance including the system load balance and the rate of successful data access.

A Cache Management Algorithm for the Heterogeneous Storage Systems

A Cache Management Algorithm for the Heterogeneous Storage Systems

Adaptive Fractional Repetition Codes for Dynamic Storage Systems

Node recovery is a challenging task in erasure-coded storage systems, which involves a considerable amount of network resources. The efficacy of failure recovery is measured by various metrics, such as repair bandwidth and computational complexity. This letter focuses on fractional repetition (FR) codes, which are a family of codes that provide uncoded exact repairs with minimum repair bandwidth. We consider the usage of FR codes in dynamic storage systems, where the system parameters can vary over time. Specifically, we introduce adaptive fractional repetition codes that can effectively adapt to system changes. The proposed codes allow update of systems without the need for frequent reconfigurations. Moreover, explicit code constructions for both homogeneous and heterogeneous storage systems are presented.

Comprehensive Monitoring for Heterogeneous Geographically Distributed Storage

Storage capacity at CMS Tier-1 and Tier-2 sites reached over 100 Petabytes in 2014, and will be substantially increased during Run 2 data taking. The allocation of storage for the individual users analysis data, which is not accounted as a centrally managed storage space, will be increased to up to 40%. For comprehensive tracking and monitoring of the storage utilization across all participating sites, CMS developed a space monitoring system, which provides a central view of the geographically dispersed heterogeneous storage systems. The first prototype was deployed at pilot sites in summer 2014, and has been substantially reworked since then. In this paper we discuss the functionality and our experience of system deployment and operation on the full CMS scale.

DLS: a cloud-hosted data caching and prefetching service for distributed metadata access

Due to the diverse approaches and characteristics of scientific research, valuable resources within most important scientific disciplines exist in heterogeneous storage systems in distributed environments as they are often created and maintained by different information providers. Considering more than half of all data access operations in traditional storage systems are metadata access operations, it is important to design a framework that can effectively query and access valuable metadata information in distributed environments. In this paper, we present a highly efficient caching and prefetching mechanism tailored to reduce metadata access latency and improve responsiveness in wide-area data transfers. We designed dynamically provisioned parallel TCP streams and non-blocking concurrent in-memory cache for the system performance. We have implemented these mechanisms in a cloud-hosted metadata retrieval, caching, and prefetching system called directory listing service (DLS) and have evaluated its performance on both local area and wide area settings.

Tradeoff between storage cost and repair cost in heterogeneous distributed storage systems

AbstractIn distributed storage systems (DSS), the storage costs and download costs with different storage nodes, in general, can be different. In such heterogeneous storage systems, how to establish a fundamental tradeoff between system storage cost and system repair cost is investigated. We formulate the problem of establishing the tradeoff between system storage cost and system repair cost as a bi‐objective linear programming problem subject to the min‐cut constraint of information flow graphs. We give a tight min‐cut bound for heterogeneous DSSs with general setting. Moreover, we show that the tradeoff between system storage cost and system repair cost of some special heterogeneous DSSs can be established in polynomial time. Copyright © 2014 John Wiley & Sons, Ltd.

An Effective Cache Algorithm for Heterogeneous Storage Systems

Modern storage environment is commonly composed of heterogeneous storage devices. However, traditional cache algorithms exhibit performance degradation in heterogeneous storage systems because they were not designed to work with the diverse performance characteristics. In this paper, we present a new cache algorithm called HCM for heterogeneous storage systems. The HCM algorithm partitions the cache among the disks and adopts an effective scheme to balance the work across the disks. Furthermore, it applies benefit-cost analysis to choose the best allocation of cache block to improve the performance. Conducting simulations with a variety of traces and a wide range of cache size, our experiments show that HCM significantly outperforms the existing state-of-the-art storage-aware cache algorithms.

ARC-H: Adaptive replacement cache management for heterogeneous storage devices

Heterogeneous storage architectures combine the strengths of different storage devices in a synergistically useful fashion, and are increasingly being used in mobile storage systems. In this paper, we propose ARC-H, an adaptive cache replacement algorithm for heterogeneous storage systems consisting of a hard disk and a NAND flash memory. ARC-H employs a dynamically adaptive management policy based on ghost buffers and takes account of recency, I/O cost per device, and workload patterns in making cache replacement decisions. Realistic trace-driven simulations show that ARC-H reduces service time by up to 88% compared with existing caching algorithms with a 20Mb cache. ARC-H also reduces energy consumption by up to 81%.

Service-Oriented Collaborative Framework for High-Performance Data Transfer in Grids

As the number of data-intensive applications increases in various domains, scientists need to save, retrieve, and analyze increasingly large datasets. The huge volume of data and the long latency of data transfer on the Internet make it very difficult to ensure high-performance access to Data Grids. Thus, data replication techniques have been widely adopted to solve the latency problem. In this paper, we propose an efficient data replication algorithm for multi-source data transfer, whereby a data replica can be assembled in parallel from multiple distributed data sources and adapted to the fluctuation of network bandwidths. The experimental results show that the proposed algorithm can obtain more aggregated bandwidth, reduce connection overheads, and achieve superior load balance. We also develop a Service-oriented Collaborative Framework that not only implements the proposed data replication algorithm in a real Grid system, but also provides a virtual data management environment that integrates distributed and heterogeneous storage systems and enables users to share their data with group members for collaborative work.