Redundant Disk Research Articles

Optimizing the restoration performance of deduplication systems through an energy-saving data layout

While data deduplication is an important data compression technique that removes copies of repeated data to enhance storage utilization, security and privacy risks arise since sensitive or delicate user data are at risk to both insider and outsider attacks. A distinct negative factor to performance of the technique is data fragmentation, which not only slows down the restoration process but also leads to massive power consumption. In this paper, we address this problem from the perspective of data layout. The kernel point of our method is a novel RAID-5-based cross grouping data layout (CGDL). We introduce a selective deduplication algorithm (SDD) to perform data replication and restoration. A new CGDL-based disk scheduling algorithm (LDP) is also proposed that predicts location dependence to save energy by eliminating the redundant disk read/write operations. We evaluate our new method on the Linux MD (multiple device) driver modules. The experiments show that, under a 10 disks 3 groups storage configuration, our method drastically (by 20%) improves restoration efficiency with only 7.6% reduction on the deduplication ratio, while reducing 23% power consumption.

Analysis of mean time to data loss of fault-tolerant disk arrays RAID-6 based on specialized Markov chain

This scientific paper is devoted to the analysis of the mean time to data loss of redundant disk arrays RAID-6 with alternation of data considering different failure rates of disks both in normal state of the disk array and in degraded and rebuild states, and also nonzero time of the disk replacement. The reliability model developed by the authors on the basis of the Markov chain and obtained calculation formula for estimation of the mean time to data loss (MTTDL) of the RAID-6 disk arrays are also presented. At last, the technique of estimation of the initial reliability parameters and examples of calculation of the MTTDL of the RAID-6 disk arrays for the different numbers of disks are also given.

Content Look-Aside Buffer for Redundancy-Free Virtual Disk I/O and Caching

Storage consolidation in a virtualized environment introduces numerous duplications in virtual disks and imposes considerable pressure on disk I/O and caching. In this paper, we present a content look-aside buffer (CLB) approach for simultaneously providing redundancy-free virtual disk I/O and caching. CLB attaches persistent fingerprints to virtual disk blocks, which enables detection of I/O redundancy before disk access. At run time, CLB exploits content pages already present in the guest disk caches to service the redundant reads through page sharing, thus eliminating both redundant I/O requests and redundant disk cache copies. For write requests, CLB uses a group invalidating writeback protocol for updating fingerprints to support crash consistency while minimizing disk write overhead. By implementing and evaluating a CLB prototype on KVM hypervisor, we demonstrate that CLB delivers considerably improved I/O performance with realistic workloads. Our CLB prototype improves the throughput of sequential and random read on duplicate data by 4.1x and 26.2x, respectively. For typical read-intensive workloads, such as booting VM and launching application, CLB's I/O deduplication and cache deduplication eliminates 94.9%--98.5% of read requests and saves 50%--100% cache memory in each VM, respectively. Compared with the QEMU's raw virtual disk format, CLB improves the per-disk VM density by 8x--16x. For mixed read-write workloads, the cost of on-line fingerprint updating offsets the read benefit; nevertheless, CLB substantially improves overall performance.

Exact Analysis of Some Split-Merge Queues

short-paper Share on Exact Analysis of Some Split-Merge Queues Authors: Pierre M. Fiorini CF Search Marketing, Portsmouth, NH CF Search Marketing, Portsmouth, NHView Profile , Lester Lipsky University of Connecticut University of ConnecticutView Profile Authors Info & Claims ACM SIGMETRICS Performance Evaluation ReviewVolume 43Issue 2September 2015 pp 51–53https://doi.org/10.1145/2825236.2825257Published:16 September 2015Publication History 9citation98DownloadsMetricsTotal Citations9Total Downloads98Last 12 Months4Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Distributed Seams for Gigapixel Panoramas

Gigapixel panoramas are an increasingly popular digital image application. They are often created as a mosaic of many smaller images. The mosaic acquisition can take many hours causing the individual images to differ in exposure and lighting conditions. A blending operation is often necessary to give the appearance of a seamless image. The blending quality depends on the magnitude of discontinuity along the image boundaries. Often, new boundaries, or seams, are first computed that minimize this transition. Current techniques based on multi-labeling Graph Cuts are too slow and memory intensive for gigapixel sized panoramas. In this paper, we present a parallel, out-of-core seam computing technique that is fast, has small memory footprint, and is capable of running efficiently on different types of parallel systems. Its maximum memory usage is configurable, in the form of a cache, which can improve performance by reducing redundant disk I/O and computations. It shows near-perfect scaling on symmetric multiprocessing systems and good scaling on clusters and distributed shared memory systems. Our technique improves the time required to compute seams for gigapixel imagery from many hours (or even days) to just a few minutes, while still producing boundaries with energy that is on-par with Graph Cuts.

Outage at UAA

With redundant hardware, it is rare that a disk failure results in downtime at the system level. System failures do sometimes occur, typically as a sequence of very rare events that leads to a catastrophic failure. This case describes how a combination of hardware and firmware failures, along with human error, led to the failure of a redundant disk storage unit, which in turn affected several enterprise systems at a major public university. Subsequently, a small number of conservative and seemingly “good” decisions in the process of restoring the system from backups led to negative outcomes, primarily additional downtime over the course of several days. The case illustrates how even well-considered and conservative decisions may seem flawed in hindsight. An important lesson from the case is that it is difficult to justify to management the provision of sufficient backup resources to prevent very low-probability failure events.

Development of Cloud Computing Based Intelligent Integrated Manufacturing System

A cloud computing based intelligent integrated manufacturing system was proposed. In order to develop an intelligent integrated manufacturing system, it is necessary to build up many database servers, which makes the system very expensive. To overcome this problem, we developed a cloud computing based system for quality analysis of the intelligent integrated manufacturing system. The architecture of the system contains: i) Cloud database structure; ii) Quality analysis engine; iii) Real-time human-machine interface. We use Microsoft SQL Azure as the cloud database to scatter and store data on redundant disk arrays. The quality analysis engine contains 12 quality input factors and 16 prediction outputs. It is used to predict manufacturing quality using neural networks. ASP.NET was used to program a human-machine interface with real-time data updating, providing a real-time data display available anytime, anywhere through the interface. Through experiments we found that that by using many database servers, improvement in a general traditional manufacturing system can be realized. The proven advantages of low cost, performance scalability, mobile management and maintenance facilities, make it important to develop a cloud computing based intelligent integrated manufacturing system using the analysis engine to predict manufacturing quality.

A New Fault Tolerance System for Cloud Storage

We present a class of array codes which are called Random Extensible Codes (RE Code). Benefit from the novel codes, we can encode an array of n disks using t redundant disks to protect the information against a failure of t/2 disks with n and t arbitrary. We can also add several extra disks to an existed disk array to improve its fault tolerant capability. This technique can be applied in a variety of occasions such as large-Scale fault-tolerant network storage system, highly unstable communication and so on, which differs itself from traditional array codes.

XHive: Efficient Cooperative Caching for Virtual Machines

Since a virtual machine independently uses its own caching policy, redundant disk operations exacerbate the I/O virtualization overhead when virtual machines access large amounts of data on shared storage. This paper presents XHive, an efficient cooperative caching system that is implemented at the virtualization layer, for consolidated environments. Our proposed scheme globally manages buffer caches of consolidated virtual machines in order to accommodate a shared working set in machine memory. A singlet, which is a block cached solely by a virtual machine, is preferentially given more chances to be cached in machine memory by XHive, when it is evicted by a guest operating system. For efficient use of limited memory, singlets are cached in memory that is collaboratively donated from idle memory of virtual machines. Our evaluation shows that XHive significantly reduces disk I/O operations for shared working sets, thereby achieving high read performance and scalability. Improved scalability enables a high degree of workload consolidation with respect to virtual machines that have shared working sets.

All-nearest-neighbors finding based on the Hilbert curve

An all-nearest-neighbors (ANN) query retrieves all nearest neighbors to all query objects. We may perform large number of one-nearest-neighbor queries to answer such an ANN query. Due to no total ordering of spatial proximity among spatial objects, the Hilbert curve approach has proposed to preserve the spatial locality. Chen and Chang have proposed a neighbor finding strategy (denoted as the CCSF strategy) based on the Hilbert curve to compute the absolute location of the neighboring blocks. However, it costs much time during the transformation between the Hilbert curve and the Peano curve. On the other hand, in the strategy based on R or R∗-trees for an ANN query, large number of unnecessary distance comparisons have to be done due to the problem of overlaps within the R-tree, resulting in many redundant disk accesses. Therefore, in this paper, we first propose the one-nearest-neighbor finding strategy directly based on the Hilbert curve (denoted as the ONHC strategy) for a one-nearest-neighbor query. By relations among orientations, orders, and quaternary numbers, we compute the relative locations of the query block and the neighboring block in the Hilbert curve. Then, the nearest neighbor of one query point can be found directly from these neighboring blocks. Next, by using our ONHC strategy, we propose the all-nearest-neighbors finding strategy based on the Hilbert curve (denoted as the ANHC strategy) for an ANN query. Finally, from the simulation result, we show that our ONHC strategy needs less response time (the CPU-time and the I/O time) than the CCSF strategy for the one-nearest-neighbor query. We also show that our ANHC strategy needs less response time than the strategy based on R∗-trees for an ANN query.

Exploiting In-Memory and On-Disk Redundancy to Conserve Energy in Storage Systems

Today's storage systems place an imperative demand on energy efficiency. A storage system often places single-rotation- rate disks into standby mode by stopping them from spinning to conserve energy when the workload is not heavy. The major obstacle of this method is a high spin-up cost introduced by passively waking up the standby disk to service requests. In this paper, we propose a redundancy-based hierarchical I/O cache architecture called RIMAC to solve the problem. The idea of RIMAC is to enable data on the standby disk(s) to be recovered by accessing a two-level I/O cache and/or active disks if needed. In parity-based redundant disk arrays, RIMAC exploits parity redundancy to dynamically XOR-reconstruct the data being accessed toward standby disk(s) at both the cache and disk levels. By avoiding passive spin-ups, RIMAC can significantly improve both energy efficiency and performance. In RIMAC, we developed 1) two power-aware read request transformation schemes called Transformable Read in Cache (TRC) and Transformable Read on Disk (TRD), 2) a power-aware write request transformation policy for parity update, and 3) a second-chance parity cache replacement algorithm to favor the request transformation rate. We evaluated RIMAC by augmenting a validated storage system simulator, DiskSim, and tested three real-life server traces, including HP's cello99, UlUC's OLTP, and SPC's search engine. Comprehensive results indicate that RIMAC is able to reduce energy consumption by up to 18 percent and simultaneously improve the average response time by up to 34 percent compared with threshold-based power management schemes for single-rotation-rate disk- based RAIDs.

ERAID: Conserving Energy in Conventional Disk-Based RAID System

Recently, high-energy consumption has become a serious concern for both storage servers and data centers. Recent research studies have utilized the short transition times of multispeed disks to decrease energy consumption. Manufacturing challenges and costs have so far prevented commercial deployment of multispeed disks. In this paper, we propose an energy saving policy, eRAID (energy-efficient RAID), for conventional disk-based mirrored and parity redundant disk array architectures. eRAID saves energy by spinning down partial or the entire mirror disk group with constraints of acceptable performance degradation. We first develop a multiconstraint energy-saving model for the RAID environment by considering both disk characteristics and workload features. Then, we develop a performance (response time and throughput) control scheme for eRAID based on the analytical model. Experimental results show that eRAID can save up to 32 percent energy while satisfying the predefined performance requirement.

Hard Disk Drives: The Good, the Bad and the Ugly!

HDDs are like the bread in a peanut butter and jelly sandwich—sort of an unexciting piece of hardware necessary to hold the “software.” They are simply a means to an end. HDD reliability, however, has always been a significant weak link, perhaps the weak link, in data storage. In the late 1980s people recognized that HDD reliability was inadequate for large data storage systems so redundancy was added at the system level with some brilliant software algorithms, and RAID (redundant array of inexpensive disks) became a reality. RAID moved the reliability requirements from the HDD itself to the system of data disks. Commercial implementations of RAID range from n+1 configurations (mirroring) to the more common RAID-4 and RAID-5, and recently to RAID-6, the n+2 configuration that increases storage system reliability using two redundant disks (dual parity). Additionally, reliability at the RAID group level has been favorably enhanced because HDD reliability has been improving as well.

RIMAC

Energy efficiency becomes increasingly important in today's high-performance storage systems. It can be challenging to save energy and improve performance at the same time in conventional (i.e. single-rotation-rate) disk-based storage systems. Most existing solutions compromise performance for energy conservation. In this paper, we propose a redundancy-based, two-level I/O cache architecture called RIMAC to address this problem. The idea of RIMAC is to enable data on the standby disk to be recovered by accessing data in the two-level I/O cache or on currently active/idle disks. At both cache and disk levels, RIMAC dynamically transforms accesses toward standby disks by exploiting parity redundancy in parity-based redundant disk arrays. Because I/O requests that require physical accesses on standby disks involve long waiting time and high power consumption for disk spin-up (tens of seconds for SCSI disks), transforming those requests to accesses in a two-level, collaborative I/O cache or on active disks can significantly improve both energy efficiency and performance.In RIMAC, we developed i) two power-aware read request transformation schemes called Transformable Read in Cache (TRC) and Transformable Read on Disk (TRD), ii) a power-aware write request transformation policy for parity update and iii) a second-chance parity cache replacement algorithm to improve request transformation rate. We evaluated RIMAC by augmenting a validated storage system simulator, disksim. For several real-life server traces including HP's cello 99, TPC-D and SPC's search engine, RIMAC is shown to reduce energy consumption by up to 33% and simultaneously improve the average response time by up to 30%.

Let's Acquire Data!

In fusion experiments, diagnostic control and logging devices are usually connected through the field bus, e.g. GP-IB. Internet technologies are often applied for their remote operation. All equipment and digitizers are driven by pre-programmed sequences, in which clocks and triggers give the essential timing for data acquisition. Data production rate and amount must be checked in comparison with the transfer and store rates. To store binary raw data safely, journaling file systems are preferably used with redundant disks (RAID) or mirroring mechanism, such as “rsync”. A proper choice of the data compression method not only reduces the storage size but also improves the I/O throughputs. DBMS is even applicable to quick search or security around the table data.

2 years experience of a public domain PACS at a German university clinic

Ziele: The purpose of this endeavour was to combine the price drop in harddisk storage with freeware to build a PACS solution for our university hospital. Methode: The used program for DICOM communication and archiving is the freely available Conquest DICOM Server software running against a Microsoft SQL Server database on Windows 2000 Professional. As hardware we chose the personal computer (PC) platform. The computer was ordered and built at a local hardware store with our description. It uses dual AMD Athlons 2 Gbyte RAM. An external IDE RAID system with 12 160Gbyte hard disks is connected by SCSI interface. The RAID system is configured as RAID 5 with a hot spare disk resulting in 1,5 Tbyte of available and redundant disk space. Backup of the data is done on a CD-R robotic system. The price of the hardware was 15.000$ at the time of purchase (6/2002). Ergebnis: The system runs now for over two years flawlessly. Connected to the server are 2 CT scanners (1 Multislice, 1 spiral CT), 2 MR Scanners, 2 DSA workplaces and 2 digital flouroscopy workplaces. They produce a total of 60 Gbyte of data per month. 20 reading workstations are connected to the system running eFilm software from Merge. Schlussfolgerung: Our Public Domain and Open Source Software based PACS proved to be highly reliable. No data loss occurred. We fully recommend this PACS solution.

A cost effective fault-tolerant scheme for RAIDs

The rapid progress in mass storage technology has made it possible for designers to implement large data storage systems for a variety of applications. One of the efficient ways to build large storage systems is to use RAIDs as basic storage modules. In general, the data can be recovered in RAIDs only when one error occurs. But in large RAIDs systems, the fault probability will increase when the number of disks increases, and the use of disks with big storage capacity will cause the recovering time to prolong, thus the probability of the second disk's fault will increase. Therefore, it is necessary to develop methods to recover data when two or more errors have occurred. In this paper, a fault tolerant scheme is proposed based on extended Reed-Solomon code, a recovery procedure is designed to correct up to two errors which is implemented by software and hardware together, and the scheme is verified by computer simulation. In this scheme, only two redundant disks are used to recover up to two disks' fault. The encoding and decoding methods, and the implementation based on software and hardware are described. The application of the scheme in software RAIDs that are built in cluster computers are also described. Compared with the existing methods such as EVENODD and DH, the proposed scheme has distinct improvement in implementation and redundancy.

A fault-tolerant continuous media disk array under arbitrary-rate search

Multimedia data, especially continuous media including video and audio objects, represent a rich and natural stimulus for humans, but require a large amount of storage capacity and real-time processing. End user operations consist of arbitrary-rate search, pause, and others as well as normal-rate play. FF (fast-forward) and FB (fast-backward) among those operations are desirable to find out the scene of interest, but they require non-sequential accesses of disks. When data accesses are clustered to several disks without considering load balance, high quality services in playback may not be available. In this paper, we propose a new disk placement scheme, called PRR/sub gp/ (prime round robin with grouped parities), to enhance reliability using the wasted disk storage spaces, in which continuous media are placed on a disk array to distribute disk accesses uniformly. The PRR/sub gp/ can not only achieve load balance of disks consisting of a disk array under arbitrary-rate search like PRR (prime round robin), but also improve reliability by storing parity information on the redundant disk spaces appropriately. We use combinatorial and Markov models to evaluate the reliability of a disk array. The previous PRR scheme cannot tolerate more than two simultaneous faults. However, about 30% of the simultaneous faults can be recovered using our PRR/sub gp/ scheme.

Declustered disk array architectures with optimal and near-optimal parallelism

This paper investigates the placement of data and parity on redundant disk arrays. Declustered organizations have been traditionally used to achieve fast reconstruction of a failed disk's contents. In previous work, Holland and Gibson identified six desirable properties for ideal layouts; however, no declustered layout satisfying all properties has been published in the literature. We present a complete, constructive characterization of the collection of ideal declustered layouts possessing all six properties. Given that ideal layouts exist only for a limited set of configurations, we also present two novel layout families. P RIME and R ELPR can tolerate multiple failures in a wide variety of configurations with slight deviations from the ideal. Our simulation studies show that the new layouts provide excellent parallel access performance and reduced incremental loads during degraded operation, when compared with previously published layouts. For large accesses and under high loads, response times for the new layouts are typically smaller than those of previously published declustered layouts by a factor of 2.5.

RAID keeps going and going and... [magnetic disk storage

From its conception as a small, simple, inexpensive array of redundant magnetic disks, redundant arrays of inexpensive disks (RAID) has grown into a sophisticated technology. Here, the author describes how the application of RAID has grown and grown until it has become a technology for making data constantly available.