RAID System Research Articles

Deconstructing storage arrays

We introduce Shear, a user-level software tool that characterizes RAID storage arrays. Shear employs a set of controlled algorithms combined with statistical techniques to automatically determine the important properties of a RAID system, including the number of disks, chunk size, level of redundancy, and layout scheme. We illustrate the correctness of Shear by running it upon numerous simulated configurations, and then verify its real-world applicability by running Shear on both software-based and hardware-based RAID systems. Finally, we demonstrate the utility of Shear through three case studies. First, we show how Shear can be used in a storage management environment to verify RAID construction and detect failures. Second, we demonstrate how Shear can be used to extract detailed characteristics about the individual disks within an array. Third, we show how an operating system can use Shear to automatically tune its storage subsystems to specific RAID configurations.

Deconstructing storage arrays

We introduce Shear, a user-level software tool that characterizes RAID storage arrays. Shear employs a set of controlled algorithms combined with statistical techniques to automatically determine the important properties of a RAID system, including the number of disks, chunk size, level of redundancy, and layout scheme. We illustrate the correctness of Shear by running it upon numerous simulated configurations, and then verify its real-world applicability by running Shear on both software-based and hardware-based RAID systems. Finally, we demonstrate the utility of Shear through three case studies. First, we show how Shear can be used in a storage management environment to verify RAID construction and detect failures. Second, we demonstrate how Shear can be used to extract detailed characteristics about the individual disks within an array. Third, we show how an operating system can use Shear to automatically tune its storage subsystems to specific RAID configurations.

Deconstructing storage arrays

We introduce Shear, a user-level software tool that characterizes RAID storage arrays. Shear employs a set of controlled algorithms combined with statistical techniques to automatically determine the important properties of a RAID system, including the number of disks, chunk size, level of redundancy, and layout scheme. We illustrate the correctness of Shear by running it upon numerous simulated configurations, and then verify its real-world applicability by running Shear on both software-based and hardware-based RAID systems. Finally, we demonstrate the utility of Shear through three case studies. First, we show how Shear can be used in a storage management environment to verify RAID construction and detect failures. Second, we demonstrate how Shear can be used to extract detailed characteristics about the individual disks within an array. Third, we show how an operating system can use Shear to automatically tune its storage subsystems to specific RAID configurations.

X-RAY

RAID storage arrays often possess gigabytes of RAM forcaching disk blocks. Currently, most RAID systems use LRUor LRU-like policies to manage these caches. Since these arraycaches do not recognize the presence of file system buffer caches,they redundantly retain many of the same blocks as those cachedby the file system, thereby wasting precious cache space. In thispaper, we introduce X-RAY, an exclusive RAID array cachingmechanism. X-RAY achieves a high degree of (but not perfect) exclusivitythrough gray-box methods: by observing which files havebeen accessed through updates to file system meta-data, X-RAYconstructs an approximate image of the contents of the file systemcache and uses that information to determine the exclusive set ofblocks that should be cached by the array. We use microbenchmarksto demonstrate that X-RAY's prediction of the file systembuffer cache contents is highly accurate, and trace-based simulationto show that X-RAY considerably outperforms LRU andperforms as well as other more invasive approaches. The mainstrength of the X-RAY approach is that it is easy to deploy - allperformance gains are achieved without changes to the SCSI protocolor the file system above.

2 years experience of a public domain PACS at a German university clinic

Ziele: The purpose of this endeavour was to combine the price drop in harddisk storage with freeware to build a PACS solution for our university hospital. Methode: The used program for DICOM communication and archiving is the freely available Conquest DICOM Server software running against a Microsoft SQL Server database on Windows 2000 Professional. As hardware we chose the personal computer (PC) platform. The computer was ordered and built at a local hardware store with our description. It uses dual AMD Athlons 2 Gbyte RAM. An external IDE RAID system with 12 160Gbyte hard disks is connected by SCSI interface. The RAID system is configured as RAID 5 with a hot spare disk resulting in 1,5 Tbyte of available and redundant disk space. Backup of the data is done on a CD-R robotic system. The price of the hardware was 15.000$ at the time of purchase (6/2002). Ergebnis: The system runs now for over two years flawlessly. Connected to the server are 2 CT scanners (1 Multislice, 1 spiral CT), 2 MR Scanners, 2 DSA workplaces and 2 digital flouroscopy workplaces. They produce a total of 60 Gbyte of data per month. 20 reading workstations are connected to the system running eFilm software from Merge. Schlussfolgerung: Our Public Domain and Open Source Software based PACS proved to be highly reliable. No data loss occurred. We fully recommend this PACS solution.

Non-restrictive MDS Array Codes for Correcting Multiple Column Erasures

A new family of MDS array codes of size m × n, that we call Br (m, n, t), for correcting multiple column erasures is proposed. Br(m, n, t) codes achieve the maximum possible correcting capability, i.e., they are MDS codes with dmin = r + 1. The key novelty in Br(m, n, t) codes is that they can be constructed for all possible code length up to (2m–l)and for any column size; thus overcoming constrains on the code parameters of conventional array codes. Br(m, n, t) codes have a simple structure, which is based on exclusive-OR operations, avoiding computations over finite fields. Analytical results show that the complexity of the proposed decoding algorithm is proportional to rm2(n–l), where r is the number of correctable erasures, i.e., is more efficient than the Forney decoding algorithm for Reed-Solomon codes. The proposed code can be used in any system requiring large symbols, for instance multitrack magnetic recording and RAID systems.

A multiple disk failure recovery scheme in RAID systems

In this paper, we propose a practical disk error recovery scheme tolerating multiple simultaneous disk failures in a typical RAID system, resulting in improvement in availability and reliability. The scheme is composed of the encoding and the decoding processes. The encoding process is defined by making one horizontal parity and a number of vertical parities. The decoding process is defined by a data recovering method for multiple disk failures including the parity disks. The proposed error recovery scheme is proven to correctly recover the original data for multiple simultaneous disk failures regardless of the positions of the failed disks. The proposed error recovery scheme only uses exclusive OR operations and simple arithmetic operations, which can be easily implemented on current RAID systems without hardware changes.

A Fault-Tolerant Architecture for Supporting Large Scale Digital Libraries

At the Research Library of Los Alamos National Laboratory (LANL), we have developed an affordable multi-terabyte redundant RAID system -- running on Linux -- that addresses the needs of a constantly growing demand for electronic journals, conferences and standards. Although there are several practical issues associated with systems supporting electronic content -- such as acquisition, archiving, access and cataloging -- this paper will focus on archiving and providing access to electronic journals. This article explains the design and development of a scalable, fault-tolerant and affordable system, presents an overview of the architecture, summarizes the problems and architectural issues encountered, and describes the technical solutions adopted.

데이터베이스 벤치마크를 위한 데이터 생성기와 성능 평가

데이터베이스 벤치마크는 많은 양의 데이터에 대한 효과적 생성을 요구한다. 본 논문은 개발된 데이터 생성기의 시스템 구성도, 데이터 생성 동작, 특성을 기술한다. 본 데이터 생성기 특성은 대용량 데이터 생성 지원, 칼럼 단위 데이터 생성, 다양항 데이터 분포 지원 및 검증, 실데이터 생성 등이다. 또한 타 데이터 생성기들과 기능 관점에서 비교하였다. 마지막으로 대용량 데이터 생성시의 입출력 병목현상 해결을 위해 RAID 시스템과 비 RAID 시스템간의 성능을 실험적으로 비교하였으며, 실험 결과에 근거하여 시스템 구성 가이드를 제시한다. Database benchmarks require efficient of large-scale data. This presents the system architecture, control flows, and characteristics of the data generator we have developed. The data generator features generation of large-scale data, column-by-column data generation, a number of data distributions and verification, and real data generation. An extensive conparison with other data generators in terms of function is also presented. Finally, empirical performance experiments between RAID systems and non-RAID one have been conducted to alleviate I/O bottleneck. The test results can serve as guidelines to help confifure system architecture.

내장형 리눅스 기반 RAID 시스템의 구현 및 성능평가

본 논문에서는 내장형 리눅스를 이용한 RAID 시스템의 하드웨어 구현 방법을 제시하고 성능을 측정하여 본 시스템의 장단점을 제시한다. 파이버 채널 디스크와 호스트와의정합을 위하여 세 개리 파이버 채널 제어기를 포함하는 하드웨어를 설계 제작하였으며, 이하드웨어 위에 내장형 리눅스를 이식하고 RAID 소프트웨어를 구현하였다. RAID 기능을 위하여 SCSI 목표 모드 디바이스 드라이버와 목표 모드 SCSI 모듈을 구현하여 호스트 컴퓨터에게 SCSI 블록 디바이스를 제공한다. 또한 RAID의 기능을 위하여 리눅스의 Multi-device 모듈을 사용하였고, 높은 성능을 제공하기 위하여 Multi-device 모듈과 목표 모드 SCSI 모듈의 사이에서 연동하는 데이터 캐쉬 모듈을 구현하였다. 리눅스의 RAID 5 모듈을 수정하여, 읽기 성능을 대폭 향상시켰다. 벤치마크는 새로운 RAID 5모듈이 기존의방법보다 전체적인 성능에서 우수함을 보여준다. In this article, we present, design, and implement a software and hardware for an embedded RAID system. The merits and drawbacks of our system are presented by performance evaluation. The proposed hardware system consists of three fibre channel controllers for the interface with fibre channel disks and hosts. Embedded Linux in which a RAID software is implemented is ported to the hardware. A SCSI target mode device driver and a target mode SCSI module are designed for that our RAID system is considered as a block device to a host computer. Linux Multi-device is used as RAID functions of this system. A data cache module is implemented for high performance and the interconnection between Linux Multi-device and the target mode SCSI module. The RAID 5 module of Multi-device is modified for improvement of read performance. The benchmark shows that the new RAID 5 module is superior to the original one in overall performance.

Experiences with ATM in a multivendor pilot system at Forschungszentrum Julich

The ATM technology for high speed serial transmission provides a new quality of communication by introducing novel features in a LAN environment, especially support of real time communication, of both LAN and WAN communication and of multimedia streams. In order to evaluate ATM for future DAQ systems and remote control systems as well as for a high speed picture archiving and communications system for medical images, Forschungszentrum Julich has build up a pilot system for the evaluation of ATM and standard low cost multimedia systems. It is a heterogeneous multivendor system containing a variety of switches and desktop solutions, employing different protocol options of ATM. The tests conducted in the pilot system revealed major difficulties regarding stability, interoperability and performance. The paper presents motivations, layout and results of the pilot system. Discussion of results concentrates on performance issues relevant for realistic applications, e.g., connection to a RAID system via NFS over ATM.

DVDs: Much Needed “Shot in the Arm” for Video Servers

The release of DVD-ROMs (Digital Versatile Disk—Read Only Memory), capable of storing an entire 133 minute MPEG-2 movie, coupled with the gradual deployment of high speed networks will give a much needed impetus to Video-on-Demand systems, especially movie-on-demand services. Various memory components such as DVD-ROMs, magnetic RAID towers, and RAM are available, each with different storage and bandwidth capabilities, and each with different costs. To design a video server, we need to optimize the cost factor, yet meet the massive storage, high bandwidth and continuity requirements of video stream delivery. In the first part of this paper we compare DVD-ROM and RAID systems on the basis of two factors—a storage factor and a bandwidth factor. We work out relations to calculate capacity requirements and costs, of these systems given the demands of video delivery. We then propose an architecture for a video server where we deploy three layers of memory functioning as video pumps: DVD-ROM towers, RAID towers, and RAM. We then work out expressions to determine the migration strategy of a movie between these layers of storage so as to optimize on the cost of storage while satisfying the performance requirements. We then show the method by which the amount of storage required for each type of memory in the three layers can be fixed, knowing the usage pattern. In the second part of the paper, we discuss the implementation of a video pump where there is a need to integrate scheduling, admission control, VBR stream management, and handling of VCR-like requests such as fast-forward, fast-reverse and pause. We first work out general relations for calculation of the service time of a request knowing the seek overheads and playback rates. We then show that, surprisingly, the C-SCAN algorithm, which incurs the least amount of seek overhead and therefore has a lesser cycle time, is twice as efficient as SCAN in terms of buffer space requirements. We then propose a ’full-load‘ admission control and scheduling algorithm that operates on a constant cycle time basis. We test the validity of this scheme though a the simulator which we have built based on our disk model and use it to extract parameters such as disk bandwidth utilization factor and buffer sizes needed for a VBR load scenario.

Communication facilities for distributed transaction-processing systems

The study of communication designs in the context of the Raid system, a robust and adaptable distributed database system for transaction processing, is discussed. Related research work on local interprocess communication, remote interprocess communication, and communication protocols for both local area and wide area networks is briefly summarized. A series of experiments on the performance of the facilities available for building the Raid communication software is described. Raid's communication software, called Raidcomm, has evolved as a result of the knowledge gained both from other systems and from the authors' experiments. Several communication services and mechanisms that can be used to make Raid efficient are identified. >

The foraging ecology of the army ant Eciton rapax: an ergonomic enigma?

Abstract. 1. The army ant Eciton rapax (F. Smith) produces longer raid systems than any other member of its genus and it is a specialist predator of forest floor and understory ants such as species of Camponotus, Odontomachus and Pachycondyla.2. Allometrical analysis confirms that E. rapax is the only member of its genus without distinct physical castes among its workers: its foraging population is entirely monomorphic and there are no majors.3. The workers of E. rapax are distributed over a considerable size range, and there are distinct divisions of labour within these colonies: small workers tend to stay in the nests and among the larger foraging workers those retrieving prey items are significantly bigger than the rest.4. An analysis of foraging efficiency and worker performance in E. rapax suggests that transport costs, resulting from the great distances that workers travel during raids and emigrations, are one of the selection pressures that have favoured the evolution of large monomorphic workers in this species.