Tape Library Research Articles

The INFN Tier-1

INFN-CNAF is the central computing facility of INFN: it is the Italian Tier-1 for the experiments at LHC, but also one of the main Italian computing facilities for several other experiments such as BABAR, CDF, SuperB, Virgo, Argo, AMS, Pamela, MAGIC, Auger etc. Currently there is an installed CPU capacity of 100,000 HS06, a net disk capacity of 9 PB and an equivalent amount of tape storage (these figures are going to be increased in the first half of 2012 respectively to 125,000 HS06, 12 PB and 18 PB). More than 80,000 computing jobs are executed daily on the farm, managed by LSF, accessing the storage, managed by GPFS, with an aggregate bandwidth up to several GB/s. The access to the storage system from the farm is direct through the file protocol. The interconnection of the computing resources and the data storage is based on 10 Gbps technology. The disk-servers and the storage systems are connected through a Storage Area Network allowing a complete flexibility and easiness of management; dedicated disk-servers are connected, also via the SAN, to the tape library. The INFN Tier-1 is connected to the other centers via 3×10 Gbps links (to be upgraded at the end of 2012), including the LHCOPN and to the LHCONE. In this paper we show the main results of our center after 2 full years of run of LHC.

ENGLISH

This study is set out to find out the status of the multimedia resources available in the libraries of Delhi, to further investigate the diversity of nature, types, and formats of multimedia resources available in these libraries. The study also includes a brief account of facilities available for the conservation and preservation along with the multimedia production facilities available in the libraries. Findings of the study reveal that CDs/ DVDs are the most preferred physical storage form and compact shelves are the most preferred external storage form for the multimedia resources in all the libraries. The CEC (Consortium for Educational Communication)-Media Tape Library is the best multimedia library among all the libraries with all the latest equipments and best production facilities available throughout India.

TReqS: The Tape REQuest Scheduler

TReqS is a new layer in the storage infrastructure at CCIN2P3 controlling the read operations for data stored on tapes in HPSS (a HSM) by centralizing all requests. The development and deployment of TReqS at CCIN2P3 has brought major advances in data access services on several aspects. First, by sorting and scheduling tape mounts, we radically reduce the waiting time due to media movements in the tape library. Second, in a mutualised environment where tape drives are globally available to all users, TReqS ensures the fair share of the drives. Finaly, TReqS softens the impact of unpredictable grid jobs and transfers on the actual tape library and efficiently masks the data access complexity to the end user.

A high performance hierarchical storage management system for the Canadian tier-1 centre at TRIUMF

We describe in this paper the design and implementation of Tapeguy, a high performance non-proprietary Hierarchical Storage Management (HSM) system which is interfaced to dCache for efficient tertiary storage operations. The system has been successfully implemented at the Canadian Tier-1 Centre at TRIUMF. The ATLAS experiment will collect a large amount of data (approximately 3.5 Petabytes each year). An efficient HSM system will play a crucial role in the success of the ATLAS Computing Model which is driven by intensive large-scale data analysis activities that will be performed on the Worldwide LHC Computing Grid infrastructure continuously. Tapeguy is Perl-based. It controls and manages data and tape libraries. Its architecture is scalable and includes Dataset Writing control, a Read-back Queuing mechanism and I/O tape drive load balancing as well as on-demand allocation of resources. A central MySQL database records metadata information for every file and transaction (for audit and performance evaluation), as well as an inventory of library elements. Tapeguy Dataset Writing was implemented to group files which are close in time and of similar type. Optional dataset path control dynamically allocates tape families and assign tapes to it. Tape flushing is based on various strategies: time, threshold or external callbacks mechanisms. Tapeguy Read-back Queuing reorders all read requests by using an elevator algorithm, avoiding unnecessary tape loading and unloading. Implementation of priorities will guarantee file delivery to all clients in a timely manner.

The Pathologies of Big Data

What is "big data" anyway? Gigabytes? Terabytes? Petabytes? A brief personal memory may provide some perspective. In the late 1980s at Columbia University I had the chance to play around with what at the time was a truly enormous "disk": the IBM 3850 MSS (Mass Storage System). The MSS was actually a fully automatic robotic tape library and associated staging disks to make random access, if not exactly instantaneous, at least fully transparent. In Columbia’s configuration, it stored a total of around 100 GB. It was already on its way out by the time I got my hands on it, but in its heyday, the early to mid-1980s, it had been used to support access by social scientists to what was unquestionably "big data" at the time: the entire 1980 U.S. Census database.

Characterizing and designing photopolymer materials

Photopolymers are organic materials that polymerize when illuminated with a suitable wavelength of light. They have many potential applications, such as optically written waveguides for 3D photonic circuits. Perhaps the most exciting applications are in the area of data storage. Holographic data storage has been identified as a significant goal, and photopolymers are the medium of choice for the digital versatile disc (DVD)-sized devices with terabyte-plus data densities promised by these 3D optical storage techniques. High-density magnetic tapes can currently hold∼500GB (approximately 200 million pages of text). Magnetic tape libraries, sold to governments and large companies, are made by connecting cabinets that hold ∼5,000 tapes. Companies like StorageTech sell over 22,000 of these cabinets annually, so the need for high-volume storage exists. Several important technical milestones have recently been achieved in this area, such as InPhase Technologies launching their holographic disc-drive in 2008. The commercial possibilities may explain why several companies, including Bayer, DuPont, General Electric, and Sony have entered this research area. Since 2000,1 our group has been developing detailed electromagnetic and material models to describe the operation of photopolymers.2–5 The aim has been to develop simulations4 that characterize the physical and chemical effects taking place within the material during and after exposure. These efforts could allow scientists to optimize materials and simulate their applications. Figure 1 shows the material processes involved. Exposing light produces a sinusoidal interference pattern. The grating causes diffraction of the probing beam in the non-latent material. Polymerization (initiated in the bright regions of the material layer) removes the monomer, producing a concentration gradient. The monomer diffuses from the dark to the bright regions. Non-local growth of the polymer chains (away from the point of Figure 1. Material processes occurring in photopolymers during and after exposure

Frictional vibration transmission from a laterally moving surface to a traveling beam

As the density of information stored in automated magnetic tape libraries continues to increase, greater requirements are placed on the precision of mechanical positioning in order to successfully read and write data bits. The location of the read/write head in the direction across the tape's width (termed the lateral direction) is actively controlled in order to maintain alignment between the head and data tracks, even in the presence of the tape's lateral vibration. However, during repositioning, vibration is undesirably transmitted from the laterally moving head structure to the axially moving tape because of frictional contact between the two adjacent surfaces. As an analog of that interaction, a model is developed here to describe frictional vibration transmission from a surface having prescribed lateral motion to a tensioned beam that travels and slides over it. For a transport speed that is high when compared to the lateral vibration velocity, Coulomb friction between the surface and the beam can be well-approximated by an equivalent form of viscous damping. The beam is divided into contiguous regions corresponding to free spans and the beam's portion that contacts the surface. A critical engagement length between the beam and the surface exists for which vibration transmission at a particular natural frequency can be substantially reduced, and for a given mode, that length depends weakly on the surface's position along the beam's span. By contouring the surface to have portions of differing radii of curvature, the extent of vibration transmission can be reduced over a broad range of frequency.

Data Processing Model for the CDF Experiment

The data processing model for the CDF experiment is described. Data processing reconstructs events from parallel data streams taken with different combinations of physics event triggers and further splits the events into datasets of specialised physics interests. The design of the processing control system makes strict requirements on bookkeeping records, which trace the status of data files and event contents during processing and storage. The computing architecture was updated to meet the mass data flow of the Run II data collection, recently upgraded to a maximum rate of 40 MByte/sec. The data processing facility consists of a large cluster of Linux computers with data movement managed by the CDF data handling system to a multi-petaByte Enstore tape library. The latest processing cycle has achieved a stable speed of 35 MByte/sec (3 TByte/day). It can be readily scaled by increasing CPU and data-handling capacity as required

천문 계산용 고성능 클러스터 구축

A high performance computing cluster for astronomical computations has been built at Korea Astronomy Observatory. The 64 node cluster interconnected with Gigabit Ethernet is composed of 128 Intel Xeon processors, 160 GB memory, 6 TB global storage space, and an LTO (Linear Tape-Open) tape library. The cluster was installed and has been managed with the Open Source Cluster Application Resource (OSCAR) framework. Its performance for parallel computations was measured with a three-dimensional hydrodynamic code and showed quite a good scalability as the number of computational cells increases. The cluster has already been utilized for several computational research projects, some of which resulted in a few publications, even though its full operation time is less than one year. As a major resource of the testbed, the cluster has been used for Grid computations, too.

Implementation of EIDE disk array system for mass data backup

Today, EIDE hard disk drives have become the mainstream products in low-end storage applications. An inexpensive disk array system based on EIDE hard disk drives is proposed, which has not only much higher speed than a tape library but also much lower price than a general SCSI or fibre channel disk array. Each string controller in a disk array can support up to 16 hard disk drives (4TB) and multiple RAID levels (RAID 0,1,0+1,3, 5, NRAID or JBOD). It also provides multiform interfaces for the host computer, including SCSI, USB, FireWire, EIDE, Serial ATA, etc. Since this kind of disk array has excellent price performance ratio, it is worth using in a mass data backup area instead of tape equipments and other low-end secondary storage applications.

Innovations in tape storage automation at IBM

In the mid-1980s, tape storage appeared to be heading toward the graveyard. Storing information on large reels of tape that had to be manually mounted was expensive, inefficient, and prone to human error. In addition, competition from hard disk drives (HDDs) was growing. Data access on HDD was faster because the data was stored in a manner for rapid random access, not linearly as on tape, and HDD capacity was rapidly growing. Then, in 1984, IBM introduced the half-inch tape cartridge (3480), which was novel in its small form factor. In 1986, an automatic loading mechanism followed. In 1987 and 1988, competitors introduced the first automated tape libraries to use the half-inch tape cartridge. Suddenly, tape storage became considerably easier to use and more efficient. With automation, it also represented the least expensive way to store data. In 1993, IBM introduced the 3495 tape library, and now offers several different automated libraries ranging from the smaller 3575 MP and 3581/3583 models to very large libraries such as the 3494 and 3584 models. This paper examines the history of IBM tape storage and automation products, including the engineering challenges that were met in response to users' pressing requirements; it also examines the future of automated tape storage.

Tape management in a storage networking environment

Today, the paradigm shift in storage architectures from direct-attached storage to storage area network and network-attached storage, and to emerging standards such as iSCSI, has a strong impact on the requirements for storage management software. The managed storage resources could be disks (from a few disks to a large disk subsystem) or tapes (from a single tape drive to large automated tape libraries). A key challenge in this field is the management of removable media such as tape, optical, or others. Besides having to keep track of a potentially enormous number of different volumes and maintaining records of important attributes such as media owner, age, I/O errors, media-pool affiliation, and much more, the open storage networking environment raises new questions. In a scenario in which the use of SCSI devices is no longer restricted to a small number of locally attached servers, security and robustness become increasingly important issues. Access conflicts must be carefully controlled, mount operations have to run in a robust manner, and it is mandatory to comply with a security concept that protects data as well as privacy and confidentiality. This paper concerns the new requirements for tape management systems in storage networking environments. It describes the challenges and relates these requirements to the standards that exist today.

Recent changes of JT-60 data processing system

The JT-60 data processing system is a large computer complex, gradually being modernized by utilizing progressing computer and network technologies. There were three major changes in our system recently. A VME-based fast data acquisition system (FDS) has been developed to enlarge a transient mass-data storage system (TMDS). It can acquire the data of 6 MB per channel with 1 or 5 μs sampling. Now there are three FDSs with 29 channels. The TMDS, which was composed of a mini-computer and 61 channels of 6 MB memory modules with 5-μs sampling, has been replaced with a new system based on the technology of the VME-based fast data acquisition system. One of the outdated auxiliary crate controllers with micro-processor (ACM), which controls CAMAC modules in the crate on the serial highway, acquires data from them and transfers data to the main computer, has been replaced by a workstation with a VMEbus byte-serial highway driver. The combination of a workstation and a VMEbus byte-serial highway driver has been developed and used to replace many mini-computers in our system. A workstation with a digital linear tape (DLT) library has been introduced as final data storage. The conversion of a few terabytes of raw data tapes for a cartridge tape library (CTL), which is attached to a mainframe computer, Fujitsu GS8300, will be started by using data conversion software prepared to handle mainframe data on the workstation.

Fundamentals of Scheduling and Performance of Video Tape Libraries

Robotic tape libraries are popular for applications with very high storage requirements, such as video servers. Here, we study the throughput of a tape library system, we design a new scheduling algorithm, the so-called Relief, and compare it against some older/straightforward ones, like FCFS, Maximum Queue Length (MQL) and an unfair one (Bypass), roughly equivalent to Shortest Job First. The proposed algorithm incorporates an aging mechanism in order to attain fairness and we prove that, under certain assumptions, it minimizes the average start-up latency. Extensive simulation experiments show that Relief outperforms its competitors (fair and unfair alike), with up to 203% improvement in throughput, for the same rejection ratio.

Cutting edge of the Information Storage Technologies. Magnetic Tape Recording and Tape Library.

Cutting edge of the Information Storage Technologies. Magnetic Tape Recording and Tape Library.

Overview of the CDF Run II data handling system

The Collider Detector at Fermilab (CDF) collaboration records and analyses proton–anti-proton interactions with a center-of-mass energy of 2 TeV at the Tevatron. During the next collider run the experiment expects to record about 1 PetaByte of data, an increase by more than an order of magnitude in data volume compared to the existing data of the experiment. This paper gives an overview of the new data handling system. The design builds upon successful strategies used in the past but eliminates shortcomings encountered with the old system. The core of the central analysis system will be a pool of over 20 TBytes of data disks. Logically behind this disk pool will be an automated tape library with a storage capacity to hold the full 1 PByte of Run II data. Multi-processor compute systems will be located around this storage pool to provide the required analysis power of over 3000 SPECint95. Desktop systems will be integrated into the analysis system via a shared login area and user authentication. The data handling software falls into four categories: software to manage computing resources, software for the data disks and for the mass storage subsystem, software to organize and manage the meta-data information, and the I/O modules in the analysis framework for accessing data.

Management of the picture archiving and communications system archive at Texas Children's Hospital.

As hospitals convert from conventional film-based imaging to picture archival and communications systems (PACS), methods for managing an enormous library of images must change considerably. While most hospitals are required to retain general, nonmammographic, radiologic images for 7 years beyond the examination date, our pediatric hospital must maintain images until the child's eighteenth birthday, plus the 5-year statute of limitations. Although the physical extent of an electronic archive is tiny compared with a film archive, a long-range strategy is required to ensure that electronic images acquired today can be retrieved and viewed 23 years in the future. Challenges to the long-term stability of the electronic archive include the limited and uncertain shelf life of high-density electronic storage media, the finite maintainability of the electromechanical systems for reading the media, the short product lifetime of software for accessing the images, rapid development of higher density storage products, and the exponential advancement of computer and networking technology that fuels product obsolescence. Since we cannot assure the function of our current archive in two decades, we are committed to a continual process of migration of old electronic image data to newer media and systems. As an early-adopter of PACS technology, Texas Children's Hospital's (TCH) archive management experience is relevant to others. Although not filled to capacity, our first digital archive, based on phase-change write-once-read-many (WORM) technology, was forced into an inactive status by software and hardware changes. Our second set of archives was partially filled with low-density magneto-optical disk (MOD) media, when the drives were upgraded to high density and then filled to capacity. This undesirable situation forced us into shelf management of media. Our third-generation archive is based on a helical tape library with the capacity to contain 7 years of examinations. We will describe the motivation for data migration, limitations in the methods available to perform the migration, and unanticipated benefits of the migration process.

Multimedia data storage and representation issues on tertiary storage subsystems

Storage of multimedia data is a critical issue for the overall system's performance and functionality. Multimedia applications must be able to support a variety of data defined as multimedia objects. Multimedia data consist of various objects such as text, image, video and graphics, which need to be synchronized and meet certain timing requirements. The development and evolution of these new applications characterized by high storage needs resulted in strengthening the role and importance of Tertiary Storage Systems. Recent technological advances have resulted in wide availability of commercial products offering near-line, robot-based, tertiary storage libraries. Therefore, such libraries have become an important component of modern large-scale storage servers. The issue of optimal data placement strategies in tertiary storage is considered below. Different configurations of tape libraries and magnetic tape technologies as well as the special attributes of stored data have an impact on the optimal placement.

Continuous Data Block Placement in and Elevation from Tertiary Storage in Hierarchical Storage Servers

Given the cost of memories and the very large storage and bandwidth requirements of large-scale multimedia databases, hierarchical storage servers (which consist of disk-based secondary storage and tape-library-based tertiary storage) are becoming increasingly popular. Such server applications rely upon tape libraries to store all media, exploiting their excellent storage capacity and cost per MB characteristics. They also rely upon disk arrays, exploiting their high bandwidth, to satisfy a very large number of requests. Given typical access patterns and server configurations, the tape drives are fully utilized uploading data for requests that “fall through” to the tertiary level. Such upload operations consume significant secondary storage device and bus bandwidth. In addition, with present technology (and trends) the disk array can serve fewer requests to continuous objects than it can store, mainly due to IO and/or backplane bus bandwidth limitations. In this work we address comprehensively the performance of these hierarchical, continuous-media, storage servers by looking at all three main system resources: the tape drive bandwidth, the secondary-storage bandwidth, and the host's RAM. We provide techniques which, while fully utilizing the tape drive bandwidth (an expensive resource) they introduce bandwidth savings, which allow the secondary storage devices to serve more requests and do so without increasing demands for the host's RAM space. Specifically, we consider the issue of elevating continuous data from its permanent place in tertiary for display purposes. We develop algorithms for sharing the responsibility for the playback between the secondary and tertiary devices and for placing the blocks of continuous objects on tapes, and show how they achieve the above goals. We study these issues for different commercial tape library products with different bandwidth and tape capacity and in environments with and without the multiplexing of tape libraries.

Hierarchical storage management: leveraging new capabilities

E-commerce has created new problems in data management as mission-critical data swells a business' database into the terabytes. IT shops must efficiently manage data and still provide a rock-solid system with guaranteed 24/7 access. Finding a way to effectively fulfil this mission requires rethinking traditional data management methods. Among the many tools already available, hierarchical storage management (HSM) is an effective data management solution for this demanding new environment. HSM enables centralized data management across a variety of storage platforms, such as servers, RAID (redundant arrays of inexpensive disks), magneto-optical jukeboxes, and tape libraries. It maintains data through a three-level management system. Level one is online data maintained in an online server volume or RAID. Level two is infrequently accessed data; so HSM maintains it in slower storage platforms. Level three is offline data stored in a tape library or on shelved media. The business rarely accesses such data, basically keeping it to comply with record retention rules.