ZettaDS: A Light-weight Distributed Storage System for Cluster

Abstract

We have designed and implemented the Zetta data storage system (ZettaDS), a light-weight scalable distributed data storage system for cluster. While sharing many common characters with some of modern distributed data storage systems such as single meta server architecture, running on inexpensive commodity components, our system is a very light-weight one and aims to handle lots of small files efficiently. The emphases of our design are on scalability of storage capacity and manageability. Throughput and performance are considered secondary. Furthermore, ZettaDS is designed to minimize the resource consumption due to running on a non-dedicated system.The paper describes the details and rationales of the design and implementation. Also, we evaluate our system by some experiments. The results demonstrate that our system can use the storage spaces more efficiently and achieve better transfer performance when facing a large number of small files.