Hashing Based Data Distribution in Heterogeneous Storage

Abstract

Storage systems are important infrastructures of cloud computing in data centers. They demand the efficiency to distribute data and provide high I/O performance. The consistent hashing algorithm is widely used in modern storage systems due to its decentralized design, scalability, and adaptability to node changes. However, it lacks efficiency in a heterogeneous environment. In this study, we propose a Dynamically Attributed Consistent Hashing (DACH), to overcome this deficiency. DACH manages heterogeneous storage resources on consistent hashing ring and maintains multiple, dynamic attributes for nodes to characterize their distinct features. It places data on the ring and selects nodes with a balanced, weighted data distribution algorithm by taking full advantage of node attributes. By considering attribute variation, such as remaining capacity and workload changes, DACH can further optimize data layout. Extensive evaluation results show that, by well exploiting storage heterogeneity, DACH achieves adaptive and efficient data distribution for heterogeneous storage systems.