LaMeta: An efficient locality aware metadata management technique for an ultra-large distributed storage system

Abstract

The locality of metadata is a critical parameter of data storage and retrieval performance for an ultra-large distributed storage system. The data of an ultra-large distributed storage system is accessed through its distributed metadata servers (MDSs). The distributed MDSs store and cache partial metadata for scalability and performance. The MDSs are connected through varied latencies multipath network connections of using various metadata distribution techniques. The locality of MDS in the metadata distribution techniques improves the metadata query routing and enhances the performance of the distributed storage system. Motivated by this insight, this work introduces an intelligent locality-aware metadata management technique based on recommendation system that exploits MDS path ratings in a globally distributed ultra-large storage system and proposes a novel locality-aware metadata query routing algorithm. The proposed metadata server locality aware employs collaborative filtering with Stacked AutoEncoder to address the sparsity of the path rating matrix and its overfitting issue. The extensive experiments on real datasets show that the proposed metadata server locality aware technique enhances the locality of metadata operations and improves aggregate operations throughput by 12% to 21% of contemporary distributed metadata management techniques in ultra-large distributed storage systems.