CHMasters: A Scalable and Speed-Efficient Metadata Service in Distributed File System

Abstract

Distributed file system (DFS) is playing important roles of supporting large distributed data-intensive applications to meet storage needs. Typically, the design of DFS, such as GFS in Google, DMS in Cisco and TFS in Alibaba, is driven by observations of specific application workloads, internal demands and technological environment. In such systems, the metadata service is a critical factor that can affect the file system performance and availability to a great degree. Five requirements have been summarized for the metadata service: location transparent file service, smart director, efficient speed, strong scalability and friendly collaborator. In this paper, we present metadata service module called CH Masters in our DFS. Consistent hashing protocol is used to relieve potential hot spots on name servers. Files' metadata and master nodes are mapped into the same hash space by consistent hash function. And then files' metadata are scattered to master nodes by clockwise closest principle. Chunk server acts as a client when report its chunks info. Only a small proportion of files' metadata will be rehashed when master nodes state change. A new scalable file mapping strategy is also proposed to map file sizes from few MB to several GB efficiently. After intensive experiments, it shows CH Masters is satisfying the above five requirements.