MyStore: A High Available Distributed Storage System for Unstructured Data

Abstract

Although some NoSQL systems such as Dynamo, Cassandra, MongoDB have provided different advantages for unstructured data management, no one can provide flexible query functions like MongoDB, while guarantee the availability and scalability as Cassandra simultaneously. This paper introduces a new methodology and implementation for improving the availability of unstructured data by presenting a new distributed storage system called MyStore, based on the combination of MongoDB and some advantages from other NoSQL systems. Consistent hash is used to distribute data on multiple MongoDB nodes, NWR mode is applied to provide automatic backup operation and guarantee data consistency. Gossip protocol is taken for exchanging information of failures in the system. Based on above strategies, a high available and scalable system for storing unstructured data is realized, which can also provide complex query functions like rational database. Moreover, this system is applied in a multi-discipline virtual experiment platform named VeePalms that requires high availability and high access efficiency for its unstructured data such as XML scene, video guideline. Experimental evaluation shows that the methodology is powerful enough not only to enhance the data availability, but also to improve the server's scalability.