Multi-index technique for metadata management in private cloud storage

Abstract

Cloud computing provides computing, storage and software resources as services to users on-demand over the Internet. Cloud computing has been increasingly gaining popularity due to the benefits such as its access capability from any location, sharing of data across multiple data centers, and the management of data storage by the service providers. Since security is still an unresolved issue in public cloud storage, business critical data can be preferably maintained in private cloud storage. It is normally built with the utilization of unused commodity machines. As storage space is limited in private cloud storage, it has to be efficiently utilized. Hence, a specialized technique known as deduplication is used to reduce the data footprint in the storage. This technique splits the file into several chunks. Chunks of different files are compared against each other to ensure only unique chunks are stored in the storage. Chunk index maintains the chunkID and the location of the actual chunk for reconstructing the file. Whenever a file is stored in the storage system, corresponding metadata is also placed along with it to facilitate the retrieval of that file. Metadata generally includes the properties of the file and the beginning location of physical block of the file. In deduplication enabled cloud storage system, since the file is stored in the form of a collection of chunks, chunk index also need to be maintained in addition to metadata. This improves the search time of a particular file. Thus, the efficiency of the chunk indexing mechanism will directly influence the file retrieval time. In order to achieve faster retrieval time, this paper proposes multi-index structure for metadata management.