Design and optimization for distributed indexing scheme in switch-centric cloud storage system

Abstract

The capacity of data management and the query performance are two main metrics for today's cloud storage system. In this paper, we design a two-layer indexing scheme of distributed secondary index on switch-centric data center network (DCN) topologies. We take advantage of the desirable features of switch-centric topologies, such as stability, scalability, and fault tolerance, and successfully improve the query efficiency for cloud storage system. We choose B+-tree as the local layer index to locate data in each local host, while implement segment tree as the global layer index to manage a portion of metadata published from local hosts. We also optimize the query processing protocols to reduce false positives and network cost. In addition, we propose a top-down index selection method for maintenance. We then analyze the efficiency of query processing theoretically and calculate the expected number of routing cost for each query precisely. Finally we validate the efficiency of our design by experiments and comparison with a previous work.