Replica parallelism to utilize the granularity of data

Abstract

As the volume of relational data is increased significantly, big data technologies have been noticed for recent years. Hadoop File System (HDFS) [14] is a basis of several big data systems and enables large data sets to be stored across the big data environment which is composed of many computers. HDFS divides large data into several blocks and each block is distributed and stored in a computer. To support reliability of data, HDFS replicates the data block. Generally, HDFS provides high-throughput when a client accesses to data. However, the architecture of HDFS is mainly designed to process data whose pattern is large and sequential. A data input whose pattern is small and random is not appropriate for applying HDFS. The data can result in several weak points of HDFS in terms of performance. HBase [2], one of Hadoop eco-systems, is a distributed data store which can process random, small read/write data efficiently. HBase utilizes HDFS structure but the block size of HBase is smaller than one of HDFS and a file of HBase is composed by blocks which is arranged by index structure. As many softwares which are related with big data science have emerged, many researches improving the system performance also have emerged steadily. Because a read workload is dominant in most big data workloads, read performance in the big data systems is crucial. To improve the read performance of HBase, we propose a replica parallelism algorithm utilizing the fact that block size of HDFS and HBase are different. When a read I/O is requested, HDFS generally selects a best node which is closest to client and I/O tend to converge on the specific node. We suggest distributing I/O jobs to replicas of HDFS based on sub-region of file to reduce a seek time and utilize the full bandwidth of all replicas. We analyzed the performance results of original Hbase using YCSB benchmark known as a well-known benchmark for distributed data store. In additional, we predict the performance of the replica parallelism comparing with original Hbase. According to predicted result of experiment, the replica parallelism can improve the read performance up to 1.5x ∼ 2.5x comparing with the conventional Hbase. Additionally, highly random read requests can intensify the effectiveness of the replica parallelism.