OptRS: An Optimized Algorithm Based on CRS Codes in Big Data Storage Systems

Abstract

It is well-known that erasure codes, such as Reed-Solomon (RS) and Cauchy RS (CRS) codes, have played an important roles in big data storage systems to both industry and academia. While RS and CRS codes provide significant saving in storage space, they can impose a huge burden of systems performance while encoding and decoding. By studying existing high reliability and space saving rate of coding technologies, it is urgent to deploy an efficient erasure coding mechanism into distributed storage systems, which is the main storage architecture in big data era.This paper puts forward an optimized algorithm named OptRS (Optimized RS), which can not only guarantee the system’s reliability, but also enhance the efficiency and utilization of storage space. The dominant type of encoding and decoding inside erasure codes is matrix computation. In order to accelerate the speed of calculation, OptRS transferred the computation of matrix Galois field mapping into the XOR operation. Additionally, OptRS has developed the elimination schemes to minimize the numbers of XOR. Through theory analysis, we can conclude that OptRS algorithm improved the performance of encoding and decoding lead to shorten the computation time the same as verified by the test. The encoding efficiency with OptRS coding achieves 36.1 % and 58.2 % acceleration than using CRS and RS coding, respectively. The decoding rate by using OptRS can increase 19.3 % and 33.1 % compared with CRS and RS averagely by quantitative studying.