Erasure coded storage systems for cloud storage — challenges and opportunities

Abstract

Erasure coded storage schemes offer a promising future for cloud storage. Highlights of erasure coded storage systems are that these offer the same level of fault tolerance as that of replication, at lower storage footprints. In the big data era, cloud storage systems based on data replication are of dubious usability due to 200% storage overhead in data replication systems. This has prompted storage service providers to use erasure coded storage as an alternative to replication. Refinements are required in various aspects of erasure coded storage systems to make it a real contender against data replication based storage systems. Streamlining huge bandwidth requirements during the recovery of failed nodes, inefficient update operations, effect of topology in recovery and consistency requirements of erasure coded storage systems, are some areas which need attention. This paper presents an in-depth study on the challenges faced, and research pursued in some of these areas. The survey shows that more research is required to improve erasure coded storage system from being bandwidth crunchers to efficient storage systems. Another challenge that has emerged from the study is the requirement of elaborate research for upgrading the erasure coded storage systems from being mere archival storage systems by providing better update methods. Provision of multiple level consistency in erasure coded storage is yet another research opportunity identified in this work. A brief introduction to open source libraries available for erasure coded storage is also presented in the paper.