An Adaptive Erasure Code for JointCloud Storage of Internet of Things Big Data

Abstract

JointCloud is a cross-cloud cooperation architecture for integrated Internet service customization. The customized cross-cloud storage service based on this architecture is called JointCloud storage. Storing the Internet of Things (IoT) big data in erasure-coded JointCloud storage systems ensures that data can be accessed when several cloud services interrupt. However, because existing erasure codes cannot adapt the generator matrix and data placement scheme to different network environments and encoding parameters, they usually incur a large network resource consumption (NRC) for repairing data in JointCloud storage systems. As a result, the availability of IoT applications running on JointCloud storage systems is impaired. In this article, to minimize the NRC of repairing data, we propose an adaptive erasure code for JointCloud storage of IoT big data called ACIoT. Specifically, we first propose the concept of average weighted locality (AWL) of a stripe of erasure-coded data, which is proportional to the average NRC of repairing this stripe in JointCloud storage systems. Then, we propose an active parallel trial-and-error algorithm to calculate the optimal generator matrix and data placement scheme to achieve the lowest AWL, under different network environments and encoding parameters. By encoding and placing each stripe of data with the optimal generator matrix and data placement scheme, ACIoT can achieve the minimum NRC. The experiments show that, compared with several state-of-the-art erasure codes, ACIoT reduces the NRC by 26.4%–44.7%.