StructMesh: A storage framework for serverless computing continuum

Abstract

Computing continuum is becoming a solution for organizations to process and analyze data for supporting decision-making processes. In this context, serverless paradigm is arising as a solution to manage continuum computing. However, the management of data storage still represents an obstacle for integrating continuum computing and serverless paradigms into a single solution, as this has to be performed transparently to users through multiple infrastructures. This paper presents StructMesh, a storage framework for serverless continuum systems. This framework is based on a processing plane where functions are managed as patterns, and a data plane based on storage meshes that represent maps of storage resources available in a given infrastructure. The logical interconnection of storage meshes enables organizations to integrate storage resources into a single unified storage service, which creates data exchange channels for continuum processing throughout multiple infrastructures. These meshes include load-balancing and data allocation/location algorithms for transparently and automatically managing the inputs/outputs of functions throughout these channels, as well as non-functional requirement schemes for organizations to manage sensitive data. We developed a framework prototype that harmonizes processing serverless functions with storage functions for building serverless pipeline services. A case study was conducted by using these services for processing meteorological and earth observation data throughout multiple infrastructures. The evaluation revealed the efficiency of StructMesh when managing data through fog and cloud infrastructures. It also showed the feasibility of StructMesh to create and enable continuum data exchange channels for serverless pipelines.