Holistic cold-start management in serverless computing cloud with deep learning for time series

Abstract

Serverless computing is the most innovative and affordable cloud service model that improves three strategic business objectives: flexibility, efficiency, and innovation. Compared to monolithic applications, serverless computing applications scale better as the applications are further broken down into small building blocks of functions. Resources are assigned per function and can scale up as function requests increase. Hence, serverless computing has been increasingly employed in high-impact areas such as banking and health care. An issue affecting serverless computing’s performance is cold-start — the delays in assigning resources for function execution. Such delays can violate the Service Level Agreements especially those of time-sensitive applications. Current cold-start management policies operate in silos with narrow focuses on function, application, runtime, or infrastructure levels. Such policies do not incorporate higher-level requirements like those in the Service Level Agreements. Additionally, operating in silos does not follow the principle of continuous feedback loops among management policies at different system levels, which is essential for AI-driven self-adaptive future computing systems. Therefore, this paper proposes a novel 2-prong cold-start management policy that allows feedback loops with other higher management policies and orchestrates lower-level cold-start optimization policies. The main policy driver is the temporal convolutional network (TCN) model that can predict function instance arrivals from 5 to 15 min into the future. Evaluation strategies were proposed for both the TCN model and the management policy. Evaluation results show that the TCN model performs reliably across two trace datasets from leading serverless computing providers. Notably, the proposed cold-start policy evaluation plan allows organizations to realistically evaluate their cold-start management in ways that have never been done before.