Intelligent Horizontal Autoscaling in Edge Computing using a Double Tower Neural Network

Abstract

Edge computing is characterized by varying workload intensities that have a strong effect on applications’ performance and requirements in terms of resources. Thus, in order to maintain a sustainable performance a resource autoscaling mechanism that will automatically add or remove computational nodes is needed. This autoscaling mechanism must ensure that the user’s requirements in terms of Quality of Service (QoS) are satisfied, while respecting the Service Level Agreements (SLAs) and delivering always-on services using affordable on-demand computing solutions. The autoscaling could take place proactively or reactively and the adjustability of the resources could be static or dynamic. To this end, the research goal of this paper is to design a novel autoscaling mechanism that minimizes the task execution times and maximizes the corresponding throughput while using the minimum number of resources. In this direction, we propose an Intelligent Horizontal Proactive Autoscaling (IHPA) mechanism that leverages resource usage metrics of processing edge nodes such as CPU, RAM and Bandwidth in order to provide timely scale up and scale down decisions. The IHPA is based on a double tower Deep Learning (DL) architecture. In order to find a close to optimal DL architecture and guarantee the generality of our approach we also propose the innovative hybrid Bayesian Evolution Strategy method. We conduct an extensive simulation of the IHPA with three baseline task offloading mechanisms keeping the same statistical task generation distribution. We also compare the performance of our proposed IHPA against a reactive autoscaling approach and the Kubernetes Horizontal autoscaling mechanism. The results show that our framework shows significant improvements in terms of latency and throughput, while making an optimal use of the available resources.