Realizing dynamic resource orchestration on cloud systems in the cloud-to-edge continuum

Abstract

Cloud computing has been widely utilized to handle the huge volume of data from many cutting-edge research areas such as Big Data and Internet of Things (IoT). The fast growing of edge devices makes it difficult for cloud systems to process all data and jobs originating from edge devices, which leads to the development of edge computing by completing jobs on edges instead of clouds. Unfortunately, edge devices generally possess only limited computing power. Therefore, jobs demanding heavy computation under strict time constraints could have more difficulties to successfully complete their work on edges than on clouds in the Cloud-to-Edge continuum. If cloud systems could dynamically orchestrate cloud resources to expedite the execution of those jobs, not only their timely execution could be assured, also the loading of edge devices could be reduced. The Apache Hadoop is considered one of the most popular cloud systems in industry and academia. However, it does not support dynamic resource allocation. Previously we proposed and implemented a new model which can dynamically adjust the computing resources assigned to given jobs in the Hadoop cloud system to speed up their execution. Like other computer software, cloud systems completely rely on their underlying operating systems to access hardware components such as CPUs and hard drives. In this paper, we report our efforts to improve our model to collaborate with the Linux operating system to accelerate the execution of jobs with high priority to a greater extent. Compared with what our original model achieved, experiments show that our ameliorated model could further quicken the execution of prioritized jobs in Hadoop by up to around 21%. As a result, jobs from edges that require substantial computing resources promptly could have better chances to get accomplished on cloud systems.