DAScheduler: Dependency-Aware Scheduling Algorithm for Containerized Dependent Jobs

Abstract

Containers have emerged recently as a cloud technology for improving and managing cloud resources. They improve resource sharing by allowing instances to run on top of the host’s operating system. Container-based virtualization runs and manages hosted instances via the host kernel. Resource sharing can cause resource contention. In addition, dependent jobs, which may be deployed across multiple hosts, require frequent communication, resulting in a high volume of network traffic and network contention. The majority of existing research focuses on load balancing, with no consideration for the fact that network contention also plays a significant role in container performance. In this research, we propose a Dependency-aware Scheduling algorithm (DAScheduler) that deploys jobs into containers while accounting for both load balancing and job dependencies. The experimental results show that DAScheduler reduces network traffic by more than half and balances the loads. In comparison to one of the existing state-of-the-art techniques, DAScheduler improves overall cloud performance.