No such thing as a “free launch”? Systematic benchmarking of containers

Abstract

Docker containers have recently become an extremely popular means of building robust, modular systems. Essentially, many architectures leverage lightweight virtualization to manage micro-services. Surprisingly however, there are very few studies revealing the overheads, such as starting new containers in orchestration systems, such as Kubernetes. Though traditional Virtual Machines (VMs) can take on the order of minutes to launch, containers are much faster and the launch times can be on the order of seconds. These overheads typically considered to be negligible compared with the benefits of container-based systems, however, are the predictable? Our work investigates these costs in a systematic study within a private cloud platform. The evaluation outlines a process for studies of this kind. Our results confirm that launch times of VMs are in the range of minutes, whereas containers typically only take seconds. However, these results also show that launch times for new containers do not always scale linearly. Specifically, we identify our system organized by Minikube, a tool that eases local deployment of Kubernetes, introduces a penalty on launch times once the number of containers exceeds 80% of the maximum number of pods available for the cluster. This work demonstrates the presence of unexpected overheads and the need for our proposed systematic infrastructure for testing deployments of containerized services at scale.