Linux Containers Research Articles

Plancton: an opportunistic distributed computing project based on Docker containers

The computing power of most modern commodity computers is far from being fully exploited by standard usage patterns. In this work we describe the development and setup of a virtual computing cluster based on Docker containers used as worker nodes. The facility is based on Plancton: a lightweight fire-and-forget background service. Plancton spawns and controls a local pool of Docker containers on a host with free resources, by constantly monitoring its CPU utilisation. It is designed to release the resources allocated opportunistically, whenever another demanding task is run by the host user, according to configurable policies. This is attained by killing a number of running containers. One of the advantages of a thin virtualization layer such as Linux containers is that they can be started almost instantly upon request. We will show how fast the start-up and disposal of containers eventually enables us to implement an opportunistic cluster based on Plancton daemons without a central control node, where the spawned Docker containers behave as job pilots. Finally, we will show how Plancton was configured to run up to 10 000 concurrent opportunistic jobs on the ALICE High-Level Trigger facility, by giving a considerable advantage in terms of management compared to virtual machines.

Yadage and Packtivity – analysis preservation using parametrized workflows

Preserving data analyses produced by the collaborations at LHC in a parametrized fashion is crucial in order to maintain reproducibility and re-usability. We argue for a declarative description in terms of individual processing steps – “packtivities” – linked through a dynamic directed acyclic graph (DAG) and present an initial set of JSON schemas for such a description and an implementation – “yadage” – capable of executing workflows of analysis preserved via Linux containers.

LHCb Dockerized Build Environment

Used as lightweight virtual machines or as enhanced chroot environments, Linux containers, and in particular the Docker abstraction over them, are more and more popular in the virtualization communities.The LHCb Core Software team decided to investigate how to use Docker containers to provide stable and reliable build environments for the different supported platforms, including the obsolete ones which cannot be installed on modern hardware, to be used in integration builds, releases and by any developer.We present here the techniques and procedures set up to define and maintain the Docker images and how these images can be used to develop on modern Linux distributions for platforms otherwise not accessible.

LXCloud-CR: Towards LinuX Containers Distributed Hash Table based Checkpoint-Restart

Infrastructure-as-a-Service (IaaS) container-based virtualization technology is gaining, over these years, significant interest in industry as an alternative platform for running distributed applications. They present an interesting alternative to virtual machines in the Cloud due to newer advances in container-based virtualization as a new technology that simplify the deployment of applications. As the cloud architectures continue to grow in scale and complexity, faults become very recurrent which make reliability a true challenge. Given the dynamic nature of IaaS clouds and the pay-as-you-go cloud model where the costs are directly proportional to the resource usage, a Checkpoint Restart mechanism is essential in this context. We propose LXCloud-CR our new decentralized Checkpoint-Restart model based on a distributed checkpoints repository using key–value store on a Distributed Hash Table (DHT). It is able to take snapshots of the whole Linux Container (LXC) instances. LXCloud-CR aims to reduce the runtime and storage overheads of checkpointing. Large scale experiments on the Grid’5000 testbed demonstrate the benefits of our proposal. Obtained results validate our model and improve the performance of applications.

Performance analysis of Linux containers for high performance computing applications

Although cloud infrastructures can be used as High Performance Computing (HPC) platforms, many issues from virtualisation overhead have kept them almost unrelated. However, with advent of container-based virtualisation, this scenario acquires new perspectives because this technique promises to decrease the virtualisation overhead, achieving a near-native performance. In this work, we analyse the performance of a container-based virtualisation solution - Linux Container (LXC) - against a hypervisor-based virtualisation solution - KVM - under HPC activities. For our experiments, we consider CPU and (network and inter-process) communication performance. Results show that hypervisor type can impact distinctly in performance according to resource used by HPC application.

Performance analysis of Linux containers for high performance computing applications

Performance analysis of Linux containers for high performance computing applications

Containers for Portable, Productive, and Performant Scientific Computing

Containers are an emerging technology that hold promise for improving productivity and code portability in scientific computing. We examine Linux container technology for the distribution of a non-trivial scientific computing software stack and its execution on a spectrum of platforms from laptop computers through to high performance computing (HPC) systems. We show on a workstation and a leadership-class HPC system that when deployed appropriately there are no performance penalties running scientific programs inside containers. For Python code run on large parallel computers, the run time is reduced inside a container due to faster library imports. The software distribution approach and data that we present will help developers and users decide on whether container technology is appropriate for them. We also provide guidance for the vendors of HPC systems that rely on proprietary libraries for performance on what they can do to make containers work seamlessly and without performance penalty.

Simplifying virtual linux containers using docker.

Docker is a relatively new take on Linux container based virtualization. Docker is quickly being adopted and used worldwide for its ease of use. It makes use of Linux kernel namespaces and control-groups to isolate running container instances, which are self-contained and fully-functioning systems. This makes it possible for multiple container instances to run on the same host natively using the same host Linux kernel. This contrasts with the current trend of hypervisor based virtualization, which uses a virtual guest operating system running on top of the host operating system. Because each system has its own configuration, porting hefty applications can be troublesome. Using Docker, applications can be ported from a development environment to a production environment quickly and seamlessly. Docker packages this technology into an easy-to-manage system which can be maintained by both system administrators and developers alike. By utilizing Docker, businesses can increase their efficiency and productivity by saving time which is usually reserved for systems configuration.

Secure yet usable: Protecting servers and Linux containers

Many computer security systems are considered a burden. Their inherent intrusiveness may often have an impact on the overall system stability and may conflict with a continuous stream of updates to a server operating system and components. Additionally, their complexity, and the lack of sufficient understanding of how to operate them efficiently, leads to subpar utilization of their full potential. We claim that a computer security system must make usability one of its top priorities, arguably the first, to have any chance of being correctly and fully used. In this paper, we describe Starlight, a protection tool that has usability as its core trait. We discuss the tradeoffs between security and usability and how we addressed them. Starlight monitors the behavior of a running system and creates a customized security policy, a set of operating system execution rules that accurately defines the execution boundaries of the system. We demonstrate the capabilities of our system to protect the runtime environments of servers with Linux ® containers, which add kernel exploits risks via exposure to vulnerable or rogue applications.

MultiLanes

OS-level virtualization is often used for server consolidation in data centers because of its high efficiency. However, the sharing of storage stack services among the colocated containers incurs contention on shared kernel data structures and locks within I/O stack, leading to severe performance degradation on manycore platforms incorporating fast storage technologies (e.g., SSDs based on nonvolatile memories). This article presents MultiLanes, a virtualized storage system for OS-level virtualization on manycores. MultiLanes builds an isolated I/O stack on top of a virtualized storage device for each container to eliminate contention on kernel data structures and locks between them, thus scaling them to manycores. Meanwhile, we propose a set of techniques to tune the overhead induced by storage-device virtualization to be negligible, and to scale the virtualized devices to manycores on the host, which itself scales poorly. To reduce the contention within each single container, we further propose SFS, which runs multiple file-system instances through the proposed virtualized storage devices, distributes all files under each directory among the underlying file-system instances, then stacks a unified namespace on top of them. The evaluation of our prototype system built for Linux container (LXC) on a 32-core machine with both a RAM disk and a modern flash-based SSD demonstrates that MultiLanes scales much better than Linux in micro- and macro-benchmarks, bringing significant performance improvements, and that MultiLanes with SFS can further reduce the contention within each single container.

Lark: An effective approach for software-defined networking in high throughput computing clusters

High throughput computing (HTC) systems are widely adopted in scientific discovery and engineering research. They are responsible for scheduling submitted batch jobs to utilize the cluster resources. Current systems mostly focus on managing computing resources like CPU and memory; however, they lack flexible and fine-grained management mechanisms for network resources. This has increasingly been an urgent need as current batch systems may be distributed among dozens of sites around the globe like Open Science Grid. The Lark project was motivated by this need to re-examine how the HTC layer interacts with the network layer.In this paper, we present the system architecture of Lark and its implementation as a plugin of HTCondor which is a popular HTC software project. Lark achieves lightweight network virtualization at per-job granularity for HTCondor by utilizing Linux container and virtual Ethernet devices; this provides each batch job with a unique network address in a private network namespace. We extended HTCondor’s description language, ClassAds, so users can specify networking requirements in the job submission script. HTCondor can perform matchmaking to make sure user-specified network requirements and resource-specific policies are fulfilled. We also extended the job agent, condor_starter, so that it can manage and configure the job’s network environment. Given this important building block as the core, we implement bandwidth management functionality at both the host and network levels utilizing software-defined networking (SDN). In addition to HTCondor, Wide area network bandwidth management for GridFTP traffic is designed and implemented. Our experiments and evaluations show that Lark can effectively manage network resources simultaneously for both applications inside the cluster environment. By not resorting to heavyweight VMs, we keep startup overheads minimal compared to “regular” batch jobs. This mechanism provides the users with better predictability of their job execution and the administrators more policy flexibility in allocation of network resources.

Analysis of a Network IO Bottleneck in Big Data Environments Based on Docker Containers

Abstract We live in a world increasingly driven by data with more information about individuals, companies and governments available than ever before. Now, every business is powered by Information Technology and generating Big data. Future Business Intelligence can be extracted from the big data. NoSQL [1] and Map-Reduce [2] technologies find an efficient way to store, organize and process the big data using Virtualization and Linux Container (a.k.a. Container) [3] technologies. Provisioning containers on top of virtual machines is a better model for high resource utilization. As the more containers share the same CPU, the context switch latency for each container increases significantly. Such increase leads to a negative impact on the network IO throughput and creates a bottleneck in the big data environments. As part of this paper, we studied container networking and various factors of context switch latency. We evaluate Hadoop benchmarks [5] against the number of containers and virtual machines. We observed a bottleneck where Hadoop [4] cluster throughput is not linear with the number of nodes sharing the same CPU. This bottleneck is due to virtual network layers which adds a significant delay to Round Trip Time (RTT) of data packets. Future work of this paper can be extended to analyze the practical implications of virtual network layers and a solution to improve the performance of big data environments based on containers.

On the Efficiency of Executing Hydro-environmental Models on Cloud

Optimizing high-performance computing applications requires understanding of both the application and its parallelization approach, the system software stack and the target architecture. Traditionally, performance tuning of parallel applications involves consideration of the underlying machine architecture, including floating point performance, memory hierarchies and bandwidth, interconnect architecture, data placement – among others. The shift to the utility computing model through cloud has created tempting economies of scale across IT and domains, not leaving HPC as an exception as a candidate beneficiary. Nevertheless, the infrastructure abstraction and multi-tenancy inherent to cloud offerings poses great challenges to HPC workloads, requiring a dedicated study of applicability of cloud computing as a viable time-to-solution and efficiency platform. In this paper, we present the evaluation of a widely used hydro-environmental code, EFDC, on a cloud platform. Specifically, we evaluate the target parallel application on Linux containers managed by Docker. Unlike virtualization- based solutions that have been widely used for HPC cloud explorations, containers are more fit-for-purpose, sporting among others native execution and lightweight resource consumption. Many-core capability is provided by the OpenMP library in a hybrid configuration with MPI for cross-node data movement, and we explore the combination of these in the target setup. For the MPI part, the work flow is implemented as a data-parallel execution model, with all processing elements performing the same computation, on different sub-domains with thread-level, fine-grain parallelism provided by OpenMP. Optimizing performance requires consideration of the overheads introduced by the OpenMP paradigm such as thread initialization and synchronization. Features of the application make it an ideal test case for deployment on modern cloud architectures, including that it: 1) is legacy code written in Fortran 77, 2) has an implicit solver requiring non-local communication that poses a challenge to traditional partitioning methods, communication optimization and scaling and, 3) is a legacy code across academia, research organizations, governmental agencies, and consulting firms. These technical and practical considerations make this study a representative assessment of migrating legacy codes from traditional HPC systems to the cloud. We finally discuss challenges that stem from the containerized nature of the platform; the latter forms another novel contribution of this paper.

Strengthening MT6D Defenses with LXC-Based Honeypot Capabilities

Moving Target IPv6 Defense (MT6D) imparts radio-frequency hopping behavior to IPv6 networks by having participating nodes periodically hop onto new addresses while giving up old addresses. Our previous research efforts implemented a solution to identify and acquire these old addresses that are being discarded by MT6D hosts on a local network besides being able to monitor and visualize the incoming traffic on these addresses. This was essentially equivalent to forming a darknet out of the discarded MT6D addresses, but the solution presented in the previous research effort did not include database integration for it to scale and be extended. This paper presents a solution with a new architecture that not only extends the previous solution in terms of automation and database integration but also demonstrates the ability to deploy a honeypot on a virtual LXC (Linux Container) on-demand based on any interesting traffic pattern observed on a discarded address. The proposed architecture also allows an MT6D host to query the solution database for network activity on its relinquished addresses as a JavaScript Object Notation (JSON) object. This allows an MT6D host to identify suspicious activity on its discarded addresses and strengthen the MT6D scheme parameters accordingly. We have built a proof-of-concept for the proposed solution and analyzed the solution’s feasibility and scalability.

Borg, Omega, and Kubernetes

Though widespread interest in software containers is a relatively recent phenomenon, at Google we have been managing Linux containers at scale for more than ten years and built three different container-management systems in that time. Each system was heavily influenced by its predecessors, even though they were developed for different reasons. This article describes the lessons we’ve learned from developing and operating them.

BioShaDock: a community driven bioinformatics shared Docker-based tools registry

Linux container technologies, as represented by Docker, provide an alternative to complex and time-consuming installation processes needed for scientific software. The ease of deployment and the process isolation they enable, as well as the reproducibility they permit across environments and versions, are among the qualities that make them interesting candidates for the construction of bioinformatic infrastructures, at any scale from single workstations to high throughput computing architectures. The Docker Hub is a public registry which can be used to distribute bioinformatic software as Docker images. However, its lack of curation and its genericity make it difficult for a bioinformatics user to find the most appropriate images needed. BioShaDock is a bioinformatics-focused Docker registry, which provides a local and fully controlled environment to build and publish bioinformatic software as portable Docker images. It provides a number of improvements over the base Docker registry on authentication and permissions management, that enable its integration in existing bioinformatic infrastructures such as computing platforms. The metadata associated with the registered images are domain-centric, including for instance concepts defined in the EDAM ontology, a shared and structured vocabulary of commonly used terms in bioinformatics. The registry also includes user defined tags to facilitate its discovery, as well as a link to the tool description in the ELIXIR registry if it already exists. If it does not, the BioShaDock registry will synchronize with the registry to create a new description in the Elixir registry, based on the BioShaDock entry metadata. This link will help users get more information on the tool such as its EDAM operations, input and output types. This allows integration with the ELIXIR Tools and Data Services Registry, thus providing the appropriate visibility of such images to the bioinformatics community.

BioShaDock: a community driven bioinformatics shared Docker-based tools registry.

Linux container technologies, as represented by Docker, provide an alternative tocomplex and time-consuming installation processes needed for scientific software. The ease of deployment and the process isolation they enable, as well as the reproducibility they permit across environments and versions, are among the qualities that make them interesting candidates for the construction of bioinformatic infrastructures, at any scale from single workstations to high throughput computing architectures. The Docker Hub is a public registry which can be used to distribute bioinformatic software as Docker images. However, its lack of curation and its genericity make it difficult for a bioinformatics user to find the most appropriate images needed. BioShaDock is a bioinformatics-focused Docker registry, which provides a local and fully controlled environment to build and publish bioinformatic software as portable Docker images. It provides a number of improvements over the base Docker registry on authentication and permissions management, that enable its integration in existing bioinformatic infrastructures such as computing platforms. The metadata associated with the registered images are domain-centric, including for instance concepts defined in the EDAM ontology, a shared and structured vocabulary of commonly used terms in bioinformatics. The registry also includes user defined tags to facilitate its discovery, as well as a link to the tool description in the ELIXIR registry if it already exists. If it does not, the BioShaDock registry will synchronize with the registry to create a new description in the Elixir registry, based on the BioShaDock entry metadata. This link will help users get more information on the tool such as its EDAM operations, input and output types. This allows integration with the ELIXIR Tools and Data Services Registry, thus providing the appropriate visibility of such images to the bioinformatics community.

Evaluation of containers as a virtualisation alternative for HEP workloads

In this paper the emerging technology of Linux containers is examined and evaluated for use in the High Energy Physics (HEP) community. Key technologies required to enable containerisation will be discussed along with emerging technologies used to manage container images. An evaluation of the requirements for containers within HEP will be made and benchmarking will be carried out to asses performance over a range of HEP workflows. The use of containers will be placed in a broader context and recommendations on future work will be given.

A simulation as a service cloud middleware

Many seemingly simple questions that individual users face in their daily lives may actually require substantial number of computing resources to identify the right answers. For example, a user may want to determine the right thermostat settings for different rooms of a house based on a tolerance range such that the energy consumption and costs can be maximally reduced while still offering comfortable temperatures in the house. Such answers can be determined through simulations. However, some simulation models as in this example are stochastic, which require the execution of a large number of simulation tasks and aggregation of results to ascertain if the outcomes lie within specified confidence intervals. Some other simulation models, such as the study of traffic conditions using simulations may need multiple instances to be executed for a number of different parameters. Cloud computing has opened up new avenues for individuals and organizations with limited resources to obtain answers to problems that hitherto required expensive and computationally-intensive resources. This paper presents SIMaaS, which is a cloud-based Simulation-as-a-Service to address these challenges. We demonstrate how lightweight solutions using Linux containers (e.g., Docker) are better suited to support such services instead of heavyweight hypervisor-based solutions, which are shown to incur substantial overhead in provisioning virtual machines on-demand. Empirical results validating our claims are presented in the context of two case studies.

Sharing and reproducing database applications

Sharing and repeating scientific applications is crucial for verifying claims, reproducing experimental results (e.g., to repeat a computational experiment described in a publication), and promoting reuse of complex applications. The predominant methods of sharing and making applications repeatable are building a companion web site and/or provisioning a virtual machine image (VMI). Recently, application virtualization (AV), has emerged as a light-weight alternative for sharing and efficient repeatability. AV approaches such as Linux Containers create a chroot-like environment [4], while approaches such as CDE [1] trace system calls during application execution to copy all binaries, data, and software dependencies into a self-contained package.