Container Runtime Research Articles

DMSCTS: Dynamic measurement scheme for the containers-hybrid-deployment based on trusted subsystem

Hybrid deployment of containers with different kernel types offers a novel solution for cloud service providers. While extensive research has been conducted on shared kernel containers, the security risks associated with diverse kernel types in hybrid deployment scenarios present more complex challenges. Establishing trusted relationships from hardware to containers for hybrid deployment has become a primary concern. Additional challenges include the lack of measurement and communication methods for independent kernel containers and insufficient dynamic measurement capabilities for containers. To address these issues, we propose a novel approach of achieving secure hybrid deployment of containers through the provision of trusted assurance in three layers: container infrastructure, container application environment, and container runtime. We propose the corresponding measurement schemes for each trust layer. Through functional verification and performance evaluation, we demonstrate that our architecture exhibits improved feasibility and effectiveness.

PASS4SWAT: Orchestration of containerized SWAT for facilitating computational reproducibility of model calibration and uncertainty analysis

When performing modeling routines with iterative simulations, a research gap exists in developing the parallelizing frameworks that are adaptable to the increasing complexities of the hydrologic models. This study addresses this gap by proposing a parallel simulation stack for SWAT (PASS4SWAT) that leverages the power of cloud-native infrastructure. Cloud-native applications are designed to be scalable, agile, and resilient, making them ideal for computationally intensive tasks such as hydrologic modeling. To achieve this goal, PASS4SWAT was created by integrating a message broker, a container runtime and orchestration tool, and certain components from SWAT-CUP. We evaluated PASS4SWAT using the Jinjiang watershed model and a synthetic model with high input/output demands to demonstrate its effectiveness. The results show that PASS4SWAT can consistently replicate the results obtained with SWAT-CUP and significantly reduce the runtime, achieving 91.5% and 93.2% reductions in single-node and multinode Kubernetes clusters, respectively. Therefore, in this paper, we conclude that PASS4SWAT can effectively address the high computational demands of the SWAT model by scaling out parallel tasks on a cluster, and can adapt flexibly to diverse environments, including single-node clusters, multinode clusters, and potential cloud platforms.

Digital twins, the journey of an operational weather prediction system into the heart of Destination Earth

Moving a world leading numerical weather prediction system of the European Center for Medium-Range Weather Forecasts (ECMWF), that runs on a dedicated, bespoke, high performance computing cluster and supporting infrastructure, into the heart of the digital twin for climate change adaptation and extreme weather events, developed in the Destination Earth (DestinE) initiative of the European Commission, has been a challenging and exciting journey. In this paper we describe this journey with a focus on those aspects required to leverage the pre-exascale EuroHPC systems that have been made available to DestinE to run its computational representation [1]. EuroHPC systems can be effectively used for DestinE and are in fact key assets to deliver the computational power required for Earth system digital twins at global km-scale resolution. Each of these systems is operated by a national hosting entity that implements its own procedures, e.g. for identity and access management, specific system configuration like schedulers, filesystems, software management systems, and specific, sometimes vendor associated, toolchains, tooling, and container runtimes. In particular, the different scheduling policies encountered required us to adapt our workflows for each site. We found that having dedicated resources available, which was trialed in a period from 16th February to 14th April on LUMI, allowed to achieve high occupation rates, with 92% on the reserved GPU allocation and greater than 97% efficiency on the CPU reservation. Also, a stronger focus on federation of these systems, with a focus not only on federation of identities and accounts, but also in the areas of data ownership/transfer, observability, services and service accounts, maintenance coordination and performance portability could benefit DestinE. In general, it would be beneficial to make it easier to transfer a workload, or a digital twin system, from one EuroHPC site to the next and run and maintain them across several sites concurrently.

Apptainer Without Setuid

Apptainer (formerly known as Singularity) since its beginning implemented many of its container features with the assistance of a setuidroot program. It still supports that mode, but as of version 1.1.0 it no longer uses setuid by default. This is feasible because it now can mount squashfs filesystems, ext3 filesystems, and overlay filesystems using unprivileged user namespaces and FUSE. It also now enables unprivileged users to build containers, even without requiring system administrators to configure /etc/subuid and /etc/subgid unlike other “rootless” container systems. As a result, all the unprivileged functions can be used nested inside of another container, even if the container runtime prevents any elevated privileges. As of version 1.2.0 Apptainer also supports completely unprivileged encryption of Singularity Image Format (SIF) container files. Performance with a particularly challenging HEP benchmark using the FUSE-based mounts both with and without encryption is essentially identical to the previous methods that required elevated privileges to use the Linux kernel-based counterparts.

Exploring the Performance of Container Runtimes within Kubernetes Clusters

The advent of cloud computing, with its Pay-As-You-Go model, has significantly simplified IT maintenance and revolutionized the industry. In the era of Microservices, containerized deployment and Kubernetes orchestration have permeated almost every working domain, drastically reducing the time to market for software releases. Kubernetes utilizes container runtimes to manage Containers, with the Container Runtime Interface (CRI) serving as a communication medium with low-level container runtimes such as runc and kata container. With the deprecation of Dockershim, developers are left to choose between CRI-O and Containerd, two CRI implementations. This study configures a Kubernetes cluster with both Containerd and CRI-O separately and analyzes performance parameters such as throughput, response time, CPU, memory, and network utilization. Additionally, we examine the impact of using runc and kata container runtimes together within the cluster. The study, conducted using a performance script created by JMeter, reveals that different container runtimes cater to distinct business use-cases and can complement each other when used together in a cluster environment. High compute applications are best run using runc, while high-security requirements are fulfilled by kata. The study provides a comprehensive performance comparison between Containerd and CRI-O, shedding light on the depth and versatility of container runtimes.

Impact of Secure Container Runtimes on File I/O Performance in Edge Computing

Containers enable high performance and easy deployment due to their lightweight architecture, thus facilitating resource utilization in edge computing nodes. Secure container runtimes have attracted significant attention because of the necessity for overcoming the security vulnerabilities of containers. As the runtimes adopt an additional layer such as virtual machines and user-space kernels to enforce isolation, the container performance can be degraded. Even though previous studies presented experimental results on performance evaluations of secure container runtimes, they lack a detailed analysis of the root causes that affect the performance of the runtimes. This paper explores the architecture of three secure container runtimes in detail: Kata containers, gVisor, and Firecracker. We focus on file I/O, which is one of the key aspects of container performance. In addition, we present the results of the user- and kernel-level profiling and reveal the major factors that impact the file I/O performance of the runtimes. As a result, we observe three key findings: (1) Firecracker shows the highest file I/O performance as it allows for utilizing the page cache inside VMs, and (2) Kata containers offer the lowest file I/O performance by consuming the largest amount of CPU resources. Also, we observe that gVisor scales well as the block size increases because the file I/O requests are mainly handled by the host OS similar to native applications.

Malware detection for container runtime based on virtual machine introspection

Malware detection for container runtime based on virtual machine introspection

Cloud Container Service Orchestrated with Kubernetes: a State-of-the-Art Technology Review and Application Proposal

This paper discusses specific technology related issues with the latest release of the container orchestrator Kubernetes and combining it with certain container runtimes. We provide a technology-focused state-of-the-art review, addressing identified research gaps in this area. Following the depreciation of Dockershim in the latest K8s release 1.26, we’ve tested the migration to another container management tool, that enables using the latest functions or the Containerd runtime (Nerdctl). At the same time this migration also allows running containers in a secure and resource effective way and managing them together with container images. The application we study is orchestrating a container cluster-based cloud NIDS service by using the latest Snort 3 system. We’ve also outlined directions for future research.

PIMS: An Efficient Process Integrity Monitoring System Based on Blockchain and Trusted Computing in Cloud-Native Context

With the advantages of lightweight and high resource utilization, cloud-native technology with containers as the core is gradually becoming the mainstream technical architecture for information infrastructure. However, malware attacks such as Doki and Symbiote threaten the container runtime’s security. Malware initiates various types of runtime anomalies based on process form (e.g., modifying the process of a container, and opening the external ports). Fortunately, dynamic monitoring mechanisms have proven to be a feasible solution for verifying the trusted state of containers at runtime. Nevertheless, the current routine dynamic monitoring mechanisms for baseline data protection are still based on strong security assumptions. As a result, the existing dynamic monitoring mechanism is still not practical enough. To ensure the trustworthiness of the baseline value data and, simultaneously, to achieve the integrity verification of the monitored process, we combine blockchain and trusted computing to propose a process integrity monitoring system named IPMS. Firstly, the hardware TPM 2.0 module is applied to construct a trusted security foundation for the integrity of the process code segment due to its tamper-proof feature. Then, design a new format for storing measurement logs, easily distinguishing files with the same name in different containers from log information. Meanwhile, the baseline value data is stored on the blockchain to avoid malicious damage. Finally, trusted computing technology is used to perform fine-grained integrity measurement and remote attestation of processes in a container, detect abnormal containers in time and control them. We have implemented a prototype system and performed extensive simulation experiments to test and analyze the functionality and performance of the PIMS. Experimental results show that PIMS can accurately and efficiently detect tampered processes with only 3.57% performance loss to the container.

HACM: High Availability Control Method in Container-Based Microservice Applications Over Multiple Clusters

Emerging cloud-native technologies, such as container runtime and container orchestrator, offer unprecedented agility in developing and running applications, especially when combined with microservice-style architecture. Several commercial Samsung Network products such as Samsung Element Management System (S-EMS), 5G Radio Access Network (RAN) & Core network elements are being redesigned to fit the microservice paradigm. The cloud environment allows enterprises to scale their applications on-demand with minimum cost; however, it is often difficult to use containers without sacrificing the many benefits offered by container technology. S-EMS manages 5G RAN & Core network elements (NEs) deployed nationwide, and systematically stores a huge volume of stateful data per second. Containers are characterized to have an ephemeral state, hence ‘stateful-ness’ aspect of S-EMS makes management more complex. The existing system in a container-based application does not support geo-redundancy where services/data are stateful/state dependent. In this paper, different challenges around geo-redundancy between different independent Kubernetes set up with active and standby modes between the sites where state-dependent data is stored in each site are described. To overcome these challenges, we propose the High Availability Control Method (HACM) - which enables the Kubernetes cluster to be active and standby, where state-dependent data and context-based operations are intrinsically supported by the underlying S-EMS container application. Our approach has been designed to maintain the geo-redundancy philosophy of cloud-native by associating the status of each site using high availability (HA), switching over services based on the health of applications, deciding state when there are conflicts in site state, and the option to auto fallback based on user preference and services are transferred between sites without user intervention with optimized storage that ensures consistency, persistence, reliability, and availability. Through evaluation, we show that HACM with S-EMS can facilitate geo-redundancy HA, while not posing a significant burden on the Cloud.

Container Performance and Vulnerability Management for Container Security Using Docker Engine

Containers have evolved to support microservice architecture as a low-cost alternative to virtual machines. Containers are increasingly prevalent in the virtualization landscape because of better working; containers can bear considerably less overhead than the conventional hypervisor-based component virtual machines. However, containers directly communicate with the host kernel, and attackers can co-locate containers in the host system quicker than virtual machines. This causes significant security issues in container technology. The security hardening system is currently targeted at implementing universal access management regulations that make it difficult to assess the required procedure for accessing containers. Security mechanisms include an explicit awareness of the purpose and actions of the container and entail manual interaction and configuration. A user-friendly container protection scheme implemented an access policy to comply with its anticipated and legitimate application performance. In this study, container technology constraints have been overcome by proposing a unique Docker-sec mechanism. Docker-sec uses four mechanisms; the original collection has been improved during container runtime by additional rules that constrain the capacity of the container, further representing the applications in practice, file system, processes, network isolation, and vulnerability scanning of Docker images over different workload. Different vulnerabilities have been scanned with a CVE severity level. Results showed that inter-container communication with the system is more secure containers from zero vulnerabilities with an overhead of 3.45%.

Container Technologies for ARM Architecture: A Comprehensive Survey of the State-of-the-Art

Container technology is becoming increasingly popular as an alternative to traditional virtual machines because it provides a faster, lighter, and more portable runtime environment for the applications. A container bundles the application and its binary code, libraries, and configuration files together while sharing the host operating system image. Accordingly, containers efficiently share resources and operate small micro-services, software programs, and even more extensive applications with less overhead than virtual machines. There are many container technologies available with Docker being the most popular and many technologies support multiple architectures, including the ARM architecture. Due to its energy efficiency and high-performance, which are crucial parameters in containerization, ARM architecture is becoming prevalent in container technologies. In this paper, we explore various container technologies that support ARM architecture and investigate the pros and cons of each technology. Moreover, we provide a comparative analysis of both container orchestrators and container runtimes that are most prominent competitors of Docker. We also consider security of container technologies with particular focus on the image scanning tools that supports ARM architecture. Our survey reveals that ARM technology is gaining popularity in containerization and almost all recent technologies support ARM architecture.

Orchestrated sandboxed containers, unikernels, and virtual machines for isolation‐enhanced multitenant workloads and serverless computing in cloud

AbstractContainers emerge as the prevalent virtualization technology in cloud computing. Containers are more light‐weight and agile compared to traditional virtual machines (VMs), since they provide virtualization at the operating system level. There are specific factors driving the container adoption in cloud, however, the main disadvantage of container‐based virtualization technologies is poor isolation. To address isolation and security‐related issues, new container runtimes appeared. In this study we present and evaluate the most common security‐oriented runtimes, Kata, gVisor and Nabla, running with Docker, Containerd, and CRI‐O. We deploy containers for all the aforementioned container solutions, as well as the default Kubernetes runtime runc, on a Kubernetes cluster and additionally on a Docker node. Moreover, in a similar way to containers, we deploy and evaluate unikernels and security‐oriented lightweight Linux‐based VMs running on Kubernetes cluster. To evaluate container runtimes, we take under consideration Firecracker microVM too. To automate the deployment of high isolated containers and VMs on Kubernetes clusters, we have developed our own tool. Finally, we recognize the increasing interest on Function‐as‐a‐Service and other serverless architectures. In the same direction with these emerging cloud computing services, we have extended the Kubeless serverless framework to support security‐oriented container runtimes.

Container orchestration on HPC systems through Kubernetes

Containerisation demonstrates its efficiency in application deployment in Cloud Computing. Containers can encapsulate complex programs with their dependencies in isolated environments making applications more portable, hence are being adopted in High Performance Computing (HPC) clusters. Singularity, initially designed for HPC systems, has become their de facto standard container runtime. Nevertheless, conventional HPC workload managers lack micro-service support and deeply-integrated container management, as opposed to container orchestrators. We introduce a Torque-Operator which serves as a bridge between HPC workload manager (TORQUE) and container orchestrator (Kubernetes). We propose a hybrid architecture that integrates HPC and Cloud clusters seamlessly with little interference to HPC systems where container orchestration is performed on two levels.

Security Assessment Technique of a Container Runtime Using System Call Weights

Security Assessment Technique of a Container Runtime Using System Call Weights

CernVM-FS powered container hub

Containers became the de-facto standard to package and distribute modern applications and their dependencies. The HEP community demonstrates an increasing interest in such technology, with scientists encapsulating their analysis workflow and code inside a container image. The analysis is first validated on a small dataset and minimal hardware resources to then run at scale on the massive computing capacity provided by the grid. The typical approach for distributing containers consists of pulling their image from a remote registry and extracting it on the node where the container runtime (e.g., Docker, Singularity) runs. This approach, however, does not easily scale to large images and thousands of nodes. CVMFS has long been used for the efficient distribution of software directory trees at a global scale. In order to extend its optimized caching and network utilization to the distribution of containers, CVMFS recently implemented a dedicated container image ingestion service together with container runtime integrations. CVMFS ingestion is based on per-file deduplication, instead of the per-layer deduplication adopted by traditional container registries. On the client-side, CVMFS implements on-demand fetching of the chunks required for the execution of the container instead of the whole image.

Assessing Container Network Interface Plugins: Functionality, Performance, and Scalability

Kubernetes, an open-source container orchestration platform, has been widely adopted by cloud service providers (CSPs) for its advantages in simplifying container deployment, scalability, and scheduling. Networking is one of the central components of Kubernetes, providing connectivity between different Pods (a group of containers) both within the same host and across hosts. To bootstrap Kubernetes networking, the Container Network Interface (CNI) provides a unified interface for the interaction between container runtimes. There are several CNI implementations, available as open-source ‘CNI plugins’. While they differ in functionality and performance, it is a challenge for a cloud provider to differentiate and choose the appropriate plugin for their environment. In this article, we compare the various open-source CNI plugins available from the community, qualitatively, and through detailed quantitative measurements. With our experimental evaluation, we analyze the overheads and bottlenecks for each CNI plugin, especially because of the interaction with the datapath/iptables as well as the host network stack. Overlay tunnel offload support in the network interface card plays a significant role in achieving the good performance of CNIs that use overlay tunnels for inter-host Pod-to-Pod communication. We also study scalability with an increasing number of Pods, as well as with HTTP workloads, and briefly evaluate Pod startup latency. Our measurement results inform the outline of an ideal CNI environment for Kubernetes.

Extending Kubernetes Clusters to Low-Resource Edge Devices Using Virtual Kubelets

In recent years, containers have gained popularity as a lightweight virtualization technology. This rise in popularity has gone hand in hand with the adoption of microservice architectures, mostly thanks to the scalable, ethereal, and isolated nature of containers. More recently, edge devices have become powerful enough to be able to run containerized microservices, while remaining flexible enough in terms of size and power to be deployed almost anywhere. This has triggered research into several container placement strategies involving edge networks, leading to concepts such as osmotic computing. While these container placement strategies are optimal in terms of workload placement, current container orchestrators are often not suitable for running on edge devices due to their high resource requirements. In this article, FLEDGE is presented as a Kubernetes-compatible container orchestrator based on Virtual Kubelets, aimed primarily at container orchestration on low-resource edge devices. Several aspects of low-resource container orchestration are examined, such as the choice of container runtime and how to realize container networking. A number of evaluations are performed to determine how FLEDGE compares to Kubernetes and K3S in terms of resource requirements, showing that it needs around 60MiB memory and 78MiB storage to run on a Raspberry Pi 3, including all dependencies, which is significantly less than both studied alternatives.

Exploring the support for high performance applications in the container runtime environment

Cloud computing is the driving power behind the current technological era. Virtualization is rightly referred to as the backbone of cloud computing. Impacts of virtualization employed in high performance computing (HPC) has been much reviewed by researchers. The overhead in the virtualization layer was one of the reasons which hindered its application in the HPC environment. Recent developments in virtualization, especially the OS container based virtualization provides a solution that employs a lightweight virtualization layer and promises lesser overhead. Containers are advantageous over virtual machines in terms of performance overhead which is a major concern in the case of both data intensive applications and compute intensive applications. Currently, several industries have adopted container technologies such as Docker. While Docker is widely used, it has certain pitfalls such as security issues. The recently introduced CoreOS Rkt container technology overcomes these shortcomings of Docker. There has not been much research on how the Rkt environment is suited for high performance applications. The differences in the stack of the Rkt containers suggest better support for high performance applications. High performance applications consist of CPU-intensive and data-intensive applications. The High Performance Linpack Library and the Graph500 are the commonly used computation intensive and data-intensive benchmark applications respectively. In this work, we explore the feasibility of this inter-operable Rkt container in high performance applications by running the HPL and Graph500 applications and compare its performance with the commonly used container technologies such as LXC and Docker containers.