Linux Containers Research Articles

Envyr: Instant Execution with Smart Inference

This paper introduces a novel framework that eliminates the often cumbersome "build and install" step when running software. Our framework packages a collection of techniques to automatically infer and generate sandboxes, specifically Linux containers, for running applications directly from their source code. This approach significantly simplifies software execution, improving developer productivity and promoting reuse. We demonstrate the framework’s effectiveness through real-world examples and use cases implemented in Python, NodeJS, and Bash.

A Comparative Analysis of High Availability for Linux Container Infrastructures

In the current era of prevailing information technology, the requirement for high availability and reliability of various types of services is critical. This paper focusses on the comparison and analysis of different high-availability solutions for Linux container environments. The objective was to identify the strengths and weaknesses of each solution and to determine the optimal container approach for common use cases. Through a series of structured experiments, basic performance metrics were collected, including average service recovery time, average transfer rate, and total number of failed calls. The container platforms tested included Docker, Kubernetes, and Proxmox. On the basis of a comprehensive evaluation, it can be concluded that Docker with Docker Swarm is generally the most effective high-availability solution for commonly used Linux containers. Nevertheless, there are specific scenarios in which Proxmox stands out, for example, when fast data transfer is a priority or when load balancing is not a critical requirement.

Automating Deployment and Scaling of ERP Applications Using Linux Containers and Kubernetes

Enterprise Resource Planning (ERP) systems are crucial for the efficient management of business processes across various sectors. However, deploying and scaling these systems traditionally involve complex procedures and significant resource allocation. This paper explores the automation of ERP application deployment and scaling using Linux containers and Kubernetes, highlighting how these technologies simplify and enhance system management. By leveraging containerization and orchestration platforms, organizations can achieve greater agility, scalability, and resource efficiency. This study provides an overview of Linux containers and Kubernetes, detailing their architecture, advantages, and application in ERP systems. It includes a comprehensive methodology for automating deployment and scaling, supported by real-world case studies and data analysis. The results demonstrate significant improvements in deployment time, scalability, and cost savings. This paper concludes by discussing the implications of these technologies in the ERP domain, highlighting future research opportunities to address current limitations.

Don’t Look UB: Exposing Sanitizer-Eliding Compiler Optimizations

Sanitizers are widely used compiler features that detect undefined behavior and resulting vulnerabilities by injecting runtime checks into programs. For better performance, sanitizers are often used in conjunction with optimization passes. But doing so combines two compiler features with conflicting objectives. While sanitizers want to expose undefined behavior, optimizers often exploit these same properties for performance. In this paper, we show that this clash can have serious consequences: optimizations can remove sanitizer failures, thereby hiding the presence of bugs or even introducing new ones. We present LookUB, a differential-testing based framework for finding optimizer transformations that elide sanitizer failures. We used our method to find 17 such sanitizer-eliding optimizations in Clang. Next, we used static analysis and fuzzing to search for bugs in open-source projects that were previously hidden due to sanitizer-eliding optimizations. This led us to discover 20 new bugs in Linux Containers, libmpeg2, NTFS-3G, and WINE. Finally, we present an effective mitigation strategy based on a customization of the Clang optimizer with an overhead increase of 4%.

Automated Discovery of Container Executables

Linux container technologies such as Docker and Singularity offer encapsulated environments for easy execution of software. In high performance computing, this is especially important for evolving and complex software stacks with conflicting dependencies that must co-exist. Singularity Registry HPC (“shpc”) was created as an effort to install containers in this environment as modules, seamlessly allowing for typically hidden executables inside containers to be presented to the user as commands, and as such significantly simplifying the user experience. A remaining challenge, however, is deriving the list of important executables in the container. In this work, we present a new modular methodology that allows for discovering new containers in large community sets, deriving container entries with relevant executables therein, and fully automating both recipe generation and updates over time. As an exemplar outcome, we have employed this methodology to add to the Registry over 8,000 containers from the BioContainers community that can be maintained and updated by the software automation. All software is publicly available on the GitHub platform, and can be beneficial to container registries and infrastructure providers for automatically generating container modules, thus lowering the usage entry barrier and improving user experience.

SchedGuard++: Protecting against Schedule Leaks Using Linux Containers on Multi-Core Processors

Timing correctness is crucial in a multi-criticality real-time system, such as an autonomous driving system. It has been recently shown that these systems can be vulnerable to timing inference attacks, mainly due to their predictable behavioral patterns. Existing solutions like schedule randomization cannot protect against such attacks, often limited by the system’s real-time nature. This article presents “ SchedGuard++ ”: a temporal protection framework for Linux-based real-time systems that protects against posterior schedule-based attacks by preventing untrusted tasks from executing during specific time intervals. SchedGuard++ supports multi-core platforms and is implemented using Linux containers and a customized Linux kernel real-time scheduler. We provide schedulability analysis assuming the Logical Execution Time (LET) paradigm, which enforces I/O predictability. The proposed response time analysis takes into account the interference from trusted and untrusted tasks and the impact of the protection mechanism. We demonstrate the effectiveness of our system using a realistic radio-controlled rover platform. Not only is “ SchedGuard++ ” able to protect against the posterior schedule-based attacks, but it also ensures that the real-time tasks/containers meet their temporal requirements.

Nanopore Direct RNA Sequencing Data Processing and Analysis Using MasterOfPores.

This chapter describes MasterOfPores v.2 (MoP2), an open-source suite of pipelines for processing and analyzing direct RNA Oxford Nanopore sequencing data. The MoP2 relies on the Nextflow DSL2 framework and Linux containers, thus enabling reproducible data analysis in transcriptomic and epitranscriptomic studies. We introduce the key concepts of MoP2 and provide a step-by-step fully reproducible and complete example of how to use the workflow for the analysis of S. cerevisiae total RNA samples sequenced using MinION flowcells. The workflow starts with the pre-processing of raw FAST5 files, which includes basecalling, read quality control, demultiplexing, filtering, mapping, estimation of per-gene/transcript abundances, and transcriptome assembly, with support of the GPU computing for the basecalling and read demultiplexing steps. The secondary analyses of the workflow focus on the estimation of RNA poly(A) tail lengths and the identification of RNA modifications. The MoP2 code is available at https://github.com/biocorecrg/MOP2 and is distributed under the MIT license.

CWL-PLAS: Task Workflows Assisted by Data Science Cloud Platforms

The Common Workflow Language (CWL) is a platform-independent description language for the representation of data science workflows consisting of a set of tasks that interact with each other to perform scientific analysis. The tasks can be packaged as Linux containers. On the one hand, using containers ensures the reproducibility and portability of workflows. Still, on the other hand, it limits each task to exploiting, at most, the resources of the host where its container runs. In this paper, we propose CWL-PLAS, an extension of CWL that allows a task to instantiate and temporarily use a supporting cloud platform for parallel computing, which is specialized for the task’s activity. In this way, tasks can leverage the resources of multiple hosts in parallel, reducing the duration of the workflow. We implemented an open-source workflow manager that supports CWL-PLAS workflows and exploits a Kubernetes back-end. We used this workflow manager to evaluate the performance of CWL-PLAS in a couple of machine learning workflows.

<i>Verikube</i>: Automatic and Efficient Verification for Container Network Policies

Recently, Linux Container has been the de-facto standard for a cloud system, enabling cloud providers to create a virtual environment in a much more scaled manner. However, configuring container networks remains immature and requires automatic verification for efficient cloud management. We propose Verikube, which utilizes a novel graph structure representing policies to reduce memory consumption and accelerate verification. Moreover, unlike existing works, Verikube is compatible with the complex semantics of Cilium Policy which a cloud adopts from its advantage of performance. Our evaluation results show that Verikube performs at least seven times better for memory efficiency, at least 1.5 times faster for data structure management, and 20K times better for verification.

Mechanisms and techniques to enhance the security of big data analytic framework with MongoDB and Linux Containers

The frequency and scale of unauthorized access cases and misuses of data access privileges are a growing concern of many organizations. The protection of confidential data, such as social security numbers, financial information, etc., of the customers and/or employees is among the key responsibilities of any organization, and damage to such sensitive data can easily pose a threat to the future of a business and the security of the customers. Therefore, this paper proposes and implements some security mechanisms and techniques, such as secure authentication, secure authorization, and encryption, to assure the overall security of a big data analytic framework with MongoDB free community edition. This paper presents the fourth phase of our continuous research where in the first phase we proposed a data analytic framework with MongoDB and Linux Containers (LXCs) with basic security requirements. Next, in the second phase we proposed a vulnerability analysis testbed to find vulnerabilities associated with the system. Finally, in the third phase we discussed in detail root causes and some prevention techniques of vulnerabilities found in the system. In addition, this paper introduces a new security mechanism for privacy preserving data handling with MongoDB to ensure the privacy of the data before being processed. Our results show, with our initial model of the analytic framework, how well our newly introduced security mechanisms work and how these security mechanisms and techniques can be used to assure the confidentiality, integrity, and availability (CIA) of any data science project conducted on our proposed analytic framework. In addition, these security mechanisms and techniques help us to strengthen the current system against zero-day attacks where attacks on vulnerabilities that have not been patched or made public yet. Therefore, our vulnerability analysis testbed which is proposed in the second phase of this research will not be able to finds vulnerabilities related to zero-day attacks.

Reducing response latency of composite functions-as-a-service through scheduling

In Function-as-a-Service (FaaS) clouds, customers deploy to cloud individual functions, in contrast to complete virtual machines (IaaS) or Linux containers (PaaS). FaaS offerings are available in the largest public clouds (Amazon Lambda, Google Cloud Functions, Azure Serverless); there are also popular open-source implementations (Apache OpenWhisk) with commercial offerings (Adobe I/O Runtime, IBM Cloud Functions). A recent addition to FaaS is the ability to compose functions: a function may call another functions, which, in turn, may call yet another function — forming a directed acyclic graph (DAG) of invocations. From the perspective of the infrastructure, a composed function is less opaque than a virtual machine or a container. We show that this additional information about the internal structure of the function enables the infrastructure provider to reduce the response latency. In particular, knowing the successors of a function in a DAG, the infrastructure can schedule these future invocations along with necessary preparation of environments.We model resource management in FaaS as a scheduling problem combining (1) sequencing of invocations; (2) deploying execution environments on machines; and (3) allocating invocations to deployed environments. For each aspect, we propose heuristics that employ FaaS-specific features. We explore their performance by simulation on a range of synthetic workloads and on workloads inspired by trace from existing system. Our results show that if the setup times are long compared to invocation times, algorithms that use information about the composition of functions consistently outperform greedy, myopic algorithms, leading to significant decrease in response latency.

360^{circ } real-time and power-efficient 3D DAMOT for autonomous driving applications

Autonomous Driving (AD) promises an efficient, comfortable and safe driving experience. Nevertheless, fatalities involving vehicles equipped with Automated Driving Systems (ADSs) are on the rise, especially those related to the perception module of the vehicle. This paper presents a real-time and power-efficient 3D Multi-Object Detection and Tracking (DAMOT) method proposed for Intelligent Vehicles (IV) applications, allowing the vehicle to track 360^{circ } surrounding objects as a preliminary stage to perform trajectory forecasting to prevent collisions and anticipate the ego-vehicle to future traffic scenarios. First, we present our DAMOT pipeline based on Fast Encoders for object detection and a combination of a 3D Kalman Filter and Hungarian Algorithm, used for state estimation and data association respectively. We extend our previous work ellaborating a preliminary version of sensor fusion based DAMOT, merging the extracted features by a Convolutional Neural Network (CNN) using camera information for long-term re-identification and obstacles retrieved by the 3D object detector. Both pipelines exploit the concepts of lightweight Linux containers using the Docker approach to provide the system with isolation, flexibility and portability, and standard communication in robotics using the Robot Operating System (ROS). Second, both pipelines are validated using the recently proposed KITTI-3DMOT evaluation tool that demonstrates the full strength of 3D localization and tracking of a MOT system. Finally, the most efficient architecture is validated in some interesting traffic scenarios implemented in the CARLA (Car Learning to Act) open-source driving simulator and in our real-world autonomous electric car using the NVIDIA AGX Xavier, an AI embedded system for autonomous machines, studying its performance in a controlled but realistic urban environment with real-time execution (results).

Ant Colony Optimization-based Light Weight Container (ACO-LWC) Algorithm for Efficient Load Balancing

Container technology is the latest lightweight virtualization technology which is an alternate solution for virtual machines. Docker is the most popular container technology for creating and managing Linux containers. Containers appear to be the most suitable medium for use in dynamic development, packaging, shipping and many other information technology environments. The portability of the software through the movement of containers is appreciated by businesses and IT professionals. In the docker container, one or more processes may run simultaneously. The main objective of this work is to propose a new algorithm called Ant Colony Optimization-based Light Weight Container (ACO-LWC) load balancing scheduling algorithm for scheduling various process requests. This algorithm is designed such that it shows best performance in terms of load balancing. The proposed algorithm is validated by comparison with two existing load balancing scheduling algorithms namely, least connection algorithm and round robin algorithm. The proposed algorithm is validated using metrics like response time (ms), mean square error (MSE), node load, largest Transactions Per Second (TPS) of cluster (fetches/sec), average response time for each request (ms) and run time (s). Quantitative analysis show that the proposed ACO-LWC scheme achieves best performance in terms of all the metrics compared to the existing algorithms. In particular, the response time for least connection, round robin and the proposed ACO-LWC algorithm are 58, 60 and 48 ms respectively when 95% requests are finished. Similarly, the error for scheduling 120 requests using least connection, round robin and the proposed ACO-LWC algorithm are 0.15, 0.11 and 0.06 respectively.

ISOBlue Avena: A Framework for Agricultural Edge Computing and Data Sovereignty

While telematics and edge computing in agriculture have evolved significantly over the past 30 years, the same fundamental problems of vendor lock-in and incompatibility remain. In this article, we introduce Avena, our new open source framework for building an ecosystem around agricultural edge computing that embraces data sovereignty while still encouraging participation from the community’s stakeholders. Built on Linux containers and the technology of modern cloud systems, Avena brings a new way of thinking to agricultural machines.

ShareOn: Shared Resource Dynamic Container Migration Framework for Real-Time Support in Mobile Edge Clouds

Mobile Edge Cloud (MEC) technology is envisioned to play a key role in next generation mobile networks by supporting low-latency applications using geographically distributed local cloud clusters. However, MEC faces challenges of resource assignment and load balancing to support user mobility and latency-sensitive applications. Virtualized resource reallocation techniques including dynamic service migration are evolving to achieve load balance, fault tolerance and system maintenance objectives for resource constrained edge nodes. In this work, a compute and network-aware lightweight resource sharing framework with dynamic container migration, ShareOn, is proposed. The migration framework is validated using a set of heterogeneous edge cloud nodes distributed in San Francisco city, serving mobile taxicab users across that region. The end-to-end system is implemented using a container hypervisor called LXD (Linux Container Hypervisor) executing a real-time application to detect license number plates in automobiles. The system is evaluated based on key metrics associated with application quality-of-service (QoS) and network efficiency such as the average system response time and the migration cost for different combinations of load, compute resources, inter-edge cloud bandwidth, network and user latency. A detailed migration cost analysis enables evaluation of migration strategies to improve ShareOn’s performance in comparison to alternative migration techniques, achieving a gain of 15–22% in system response time for highly loaded edge cloud nodes.

Advances in parallel and distributed computing and its applications

Advances in parallel and distributed computing and its applications

PEPATAC: an optimized pipeline for ATAC-seq data analysis with serial alignments.

As chromatin accessibility data from ATAC-seq experiments continues to expand, there is continuing need for standardized analysis pipelines. Here, we present PEPATAC, an ATAC-seq pipeline that is easily applied to ATAC-seq projects of any size, from one-off experiments to large-scale sequencing projects. PEPATAC leverages unique features of ATAC-seq data to optimize for speed and accuracy, and it provides several unique analytical approaches. Output includes convenient quality control plots, summary statistics, and a variety of generally useful data formats to set the groundwork for subsequent project-specific data analysis. Downstream analysis is simplified by a standard definition format, modularity of components, and metadata APIs in R and Python. It is restartable, fault-tolerant, and can be run on local hardware, using any cluster resource manager, or in provided Linux containers. We also demonstrate the advantage of aligning to the mitochondrial genome serially, which improves the accuracy of alignment statistics and quality control metrics. PEPATAC is a robust and portable first step for any ATAC-seq project. BSD2-licensed code and documentation are available at https://pepatac.databio.org.

Edge Container for Speech Recognition

Containerization has been mainly used in pure software solutions, but it is gradually finding its way into the industrial systems. This paper introduces the edge container with artificial intelligence for speech recognition, which performs the voice control function of the actuator as a part of the Human Machine Interface (HMI). This work proposes a procedure for creating voice-controlled applications with modern hardware and software resources. The created architecture integrates well-known digital technologies such as containerization, cloud, edge computing and a commercial voice processing tool. This methodology and architecture enable the actual speech recognition and the voice control on the edge device in the local network, rather than in the cloud, like the majority of recent solutions. The Linux containers are designed to run without any additional configuration and setup by the end user. A simple adaptation of voice commands via configuration file may be considered as an additional contribution of the work. The architecture was verified by experiments with running containers on different devices, such as PC, Tinker Board 2, Raspberry Pi 3 and 4. The proposed solution and the practical experiment show how a voice-controlled system can be created, easily managed and distributed to many devices around the world in a few seconds. All this can be achieved by simple downloading and running two types of ready-made containers without any complex installations. The result of this work is a proven stable (network-independent) solution with data protection and low latency.

Even Lower Latency in IIoT: Evaluation of QUIC in Industrial IoT Scenarios.

In this paper we analyze the performance of QUIC as a transport alternative for Internet of Things (IoT) services based on the Message Queuing Telemetry Protocol (MQTT). QUIC is a novel protocol promoted by Google, and was originally conceived to tackle the limitations of the traditional Transmission Control Protocol (TCP), specifically aiming at the reduction of the latency caused by connection establishment. QUIC use in IoT environments is not widespread, and it is therefore interesting to characterize its performance when in over such scenarios. We used an emulation-based platform, where we integrated QUIC and MQTT (using GO-based implementations) and compared their combined performance with the that exhibited by the traditional TCP/TLS approach. We used Linux containers as end devices, and the ns-3 simulator to emulate different network technologies, such as WiFi, cellular, and satellite, and varying conditions. The results evince that QUIC is indeed an appropriate protocol to guarantee robust, secure, and low latency communications over IoT scenarios.

Implementation of the Moodle e-learning platform from server selection to configuration

Through this article which concerns the implementation of the Moodle e-learning platform in a server, we will first present an example of a Web server architecture, then we propose the adopted architecture which is based on Linux containers. Afterwards, we propose a description of all the necessary tools chosen for the implementation of the platform in a Web server. Then, we propose through figures the installation of the different technological tools and the Moodle platform. Finally, we propose the configuration of our Moodle platform according to our needs.