Linux System Research Articles

Comparative analysis of package managers Flatpak and Snap used for open-source software distribution

This article presents the result of a research of the Flatpak and Snap package managers used to distribute open-source software on Linux systems. Both package managers are characterised by their versatility and implementation of sandboxing. As part of the research, a test application was prepared, which was built in the Flatpak and Snap formats and published in the official software repositories, where for Flatpak it is Flathub and for Snap it is the Snap Store. The prepared application was first used to test and compare the implementation of sandboxing rules. This was followed by tests of RAM usage and start-up time by the application installed in both formats. The result of the research is an analysis of the measurement results and the drawing of conclusions.

PhytoPipe: a phytosanitary pipeline for plant pathogen detection and diagnosis using RNA-seq data

BackgroundDetection of exotic plant pathogens and preventing their entry and establishment are critical for the protection of agricultural systems while securing the global trading of agricultural commodities. High-throughput sequencing (HTS) has been applied successfully for plant pathogen discovery, leading to its current application in routine pathogen detection. However, the analysis of massive amounts of HTS data has become one of the major challenges for the use of HTS more broadly as a rapid diagnostics tool. Several bioinformatics pipelines have been developed to handle HTS data with a focus on plant virus and viroid detection. However, there is a need for an integrative tool that can simultaneously detect a wider range of other plant pathogens in HTS data, such as bacteria (including phytoplasmas), fungi, and oomycetes, and this tool should also be capable of generating a comprehensive report on the phytosanitary status of the diagnosed specimen.ResultsWe have developed an open-source bioinformatics pipeline called PhytoPipe (Phytosanitary Pipeline) to provide the plant pathology diagnostician community with a user-friendly tool that integrates analysis and visualization of HTS RNA-seq data. PhytoPipe includes quality control of reads, read classification, assembly-based annotation, and reference-based mapping. The final product of the analysis is a comprehensive report for easy interpretation of not only viruses and viroids but also bacteria (including phytoplasma), fungi, and oomycetes. PhytoPipe is implemented in Snakemake workflow with Python 3 and bash scripts in a Linux environment. The source code for PhytoPipe is freely available and distributed under a BSD-3 license.ConclusionsPhytoPipe provides an integrative bioinformatics pipeline that can be used for the analysis of HTS RNA-seq data. PhytoPipe is easily installed on a Linux or Mac system and can be conveniently used with a Docker image, which includes all dependent packages and software related to analyses. It is publicly available on GitHub at https://github.com/healthyPlant/PhytoPipe and on Docker Hub at https://hub.docker.com/r/healthyplant/phytopipe.

An Experimental Assessment of Inconsistencies in Memory Forensics

Memory forensics is concerned with the acquisition and analysis of copies of volatile memory (memory dumps). Based on an empirical assessment of observable inconsistencies in 360 memory dumps of a running Linux system, we confirm a state of overwhelming inconsistency in memory forensics: almost a third of these dumps had an empty process list and was therefore obviously incomplete. Out of those dumps that were analyzable, almost every second dump showed some form of inconsistency that potentially impacts the interpretation of the dump in a forensic investigation. These results are based on a new way to estimate the level of causal consistency of a memory dump. The factors influencing these inconsistencies are less clear but in general correlate with the level of concurrency (system load and number of threads).

Kernel vs. User-Level Networking: Don't Throw Out the Stack with the Interrupts

This paper reviews the performance characteristics of network stack processing for communication-heavy server applications. Recent literature often describes kernel-bypass and user-level networking as a silver bullet to attain substantial performance improvements, but without providing a comprehensive understanding of how exactly these improvements come about. We identify and quantify the direct and indirect costs of asynchronous hardware interrupt requests (IRQ) as a major source of overhead. While IRQs and their handling have a substantial impact on the effectiveness of the processor pipeline and thereby the overall processing efficiency, their overhead is difficult to measure directly when serving demanding workloads. This paper presents an indirect methodology to assess IRQ overhead by constructing preliminary approaches to reduce the impact of IRQs. While these approaches are not suitable for general deployment, their corresponding performance observations indirectly confirm the conjecture. Based on these findings, a small modification of a vanilla Linux system is devised that improves the efficiency and performance of traditional kernel-based networking significantly, resulting in up to 45% increased throughput without compromising tail latency. In case of server applications, such as web servers or Memcached, the resulting performance is comparable to using kernel-bypass and user-level networking when using stacks with similar functionality and flexibility.

Lifex-cfd: An open-source computational fluid dynamics solver for cardiovascular applications

Computational fluid dynamics (CFD) is an important tool for the simulation of the cardiovascular function and dysfunction. Due to the complexity of the anatomy, the transitional regime of blood flow in the heart, and the strong mutual influence between the flow and the physical processes involved in the heart function, the development of accurate and efficient CFD solvers for cardiovascular flows is still a challenging task. In this paper we present life▪-cfd, an open-source CFD solver for cardiovascular simulations based on the life▪ finite element library, written in modern C++ and exploiting distributed memory parallelism. We model blood flow in both physiological and pathological conditions via the incompressible Navier-Stokes equations, accounting for moving cardiac valves, moving domains, and transition-to-turbulence regimes. In this paper, we provide an overview of the underlying mathematical formulation, numerical discretization, implementation details and examples on how to use life▪-cfd. We verify the code through rigorous convergence analyses, and we show its almost ideal parallel speedup. We demonstrate the accuracy and reliability of the numerical methods implemented through a series of idealized and patient-specific vascular and cardiac simulations, in different physiological flow regimes. The life▪-cfd source code is available under the LGPLv3 license, to ensure its accessibility and transparency to the scientific community, and to facilitate collaboration and further developments. Program summaryProgram Title:life▪-cfdCPC Library link to program files:https://doi.org/10.17632/hzsnc3jgds.1Developer's repository link:https://gitlab.com/lifex/lifex-cfdLicensing provisions:LGPLv3Programming language: C++ (standard ≥17)Supplementary material:https://doi.org/10.5281/zenodo.7852088 contains the application executable in binary form, compatible with any recent enough x86-64 Linux system, assuming that glibc version ≥ 2.28 is installed. Data and parameter files necessary to replicate the test cases described in this manuscript are also available.Nature of problem: The program allows to run computational fluid dynamics simulations of cardiovascular blood flows in physiological and pathological conditions, modeled through incompressible Navier-Stokes equations, including moving cardiac valves, moving domains (such as contracting cardiac chambers) in the arbitrary Lagrangian-Eulerian framework, and transition-to-turbulence flow. Given the scale of the typical applications, the program is designed for parallel execution.Solution method: The equations are discretized using the Finite Element method, on either tetrahedral or hexahedral meshes. The software builds on top of deal.II, implementing the mathematical models and numerical methods specific for CFD cardiovascular simulations. Parallel execution exploits the MPI paradigm. The software supports both Trilinos and PETSc as linear algebra backends.Additional comments including restrictions and unusual features: The program provides a general-purpose executable that can be used to run CFD simulations without having to access or modify the source code. The program allows to setup simulations through a user-friendly yet flexible interface, by means of readable and self-documenting parameter files. On top of that, more advanced users can modify the source code to implement more sophisticated test cases. life▪-cfd supports checkpointing, i.e. simulations can be stopped and restarted at a later time.

ROS2 Real-time Performance Optimization and Evaluation

Real-time interaction with uncertain and dynamic environments is essential for robotic systems to achieve functions such as visual perception, force interaction, spatial obstacle avoidance, and motion planning. To ensure the reliability and determinism of system execution, a flexible real-time control system architecture and interaction algorithm are required. The ROS framework was designed to improve the reusability of robotic software development by providing a distributed structure, hardware abstraction, message-passing mechanism, and application prototypes. Rich ecosystems for robotic development have been built around ROS1 and ROS2 architectures based on the Linux system. However, because of the fairness scheduling principle of the default Linux system design and the complexity of the kernel, the system does not have real-time computing. To achieve a balance between real-time and non-real-time computing, this paper uses the transmission mechanism of ROS2, combines it with the scheduling mechanism of the Linux operating system, and uses Preempt_RT to enhance the real-time computing of ROS1 and ROS2. The real-time performance evaluation of ROS1 and ROS2 is conducted from multiple perspectives, including throughput, transmission mode, QoS service quality, frequency, number of subscription nodes and EtherCAT master. This paper makes two significant contributions: firstly, it employs Preempt_RT to optimize the native ROS2 system, effectively enhancing the real-time performance of native ROS2 message transmission; secondly, it conducts a comprehensive evaluation of the real-time performance of both native and optimized ROS2 systems. This comparison elucidates the benefits of the optimized ROS2 architecture regarding real-time performance, with results vividly demonstrated through illustrative figures.

Receiving ADS-B Signals on Embedded Linux using RTL-SDR: A Practical Guide

This paper presents a solution for ADS-B signal reception with RTL-SDR based on embedded Linux. This system utilizes an inexpensive RTL-SDR radio receiver for real-time reading of aeronautical messages, and it is experimentally demonstrated that this combination provides a significantly cost-effective way to track aircraft. By using an embedded Linux system, this paper demonstrates the high degree of portability and automation that can be achieved with RTL-SDR and its built-in ADS-B signal reception capability. The solution can be used in a wide range of applications and has high practical value.

Tissue Forge: Interactive biological and biophysics simulation environment.

Tissue Forge is an open-source interactive environment for particle-based physics, chemistry and biology modeling and simulation. Tissue Forge allows users to create, simulate and explore models and virtual experiments based on soft condensed matter physics at multiple scales, from the molecular to the multicellular, using a simple, consistent interface. While Tissue Forge is designed to simplify solving problems in complex subcellular, cellular and tissue biophysics, it supports applications ranging from classic molecular dynamics to agent-based multicellular systems with dynamic populations. Tissue Forge users can build and interact with models and simulations in real-time and change simulation details during execution, or execute simulations off-screen and/or remotely in high-performance computing environments. Tissue Forge provides a growing library of built-in model components along with support for user-specified models during the development and application of custom, agent-based models. Tissue Forge includes an extensive Python API for model and simulation specification via Python scripts, an IPython console and a Jupyter Notebook, as well as C and C++ APIs for integrated applications with other software tools. Tissue Forge supports installations on 64-bit Windows, Linux and MacOS systems and is available for local installation via conda.

CluMP: Clustered Markov Chain for Storage I/O Prefetch

Due to advancements in CPU and storage technologies, the processing speed of tasks has been increasing. However, there has been a relative slowdown in the data transfer speeds between disks and memory. Consequently, the issue of I/O processing speed has become a significant concern in I/O-intensive tasks. This research paper proposes CluMP, which predicts the next block to be requested within a process using a clustered Markov chain. Compared to the simple read-ahead approach commonly used in Linux systems, CluMP can predict prefetching more accurately and requires less memory for the prediction process. CluMP demonstrated a maximum memory hit ratio improvement of 191.41% in the KVM workload compared to read-ahead, as well as a maximum improvement of 130.81% in the Linux kernel build workload. Additionally, CluMP provides the advantage of adaptability to user objectives and utilized workloads by incorporating several parameters, thereby allowing for optimal performance across various workload patterns.

Design and study of a hydrogen concentration online monitoring system

This paper designs a hydrogen concentration online monitoring system, mainly completes the construction of the software and hardware platform, select the I.MX6ULL embedded processor as the core hardware platform and embedded Linux system transplantation, root file system construction, hardware peripherals driver design, complete the human-computer interface based on QT application framework and save the collected data to the memory of the embedded motherboard and send to the cloud in real time through the network port.

Lifex-fiber: an open tool for myofibers generation in cardiac computational models

BackgroundModeling the whole cardiac function involves the solution of several complex multi-physics and multi-scale models that are highly computationally demanding, which call for simpler yet accurate, high-performance computational tools. Despite the efforts made by several research groups, no software for whole-heart fully-coupled cardiac simulations in the scientific community has reached full maturity yet.ResultsIn this work we present texttt {life}^{texttt {x}}-fiber, an innovative tool for the generation of myocardial fibers based on Laplace-Dirichlet Rule-Based Methods, which are the essential building blocks for modeling the electrophysiological, mechanical and electromechanical cardiac function, from single-chamber to whole-heart simulations. texttt {life}^{texttt {x}}-fiber is the first publicly released module for cardiac simulations based on texttt {life}^{texttt {x}}, an open-source, high-performance Finite Element solver for multi-physics, multi-scale and multi-domain problems developed in the framework of the iHEART project, which aims at making in silico experiments easily reproducible and accessible to a wide community of users, including those with a background in medicine or bio-engineering.ConclusionsThe tool presented in this document is intended to provide the scientific community with a computational tool that incorporates general state of the art models and solvers for simulating the cardiac function within a high-performance framework that exposes a user- and developer-friendly interface. This report comes with an extensive technical and mathematical documentation to welcome new users to the core structure of texttt {life}^{texttt {x}}-fiber and to provide them with a possible approach to include the generated cardiac fibers into more sophisticated computational pipelines. In the near future, more modules will be successively published either as pre-compiled binaries for x86-64 Linux systems or as open source software.

Autonomous landing algorithm of UAV based on Target tracking

The target tracking technology in the UAV autonomous landing process is studied. Based on the analysis of the occlusion, background change, and variable scale problems in the target tracking process, a scale adaptive method for UAV target tracking based on KCF (Kernelized Correlation Filter) algorithm is proposed. The original KCF target tracking framework is adopted to achieve the target position prediction function. Secondly, to address the variable scale problem of moving targets and the target loss problem caused by scale changes, the scale filter of the DSST target tracking algorithm is used. The algorithm is for estimating the scale of moving targets and generating a small-scale filter estimate to improve the tracking efficiency of the UAV for moving targets and reduce the target loss rate. The final test on the OTB dataset showed a 4% improvement in variable scale accuracy and a 2.9% improvement in variable scale success rate. A Gazebo 3D environment scene was set up under the Linux system, and UAV autonomous landing simulation experiments were conducted to verify the improved algorithm’s real-time effectiveness in the UAV autonomous landing application. The simulation results show that the improved KCF target tracking algorithm has real-time performance in the UAV autonomous landing trajectory tracking process.

Portable hearing laboaratory for developping and evaluating novel hearing aid algorithms

We report here on the further development of our portable hearing aid research platform achived in Phase III NIDCD project R44DC016247. We also report on a large number of academic studies using the platform around the world. The project builds on the openMHA distribution for ARM processors included in an optimized Linux system developed in our other project R01DC015429. Thus we provide a complete wearable software-hardware fast hearing aid device needed for development of new innovative solutions for assisted hearing. We refer to this device as the PHL, short for Portable Hearing Laboratory. Throught the project we have followed the requirement that the platform should be “acoustically perfect.” That is able to transmit and process signals without audible distortion or noise beyond what is required to compensate for hearing loss. Any new sophisticated algorithms (such as those based on the AI) if run on a platform of inferior quality will adapt for that inferior structire and fail to capitalize on the reach processing capabilities of the modern hearing aids. We also report on a family of devices that can be used either with PHL or with other processors to provide supperior sound; such as bunaural BTE multimicrophone headsets. Last but not least we will present new devices connecting these auxiliary devices to powefull desktop computers for laboratory studies.

SMT: Software-Defined Memory Tiering for Heterogeneous Computing Systems With CXL Memory Expander

The rapid development of data-intensive technologies has driven an increasing demand for new architectural solutions with scalable, composable, and coherent computing environments. Compute Express Link (CXL), an open-standard interconnect protocol, overcomes architectural limitations by efficiently expanding memory capacity and bandwidth. In this work, we propose a power-efficient and cost-effective solution consisting of CXL-attached memory hardware and a software suite. The memory module hardware integrated with double-data-rate (DDR) dynamic random access memory and a CXL controller expands bandwidth by dozens of gigabytes per second and increases memory capacity by a few terabytes. Our software suite, scalable memory development kit, inherits and expands the traditional memory management architecture of existing Linux systems. We proved the functionality of the proposed solution by integrating renowned data centers and computing-intensive applications. The proposed CXL solution improved throughput for in-memory database and artificial intelligence applications by 1.5-fold and 1.99-fold, respectively, compared with the conventional DDR-only memory system.

Deployment of High Energy Physics software with a standard method

The installation and maintenance of scientific software for research in experimental, phenomenological, and theoretical High Energy Physics (HEP) requires a considerable amount of time and expertise. While many tools are available to make the task of installation and maintenance much easier, many of these tools require maintenance on their own, have little documentation and very few are used outside of HEP community.For the installation and maintenance of the software, we rely on the well tested, extensively documented, and reliable stack of software management tools with the RPM Package Manager (RPM) at its core. The precompiled HEP software packages can be deployed easily and without detailed Linux system knowledge and are kept up-to-date through the regular system update process. The precompiled packages were tested on multiple installations of openSUSE, RHEL clones, and Fedora. As the RPM infrastructure is adopted by many Linux distributions, the approach can be used on more systems.In this contribution, we discuss our approach to software deployment in detail, present the software repositories for multiple RPM-based Linux distributions to a wider public and call for a collaboration for all the interested parties.

Bash in the Wild: Language Usage, Code Smells, and Bugs

The Bourne-again shell (Bash) is a prevalent scripting language for orchestrating shell commands and managing resources in Unix-like environments. It is one of the mainstream shell dialects that is available on most GNU Linux systems. However, the unique syntax and semantics of Bash could easily lead to unintended behaviors if carelessly used. Prior studies primarily focused on improving the reliability of Bash scripts or facilitating writing Bash scripts; there is yet no empirical study on the characteristics of Bash programs written in reality, e.g., frequently used language features, common code smells, and bugs. In this article, we perform a large-scale empirical study of Bash usage, based on analyses over one million open source Bash scripts found in Github repositories. We identify and discuss which features and utilities of Bash are most often used. Using static analysis, we find that Bash scripts are often error-prone, and the error-proneness has a moderately positive correlation with the size of the scripts. We also find that the most common problem areas concern quoting, resource management, command options, permissions, and error handling. We envision that these findings can be beneficial for learning Bash and future research that aims to improve shell and command-line productivity and reliability.

NetAP-ML: Machine Learning-Assisted Adaptive Polling Technique for Virtualized IoT Devices.

To maximize the performance of IoT devices in edge computing, an adaptive polling technique that efficiently and accurately searches for the workload-optimized polling interval is required. In this paper, we propose NetAP-ML, which utilizes a machine learning technique to shrink the search space for finding an optimal polling interval. NetAP-ML is able to minimize the performance degradation in the search process and find a more accurate polling interval with the random forest regression algorithm. We implement and evaluate NetAP-ML in a Linux system. Our experimental setup consists of a various number of virtual machines (2-4) and threads (1-5). We demonstrate that NetAP-ML provides up to 23% higher bandwidth than the state-of-the-art technique.

Оценка влияния протокола Openflow на производительность сетевых устройств

A very promising area of computer network technologies is the use of Software Defined Network (SDN). The principles of SDN first emerged in the Stanford and Berkeley research laboratories and are currently being developed as part of the Open Network Foundation consortium and the European project OFELIA. The openness of the OpenFlow standard and the introduction of control logic into a separate controller simplifies the software and hardware of active network equipment, which will allow manufacturers to reduce its cost in the future, and therefore reduce the cost of creating computer networks. Therefore, research in this area is very promising. This work is devoted to the study of the performance of SDN, namely, how the implementation of the OpenFlow protocol affects the quality of data transfer between network devices. Using a single stream, the maximum throughput is estimated. To do this, three experimental installations of a network with different network devices, a switch, a router and an OpenFlow device, as well as a dedicated Linux server with support for the OpenFlow protocol, are assembled. The essence of the experiments is to change the size of packets and the size of the forwarding table to assess the performance of SDN with different network devices. The results of the analysis showed that the implementation of OpenFlow on Linux systems is able to offer very good performance. OpenFlow shows good performance with a large number of threads.

DEVELOPMENT OF A TEXT EDITOR FOR UNIX OPERATING SYSTEM

The task of developing and implementing a software application for creating, editing, displaying on the screen, as well as saving in the form of files of various formats, which in turn will be used in programming in languages such as C, C++, Python, and the Linux system language – Bash, is being considered. The developed software application is designed to enable the user to interact with files of various extensions, edit and save changes, work with the content of files for further development of software code in languages such as C, C++, Python. The software application can be used both as a console application and with the use of a graphical interface on UNIX and MS Windows operating systems. During development, the UNIX operating system was used, namely the Linux – Ubuntu distribution, which made it possible to use the created text editor on such operating systems as Ubuntu, Linux Mint, Kali Linux, Raspberry Pi Ubuntu. A free cross–platform multimedia package of libraries – SFML was also used, which made it possible to use a text editor on the MS Windows platform. The source code of the SFML library is provided under the ZLIB/PNG Free Software Distribution License. During development, a method of processing input information, as well as saving it in a file of a separate format, was defined and implemented. The following programming languages were used: C, C++11, a convenient software application architecture was developed, which allows you to easily maintain and improve the program in the future. The use of a virtual machine was an integral part of the project. By using a virtual machine, many computer resources are utilized, such as: RAM, number of cores, data storage and discrete desktop acceleration in the desired operating system. A friendly graphical interface for user interaction has been implemented.

UAF-GUARD: Defending the use-after-free exploits via fine-grained memory permission management

The defense of Use-After-Free (UAF) exploits generally could be guaranteed via static or dynamic analysis, however, both of which are restricted to intrinsic deficiency. The static analysis has limitations in loop handling, optimization of memory representation and constructing a satisfactory test input to cover all execution paths. While the lack of maintenance of pointer information in dynamic analysis may lead to defects that cannot accurately identify the relationship between pointers and memory. In order to successfully exploit a UAF vulnerability, attackers need to reference freed memory. However, main existing schemes barely defend all types of UAF exploits because of the incomplete check of pointers. To solve this problem, we propose UAF-GUARD to defend against the UAF exploits via fine-grained memory permission management. Specially, we design two key data structures to enable the fine-grained memory permission management to support efficient relationship search for pointers and memory, which is the key design of our defending scheme against UAF exploits. In addition, UAF-GUARD can precisely locate the position of UAF vulnerabilities, so that malicious programs can be terminated in the place where the abnormality is discovered. We implement UAF-GUARD on a 64-bit Linux system, and further use UAF-GUARD to transform a program into a suitable version that can defend against UAF vulnerabilities exploits. Compared with main existing schemes UAF-GUARD is able to effectively and efficiently defend against all the three types of UAF exploits with acceptable space overhead (26.4% for small programs and 0.3% for large programs) and time complexity (21.9%).