RT Patch Research Articles

Real-Time Performance Analysis and Tuning of Embedded System Virtualization Architecture Based on KVM

Embedded systems are proceeding towards exploiting virtualization technology to have the benefits of Real-Time Operating System (RTOS) and General-Purpose Operating System (GPOS) in the same system. This combination provides both a timely and deterministic behavior and a general-purpose application codebase. There still exist concerns about the real-time responsiveness of RTOS running inside a Virtual Machine (VM). In this paper, the real-time performance of Kernel-based Virtual Machine (KVM) virtualization architecture is analyzed on a multi-core system. Here, a preemptible Linux kernel with the PREEMPT_RT patch is used for RTOS, while a standard Linux kernel is used for GPOS. The interrupt latency inside the real-time guest VM is analyzed by applying various amounts of CPU, memory, and I/O stresses on the guest and host systems. A VM resource monitoring tool ‘VM_stat’ is developed to know the resource usage of the guest VMs, which is useful for effectively tuning the system. Different real-time tuning measures are applied on the host/guest systems and the performance is analyzed.

Performance Assessment of Linux Kernels with PREEMPT_RT on ARM-Based Embedded Devices

This work investigates the real-time performance of Linux kernels and distributions with a PREEMPT_RT real-time patch on ARM-based embedded devices. Experimental measurements, which are mainly based on heuristic methods, provide novel insights into Linux real-time performance on ARM-based embedded devices (e.g., BeagleBoard and RaspberryPi). Evaluations of the Linux real-time performance are based on specific real-time software measurement modules, developed for this purpose, and the use of a standard benchmark tool, cyclictest. Software modules were designed upon the introduction of a new response task model, an innovative aspect of this work. Measurements include the latency of response tasks at user and kernel space, the response on the execution of periodic tasks, the maximum sustained frequency and general latency performance metrics. The results show that in such systems the PREEMPT_RT patch provides more improved real-time performance than the default Linux kernels. The latencies and particularly the worst-case latencies are reduced with real-time support, thus making such devices running Linux with PREEMPT_RT more appropriate for use in time-sensitive embedded control systems and applications. Furthermore, the proposed performance measurements approach and evaluation methodology could be applied and deployed on other Linux-based real-time platforms.

Real-Time Performance and Response Latency Measurements of Linux Kernels on Single-Board Computers

This research performs real-time measurements of Linux kernels with real-time support provided by the PREEMPT_RT patch on embedded development devices such as BeagleBoard and Raspberry Pi. The experimental measurements of the Linux real-time performance on these devices are based on real-time software modules developed specifically for the purposes of this research. Taking in consideration the constraints of the specific hardware platforms under investigation, new measurements software was developed. The measurement algorithms are designed upon response and periodic task models. Measurements investigate latencies of real-time applications at user and kernel space. An outcome of this research is that the proposed performance measurements approach and evaluation methodology could be applied and deployed on other Linux-based boards and platforms. Furthermore, the results demonstrate that the PREEMPT_RT patch overall improves the Linux kernel real-time performance compared to the standard one. The reduced worst-case latencies on such devices running Linux with real-time support could make them potentially more suitable for real-time applications as long as a latency value of about 160 μs, as an upper bound, is an acceptable safety margin.

Probabilistic-WCET reliability: Statistical testing of EVT hypotheses

In recent years, the interest in probabilistic real-time has grown, as a response to the limitations of traditional static Worst-Case Execution Time (WCET) methods, in performing timing analysis of applications running on complex systems, like multi/many-cores and COTS platforms. The probabilistic theory can partially solve this problem, but it requires strong guarantees on the execution time traces, in order to provide safe probabilistic-WCET estimations. These requirements can be verified through suitable statistical tests, as described in this paper. In this work, we identify also challenges and problems of using statistical testing procedures in probabilistic real-time computing, proposing a unified test procedure based on a single index called Probabilistic Predictability Index (PPI). An experimental campaign has been carried out, considering both synthetic and realistic datasets, and the analysis of the impact of the Linux PREEMPT_RT patch on a modern complex platform as a use-case of the proposed index.

Research on Linux Real-time and performance Evaluation for Loongson 3A3000 processor

In order to solve the real-time problem of Linux system for loongson3A, the real-time enhancement design of Linux kernel is carried out by using the idea of PREEMPT_RT patch, the performance analysis support based on ftrace tool is improved, and the real-time optimization method based on reserved core scheduling is proposed. The real-time performance of the system is tested from four aspects: context switching, interrupt response, scheduling latency and network latency. Five context switching methods are proposed, including active switching, priority preemptive switching, semaphores, message queuing and signal distribution latency. A test method based on the internal timer of the PCI board is designed. The test results show that the maximum latency of real-time Linux system based on Loongson3A3000 is less than 30us, the reserved core scheduling latency is the minimum, and the maximum jitter of network communication latency is less than 120us. In conclusion, the system based on loongson3A3000 can meet the demand for hard real-time applications at millisecond level.

The Real-Time Linux Kernel

The increasing functional and nonfunctional requirements of real-time applications, the advent of mixed criticality computing, and the necessity of reducing costs are leading to an increase in the interest for employing COTS hardware in real-time domains. In this scenario, the Linux kernel is emerging as a valuable solution on the software side, thanks to the rich support for hardware devices and peripherals, along with a well-established programming environment. However, Linux has been developed as a general-purpose operating system, followed by several approaches to introduce actual real-time capabilities in the kernel. Among these, the PREEMPT_RT patch, developed by the kernel maintainers, has the goal to increase the predictability and reduce the latencies of the kernel directly modifying the existent kernel code. This article aims at providing a survey of the state-of-the-art approaches for building real-time Linux-based systems, with a focus on PREEMPT_RT, its evolution, and the challenges that should be addressed in order to move PREEMPT_RT one step ahead. Finally, we present some applications and use cases that have already benefited from the introduction of this patch.

Analysis and Design of Computerized Numerical Controls for Execution on Multi-core Systems

Multi-core processors offer a performance increment compared to single-core platforms. This leap in performance is desired to be ported to computerized numerical controls. However, in order to profit of the benefits multi-core processors can bring, the software has to be tailored to real parallel execution. In this paper a concept is proposed for partitioning numerical control software functions for being concurrently executed on multi-core systems. Specifically, the interpreter and the cutter radius compensation modules have been analyzed for devising a feasible parallel architecture. The parser algorithm has been implemented following the proposed scheme in a thread-based approach. Experiments were conducted under a real time Linux kernel extension utilizing the PREEMPT_RT patch. The results were compared against its serial version in terms of execution times to validate the concept.

Performance of a Real-Time EtherCAT Master Under Linux

The adoption of open-source operating systems for the execution of real-time applications is gaining popularity, even in the networked control systems domain, due to cost and flexibility reasons. However, as opposed to their commercial counterparts, the actual performance level to be expected from them is still little known and may often depend on the kind of real-time extension being used, its configuration, and the overall software load of the system, including best-effort components. In this paper, an open-source EtherCAT master supported by a popular real-time extension for Linux, the RT Patch, is thoroughly evaluated with long-term measurements, which build confidence on the suitability of the proposed approach for real-world applications. Special attention is devoted to the unexpected, adverse effect that some best-effort components, for instance, graphics applications, may have on the overall real-time characteristics of the system. For reference, the proposed approach is also compared with RTAI, a more traditional and well-known real-time extension for Linux already in use for demanding applications.

On Improving Real-Time Interrupt Latencies of Hybrid Operating Systems with Two-Level Hardware Interrupts

In this paper, we propose to implement hybrid operating systems based on two-level hardware interrupts. We analyze and model the worst-case real-time interrupt latency for RTAI and identify the key component for its optimization. Then, we propose our methodology to implement hybrid operating systems with two-level hardware interrupts by combining the real-time kernel and the time sharing OS (Operating System) kernel. Based on the methodology, we discuss the important issues for the implementation. Finally, we implement a hybrid system called RTLinux-THIN (Real-Time LINUX with Two-level Hardware INterrupts) on the ARM architecture by combining ARM Linux kernel 2.6.9 and μC/OS-II. We conduct experiments on a set of real application programs including mplayer, Bonnie, and iperf, and compare the interrupt latency and interrupt task distributions for RTLinux-THIN (with and without cache locking), RTAI, Linux, and Linux with RT patch on a hardware platform based on Intel PXA270 processor. The results show that our scheme not only provides an easy method for implementing hybrid systems but also achieves the performance improvement for both the time sharing and real-time subsystems.

Real-time Linux in real time

The increasing market demand for systems characterized by low-latency, deterministic behavior and the emphasis on the use of commodity hardware and software have led to a new breed of real-time operating systems (OSs), known as enterprise real-time OSs. In response to the demand for accelerated access to such features in a Linux™ kernel, the IBM Linux and Java™ Technology Centers collaborated to provide the first commercially available enterprise real-time Linux kernel with real-time Java support. Extending the PREEMPT RT patch from Ingo Molnar of Red Hat, Inc., the kernel contains additional features that were required to meet the demands of enterprise real-time OS customers. This paper describes how IBM developers helped to direct, implement, and test the real-time Linux kernel, bringing it from software patches to a finished product in nine months.