Real-time Operating System Research Articles

A Power-Aware Real-Time System for Multi-Video Treatment on FPGA with Dynamic Partial Reconfiguration and Voltage Scaling

As the energy consumption is an evaluating factor for System-On-Chip (SOC) design, this paper presents a power-aware architecture for a real-time multi-video system on FPGA. This architecture aims to optimize power consumption for a multi-video system on ARM-based architectures. The proposed architecture uses dynamic reconfiguration and voltage scaling to create a power-aware system for real-time multi-video processing with minimal power dissipation. Dynamic partial reconfiguration was used to optimize the utilization of resources and reduce dynamic power consumption. Voltage scaling was also used to optimize dynamic power consumption, by configuring the blocks to use the minimum necessary voltage for normal operating conditions. The proposed architecture focused on the Zynq platform. The results showed power savings of up to 70% concerning performance and real-time constraints.

Neural Comb Filtering Using Sliding Window Attention Network for Speech Enhancement

In this paper, we demonstrate the significance of restoring harmonics of the fundamental frequency (pitch) in the deep neural network (DNN)-based speech enhancement. The parameters of the DNN can be estimated by minimizing the mask loss, but it does not restore the pitch harmonics, especially at higher frequencies. In this paper, we propose to restore the pitch harmonics in the spectral domain by minimizing cepstral loss around the pitch peak. Restoring the cepstral pitch peak, in turn, helps in restoring the pitch harmonics in the enhanced spectrum. The proposed cepstral pitch-peak loss acts as an adaptive comb filter on voiced segments and emphasizes the pitch harmonics in the speech spectrum. The network parameters are estimated using a combination of mask loss and cepstral pitch-peak loss. We show that this combination offers the complementary advantages of enhancing both the voiced and unvoiced regions. The DNN-based methods primarily rely on the network architecture, and hence, the prediction accuracy improves with the increasing complexity of the architecture. The lower complex models are essential for real-time processing systems. In this work, we propose a compact model using a sliding-window attention network (SWAN). The SWAN is trained to regress the spectral magnitude mask (SMM) from the noisy speech signal. Our experimental results demonstrate that the proposed approach achieves comparable performance with the state-of-the-art noncausal and causal speech enhancement methods with much lesser computational complexity. Our three-layered noncausal SWAN achieves 2.99 PESQ on the Valentini database with only \(10^9\) floating-point operations (FLOPs).

Designing a Custom CPU Architecture Based on Hardware RTOS and Dynamic Preemptive Scheduler

The current trend in real-time operating systems involves executing many tasks using a limited hardware platform. Thus, a single processor system has to execute multiple tasks with different priorities in different real-time system (RTS) work modes. Hardware schedulers can greatly reduce event trigger latency and successfully remove most of the scheduling overhead, providing more computing cycles for applications. In this paper, we present a hardware-accelerated RTOS based on the replication of resources such as program counters, general purpose registers (GPRs) and pipeline registers. The implementation of this new concept, based on real-time event handling implemented in hardware, is intended to meet the current rigorous requirements imposed by critical real-time systems. The most important attribute of this FPGA implementation is the time required for task context switching, which is only one clock cycle or three clock cycles when working with the atomic instructions used in the case of inter-task synchronization and communication mechanisms. The main contribution of this article is its focus on mutexes and the speed of response associated with related events. Thus, fast switching between threads is also validated, considering the handling of events in the hardware using HW_nMPRA_RTOS (HW-RTOS). The proposed architecture implements inter-task synchronization and communication mechanisms with high performance, improving the overall response time when the mutex or message is expected to relate to a higher-priority task.

An Efficient Sparse Bayesian Learning STAP Algorithm with Adaptive Laplace Prior

Space-time adaptive processing (STAP) encounters severe performance degradation with insufficient training samples in inhomogeneous environments. Sparse Bayesian learning (SBL) algorithms have attracted extensive attention because of their robust and self-regularizing nature. In this study, a computationally efficient SBL STAP algorithm with adaptive Laplace prior is developed. Firstly, a hierarchical Bayesian model with adaptive Laplace prior for complex-value space-time snapshots (CALM-SBL) is formulated. Laplace prior enforces the sparsity more heavily than Gaussian, which achieves a better reconstruction of the clutter plus noise covariance matrix (CNCM). However, similar to other SBL-based algorithms, a large degree of freedom will bring a heavy burden to the real-time processing system. To overcome this drawback, an efficient localized reduced-dimension sparse recovery-based space-time adaptive processing (LRDSR-STAP) framework is proposed in this paper. By using a set of deeply weighted Doppler filters and exploiting prior knowledge of the clutter ridge, a novel localized reduced-dimension dictionary is constructed, and the computational load can be considerably reduced. Numerical experiments validate that the proposed method achieves better performance with significantly reduced computational complexity in limited snapshots scenarios. It can be found that the proposed LRDSR-CALM-STAP algorithm has the potential to be implemented in practical real-time processing systems.

FPGA-based electronic system for the control and readout of superconducting quantum processors.

Electronic systems for qubit control and measurement serve as a bridge between quantum programming language and quantum information processors. With the rapid development of superconducting quantum circuit technology, synchronization in a large-scale system, low-latency execution, and low noise are required for electronic systems. Here, we present a field-programmable gate array (FPGA)-based electronic system with a distributed synchronous clock and trigger architecture. The system supports synchronous control of qubits with jitters of ∼5ps. We implement a real-time digital signal processing system in the FPGA, enabling precise timing control, arbitrary waveform generation, in-phase and quadrature demodulation for qubit state discrimination, and the generation of real-time qubit-state-dependent trigger signals for feedback/feedforward control. The hardware and firmware low-latency design reduces the feedback/feedforward latency of the electronic system to 125ns, significantly less than the decoherence times of the qubit. Finally, we demonstrate the functionalities and low-noise performance of this system using a fluxonium quantum processor.

A Visual Human-Computer Interaction System Based on Hybrid Visual Model

The traditional human-computer interaction is mainly through the mouse, keyboard, remote control, and other peripheral equipment electromagnetic signal transmission. This paper aims to build a visual human-computer interaction system through a series of deep learning and machine vision models, so that people can achieve complete human-computer interaction only through the camera and screen. The established visual human-computer interaction system mainly includes the function modes of three basic peripherals in human-computer interaction: keyboard, mouse (X-Y position indicator), and remote control. The convex hull method was used to switch between these three modes. After issuing the mode command, Gaussian mixture was used to quickly identify the moving human body to narrow the scope of our image processing. Subsequently, finger detection in human body was realized based on the Faster-RCNN-ResNET50-FPN model structure, and realized the function of moving mouse and keyboard through the relationship between different fingers. At the same time, the recognition of human body posture was done by using MediaPipe BlazePose, and the action classification models were established through the Angle between body movements so as to realize the control function of remote control. In order to ensure the real-time performance of the interactive system, according to the characteristics of different data processing processes, CPU and GPU computing power resources are used to cross-process images to ensure the real-time performance. The recognition accuracy of the human-computer interaction system is above 0.9 for the key feature points of human body, and above 0.87 for the recognition accuracy of four kinds of command actions. It is hoped that vision-based human-computer interaction will become a widely used interaction mode in the future.

Real-Time Performance Analysis and Tuning of Embedded System Virtualization Architecture Based on KVM

Embedded systems are proceeding towards exploiting virtualization technology to have the benefits of Real-Time Operating System (RTOS) and General-Purpose Operating System (GPOS) in the same system. This combination provides both a timely and deterministic behavior and a general-purpose application codebase. There still exist concerns about the real-time responsiveness of RTOS running inside a Virtual Machine (VM). In this paper, the real-time performance of Kernel-based Virtual Machine (KVM) virtualization architecture is analyzed on a multi-core system. Here, a preemptible Linux kernel with the PREEMPT_RT patch is used for RTOS, while a standard Linux kernel is used for GPOS. The interrupt latency inside the real-time guest VM is analyzed by applying various amounts of CPU, memory, and I/O stresses on the guest and host systems. A VM resource monitoring tool ‘VM_stat’ is developed to know the resource usage of the guest VMs, which is useful for effectively tuning the system. Different real-time tuning measures are applied on the host/guest systems and the performance is analyzed.

High-precision luminosity instrumentation for the CMS detector at the HL-LHC

The high-luminosity upgrade of the LHC (HL-LHC) is foreseen to reach an instantaneous luminosity a factor of five to seven times the nominal LHC design value. This creates the need for new high-precision instrumentation at CMS, using radiation-hard detector technologies. This contribution introduces the instrumentation for bunch-by-bunch online luminosity and beam-induced background measurements in CMS. The Tracker Endcap Pixel Detector (TEPX) will be adapted for precision luminometry by implementing a dedicated trigger type and a real-time data processing system performing clustering on FPGA. The innermost ring of the last TEPX layer (D4R1) will be operated independently from the rest of TEPX, enabling beam monitoring during the LHC ramp and during unqualified beam conditions with a dedicated timing and trigger infrastructure. A new standalone luminometer, the Fast Beam Condition Monitor (FBCM), which will be fully independent from the central trigger and data acquisition services and will be able to operate during all times is also introduced. FBCM is foreseen to utilize silicon-pad sensors with sub-bunch-crossing time resolution enabling also the measurement of beam-induced background. The potential of the exploitation of other CMS subsystems such as the outer tracker, the barrel muon detectors and the 40 MHz L1 trigger scouting system using a common histogramming firmware is also discussed.

Construction of Wireless Sensor Network Video Surveillance System for Multimedia Classroom Education and Teaching under 5G Communication Network

Research on wireless sensor network video surveillance system is a hot field of media classroom education and teaching. Based on the 5G communication network theory, this paper constructs a wireless sensor network video monitoring system for multimedia classroom education and teaching, systematically analyzes the new distance education model under the condition of the integration of the two networks, and improves the interactive live classroom education model. For the classroom teaching mode, the model studies the isochronous transmission technology of data collection and video encoding and decoding, adopts hardware compression encoding, integrates key technologies such as reliable multicast and conditional access, and proposes the design of the system scheme, which solves the problem of multimedia classroom education and teaching digital issues. During the simulation process, the MATLAB software platform was used to study the effect of the number of 5G communication network nodes and node attributes (such as node position, node perception angle, instantaneous perception direction, or direction difference between neighboring nodes, node speed, etc.) on the target area or target point. The experimental results show that the performance of the joint optimization of transmission video quality and network lifetime performs well under multipath conditions. The total utility under multipath finally converges to 8.3, while the single path finally converges to 6.5, which further promotes the real-time performance of the multimedia network classroom teaching system.

Embedded Real-time Operating System in Remote Viewing of Electrical Automation Equipment

Embedded Real-time Operating System in Remote Viewing of Electrical Automation Equipment

Quadrotor Real-Time Simulation: A Temporary Computational Complexity-Based Approach

The interaction of digital systems with dynamic systems requires synchrony and the accomplishment of time constrains, so the simulation of physical processes needs an implementation by means of real-time systems (RTS). However, as it can be expected, every simulation and/or implementation might demand too many computational resources, surpassing the capacity of the processor used by computational systems. This is the reason for the need to perform a temporary computational complexity analysis based on the study of the behavior of the execution times of the implemented simulation. In this regard, the Real-Time Operating Systems (RTOS) feature time managing tools, which allow their precise measurement and the establishment of scheduling criteria in process execution. Therefore, this research proposes accomplishing a temporary computational complexity analysis of the real-time simulation by an embedded system (ES) of an unmanned aerial vehicle (UAV) propelled by four rotors. Derived from this analysis, formal definitions are elaborated and proposed, which establish a close relationship between the temporary computational complexity and typical real-time temporary constraints. To the best of the author’s knowledge, the definitions presented in this article have not been reported in the literature. Furthermore, to perform the temporary computational complexity analysis of the UAV, the mathematical modeling based on the Euler–Lagrange approach is presented in detail. Finally, simulations were performed using a real-time system implemented on the Embedded Computer System (ECS) Raspberry Pi 2 Model B+, in order to validate the suggested definitions.

Estimating crowd density with edge intelligence based on lightweight convolutional neural networks

Crowd stampedes and incidents are critical threats to public security that have caused countless deaths during the past few decades. To avoid crowd stampedes, real-time crowd density estimation can help monitor crowd movements, and thus support a timely evacuation strategy development. In previous studies, scholars and engineers developed multiple video-based crowd density estimation algorithms based on deep neural networks. The excessive computational complexity of deep learning algorithms exacerbated the algorithm’s efficiency, causing unacceptable real-time performance. In the Internet of Things era, deploying the crowd density estimation task with edge computing is an advanced strategy to maintain the real-time performance of the entire system. Considering the limited computational resources on the edge devices, deep learning-based crowd density estimation algorithms normally cannot be handled. To fulfill the deployment on the edge device, the algorithms need to be optimized with a smaller model size. Therefore, this paper proposes a lightweight Convolutional Neural Networks (CNN) based crowd density estimation model by combining the modified MobileNetv2 and the dilated convolution. Public crowd image data sets are used to conduct experiments for evaluating the performance of the proposed algorithm in terms of accuracy and inference speed. The results show that our model achieves much better inference speed accompanied by a slight increase in accuracy. The proposed method of this study can enhance the performance of the crowd monitoring system, and therefore help avoid crowd stampedes and incidents.

Design of a Laser Driver and Its Application in Gas Sensing

A laser driver which features high stability and a graphical user-interface was designed and used in trace gas sensing. The running of the laser driver was managed by an ARM processor which was embedded with a real-time operating system (RTOS). Through clicking on the touch screen that was configured with an emWin graphical user-interface (GUI), the parameters of the driving current can be graphically set and monitored. The circuit model of the distributed feedback laser diode (DFB-LD) was introduced into a TINA-SPICE simulation to evaluate the performance of the current source. Through simulation, the potential self-oscillation can be visually predicted, and the feedback loop can be appropriately compensated. To validate the applicability, the laser driver was used for driving a carefully selected DFB-LD and was employed in wavelength modulation spectroscopy (WMS) for CH4 detection at R(3) absorption line of the 2ν3 overtone. Under the conditions of room temperature, normal pressure and an effective absorption path of 15.4 cm, repetitive experiments were conducted using gas samples, with their concentrations ranging from 400 ppm to 1%, and the detection limit derived from the signal-to-noise ratio (SNR) was 7.2 ppm. The promising result indicates the high potential of this laser driver for use in absorption spectrum-based sensing applications.

Programmable optical real-time processing system for ultra-wide bandwidth electronic reconnaissance.

Electronic reconnaissance is to detect signals, extract their parameters, modulation types or direction of arrival and so on from a wide bandwidth range. It is difficult for digital signal processing device to process in real time under an ultra-wide bandwidth environment. This paper proposed a programmable optical system which can process signals from an instantaneous bandwidth up to 40GHz in real time. In the optical system, the signals are reconstructed at wavefront of a laser beam. The laser beam carrying signals passes through an optical system composed by lens, beam splitter, light modulator, etc. Signal processing operation is accomplished when laser beam arrives at a focal plane, and processing results are acquired by a high-speed camera. Typical pulse description words can be yielded from the results. The proposed optical system has a nano-second processing delay due to its meter-length light path.

System design and research of multi-channel electroacupuncture instrument

This paper introduces the development history, working principle, curative effect under different output waveforms and the hardware and software design of the system. The proportional constant current output circuit is analyzed in detail. A system structure of electroacupuncture instrument based on modular design is proposed. The system structure mainly includes multi-channel electroacupuncture control module and display control module, which are connected through CAN bus. Multi-channel electroneedle control module adopts embedded processor and real-time operating system design, display control module is application level processor, running embedded Linux system design. The system has reasonable structure design and high expansibility, which is convenient for further research.

Cross‐layered distributed data‐driven framework for enhanced smart grid cyber‐physical security

Smart Grid (SG) research and development has drawn much attention from academia, industry and government due to the great impact it will have on society, economics and the environment. Securing the SG is a considerably significant challenge due the increased dependency on communication networks to assist in physical process control, exposing them to various cyber-threats. In addition to attacks that change measurement values using False Data Injection (FDI) techniques, attacks on the communication network may disrupt the power system's real-time operation by intercepting messages, or by flooding the communication channels with unnecessary data. Addressing these attacks requires a cross-layer approach. In this paper a cross-layered strategy is presented, called Cross-Layer Ensemble CorrDet with Adaptive Statistics(CECD-AS), which integrates the detection of faulty SG measurement data as well as inconsistent network inter-arrival times and transmission delays for more reliable and accurate anomaly detection and attack interpretation. Numerical results show that CECD-AS can detect multiple False Data Injections, Denial of Service (DoS) and Man In The Middle (MITM) attacks with a high F1-score compared to current approaches that only use SG measurement data for detection such as the traditional physics-based State Estimation, ECD-AS strategy and other machine learning classification-based detection schemes.

RTOS-supported low power scheduling of periodic hardware tasks in flash-based FPGAs

As flash-based technology features inherently a lower power consumption than SRAM-based technology, flash-based FPGAs are well suited for energy-aware applications that are often found, e.g., in battery-powered embedded systems. Additionally, power gating allows to shut down large parts of an FPGA during idle times, reducing the energy consumption even further. When power gating SRAM-based FPGAs, the configuration memory is erased. The current state has to be stored and reconfigured in a time intense process before hardware accelerators can be used again. By contrast, in flash-based FPGAs the configuration memory and register contents are retained. In Microsemi / Microchip devices, the power gating mode is called Flash*Freeze. As only the complete FPGA can be put into Flash*Freeze mode, its applicability is reduced when many hardware tasks with different execution times share the same FPGA. This problem is tackled by the cluster scheduling algorithm presented in this paper. It modifies the schedule of hardware tasks under consideration of their real-time constraints in order to prolong Flash*Freeze phases. We describe its integration into the real-time operating system FreeRTOS, which allows to hide technical details from the applications and provides an easy-to-use interface. For a task set with sufficient overlapping task instances, the cluster scheduling algorithm reduces energy consumption on average by 33.5% in comparison to applications that do not employ the Flash*Freeze mode and by 22.5% in comparison to applications that use the Flash*Freeze mode with an unchanged schedule. In the best case, a prolongation of the Flash*Freeze mode in the order of n can be reached with n given hardware tasks.

Specific Electronic Platform to Test the Influence of Hypervisors on the Performance of Embedded Systems

Some complex digital circuits must host various operating systems in a single electronic platform to make real-time and not-real-time tasks compatible or assign different priorities to current applications. For this purpose, some hardware–software techniques—called virtualization—must be integrated to run the operating systems independently, as isolated in different processors: virtual machines. These are monitored and managed by a software tool named hypervisor, which is in charge of allowing each operating system to take control of the hardware resources. Therefore, the hypervisor determines the effectiveness of the system when reacting to events. To measure, estimate or compare the performance of different ways to configure the virtualization, our research team has designed and implemented a specific testbench: an electronic system, based on a complex System on Chip with a processing system and programmable logic, to configure the hardware–software partition and show merit figures, to evaluate the performance of the different options, a field that has received insufficient attention so far. In this way, the fabric of the Field Programmable Gate Array (FPGA) can be exploited for measurements and instrumentation. The platform has been validated with two hypervisors, Xen and Jailhouse, in a multiprocessor System-on-Chip, by executing real-time operating systems and application programs in different contexts.

Energy Efficient UAV Flight Control Method in an Environment with Obstacles and Gusts of Wind

This article presents an energy-efficient method of controlling unmanned aircraft (fixed-wing UAVs), which consists of three groups of algorithms: aerial vehicle route planning, in-flight control, and algorithms to correct the preplanned flight trajectory. All algorithms shall take into account the existence of obstacles that the UAV must avoid and wind gusts in the UAV’s area of operation. Tests were carried out on the basis of the UAV mathematical model, stabilization and navigation algorithms, and Dryden turbulence model, considering the parameters of the UAV’s propulsion system. The work includes a detailed description of constructing a network of connection that is used to plan a UAV mission. It presents the algorithm for determining the actual distances between the different points in the field of action, which takes into account the existence of obstacles. The algorithm shall be based on methods for determining the flight trajectory on a hexagonal grid. It presents the developed proprietary UAV path planning algorithm based on a model from a group of algorithms of mixed integer linear problem (MILP) optimization. It presents the manner in which the pre-prepared flight path was used by UAV controllers that supervised the flight along the preset path. It details the architecture of contemporary unmanned aerial vehicles, which have embedded capability to realize autonomous missions, which require the integration of UAV systems into the route planning algorithms set out in the article. Particular attention has been paid to the planning and implementation methods of UAV missions under conditions where wind gusts are present, which support the determination of UAV flight routes to minimize the vehicle’s energy consumption. The models developed were tested within a computer architecture based on ARM processors using the hardware-in-the-loop (HIL) technique, which is commonly used to control unmanned vehicles. The presented solution makes use of two computers: FCC (flight control computer) based on a real-time operating system (RTOS) and MC (mission computer) based on Linux and integrated with the Robot Operating System (ROS). A new contribution of this work is the integration of planning and monitoring methods for the implementation of missions aimed at minimizing energy consumption of the vehicle, taking into account wind conditions.

An interactive and dynamic scratchpad memory management strategy for multi-core processors

An interactive and dynamic Scratchpad memory (Scratchpad) management approach is proposed in this research manuscript to target multi-core processors. By using MMU an integrated (memory management unit), it can be useful for an existing embedded system that maps into a virtual space the physically addressed Scratchpad memory. A hardware Unit MRSU (Memory Reference Sampling Unit) is introduced depend on mass-count disparity, with a very low probability that models the memory reference data stream. One of the memory addresses is considered as a captured address that is present in a repeatedly referenced memory segment. MRSU generated a hardware interference and the captured / identified repeatedly accessed memory block is placed, through the software, into the Scratchpad space. The page table is also modified by the software so that the Scratchpad memory data block will be forwarded by the follow-up memory accesses. In multi-core processor for Scratchpad management, this proposed strategy is specifically adequate without depending on profiling information and compiler. By profiling or static analysis, the memory accesses behaviour cannot be guessed and a RTOS (Real-Time Operating System) is mostly presented in such an environment. By executing numerous operations on a small Real-Time Operating System the proposed Scratchpad allocation strategy is evaluated with preemptive scheduling. In contrast with a referenced cache-only system with just 1% performance degradation at runtime, in terms of energy consumption on average, this proposed method can achieve an 11% reduction which can be observed in experimental results.