Real-time Operating System Research Articles

VLSI Design of Saturation-Based Image Dehazing Algorithm

Hazy weather degrades the vividness of the images captured by real-time systems for applications such as object detection, remote sensing, and surveillance systems. This affects the performance of such real-time systems. The hardware implementation of a real-time haze removal system is imperative to solve these problems. Such a solution is proposed in this article. Here, the saturation-based hardware implementation of an image dehazing system is presented. To estimate atmospheric light more precisely, a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$15\ttimes 15$</tex-math> </inline-formula> window minimum filter is implemented, which uses the downsampled hazy image to estimate the atmospheric light. Furthermore, we have employed saturation-based transmission map estimation, which makes our approach pixel-based rather than patch-based. Unlike existing patch-based methods, the proposed method requires neither an edge detection unit nor an image filtering unit to suppress halo artifacts around edges. The VLSI architecture of the proposed dehazing system comprises seven pipelined stages. It is implemented on FPGA as well as ASIC (65-nm technology node) platforms. The ASIC implementation of the proposed dehazing system yielded a maximum throughput of 624 Mpixels/s, which is fast enough to process <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3840\ttimes 2160$</tex-math> </inline-formula> resolution at a rate higher than 70 fps with only 13.2k logic gates count.

Residual network-based aberration correction in a sensor-less adaptive optics system

The performance of free-space optical communication (FSOC) is often affected by atmospheric turbulence. The sensor-less adaptive optics (SLAO) system is an effective method for overcoming the effects of atmospheric turbulence. The performance of the control algorithm in the SLAO system directly determines whether the SLAO system can effectively correct wavefront aberrations. In this study, we introduce a residual network (ResNet) as a control algorithm to replace the traditional control algorithm. By lowering the number of iterations, this strategy enhances the real-time performance of the FSOC system. The final ResNet model can achieve an accuracy of 0.98 for training and 0.92 for testing. The simulation results show that stochastic parallel gradient descent (SPGD) algorithm takes 700 times longer and requires at least 500 iterations to achieve the same performance as ResNet. And we verify the feasibility of the ResNet model by setting up an experiment.

Разработка архитектурного решения программного обеспечения для устройств интернет-вещей

The article presents the development of an architectural software solution for Internet of Things (IoT) devices that implements the functionality of an automatic medical drug dispenser, based on the ESP32 hardware platform and utilizing the capabilities of the existing real-time operating systems (RTOS). The software architecture for IoT devices was designed with scalability and fault tolerance in mind. All components of the system interact with each other through asynchronous callback functions, which provides flexibility and extensibility to the architecture. Testing for system fault tolerance was conducted. The architecture can be implemented and used as the basis for any IoT device, allowing for support of modern security and functionality stacks, by implementing this functionality once in any of the devices. The process of designing the software architecture is presented, including the selection of suitable technologies and libraries. Particular attention was paid to ensuring the safety and reliability of the device, including protection against an unauthorized access and errors in operation. The experimental results show high efficiency and accuracy of the automatic medical drug dispenser based on the developed software. The practical part provides examples of implementing the proposed architecture in the C and C++ languages, with examples and basic interaction diagrams between the components. The rxcpp library was also used in writing the C++ implementation, which made it easier to write the code base for interacting with the operating system resources and reusing the multithreaded interaction with the system.

An intelligent tracking system for surgical instruments in complex surgical environment

In recent years, the problem of visual occlusion in optical surgical tracking system has attracted widespread attention. To solve this problem, this paper proposes a tracking system for surgical instruments based on data fusion of multicamera modules. Multicamera modules are used to track surgical instruments from different angles to solve the problem of tracking occlusion in monocular tracking. Firstly, occlusion of monocular tracking based on different fiducial marker is analyzed, and a fast calibration method of tracking system for surgical instrument based on multicamera module is proposed. The calibration of the tracking system is mainly divided into the calibration of surgical instruments and the fast calibration between multicamera modules. On this basis, the occlusion detection mechanism is proposed to screen out the data with large errors, and the fuzzy control theory is used to calculate the support degree between different modules. Finally, the serviceable data is fused based on the least square method and the position of surgical instruments is estimated. The average accuracy of static positioning and dynamic trajectory tracking is mm and mm respectively. The real-time performance of the proposed system is 35 frames per second (fps). Compared with the existing optical tracking system, the method proposed in this paper has stronger anti-occlusion and simpler operation.

A Smart Card-Based Two-Factor Mutual Authentication Scheme for Efficient Deployment of an IoT-Based Telecare Medical Information System

The integration of the Internet of Things (IoT) and the telecare medical information system (TMIS) enables patients to receive timely and convenient healthcare services regardless of their location or time zone. Since the Internet serves as the key hub for connection and data sharing, its open nature presents security and privacy concerns and should be considered when integrating this technology into the current global healthcare system. Cybercriminals target the TMIS because it holds a lot of sensitive patient data, including medical records, personal information, and financial information. As a result, when developing a trustworthy TMIS, strict security procedures are required to deal with these concerns. Several researchers have proposed smart card-based mutual authentication methods to prevent such security attacks, indicating that this will be the preferred method for TMIS security with the IoT. In the existing literature, such methods are typically developed using computationally expensive procedures, such as bilinear pairing, elliptic curve operations, etc., which are unsuitable for biomedical devices with limited resources. Using the concept of hyperelliptic curve cryptography (HECC), we propose a new solution: a smart card-based two-factor mutual authentication scheme. In this new scheme, HECC’s finest properties, such as compact parameters and key sizes, are utilized to enhance the real-time performance of an IoT-based TMIS system. The results of a security analysis indicate that the newly contributed scheme is resistant to a wide variety of cryptographic attacks. A comparison of computation and communication costs demonstrates that the proposed scheme is more cost-effective than existing schemes.

An Efficient On-Chip Data Storage and Exchange Engine for Spaceborne SAR System

Advancements in remote sensing technology and very-large-scale integrated circuit (VLSI) have significantly augmented the real-time processing capabilities of spaceborne synthetic aperture radar (SAR), thereby enhancing terrestrial observational capacities. However, the inefficiency of voluminous data storage and transfer inherent in conventional methods has emerged as a technical hindrance, curtailing real-time processing within SAR imaging systems. To address the constraints of a limited storage bandwidth and inefficient data transfer, this study introduces a three-dimensional cross-mapping approach premised on the equal subdivision of sub-matrices utilizing dual-channel DDR3. This method considerably augments storage access bandwidth and achieves equilibrium in two-dimensional data access. Concurrently, an on-chip data transfer approach predicated on a superscalar pipeline buffer is proposed, mitigating pipeline resource wastage, augmenting spatial parallelism, and enhancing data transfer efficiency. Building upon these concepts, a hardware architecture is designed for the efficient storage and transfer of SAR imaging system data, based on the superscalar pipeline. Ultimately, a data storage and transfer engine featuring register addressing access, configurable granularity, and state monitoring functionalities is realized. A comprehensive imaging processing experiment is conducted via a “CPU + FPGA” heterogeneous SAR imaging system. The empirical results reveal that the storage access bandwidth of the proposed superscalar pipeline-based SAR imaging system’s data efficient storage and transfer engine can attain up to 16.6 GB/s in the range direction and 20.0 GB/s in the azimuth direction. These findings underscore that the storage exchange engine boasts superior storage access bandwidth and heightened data storage transfer efficiency. This considerable enhancement in the processing performance of the entire “CPU + FPGA” heterogeneous SAR imaging system renders it suitable for application within spaceborne SAR real-time processing systems.

Real Time Cigarette Detection using Deep Learning Models

Abstract: Cigarette smoking is a significant health hazard worldwide, leading to several chronic diseases and even deaths. Detecting cigarette smoking in real-time can help prevent and reduce its harmful effects. In this research paper, we propose a real-time cigarette detection project using deep learning models. The project aims to detect cigarette smoking in real-time through a camera feed and notify authorities to take necessary actions. The proposed system uses the YOLOv3 (You Only Look Once) object detection algorithm, a state-of-the-art deep learning model for object detection. The model is trained on a dataset of images containing cigarettes and non-cigarette images. The dataset is augmented with different lighting conditions, angles, and background to increase its diversity. The system uses a camera to capture the video feed in real-time. The frames are then processed by the YOLOv3 algorithm to detect cigarettes. Once a cigarette is detected, a notification is sent to the authorities, alerting them of the potential smoking incident. The system was evaluated on a dataset of real-world smoking scenarios, achieving an accuracy of 92.5% in detecting cigarettes. The system was tested in various lighting conditions, distances, and angles, showing consistent performance. The system's real-time performance was also evaluated, achieving an average processing time of 0.3 seconds per frame

Impact in the Parallel Processing of IHM-Plasma Using the Earliest-Deadline-First Algorithm for the Task-Scheduler Realized by Hardware

Abstract—This work studies the impact in the parallel processing of the Interlocked-Hardware-Microkernel (IHM) Plasma microprocessor (IHM-Plasma) by implementing the Earliest-Deadline-First (EDF) algorithm by hardware in the task-scheduler block that belongs to those task-based operational systems, such as that related to the real-time operational systems (RTOS). IHM-Plasma presents Reduced Instruction Set Computer (RISC) architecture. The main results have observed by this study show that the IHM-Plasma running the EDF algorithm by hardware has increased the tasks executed per second in 174% while running a task-based operational system with 14 active tasks in comparison to the original Plasma running the EDF algorithm by software. The developed work has great potential use in Hard Real-Time Systems and others where a rigid control of deadlines is essential and hold many tasks.

Research on Intermittent Hypoxia Training in Sports Based on Graph Neural Network

ABSTRACT To enhance the efficacy of intermittent hypoxia training in sports, this study presents an intelligent training model that utilizes a graph neural network. The model incorporates the particle filter method to establish a real-time processing system for physiological signals generated during intermittent hypoxia training, enabling frequency tracking and network sorting. Additionally, an ARMA model is utilized to facilitate real-time carrier frequency estimation and time-hopping detection of physiological signals. An enhanced frequency tracking method is proposed based on the Graph Neural Network (GNN) and ARMA model to improve the accuracy of frequency tracking while minimizing algorithm complexity. The experimental results indicate that the fusion of the GNN and the proposed intermittent hypoxia training model can effectively enhance the effects of intermittent hypoxia training in sports.

Real-time processing system and Internet of Things application in the cultural tourism industry development

The introduction of the Internet of Things and other information technologies provides new methods for tourism information innovation. The development of computer technology has changed the traditional tourism mode and made the information-based tourism industry develop. With the improvement and development of embedded systems, embedded systems will develop in the direction of networking, intelligence, standardization and integration. In the process of innovation, the tourism system needs innovation support, support and protection, guidance and response mechanisms. The innovation of tourism system mainly relies on market demand and supply as its most important part, in which market demand is the main driving force for formation. The operation of the system is supported by the tourism support and protection subsystem, with the innovative technology subsystem as the most important content. Based on the traditional theory of tourism industry, this paper sorted out and analyzed the related needs of tourism industry, and combined with the computer industry, according to the embedded real-time network system and the convenience of the Internet of Things industry, on the relevant technology platform to establish a modern tourism industry development system. By using the powerful advantages of computer technology, this paper from the perspective of destination selection, management optimization, industrial development, consumption upgrading, service diversification and other aspects, to promote the development of modern cultural tourism industry.

SLIMBRAIN: Augmented reality real-time acquisition and processing system for hyperspectral classification mapping with depth information for in-vivo surgical procedures

Over the last two decades, augmented reality (AR) has led to the rapid development of new interfaces in various fields of social and technological application domains. One such domain is medicine, and to a higher extent surgery, where these visualization techniques help to improve the effectiveness of preoperative and intraoperative procedures. Following this trend, this paper presents SLIMBRAIN, a real-time acquisition and processing AR system suitable to classify and display brain tumor tissue from hyperspectral (HS) information. This system captures and processes HS images at 14 frames per second (FPS) during the course of a tumor resection operation to detect and delimit cancer tissue at the same time the neurosurgeon operates. The result is represented in an AR visualization where the classification results are overlapped with the RGB point cloud captured by a LiDAR camera. This representation allows natural navigation of the scene at the same time it is captured and processed, improving the visualization and hence effectiveness of the HS technology to delimit tumors. The whole system has been verified in real brain tumor resection operations.

Application of real-time data processing system of Internet of Things based on blockchain technology in the financial field of Yangtze River Delta urban agglomeration

After more than 40 years of long-term development, as a pioneer of China's reform and opening up, the Yangtze River Delta region has made tremendous economic progress and strong scientific and technological innovation strength, which has made the economy flourish and scientific and technological innovation strength strong. However, in the process of financial growth in the Yangtze River Delta, there are also financial structures that do not match the optimization of industrial structure, thus hindering the quality and efficiency of industrial structure optimization, thus affecting the strategic function of industrial development. For the development of the financial field, this paper applies the real-time data processing system of the Internet of Things based on blockchain technology to analyze the financial industry of the Yangtze River Delta urban agglomeration. The Internet of Things, based on the technology of real-time data processing system, can store different types of financial assets. These assets are a common component of supporting the application of the Internet of Things. They mix and build a powerful back-end process that can support the data processing application of the Internet of Things. Therefore, the Internet of Things data network and other information are stored in the data structure, so that data analysis and consolidation can be carried out with consensus, Thus, the processing efficiency of relevant financing data in the Yangtze River Delta region has been greatly improved, and the development process of the financial sector in the region has been effectively accelerated. Therefore, the research on this issue will enable the financial industry to give full play to its potential as a major component of national economic development, so as to maximize the improvement of the industrial structure and promote the development of blockchain and Internet of Things technologies.

An Optimized AB SYN Algorithm for Synchronization and Inter Task Communication in Real Time Operating System

An Optimized AB SYN Algorithm for Synchronization and Inter Task Communication in Real Time Operating System

Proposal and Implementation of a Procedure for Compliance Recognition of Objects with Smart Tactile Sensors.

This paper presents a procedure for classifying objects based on their compliance with information gathered using tactile sensors. Specifically, smart tactile sensors provide the raw moments of the tactile image when the object is squeezed and desqueezed. A set of simple parameters from moment-versus-time graphs are proposed as features, to build the input vector of a classifier. The extraction of these features was implemented in the field programmable gate array (FPGA) of a system on chip (SoC), while the classifier was implemented in its ARM core. Many different options were realized and analyzed, depending on their complexity and performance in terms of resource usage and accuracy of classification. A classification accuracy of over 94% was achieved for a set of 42 different classes. The proposed approach is intended for developing architectures with preprocessing on the embedded FPGA of smart tactile sensors, to obtain high performance in real-time complex robotic systems.

Soft-core processor integration based on different instruction set architectures and field programmable gate array custom datapath implementation.

One of the fundamental requirements of a real-time system (RTS) is the need to guarantee re-al-time determinism for critical tasks. Task execution rates, operating system (OS) overhead, and task context switching times are just a few of the parameters that can cause jitter and missed deadlines in RTS with soft schedulers. Control systems that are susceptible to jitter can be used in the control of HARD RTS as long as the cumulative value of periodicity deviation and worst-case response time is less than the response time required by that application. This artcle presents field-programmable gate array (FPGA) soft-core processors integration based on different instruction set architectures (ISA), custom central processing unit (CPU) datapath, dedicated hardware thread context, and hardware real-time operating system (RTOS) implementations. Based on existing work problems, one parameter that can negatively influence the performance of an RTS is the additional costs due to the operating system. The scheduling and thread context switching operations can significantly degrade the programming limit for RTS, where the task switching frequency is high. In parallel with the improvement of software scheduling algorithms, their implementation in hardware has been proposed and validated to relieve the processor of scheduling overhead and reduce RTOS-specific overhead.

High-precision plant height measurement by drone with RTK-GNSS and single camera for real-time processing

Conventional crop height measurements performed using aerial drone images require 3D reconstruction results of several aerial images obtained through structure from motion. Therefore, they require extensive computation time and their measurement accuracy is not high; if the 3D reconstruction result fails, several aerial photos must be captured again. To overcome these challenges, this study proposes a high-precision measurement method that uses a drone equipped with a monocular camera and real-time kinematic global navigation satellite system (RTK-GNSS) for real-time processing. This method performs high-precision stereo matching based on long-baseline lengths (approximately 1 m) during the flight by linking the RTK-GNSS and aerial image capture points. As the baseline length of a typical stereo camera is fixed, once the camera is calibrated on the ground, it does not need to be calibrated again during the flight. However, the proposed system requires quick calibration in flight because the baseline length is not fixed. A new calibration method that is based on zero-mean normalized cross-correlation and two stages least square method, is proposed to further improve the accuracy and stereo matching speed. The proposed method was compared with two conventional methods in natural world environments. It was observed that error rates reduced by 62.2% and 69.4%, for flight altitudes between 10 and 20 m respectively. Moreover, a depth resolution of 1.6 mm and reduction of 44.4% and 63.0% in the error rates were achieved at an altitude of 4.1 m, and the execution time was 88 ms for images with a size of 5472 × 3468 pixels, which is sufficiently fast for real-time measurement.

Implementation of multiple controllers for context-inspired collaboration between human and robot by integrating the uni-axial motion and real-time operating system

In the era of connected things, it is necessary that agents do not work in isolation instead of collaboration with others, particularly humans in the workplace. However, the success of such cooperation depends on the specific context in which humans and robots interact. To address this challenge, a novel design of the context-aware system that enables effective collaboration between humans and robots is introduced. The architecture of this system consists of multiple controllers which handle the uni-axial motion. Then, the hardware components and their connections are described in detail so that each controller can be associated with the others. Later, the software design included a mutual interface, data flow, computational algorithms, and state machine. The practical platform has been developed that realizes several motion profiles, delivering smooth performance and advanced results. Different laboratory tests have been completed for the validations of our approach. In each context, the motion performance of the proposed method is achieved and analyzed. These results show that our approach is practical, feasible, and proper for various contexts in human-robot collaboration for Industry 4.0.

Hi-ROS: Open-source multi-camera sensor fusion for real-time people tracking

This paper presents Hi-ROS (Human Interaction in ROS), an open source framework focused on real-time accurate assessment of human motion. The system offers a series of tools to track multiple people in real-time by exploiting a calibrated camera network. No assumptions are made about the typology or number of cameras, nor about the body pose estimation algorithm used to extract the 3D poses of the people in the scene. The tools provided by Hi-ROS include a Skeleton Tracker to ensure temporal consistency of the detected poses, a Skeleton Merger to fuse the tracks from multiple cameras, thus limiting flickering phenomena, a Skeleton Optimizer to ensure limb length consistency, and a Skeleton Filter to perform real-time smoothing of the detected joint trajectories. Accuracy, tracking robustness, and real-time performance of the proposed system were evaluated on a public dataset, containing both single-person and multi-person sequences with up to 4 people interacting. The results obtained using different subsets of the proposed tools show how the complete Hi-ROS pipeline provides accurate and reliable estimates also in challenging scenarios, with a reduction of the RMSE of up to 27% with respect to a pure tracking approach. This work aims to push forward the development of unobtrusive human–robot interaction applications, multi-person automated posture analyses, rehabilitation performance assessments, and any possible application enabled by real-time accurate assessment of human motion via markerless motion capture.

Using real-time operating system to control the recycling waste system in beverage industry for circular economy: Mechanical approach

Waste in the beverage industry is characterized by greater amount of products-specific waste such as plastic or equivalent. This means that the generation of these wastes is not only unavoidable, but also recycling as much as possible. They regularly entail so many years in order to self-decompose in natural conditions. Therefore, the reuse of waste products in the beverage industry has been raised in our era. In this paper, the long-term solution to recycle the plastic water can is investigated to satisfy the ultimate goal for sustainable development. Initially, several theories in mechanics have been deployed to carry out mathematical expressions. Generally, there are three chief stages in this design, for instance belt conveyor and lid opening mechanism, turnover mechanism and cleaning mechanism. Using the computational software, preliminary data is discovered to delimitate 3D model in computer. In this work, the control challenge is to handle multi-process from different mechanisms at the same time. The real-time operating system that named as FreeRTOS, is recommended to implement into hardware level. Each mechanism is managed by one process and monitored by central micro-processor. Global planning scheme plays a role as task scheduler to ensure the continuous operation. To validate the effectiveness and correctness of our design, an experimental platform has been launched. Several tests to manipulate this system in real-world environment are completely done. From these results, it could be seen clearly that the proposed design is effective, feasible and proper for a green solution on our planet.

A Measuring Tool for Interrupt Latency Based on Linux

Linux has been used more and more in the field of industrial control and robot control, the non-real-time nature of the final human-computer interaction program may cause system performance impact in industrial production. Interrupt response time is an important indicator to measure the real-time performance of real-time operating systems. Accurate test data can be used as an evaluation index for selecting the real-time performance of the operating system. In order to improve the real-time performance of Linux, analyze real-time problems, and solve possible real-time bottlenecks, this paper designs a probe-type tool that can measure the interrupt response time, output the interrupt call stack, and analyze the reasons for high interrupts. By optimizing specific limited and high-latency paths, Linux can still maintain an acceptable range of latency for a long time under various high-load scenarios. After testing, the tool has good measurement accuracy, which provides a basis for developing real-time tasks and solving real-time problems.