Real-time Performance Constraints Research Articles

Real-time motion estimation based video steganography with preserved consistency and local optimality

AbstractThis paper introduces a novel steganographic technique for H.264 video that achieves an outstanding performance against state-of-the-art steganalysis techniques while maintaining real-time encoding performance constraints. The proposed technique embeds the secret message by altering the motion vectors (MVs) while preserving their local optimality and consistency feature to withstand the recently emerged steganalysis methods. Thanks to its macro-block (MB) basis architecture, the proposed technique satisfies the real-time constraints, eliminating the need to wait for the whole frame or group of pictures (GOP) and avoiding the need to perform any additional re-encoding step(s). Additionally, altering the MVs is performed in the motion estimation (ME) sub-pixel-refinement stage through a rule-based scheme that ensures each MB’s compatibility for embedding without detection by the aforementioned steganalysis methods. The proposed technique is integrated with the OpenH264 real-time video encoder and evaluated on widely used video sequences. The results prove that the proposed technique achieves a significant security performance against the steganalysis methods while maintaining an acceptable embedding rate, outperforming other state-of-the-art MV-based steganographic methods in real-time constrained environments. The proposed technique adds about $$1-2\%$$ 1 - 2 % overhead beyond the encoder running time. The source code is publicly available here: https://github.com/HassanMohamedGit/OpenH264-RealTime-steg.

Current status and future prospects of SLAM technology in the field of blind navigation

This comprehensive review paper explores the progressive integration of Simultaneous Localization and Mapping (SLAM) technology within the realm of blind navigation. The study delves into the current landscape of research and practical implementations where SLAM techniques are harnessed to empower visually impaired individuals in navigating their surroundings. The paper takes a focused approach by elucidating the advancements achieved through RGB-D-based SLAM navigation systems and the intriguing developments within the realm of LSD-SLAM-based navigation. A paramount facet of this exploration involves the design and deployment of interactive devices that bridge the gap between blind individuals and navigation systems. Notably, the paper examines the intricacies of these interaction interfaces, highlighting their pivotal role in enhancing the efficacy and user-friendliness of SLAM-driven navigation. Furthermore, this review critically assesses the existing limitations that persist within the current application of SLAM technology for blind navigation. The discussion encompasses challenges related to environmental diversity, real-time processing constraints, and the need for robustness in complex scenarios. By acknowledging these limitations, the paper paves the way for a comprehensive examination of potential avenues for refinement and enhancement. In conclusion, this paper serves as a comprehensive guide to the progressive incorporation of SLAM technology into blind navigation. It underscores the transformative potential of RGB-D and LSD-SLAM systems, underscores the significance of interaction interfaces, and underscores the evolving trajectory towards a more inclusive and efficient navigation solution for the visually impaired.

SP-YOLOv8s: An Improved YOLOv8s Model for Remote Sensing Image Tiny Object Detection

An improved YOLOv8s-based method is proposed to address the challenge of accurately recognizing tiny objects in remote sensing images during practical human-computer interaction. In detecting tiny targets, the accuracy of YOLOv8s is low because the downsampling module of the original YOLOv8s algorithm causes the network to lose fine-grained feature information, and the neck network feature information needs to be sufficiently fused. In this method, the strided convolution module in YOLOv8s is replaced with the SPD-Conv module. By doing so, the feature map undergoes downsampling while preserving fine-grained feature information, thereby improving the learning and expressive capabilities of the network and enhancing recognition accuracy. Meanwhile, the path aggregation network is substituted with the SPANet structure, which facilitates the acquisition of more prosperous gradient paths. This substitution enhances the fusion of feature maps at various scales, reduces model parameters, and further improves detection accuracy. Additionally, it enhances the network’s robustness to complex backgrounds. Experimental verification is conducted on the following two intricate datasets containing tiny objects: AI-TOD and TinyPerson. A comparative analysis with the original YOLOv8s algorithm reveals notable enhancements in recognition accuracy. Specifically, under real-time performance constraints, the proposed method yields a 4.9% and 9.1% improvement in mAP0.5 recognition accuracy for AI-TOD and TinyPerson datasets, respectively. Moreover, the recognition accuracy for mAP0.5:0.95 is enhanced by 3.4% and 3.2% for the same datasets, respectively. The results indicate that the proposed method enables rapid and accurate recognition of tiny objects in complex backgrounds. Furthermore, it demonstrates better recognition precision and stability than other algorithms, such as YOLOv5s and YOLOv8s.

Optimized Algorithms and Hardware Implementation of Median Filter for Image Processing

Image processing algorithms are essential for clarifying the image and improving the ability to recognize distinct characteristics of the image. The field of digital image processing is widespread in several research and technology applications. In many of these applications, the existence of impulsive noise in the obtained images is one of the most frequent problems. The median filter is a strong method to remove the impulsive noise; it effectively eliminates salt and pepper noise from the image. The main target of this paper is to investigate efficient median filter units to be connected to a general-purpose processor (GPP) for FPGA-based embedded systems. The paper exposes three novel techniques, two of them specially for median filtering techniques and the third one is used to get the maximum number of any 9 elements array. The proposed algorithms are inspired by the Median Of Median (MOM) algorithm. The first two techniques are tested for filtering 3 times 3 image windows and optimized for producing the expected result in high accuracy, short time, and reduced number of comparisons. The last technique is tested for a 9 elements array for extracting the maximum number in same high efficiency manner. Furthermore, the three proposed techniques are implemented leveraging the advantage of the parallel processing and the FPGA flexible resources to satisfy the real-time processing constraints. A comparison between the first two proposed filtering units and their counterparts in the literature is included. The comparison reveals the superiority of the first technique in terms of accuracy with fewer comparators than previously published techniques. Besides, the paper illustrates how the concept beyond the proposed techniques can be used to perform the maximum pooling for convolution neural networks.

Optimized Piccolo Lightweight Block Cipher: Area Efficient Implementation

Piccolo algorithm is one of the lightweight block ciphers designed specifically for low-resource devices which present physical constraints in terms of area, power, and memory. Various hardware architectures for Piccolo block cipher have been proposed in recent years with the aim of obtaining a more appropriate low-resource design for specific constrained applications. The latter must meet real-time processing constraints without affecting the need for hardware resources. Finding a good compromise between computation time and implementation resource consumption is a major consideration in the design process. In this paper, we suggest six serial hardware architectures for Piccolo lightweight algorithm with a 128 bits key length. Proposed architectures are compared to existing designs based on hardware resource occupancy, latency, and throughput. Also, we tested the security of the Piccolo algorithm, and the obtained results show the good robustness of the Piccolo block cipher against statistical attacks. Thus, we can use the Piccolo algorithm in lightweight applications that require a high level of privacy.

Heterogeneity-Aware Scheduling on SoCs for Autonomous Vehicles

Fully autonomous vehicles (AVs) must meet stringent real-time performance and safety-criticality constraints of multiple applications simultaneously in highly dynamic environmental conditions. To enable such a system, carefully selected accelerators and general-purpose cores on Systems-on-Chips (SoCs) are required. However, schedulers that are agnostic to this heterogeneity lead to inefficient utilization of hardware resources and increase the time to complete the AV's mission. As a solution, our letter proposes a heterogeneity-aware, multi-level scheduler called HetSched. HetSched leverages run-time information about the underlying heterogeneous SoC, along with the applications’ real-time constraints to improve an AV's mission time. Our evaluation demonstrates a reduction in mission time of 2.0-4.8×, when compared against state-of-the-art (SOTA) schedulers. Furthermore, when used as part of an SoC design space exploration loop, in comparison to the SOTA schedulers, HetSched reduces the number of accelerators required by an SoC to safely complete the AV's mission by 1.9× on average.

Real-Time Vehicle Roll Angle Estimation Based on Neural Networks in IoT Low-Cost Devices.

The high rate of vehicle-crash victims has a fatal economic and social impact in today’s societies. In particular, road crashes where heavy vehicles are involved cause more severe damage because they are prone to rollover. For this reason, many researches are focused on developing RSC Roll Stability Control (RSC) systems. Concerning the design of RSC systems with an adequate performance, it is mandatory to know the dynamics of the vehicle. The main problem arises from the lack of ability to directly capture several required dynamic vehicle variables, such as roll angle, from low-cost sensors. Previous studies demonstrate that low-cost sensors can provide data in real-time with the required precision and reliability. Even more, other research works indicate that neural networks are efficient mechanisms to estimate roll angle. Nevertheless, it is necessary to assess that the fusion of data coming from low-cost devices and estimations provided by neural networks can fulfill hard real-time processing constraints, achieving high level of accuracy during circulation of a vehicle in real situations. In order to address this issue, this study has two main goals: (1) Design and develop an IoT based architecture, integrating ANN in low cost kits with different hardware architectures in order to estimate under real-time constraints the vehicle roll angle. This architecture is able to work under high dynamic conditions, by following specific best practices and considerations during its design; (2) assess that the IoT architecture deployed in low-cost experimental kits achieve the hard real-time performance constraints estimating the roll angle with the required calculation accuracy. To fulfil these objectives, an experimental environment was set up, composed of a van with two set of low-cost kits, one including a Raspberry Pi 3 Model Band the other having an Intel Edison System on Chip linked to a SparkFun 9 Degrees of Freedom module. This experimental environment be tested in different maneuvers for comparison purposes. Neural networks embedded in low-cost sensor kits provide roll angle estimations highly approximated to real values. Even more, Intel Edison and Raspberry Pi 3 Model B have enough computing capabilities to successfully run roll angle estimation based on neural networks to determine rollover risk situations, fulfilling real-time operation restrictions stated for this problem.

Design of Low-Cost Vehicle Roll Angle Estimator Based on Kalman Filters and an Iot Architecture.

In recent years, there have been many advances in vehicle technologies based on the efficient use of real-time data provided by embedded sensors. Some of these technologies can help you avoid or reduce the severity of a crash such as the Roll Stability Control (RSC) systems for commercial vehicles. In RSC, several critical variables to consider such as sideslip or roll angle can only be directly measured using expensive equipment. These kind of devices would increase the price of commercial vehicles. Nevertheless, sideslip or roll angle or values can be estimated using MEMS sensors in combination with data fusion algorithms. The objectives stated for this research work consist of integrating roll angle estimators based on Linear and Unscented Kalman filters to evaluate the precision of the results obtained and determining the fulfillment of the hard real-time processing constraints to embed this kind of estimators in IoT architectures based on low-cost equipment able to be deployed in commercial vehicles. An experimental testbed composed of a van with two sets of low-cost kits was set up, the first one including a Raspberry Pi 3 Model B, and the other having an Intel Edison System on Chip. This experimental environment was tested under different conditions for comparison. The results obtained from low-cost experimental kits, based on IoT architectures and including estimators based on Kalman filters, provide accurate roll angle estimation. Also, these results show that the processing time to get the data and execute the estimations based on Kalman Filters fulfill hard real time constraints.

Model-driven performance prediction of systems of systems

Systems of systems exhibit characteristics that pose difficulty in modelling and predicting their overall performance capabilities, including the presence of operational independence, emergent behaviour, and evolutionary development. When considering systems of systems within the autonomous defence systems context, these aspects become increasingly critical, as constraints on the performance of the final system are typically driven by hard constraints on space, weight and power. System execution modelling languages and tools permit early prediction of the performance of model-driven systems; however, the focus to date has been on understanding the performance of a model rather than determining whether it meets performance requirements, and only subsequently carrying out analysis to reveal the causes of any requirement violations. Moreover, such an analysis is even more difficult when applied to several systems cooperating to achieve a common goal—a system of systems. In this article, we propose an integrated approach to performance prediction of model-driven real-time embedded defence systems and systems of systems. Our architectural prototyping system supports a scenario-driven experimental platform for evaluating model suitability within a set of deployment and real-time performance constraints. We present an overview of our performance prediction system, demonstrating the integration of modelling, execution and performance analysis, and discuss a case study to illustrate our approach.

Dataflow object detection system for FPGA‐based smart camera

Embedded computer vision based smart systems raise challenging issues in many research fields, including real-time vision processing, communication protocols or distributed algorithms. The amount of data generated by cameras using high resolution image sensors requires powerful computing systems to be processed at digital video frame rates. Consequently, the design of efficient and flexible smart cameras, with on-board processing capabilities, has become a key issue for the expansion of smart vision systems relying on decentralised processing at the image sensor node level. In this context, field programmable gate arrays (FPGA)-based platforms, supporting massive data parallelism, offer large opportunities to match real-time processing constraints compared with platforms based on general purpose processors. In this study, the authors describe the implementation, on such a platform, of a configurable object detection application, reformulated according to the dataflow model of computation. The application relies on the computation of the histogram of oriented gradients and a linear SVM-based classification. It is described using the CAPH programming language, allowing efficient hardware descriptions to be generated automatically from high level dataflow specifications without prior knowledge of hardware description languages such as VHDL or Verilog. Results show that the performance of the generated code does not suffer from a significant overhead compared with handwritten HDL code.

Revise and resubmit: How real-time parsing limitations influence grammar acquisition

We present the results from a three-day artificial language learning study on adults. The study examined whether sentence-parsing limitations, in particular, difficulties revising initial syntactic/semantic commitments during comprehension, shape learners’ ability to acquire a language. Findings show that both comprehension and production of morphology pertaining to sentence argument structure are delayed when this morphology consistently appears at the end, rather than at the beginning, of sentences in otherwise identical grammatical systems. This suggests that real-time processing constraints impact acquisition; morphological cues that tend to guide linguistic analyses are easier to learn than cues that revise these analyses. Parallel performance in production and comprehension indicates that parsing constraints affect grammatical acquisition, not just real-time commitments. Properties of the linguistic system (e.g., ordering of cues within a sentence) interact with the properties of the cognitive system (cognitive control and conflict-resolution abilities) and together affect language acquisition.

A Novel No-Reference Video Quality Metric for Evaluating Temporal Jerkiness due to Frame Freezing

In this work, we propose a novel no-reference (NR) video quality metric that evaluates the impact of frame freezing due to either packet loss or late arrival. Our metric uses a trained neural network acting on features that are chosen to capture the impact of frame freezing on the perceived quality. The considered features include the number of freezes, freeze duration statistics, inter-freeze distance statistics, frame difference before and after the freeze, normal frame difference, and the ratio of them. We use the neural network to find the mapping between features and subjective test scores. We optimize the network structure and the feature selection through a cross validation procedure, using training samples extracted from both VQEG and LIVE video databases. The resulting feature set and network structure yields accurate quality prediction for both the training data containing 54 test videos and a separate testing dataset including 14 videos, with Pearson Correlation Coefficients greater than 0.9 and 0.8 for the training set and the testing set, respectively. Our proposed metric has low complexity and could be utilized in a system with realtime processing constraint.

Gamma Gaussian Inverse Wishart Probability Hypothesis Density for Extended Target Tracking Using X-Band Marine Radar Data

X-band marine radar systems represent a flexible and low-cost tool for the tracking of multiple targets in a given region of interest. Although suffering several sources of interference, e.g., the sea clutter, these systems can provide high-resolution measurements, both in space and time. Such features offer the opportunity to get accurate information not only about the target position/motion but also about the targets size. Accordingly, in this paper, we exploit emergent extended target tracking (ETT) methodologies in which the target state, typically position/velocity/acceleration, is augmented with the target length and width. In this paper, we propose an ETT procedure based on the popular probability hypothesis density filter, and in particular, we describe the extended target state through the gamma Gaussian inverse Wishart model. The comparative simplicity of the used models allows us to meet the real-time processing constraint required for the practical surveillance purposes. Real-world data from an experimental and operational campaign, collected during the recovery operations of the Costa Concordia wreckage in October 2013, are used to assess the performance of the proposed target tracking methodology. The full signal processing chain is implemented, and considerations of the experimental results are provided. Important nonideal effects, common to every marine radar, are observed and discussed in relation to the assumptions made for the tracking procedure.

Host-Compiled Multicore System Simulation for Early Real-Time Performance Evaluation

With increasing complexity and software content, modern embedded platforms employ a heterogeneous mix of multicore processors along with hardware accelerators in order to provide high performance in limited power budgets. To evaluate real-time performance and other constraints, full system simulations are essential. With traditional approaches being either slow or inaccurate, so-called source-level or host-compiled simulators have recently emerged as a solution for rapid evaluation of the complete system at early design stages. In such approaches, a faster simulation is achieved by abstracting execution behavior and increasing simulation granularity. However, existing source-level simulators often focus on application behavior only while neglecting the effects of hardware/software interactions and their associated speed and accuracy trade-offs. In this article, we present a host-compiled simulator that emulates software execution in a full-system context. Our simulator incorporates abstract models of both real-time operating systems (RTOSs) and multicore processors to replicate timing-accurate hardware/software interactions and to enable full system cosimulation. An integrated approach for automatic timing granularity adjustment (ATGA) uses observations of the system state to automatically control the timing model and optimally navigate speed versus accuracy conditions. Results as applied to industrial-strength platforms confirm that OS- and system-level effects can significantly contribute to overall accuracy and simulation overhead. By providing careful abstractions, our models can achieve full system simulations at equivalent speeds of more than a thousand MIPS with less than 3% timing error. Coupled with the capability to easily adjust simulation parameters and configurations, this demonstrates the benefits of our simulator for early application development and design space exploration.

Optimization of Processor Architecture for Sobel Real-Time Edge Detection Using FPGA

This paper presents an optimized processor architecture for Sobel edge detection operator on field programmable gate arrays (FPGA). The processor is optimized by the use of several optimization techniques that aim to increase the processor throughput and reduce the processor logic utilization. FPGA offers a high level of parallelism which is exploited by the processor to implement the parallel process of edge detection in order to increase the processor throughput and enable the processor to meet real-time performance constraints. To achieve real-time performance, the proposed processor consists of several Sobel instances that are able to produce massive output pixels in parallel. This parallelism enables data reuse within the processor block which reduces the number of calculations while increasing the processor throughput. Moreover, the processor gains performance with a factor equal to the number of instances contained in the processor block. By the application of the optimization techniques, the proposed Sobel processor is able to meet real-time performance constraints due to its high throughput even with a considerably low clock frequency. In addition, the logic utilization of the proposed processor is low compared to other Sobel processors when implemented on ALTERA Cyclone II DE2-70 board

A Unified FPGA-Based System Architecture for 2-D Discrete Wavelet Transform

This paper presents a novel unified and programmable 2-D Discrete Wavelet Transform (DWT) system architecture, which was implemented using a Field Programmable Gate Array (FPGA)-based Nios II soft-core processor working in combination with custom hardware accelerators generated through high-level synthesis. The proposed system architecture, synthesized on an Altera DE3 Stratix III FPGA board, was developed through an iterative design space exploration methodology using Altera's C2H compiler. Experimental results show that the proposed system architecture is capable of real-time video processing performance for grayscale image resolutions of up to 1920?×?1080 (1080p) when ran on the Altera DE3 board, and it outperforms the existing 2-D DWT architecture implementations known in literature by a considerable margin in terms of throughput. While the proposed 2-D DWT system architecture satisfies real-time performance constraints, it can also perform both forward and inverse DWT, support a number of popular DWT filters used for image and video compression and provide architecture programmability in terms of number of levels of decomposition as well as image width and height. Based from the design principles used to implement the proposed 2-D DWT system architecture, a system design guideline can be formulated for SOC designs which plan to incorporate dedicated 2-D DWT hardware acceleration.

A Parallel Hardware Architecture for Real-Time Object Detection with Support Vector Machines

Object detection applications are often associated with real-time performance constraints that stem from the embedded environment that they are often deployed in. Consequently, researchers have proposed dedicated hardware architectures, utilizing a variety of classification algorithms targeting object detection. Support Vector Machines (SVMs) is among the most popular classification algorithms used in object detection yielding high accuracy rates. However, existing SVM hardware implementations attempting to speed up SVM classification, have either targeted only simple applications, or SVM training. As such, there are limited proposed hardware architectures that are generic enough to be used in a variety of object detection applications. Hence, this paper presents a parallel array architecture for SVM-based object detection, in an attempt to show the advantages, and performance benefits that stem from a dedicated hardware solution. The proposed hardware architecture provides parallel processing, resource sharing among the processing units, and efficient memory management. Furthermore, the size of the array is scalable to the hardware demands, and can also handle a variety of applications such as multiclass classification problems. A prototype of the proposed architecture was implemented on an FPGA platform and evaluated using three popular detection applications, demonstrating real-time performance (40-122 fps for a variety of applications).

Context-Aware Adaptation in DySCAS

DySCAS is a dynamically self-configuring middleware for automotive control systems. The addition of autonomic, context-aware dynamic configuration to automotive control systems brings a potential for a wide range of benefits in terms of robustness, flexibility, upgrading etc. However, the automotive systems represent a particularly challenging domain for the deployment of autonomics concepts, hav- ing a combination of real-time performance constraints, severe resource limitations, safety-critical aspects and cost pressures. For these reasons current systems are stat- ically configured. This paper describes the dynamic run-time configuration aspects of DySCAS and focuses on the extent to which context-aware adaptation has been achieved in DySCAS, and the ways in which the various design and implementation challenges are met.

Design and Optimization of Color Lookup Tables on a Simplex Topology

An important computational problem in color imaging is the design of color transforms that map color between devices or from a device-dependent space (e.g., RGB/CMYK) to a device-independent space (e.g., CIELAB) and vice versa. Real-time processing constraints entail that such nonlinear color transforms be implemented using multidimensional lookup tables (LUTs). Furthermore, relatively sparse LUTs (with efficient interpolation) are employed in practice because of storage and memory constraints. This paper presents a principled design methodology rooted in constrained convex optimization to design color LUTs on a simplex topology. The use of n simplexes, i.e., simplexes in n dimensions, as opposed to traditional lattices, recently has been of great interest in color LUT design for simplex topologies that allow both more analytically tractable formulations and greater efficiency in the LUT. In this framework of n-simplex interpolation, our central contribution is to develop an elegant iterative algorithm that jointly optimizes the placement of nodes of the color LUT and the output values at those nodes to minimize interpolation error in an expected sense. This is in contrast to existing work, which exclusively designs either node locations or the output values. We also develop new analytical results for the problem of node location optimization, which reduces to constrained optimization of a large but sparse interpolation matrix in our framework. We evaluate our n -simplex color LUTs against the state-of-the-art lattice (e.g., International Color Consortium profiles) and simplex-based techniques for approximating two representative multidimensional color transforms that characterize a CMYK xerographic printer and an RGB scanner, respectively. The results show that color LUTs designed on simplexes offer very significant benefits over traditional lattice-based alternatives in improving color transform accuracy even with a much smaller number of nodes.

Dynamic Resource Provisioning in Massively Multiplayer Online Games

Today's Massively Multiplayer Online Games (MMOGs) can include millions of concurrent players spread across the world and interacting with each other within a single session. Faced with high resource demand variability and with misfit resource renting policies, the current industry practice is to overprovision for each game tens of self-owned data centers, making the market entry affordable only for big companies. Focusing on the reduction of entry and operational costs, we investigate a new dynamic resource provisioning method for MMOG operation using external data centers as low-cost resource providers. First, we identify in the various types of player interaction a source of short-term load variability, which complements the long-term load variability due to the size of the player population. Then, we introduce a combined MMOG processor, network, and memory load model that takes into account both the player interaction type and the population size. Our model is best used for estimating the MMOG resource demand dynamically, and thus, for dynamic resource provisioning based on the game world entity distribution. We evaluate several classes of online predictors for MMOG entity distribution and propose and tune a neural network-based predictor to deliver good accuracy consistently under real-time performance constraints. We assess using trace-based simulation the impact of the data center policies on the quality of resource provisioning. We find that the dynamic resource provisioning can be much more efficient than its static alternative even when the external data centers are busy, and that data centers with policies unsuitable for MMOGs are penalized by our dynamic resource provisioning method. Finally, we present experimental results showing the real-time parallelization and load balancing of a real game prototype using data center resources provisioned using our method and show its advantage against a rudimentary client threshold approach.