Real-time Constraints Research Articles

Efficient Optimization of Control Libraries

A popular approach to high dimensional control problems in robotics uses a library of candidate “maneuvers” or “trajectories”. The library is either evaluated on a fixed number of candidate choices at runtime (e.g. path set selection for planning) or by iterating through a sequence of feasible choices until success is achieved (e.g. grasp selection). The performance of the library relies heavily on the content and order of the sequence of candidates. We propose a provably efficient method to optimize such libraries, leveraging recent advances in optimizing submodular functions of sequences. This approach is demonstrated on two important problems: mobile robot navigation and manipulator grasp set selection. In the first case, performance can be improved by choosing a subset of candidates which optimizes the metric under consideration (cost of traversal). In the second case, performance can be optimized by minimizing the depth in the list that is searched before a successful candidate is found. Our method can be used in both on-line and batch settings with provable performance guarantees, and can be run in an anytime manner to handle real-time constraints.

LATTE: L STM Self- Att ention based Anomaly Detection in E mbedded Automotive Platforms

Modern vehicles can be thought of as complex distributed embedded systems that run a variety of automotive applications with real-time constraints. Recent advances in the automotive industry towards greater autonomy are driving vehicles to be increasingly connected with various external systems (e.g., roadside beacons, other vehicles), which makes emerging vehicles highly vulnerable to cyber-attacks. Additionally, the increased complexity of automotive applications and the in-vehicle networks results in poor attack visibility, which makes detecting such attacks particularly challenging in automotive systems. In this work, we present a novel anomaly detection framework called LATTE to detect cyber-attacks in Controller Area Network (CAN) based networks within automotive platforms. Our proposed LATTE framework uses a stacked Long Short Term Memory (LSTM) predictor network with novel attention mechanisms to learn the normal operating behavior at design time. Subsequently, a novel detection scheme (also trained at design time) is used to detect various cyber-attacks (as anomalies) at runtime. We evaluate our proposed LATTE framework under different automotive attack scenarios and present a detailed comparison with the best-known prior works in this area, to demonstrate the potential of our approach.

SAGE: A Split-Architecture Methodology for Efficient End-to-End Autonomous Vehicle Control

Autonomous vehicles (AV) are expected to revolutionize transportation and improve road safety significantly. However, these benefits do not come without cost; AVs require large Deep-Learning (DL) models and powerful hardware platforms to operate reliably in real-time, requiring between several hundred watts to one kilowatt of power. This power consumption can dramatically reduce vehicles’ driving range and affect emissions. To address this problem, we propose SAGE: a methodology for selectively offloading the key energy-consuming modules of DL architectures to the cloud to optimize edge, energy usage while meeting real-time latency constraints. Furthermore, we leverage Head Network Distillation (HND) to introduce efficient bottlenecks within the DL architecture in order to minimize the network overhead costs of offloading with almost no degradation in the model’s performance. We evaluate SAGE using an Nvidia Jetson TX2 and an industry-standard Nvidia Drive PX2 as the AV edge, devices and demonstrate that our offloading strategy is practical for a wide range of DL models and internet connection bandwidths on 3G, 4G LTE, and WiFi technologies. Compared to edge-only computation, SAGE reduces energy consumption by an average of 36.13% , 47.07% , and 55.66% for an AV with one low-resolution camera, one high-resolution camera, and three high-resolution cameras, respectively. SAGE also reduces upload data size by up to 98.40% compared to direct camera offloading.

Interpreted synchronous extension of time Petri nets

Our work is integrated into a global methodology to design synchronously executed embedded critical systems. It is used for the development of medical devices implanted into human body to perform functional electrical stimulation solutions (used in pacemakers, deep brain stimulation...). These systems are of course critical and real time, and the reliability of their behaviors must be guaranteed. These medical devices are implemented into a programmable logic circuit in a synchronous way, which allows efficient implementation (space, consumption and actual parallelism of tasks execution). This paper presents a solution that helps to prove that the behavior of the implemented system respects a set of properties, using Petri nets for modeling and analysis purposes. But one problem in formal methods is that the hardware target and the implementation strategy can have an influence on the execution of the system, but is usually not considered in the modeling and verification processes. Resolving this issue is the goal of this article. Our work has two main results: an operational one, and a theoretical one. First, we can now design critical controllers with hard safety or real time constraints, being sure the behavior is still guaranteed during the execution. Second, this work broadens the scope of expressivity and analyzability of Petri nets extensions. Until then, none managed in the same formalism, both for modeling and analysis, all the characteristics we have considered (weights on arcs, specific test and inhibitor arcs, interpretation, and time intervals, including the management of effective conflicts and the blocking of transitions).

Design and Development of a Wearable Assistive Device Integrating a Fuzzy Decision Support System for Blind and Visually Impaired People.

In this article, a new design of a wearable navigation support system for blind and visually impaired people (BVIP) is proposed. The proposed navigation system relies primarily on sensors, real-time processing boards, a fuzzy logic-based decision support system, and a user interface. It uses sensor data as inputs and provides the desired safety orientation to the BVIP. The user is informed about the decision based on a mixed voice–haptic interface. The navigation aid system contains two wearable obstacle detection systems managed by an embedded controller. The control system adopts the Robot Operating System (ROS) architecture supported by the Beagle Bone Black master board that meets the real-time constraints. The data acquisition and obstacle avoidance are carried out by several nodes managed by the ROS to finally deliver a mixed haptic–voice message for guidance of the BVIP. A fuzzy logic-based decision support system was implemented to help BVIP to choose a safe direction. The system has been applied to blindfolded persons and visually impaired persons. Both types of users found the system promising and pointed out its potential to become a good navigation aid in the future.

QoS-Based Service-Time Scheduling in the IoT-Edge Cloud.

In edge computing, scheduling heterogeneous workloads with diverse resource requirements is challenging. Besides limited resources, the servers may be overwhelmed with computational tasks, resulting in lengthy task queues and congestion occasioned by unusual network traffic patterns. Additionally, Internet of Things (IoT)/Edge applications have different characteristics coupled with performance requirements, which become determinants if most edge applications can both satisfy deadlines and each user’s QoS requirements. This study aims to address these restrictions by proposing a mechanism that improves the cluster resource utilization and Quality of Service (QoS) in an edge cloud cluster in terms of service time. Containerization can provide a way to improve the performance of the IoT-Edge cloud by factoring in task dependencies and heterogeneous application resource demands. In this paper, we propose STaSA, a service time aware scheduler for the edge environment. The algorithm automatically assigns requests onto different processing nodes and then schedules their execution under real-time constraints, thus minimizing the number of QoS violations. The effectiveness of our scheduling model is demonstrated through implementation on KubeEdge, a container orchestration platform based on Kubernetes. Experimental results show significantly fewer violations in QoS during scheduling and improved performance compared to the state of the art.

Modified lightning search algorithm for optimization

This manuscript proposes a new metaheuristic optimization algorithm for solving the problems related to constraint optimization. The proposed algorithm is based on a nature-inspired algorithm like a lightning search algorithm and differential evolution algorithm. The proposed algorithm consists of two stages. In the first/exploitation/local stage, the fitness values are calculated by using a lightning search algorithm, the inclusion of weight factor in the lightning search algorithm, and differential evolution algorithm using the same population. In the second/global/exploration stage, the minimum value of fitness for each row population by the implementation of the above three algorithms is updated with the population. The first and second stage continues until certain criteria are achieved. The final minimum value is the optimized result. Validation of the proposed methodology is performed by using 60 benchmark functions. The proposed methodology is compared with seven other well-known algorithms namely the lightning search algorithm, differential evolution algorithm cuckoo search algorithm, particle swarm optimization algorithm, seagull optimization algorithm, chimp optimization algorithm, and mayfly algorithm by computing mean, standard deviation, best and worst value for 60 benchmark functions. The proposed methodology is also validated by implementing it in seven real-time constraint optimized problems. Furthermore, the efficiency of the proposed technique has been validated statistically for unimodal and multimodal test functions. The results show a better performance of the proposed optimized algorithm compared to other algorithms.

Efficient stability analysis approaches for nonlinear weakly-hard real-time control systems

This article considers the stability analysis of nonlinear control systems with failures in the feedback loop that satisfy weakly-hard real-time (WHRT) constraints. Such WHRT constraints are window-based guarantees like for example, that the feedback loop is at least closed for n times in each window of m sampling times. WHRT constraints can, e.g., describe the packet loss properties of a communication network. We consider two different actuation strategies to deal with failures in the feedback loop: holding the last received input or setting the input to zero if no new input arrives. We present stability analysis procedures for both strategies. The procedures are based on the emulation of a sampled-data controller from continuous-time feedback. For the hold strategy, we use a graph description for the WHRT constraints and a modified version of the Viterbi algorithm to bound a Lyapunov function candidate. For the zero strategy, we employ a bound on the evolution of the Lyapunov function candidate for zero input. For both strategies, efficient stability analysis procedures based on non-monotonic Lyapunov functions are derived. In addition, we demonstrate how switched controllers, that switch depending on the past dropout sequence, can be used for the hold strategy. Such switched controllers can significantly increase the maximum allowable sampling period for the hold strategy. The proposed approaches are illustrated with a numerical example.

Real-Time Motion Planning with Dynamic Obstacles

Robust robot motion planning in dynamic environments requires that actions be selected under real-time constraints. Existing heuristic search methods that can plan high-speed motions do not guarantee real-time performance in dynamic environments. Existing heuristic search methods for real-time planning in dynamic environments fail in the high-dimensional state space required to plan high-speed actions. In this paper, we present extensions to a leading planner for high-dimensional spaces, R*, that allow it to guarantee real-time performance, and extensions to a leading real-time planner, LSS-LRTA*, that allow it to succeed in dynamic motion planning. In an extensive empirical comparison, we show that the new methods are superior to the originals, providing new state-of-the-art search performance on this challenging problem.

A classic critique on concrete adsorbing pollutants emitted by automobiles and statistical envision using trend analysis.

Globally, road transportation is responsible for about 1/3 of total air pollution, among which Co2 , NoX and SoX are major by volume which are directly responsible for ozone layer depletion. As these activities continue to rise, the nature faces the threat of an unprecedented environmental catastrophe. In this survey, the adsorbent and the methods used to adsorb the vehicle emission pollutant directly from the ambient air and enhance the air quality are reviewed. There are extensive number of adsorbents available namely titanium oxide, polyethyleneimine (PEI), activated carbon, and other natural admixtures and methods that adsorb these pollutants such as Co2; specifically, polyethyleneimine (PEI) is a polymer-based product which has a behavior of adsorbing Co2 directly from ambient air. A carbon-neutral technology for eliminating anthropogenic CO2 emissions has been proposed, trapping CO2 from the atmosphere. The major concern including Co2 adsorption rate and methods to evaluate the volume of adsorption with time has been reviewed. Trend analysis depends on the reason that can foresee what will occur later on by seeing what has happened beforehand. The trend analysis method was used in this study to model the adsorption rate of PEI based on their pore diameter. When it comes to designing regression analysis, R2 values of 0.2 to 0.3 indicate that statistical tests have no meaningful impact and further research is needed. For adsorption of NoX, a lot of adsorbents are used namely sodium bentonite, zeolite, activated carbon, and other natural minerals. Among them, activated carbon enhances the adsorption rate of Nox. Vast research has been conducted to analyze the adsorption rate, advantages, disadvantages, and the behavior of different adsorbent with the concrete. Literally for SOx, the adsorbents that are widely used are coal ash, copper oxide, and silicate. The characterization of different adsorbents with their adsorbate can be analyzed by some test methods as follows: Fourier-transform infrared spectroscopy (FTIR), BET analysis, scanning electron microscopy (SEM), field emission scanning electron microscopy (FESEM) analysis, surface area, X-ray diffraction (XRD), porosity analysis, and more according to the requirement. The quantity of adsorbent and adsorbate could be evaluated for facing the real-time constraints through a detailed research by review.

Tactics Overview for Implementing High-Performance Computing on Embedded Platforms

Future space missions will rely on novel high-performance computing to support advanced intelligent on-board algorithms with substantial workloads that mandates firm real-time and power constraints requirements. Consequently, these advanced algorithms require significantly faster processing beyond the conventional space-grade central processing unit capabilities. Moreover, they require careful selection of the target embedded platform from a diverse set of available architectures along with several implementation tactics to map the algorithms to the target architecture to fully unlock its capabilities. In this paper, we present a study of different architectures and embedded computing platforms for the satellite on-board computers. Moreover, we present a comprehensive overview of recent implementation tactics such as source code mapping and transformations. Additionally, we highlight some optimization techniques such as partitioning and co-designing using hardware accelerators. Finally, we discuss several implementation analysis methodologies to derive optimized code implementations. The top ranked YOLO-v3, as a deep learning based object detection algorithm, is selected as a case study model to be optimized using OpenVINO toolkit. The experimental results show an improvement ratios up to 73%, 41%, and 34% in terms of frames per second, CPU utilization, and cache memory, respectively. The study presented in this paper aims to guide the researchers in the field of high performance embedded computing in terms of different hardware architectures along with several implementation tactics.

ADBench: benchmarking autonomous driving systems

Driven by the improvements in a variety of domains, autonomous driving is becoming a reality and today, industry aims at moving toward fully autonomous vehicles. High-tech chip manufacturers are designing high-performance and energy-efficient platforms in accordance with safety standard requirements. However, the software used to implement advanced functionalities in autonomous vehicles challenges real-time constraints on those platforms. Hence, there is a clear need for industry-level autonomous driving benchmarks to evaluate platforms and systems. In this paper, we propose ADBench, a benchmarking approach and benchmark suite for state-of-the-art autonomous driving platforms, in accordance with the key modules, structural design and functions of AD systems, building on several industry-level autonomous driving systems. The use of standard benchmarks facilitates the design, verification and validation process of autonomous systems.

Artificial Intelligence and Data Fusion at the Edge

Artificial intelligence (AI), owing to recent breakthroughs in deep learning, has revolutionized applications and services in almost all technology domains including aerospace. AI and deep learning rely on huge amounts of training data that are mostly generated at the network edge by Internet of Things (IoT) devices and sensors. Bringing the sensed data from the edge of a distributed network to a centralized cloud is often infeasible because of the massive data volume, limited network bandwidth, and real-time application constraints. Consequently, there is a desire to push AI frontiers to the network edge toward utilizing the enormous amount of data generated by IoT devices near the data source. The merger of edge computing and AI has engendered a new discipline, that is, AI at the edge or edge intelligence. To help AI make sense of gigantic data at the network edge, data fusion is of paramount significance and goes hand in hand with AI. This article focuses on data fusion and AI at the edge. In this article, we propose a framework for data fusion and AI processing at the edge. We then provide a comparative discussion of different data fusion and AI models and architectures. We discuss multiple levels of fusion and different types of AI, and how different types of AI align with different levels of fusion. We then highlight the benefits of combining data fusion with AI at the edge. The methods of AI and data fusion at the edge detailed in this article are applicable to many application domains including aerospace systems. We evaluate the effectiveness of combined data fusion and AI at the edge using convolutional neural network models and multiple hardware platforms suitable for edge computing. Experimental results reveal that combining AI with data fusion can impart a speedup of 9.8× while reducing energy consumption up to 88.5% over AI without data fusion. Furthermore, results demonstrate that data fusion either maintains or improves the accuracy of AI in most cases. For our experiments, data fusion imparts a maximum improvement of 15.8% in accuracy to AI.

Heterogeneity-Aware Scheduling on SoCs for Autonomous Vehicles

Fully autonomous vehicles (AVs) must meet stringent real-time performance and safety-criticality constraints of multiple applications simultaneously in highly dynamic environmental conditions. To enable such a system, carefully selected accelerators and general-purpose cores on Systems-on-Chips (SoCs) are required. However, schedulers that are agnostic to this heterogeneity lead to inefficient utilization of hardware resources and increase the time to complete the AV's mission. As a solution, our letter proposes a heterogeneity-aware, multi-level scheduler called HetSched. HetSched leverages run-time information about the underlying heterogeneous SoC, along with the applications’ real-time constraints to improve an AV's mission time. Our evaluation demonstrates a reduction in mission time of 2.0-4.8×, when compared against state-of-the-art (SOTA) schedulers. Furthermore, when used as part of an SoC design space exploration loop, in comparison to the SOTA schedulers, HetSched reduces the number of accelerators required by an SoC to safely complete the AV's mission by 1.9× on average.

Fast Heuristic Search for RTS Game Combat Scenarios

Heuristic search has been very successful in abstract game domains such as Chess and Go. In video games, however, adoption has been slow due to the fact that state and move spaces are much larger, real-time constraints are harsher, and constraints on computational resources are tighter. In this paper we present a fast search method — Alpha-Beta search for durative moves— that can defeat commonly used AI scripts in RTS game combat scenarios of up to 8 vs. 8 units running on a single core in under 5ms per search episode. This performance is achieved by using standard search enhancements such as transposition tables and iterative deepening, and novel usage of combat AI scripts for sorting moves and state evaluation via playouts. We also present evidence that commonly used combat scripts are highly exploitable — opening the door for a promising line of research on opponent combat modelling.

Hierarchical Adversarial Search Applied to Real-Time Strategy Games

Real-Time Strategy (RTS) video games have proven to be a very challenging application area for artificial intelligence research. Existing AI solutionsare limited by vast state and action spaces and real-time constraints. Most implementations efficiently tackle various tactical or strategic sub-problems, but there is no single algorithm fast enough to be successfully applied to big problem sets (such as a complete instance of the StarCraft RTS game). This paper presents a hierarchical adversarial search framework which more closely models the human way of thinking --- much like the chain of command employed by the military. Each level implements a different abstraction --- from deciding how to win the game at the top of the hierarchy to individual unit orders at the bottom. We apply a 3-layer version of our model to SparCraft ---a StarCraft combat simulator --- and show that it outperforms state of the art algorithms such as Alpha-Beta, UCT, and Portfolio Search in large combat scenarios featuring multiple bases and up to 72 mobile units per player under real-time constraints of 40 ms per search episode.

Exploiting prunability for person re-identification

Recent years have witnessed a substantial increase in the deep learning (DL) architectures proposed for visual recognition tasks like person re-identification, where individuals must be recognized over multiple distributed cameras. Although these architectures have greatly improved the state-of-the-art accuracy, the computational complexity of the convolutional neural networks (CNNs) commonly used for feature extraction remains an issue, hindering their deployment on platforms with limited resources, or in applications with real-time constraints. There is an obvious advantage to accelerating and compressing DL models without significantly decreasing their accuracy. However, the source (pruning) domain differs from operational (target) domains, and the domain shift between image data captured with different non-overlapping camera viewpoints leads to lower recognition accuracy. In this paper, we investigate the prunability of these architectures under different design scenarios. This paper first revisits pruning techniques that are suitable for reducing the computational complexity of deep CNN networks applied to person re-identification. Then, these techniques are analyzed according to their pruning criteria and strategy and according to different scenarios for exploiting pruning methods to fine-tuning networks to target domains. Experimental results obtained using DL models with ResNet feature extractors, and multiple benchmarks re-identification datasets, indicate that pruning can considerably reduce network complexity while maintaining a high level of accuracy. In scenarios where pruning is performed with large pretraining or fine-tuning datasets, the number of FLOPS required by ResNet architectures is reduced by half, while maintaining a comparable rank-1 accuracy (within 1% of the original model). Pruning while training a larger CNNs can also provide a significantly better performance than fine-tuning smaller ones.

An Analysis of Model-Based Heuristic Search Techniques for StarCraft Combat Scenarios

Real-Time Strategy games have become a popular test-bed for modern AI system due to their real-time computational constraints, complex multi-unit control problems, and imperfect information. One of the most important aspects of any RTS AI system is the efficient control of units in complex combat scenarios, also known as micromanagement. Recently, a model-based heuristic search technique called Portfolio Greedy Search (PGS) has shown promisingpaper we present the first integration of PGS into the StarCraft game engine, and compare its performance to the current state-of-the-art deep reinforcement learning method in several benchmark combat scenarios. We then perform theperformance for providing real-time decision making in RTS combat scenarios, but has so far only been tested in SparCraft: an RTS combat simulator. In this same experiments within the SparCraft simulator in order to investigate any differences between PGS performance in the simulator and in the actual game. Lastly, we investigate how varying parameters of the SparCraft simulator affect the performance of PGS in the StarCraft game engine. We demonstrate that the performance of PGS relies heavily on the accuracy of the underlying model, outperforming other techniques only for scenarios where the SparCraft simulation model more accurately matches the StarCraft game engine.

An efficient and scalable parallel mapping of pulse-Doppler radar signal processing chain on a multi-core DSP

Digital Pulse-Doppler radar chain consists of signal processing algorithms that require high computing power. Multi-processor and multi-core parallel embedded machines are one of the solutions to meet real-time constraints of many radar applications. In this paper, we proposed efficient and scalable parallelization methods of the Pulse-Doppler radar signal processing chain. First, we evaluated Open Multi Processing (OpenMP) to identify its best scheduling technique in order to exploit efficiently the available computing cores. Then, we have proposed new parallel and scalable approaches based on direct memory access (DMA) and inter-processor communication (IPC) techniques, combined with the best OpenMP scheduling method to accelerate radar signal processing chain. To prove the scalability of our proposed parallel approaches, two radar use cases with different real-time and memory constraints have been experienced. We used the eight cores C6678 digital signal processor (DSP) as a target for all our implementations. The obtained results show an overall parallel efficiency of 95%, which is better than the best state-of-the-art implementations.

Total Cray Valley and Gevo to further scale up development of renewable isoamylene from fusel oil

Contemporary embedded systems work in changing environments, some features (e.g., execution time, power consumption) of the system are often not completely predictable. Therefore, for systems with strong constraints, a worst-case design is applied. We observed that by enabling the self-adaptivity we may obtain highly optimized systems still guaranteeing the high quality of service. This paper presents a method of synthesis of real-time software for self-adaptive multicore systems. The method assumes that the system specification is given as a task graph. Then, the tasks are scheduled on a multicore architecture consisting of low-power and high-performance cores. We apply the developmental genetic programming to generate the self-adaptive scheduler and the initial schedule. The initial schedule is optimized, taking into consideration the power consumption, the real-time constraints as well as the self-adaptivity. The scheduler modifies the schedule during the system execution, whenever execution time of the recently finished task occurs other than assumed during the initial scheduling. We propose two models of self-adaptivity: self-optimization of power consumption and self-adaptivity of real-time scheduling. We present some experimental results for standard benchmarks, showing the advantages of our method in comparison with the worst case design used in existing approaches.