Autonomous Driving Systems Research Articles

Stochastic MPC Based Attacks on Object Tracking in Autonomous Driving Systems

Decision making in advanced driver assistance systems involves in general the estimated trajectories of the surrounding objects. Multiple object tracking refers to the process of estimating in real time these trajectories, leveraging for this purpose sensors to detect the objects. This paper deals with devising attacks on object tracking in automated vehicles. The vehicle is assumed to have a detection-based object tracking system that relies on multiple sensors and uses an estimator such as a Kalman filter for sensor fusion and state estimation. The attack goal is to modify the object's state estimated by the victim vehicle to put the vehicle in an unsafe situation. This goal is achieved by judiciously perturbing some or all of the sensor outputs corresponding to the object of interest over a desired horizon. A stochastic model predictive control (SMPC) problem is formulated to compute the sequence of perturbations, whereby hard constraints on the perturbations and probabilistic chance constraints on the object's state are imposed. The chance constraints ensure that some desired conditions for a successful attack are satisfied with a prespecified probability. Reasonable assumptions are then made to obtain a computationally tractable linear SMPC program. The approach is demonstrated on an adaptive cruise control system in a simulation environment, where successful sequential attacks are generated, leading the victim vehicle into dangerous driving situations including collisions.

Ruling the Operational Boundaries: A Survey on Operational Design Domains of Autonomous Driving Systems

Automated driving systems (ADS) have the potential to offer a safe and efficient future for mobility. At the beginning of the design process of an ADS, the operational limits have to be defined using the operational design domain (ODD). Nonetheless, the field of ODD has only become popular recently, and the necessary regulations, standards, and development approaches are still emerging. Current research contributions in the domain of ODD have fragmented in recent years and have not followed a concrete direction. This survey examines a large proportion of the recent research in the domain of ODD by systematically identifying subject areas and categorising relevant publications, thereby integrating the approaches and showing an overview of the emerging topic area. Furthermore, it identifies existing gaps in the ODD research that need to be considered. Finally, the paper suggest relevant future development of ODD.

Efficient Attention-Convolution Feature Extractor in Semantic Segmentation for Autonomous Driving Systems

Efficient Attention-Convolution Feature Extractor in Semantic Segmentation for Autonomous Driving Systems

Quantitative Analysis of the Impact of Baidu Apollo Parameterization on Trajectory Planning in a Critical Scenario

The increasing demand for reliable and safe high-driving automation algorithms in Autonomous Driving (AD) vehicles has driven significant advancements in the industry. This paper investigates the performance of a specific AD vehicle architecture in a critical lane change maneuver, extracted from the HighD Dataset, which consists of naturalistic vehicle trajectories recorded on German highways. The analysis focuses on different parameter configurations to understand their influence on the results. The selected AD algorithm is based on the Apollo Open Autonomous Driving Platform (AOADP). Simulation results demonstrate the significant impact of path planning algorithm parameters on lane change execution. The comparison between human driving behavior and AD systems plays a crucial role in determining sensing technology specifications. This approach provides a quantitative analysis of the impact on autonomous driving measurements, contributing to the safe specification of AD functions. These findings lay the foundation for future evaluations and improvements in the selected AD architecture.

High-Level Sensor Models for the Reinforcement Learning Driving Policy Training

Performance limitations of automotive sensors and the resulting perception errors are one of the most critical limitations in the design of Advanced Driver Assistance Systems and Autonomous Driving Systems. Ability to efficiently recreate realistic error patterns in a traffic simulation setup not only helps to ensure that such systems operate correctly in presence of perception errors, but also fulfills a key role in the training of Machine-Learning-based algorithms often utilized in them. This paper proposes a set of efficient sensor models for detecting road users and static road features. Applicability of the models is presented on an example of Reinforcement-Learning-based driving policy training. Experimental results demonstrate a significant increase in the policy’s robustness to perception errors, alleviating issues caused by the differences between the virtual traffic environment used in the policy’s training and the realistic conditions.

Generation of Correction Data for Autonomous Driving by Means of Machine Learning and On-Board Diagnostics.

A highly accurate reference vehicle state is a requisite for the evaluation and validation of Autonomous Driving (AD) and Advanced Driver Assistance Systems (ADASs). This highly accurate vehicle state is usually obtained by means of Inertial Navigation Systems (INSs) that obtain position, velocity, and Course Over Ground (COG) correction data from Satellite Navigation (SatNav). However, SatNav is not always available, as is the case of roofed places, such as parking structures, tunnels, or urban canyons. This leads to a degradation over time of the estimated vehicle state. In the present paper, a methodology is proposed that consists on the use of a Machine Learning (ML)-method (Transformer Neural Network—TNN) with the objective of generating highly accurate velocity correction data from On-Board Diagnostics (OBD) data. The TNN obtains OBD data as input and measurements from state-of-the-art reference sensors as a learning target. The results show that the TNN is able to infer the velocity over ground with a Mean Absolute Error (MAE) of 0.167 kmh (0.046 ms) when a database of 3,428,099 OBD measurements is considered. The accuracy decreases to 0.863 kmh (0.24 ms) when only 5000 OBD measurements are used. Given that the obtained accuracy closely resembles that of state-of-the-art reference sensors, it allows INSs to be provided with accurate velocity correction data. An inference time of less than 40 ms for the generation of new correction data is achieved, which suggests the possibility of online implementation. This supports a highly accurate estimation of the vehicle state for the evaluation and validation of AD and ADAS, even in SatNav-deprived environments.

Multi-Attention DenseNet: A Scattering Medium Imaging Optimization Framework for Visual Data Pre-Processing of Autonomous Driving Systems

The vision system is important for almost all kinds of autonomous driving systems. However, visual data interfered by scattering media, such as smoke, haze, water, and other non-uniform media will be degraded seriously, showing the characteristics of detail loss, poor contrast, low visibility, or color distortion. These characteristics can significantly interfere with the reliability of autonomous driving systems. In real environments the image degradation mechanism is complex, and the estimation of degradation parameters is difficult. This issue remains to be solved. In this study, we employed dense blocks as the framework and introduced the attention mechanism to our model from four dimensions: Multi-scale Attention, Channel Attention, Structure Attention, and ROI (region of interest) Attention. With the help of the training data provided by the weakly supervised model, the proposed method achieved excellent performance in the task of scattering medium imaging optimization in different scenes. Comparative experiments show that the proposed method is robust, and is superior to other state-of-the-art methods in image dehazing, and underwater image enhancement tasks. It is of great significance to improve the reliability of autonomous driving systems in underwater and severe weather environments.

A Hierarchical Temporal Memory Based End-to-End Autonomous Driving System

Abstract Achieving human-level driving performance in complex environments remains a major challenge in the field of deep learning (DL)-based end-to-end autonomous driving systems (ADS). In ADS, generalization to rare edge cases poses a serious safety concern with DL-based models. The leading solution to this problem is the construction of larger models and datasets, an approach known as scaling. However, limitations in the computational power available to autonomous vehicles, coupled with the under-representation of safety-critical edge cases in large autonomous driving datasets, raise questions over the suitability of scaling for ADS. In this work, we investigate the performance of an alternate, computationally less-demanding, machine learning (ML) algorithm, hierarchical temporal memory (HTM). Existing HTM models use rudimentary encoding schemes that have thus far limited their application to simple inputs. Motivated by this shortcoming, we first propose a bespoke convolutional neural network (CNN)-based encoding scheme suited to the input data used in ADS. We then integrate this encoding scheme into a novel DL-HTM end-to-end ADS. The proposed DL-HTM-based end-to-end ADS is trained and evaluated against a conventional DL end-to-end ADS based on the widely used AlexNet model from the literature. Our evaluation results show that the proposed DL-HTM model achieves comparable performance with far fewer trainable parameters than the conventional DL-based end-to-end ADS. Results also indicate that the proposed model demonstrates a superior capacity for learning underrepresented classes, i.e., edge cases, in the dataset.

Risk Assessment Methodologies for Autonomous Driving: A Survey

Autonomous driving systems (ADS) in recent years have been the subject of focus, evolving as one of the major mobility disruptors and being a potential candidate for deployment in urban cities due to urbanization. ADS is the system within the Autonomous Vehicle (AV) that enables automation. The different ADS technologies that enable autonomous vehicles have reached a certain maturity that no longer focus on technological deployment, but rather on the safe deployment on public roads. However, existing standards that validate functional safety and Risk Assessment (RA) may not be sufficient to tackle the increased complexity of ADS compared to traditional vehicles. This demand in ADS safety is exponentially increasing in tandem with the increase of AV automation levels. ADS are exposed to diverse environmental conditions and therefore subjected to operational risks while attempting to mimic the human driver responses. Moreover, the recent use of artificial intelligence and machine learning in the industry further shapes the way how ADS development will become in the future. This paper explains the importance of RA coverage for AV and provides a comparison and summary of existing RA methodologies. Thereafter, a recommendation of RAs for AV as potential solutions in meeting ISO 26262 and ISO/PAS 21448 standards.

Context-Aware 3D Object Detection From a Single Image in Autonomous Driving

Camera sensors have been widely used in Driver-Assistance and Autonomous Driving Systems due to their rich texture information. Recently, with the development of deep learning techniques, many approaches have been proposed to detect objects in 3D from a single frame, however, there is still much room for improvement. In this paper, we generally review the recently proposed state-of-the-art monocular-based 3D object detection approaches first. Based on the analysis of the disadvantage of previous center-based frameworks, a novel feature aggregation strategy has been proposed to boost the 3D object detection by exploring the context information. Specifically, an Instance-Guided Spatial Attention (IGSA) module is proposed to collect the local instance information and the Channel-Wise Feature Attention (CWFA) module is employed for aggregating the global context information. In addition, an instance-guided object regression strategy is also proposed to alleviate the influence of center location prediction uncertainty in the inference process. Finally, the proposed approach has been verified on the public 3D object detection benchmark. The experimental results show that the proposed approach can significantly boost the performance of the baseline method on both 3D detection and 2D Bird’s-Eye View among all three categories. Furthermore, our method outperforms all the monocular-based methods (even these trained with depth as auxiliary inputs) and achieves state-of-the-art performance on the KITTI benchmark.

Vision beyond the Field-of-View: A Collaborative Perception System to Improve Safety of Intelligent Cyber-Physical Systems.

Cyber-physical systems (CPSs) that interact with each other to achieve common goals are known as collaborative CPSs. Collaborative CPSs can achieve complex goals that individual CPSs cannot achieve on their own. One of the examples of collaborative CPSs is the vehicular cyber-physical systems (VCPSs), which integrate computing and physical resources to interact with each other to improve traffic safety, situational awareness, and efficiency. The perception system of individual VCPS has limitations on its coverage and detection accuracy. For example, the autonomous vehicle’s sensor cannot detect occluded objects and obstacles beyond its field of view. The VCPS can combine its own data with other collaborative VCPSs to enhance perception, situational awareness, accuracy, and traffic safety. This paper proposes a collaborative perception system to detect occluded objects through the camera sensor’s image fusion and stitching technique. The proposed collaborative perception system combines the perception of surrounding autonomous driving systems (ADSs) that extends the detection range beyond the field of view. We also applied logistic chaos map-based encryption in our collaborative perception system in order to avoid the phantom information shared by malicious vehicles and improve safety in collaboration. It can provide the real-time perception of occluded objects, enabling safer control of ADSs. The proposed collaborative perception can detect occluded objects and obstacles beyond the field of view that individual VCPS perception systems cannot detect, improving the safety of ADSs. We investigated the effectiveness of collaborative perception and its contribution toward extended situational awareness on the road in the simulation environment. Our simulation results showed that the average detection rate of proposed perception systems was 45.4% more than the perception system of an individual ADS. The safety analysis showed that the response time was increased up to 1 s, and the average safety distance was increased to 1.2 m when the ADSs were using collaborative perception compared to those scenarios in which the ADSs were not using collaborative perception.

Explainability of Deep Vision-Based Autonomous Driving Systems: Review and Challenges

This survey reviews explainability methods for vision-based self-driving systems trained with behavior cloning. The concept of explainability has several facets and the need for explainability is strong in driving, a safety-critical application. Gathering contributions from several research fields, namely computer vision, deep learning, autonomous driving, explainable AI (X-AI), this survey tackles several points. First, it discusses definitions, context, and motivation for gaining more interpretability and explainability from self-driving systems, as well as the challenges that are specific to this application. Second, methods providing explanations to a black-box self-driving system in a post-hoc fashion are comprehensively organized and detailed. Third, approaches from the literature that aim at building more interpretable self-driving systems by design are presented and discussed in detail. Finally, remaining open-challenges and potential future research directions are identified and examined.

Overriding Autonomous Driving Systems Using Adaptive Adversarial Billboards

The success of deep neural networks (DNNs) has led to its increased deployment in various real-world applications, which provides strong incentives for motivated adversaries to manipulate the results and models generated by these algorithms. We present an automated, physically-realizable, dynamic adversarial attack to compromise an end-to-end trained DNN controlled autonomous vehicle. The attack is initiated by installing a billboard displaying videos on the roadside to incoming DNN controlled vehicles so that the vehicle tracks an adversary customized trajectory. The billboard contains an integrated camera to enable estimation of the pose of the approaching vehicle. The dynamic billboard images (i.e., a video) continuously adapt to the vehicle’s relative pose with respect to the billboard while being robust to variations in lighting, view angle, and weather. The attack’s effectiveness is shown on a recently developed off-the-shelf high-fidelity simulator, CARLA, for autonomous vehicles. CARLA utilizes an end-to-end learning-based autonomous navigation system. The proposed approach is applicable to other end-to-end trained autonomous cyber-physical systems.

Scene-Graph Augmented Data-Driven Risk Assessment of Autonomous Vehicle Decisions

There is considerable evidence that evaluating the subjective risk level of driving decisions can improve the safety of Autonomous Driving Systems (ADS) in both typical and complex driving scenarios. In this paper, we propose a novel data-driven approach that uses scene-graphs as intermediate representations for modeling the subjective risk of driving maneuvers. Our approach includes a Multi-Relation Graph Convolution Network, a Long-Short Term Memory Network, and attention layers. To train our model, we formulate subjective risk assessment as a supervised scene classification problem. We evaluate our model on both synthetic lane-changing datasets and real-driving datasets with various driving maneuvers. We show that our approach achieves a higher classification accuracy than the state-of-the-art approach on both large (96.4% vs. 91.2%) and small (91.8% vs. 71.2%) lane-changing synthesized datasets, illustrating that our approach can learn effectively even from small datasets. We also show that our model trained on a lane-changing synthesized dataset achieves an average accuracy of 87.8% when tested on a real-driving lane-changing dataset. In comparison, the state-of-the-art model trained on the same synthesized dataset only achieved 70.3% accuracy when tested on the real-driving dataset, showing that our approach can transfer knowledge more effectively. Moreover, we demonstrate that the addition of spatial and temporal attention layers improves our model’s performance and explainability. Finally, our results illustrate that our model can assess the risk of various driving maneuvers more accurately than the state-of-the-art model (86.5% vs. 58.4%, respectively).

Computational Intelligence with Wild Horse Optimization Based Object Recognition and Classification Model for Autonomous Driving Systems

Presently, autonomous systems have gained considerable attention in several fields such as transportation, healthcare, autonomous driving, logistics, etc. It is highly needed to ensure the safe operations of the autonomous system before launching it to the general public. Since the design of a completely autonomous system is a challenging process, perception and decision-making act as vital parts. The effective detection of objects on the road under varying scenarios can considerably enhance the safety of autonomous driving. The recently developed computational intelligence (CI) and deep learning models help to effectively design the object detection algorithms for environment perception depending upon the camera system that exists in the autonomous driving systems. With this motivation, this study designed a novel computational intelligence with a wild horse optimization-based object recognition and classification (CIWHO-ORC) model for autonomous driving systems. The proposed CIWHO-ORC technique intends to effectively identify the presence of multiple static and dynamic objects such as vehicles, pedestrians, signboards, etc. Additionally, the CIWHO-ORC technique involves the design of a krill herd (KH) algorithm with a multi-scale Faster RCNN model for the detection of objects. In addition, a wild horse optimizer (WHO) with an online sequential ridge regression (OSRR) model was applied for the classification of recognized objects. The experimental analysis of the CIWHO-ORC technique is validated using benchmark datasets, and the obtained results demonstrate the promising outcome of the CIWHO-ORC technique in terms of several measures.

Advanced Techniques for Perception and Localization in Autonomous Driving Systems: A Survey

Advanced Techniques for Perception and Localization in Autonomous Driving Systems: A Survey

Simulation and Validation with Radio-Controlled (RC) Autonomous Vehicles in Roundabout Situation

In the present research paper, the authors provide an extensive overview about a so far uninvestigated approach regarding the research and development (R&D) of Autonomous Vehicle Technologies, an exhaustive investigation concerning the simulation and validation of roundabout situation with radio-controlled (RC) autonomous vehicles, as well as the examination of RC vehicles' applicability in the testing processes of Connected Autonomous Vehicle (CAV) technologies. Through this research project, the authors are offering a thorough analysis concerning the most current Autonomous Driving Systems from the point of view of their operation in roundabout related environments, as well as have developed two small-scaled RC autonomous vehicle models, which are capable of being used in the testing and validation processes concerning research studies related to the field of autonomous vehicles. This paper represents the first of a two-part in-depth examination of the aforementioned subject matter, organized in six sections, each contributing significantly to the realization of the final structure of a unique and comprehensive project.

Adaptive Real-Time Object Detection for Autonomous Driving Systems.

Accurate and reliable detection is one of the main tasks of Autonomous Driving Systems (ADS). While detecting the obstacles on the road during various environmental circumstances add to the reliability of ADS, it results in more intensive computations and more complicated systems. The stringent real-time requirements of ADS, resource constraints, and energy efficiency considerations add to the design complications. This work presents an adaptive system that detects pedestrians and vehicles in different lighting conditions on the road. We take a hardware-software co-design approach on Zynq UltraScale+ MPSoC and develop a dynamically reconfigurable ADS that employs hardware accelerators for pedestrian and vehicle detection and adapts its detection method to the environment lighting conditions. The results show that the system maintains real-time performance and achieves adaptability with minimal resource overhead.

A Practical and Economical Ultra-wideband Base Station Placement Approach for Indoor Autonomous Driving Systems

Automated valet parking (AVP) has attracted much attention as the entry point to autonomous driving. In an indoor environment, high-precision positioning systems are essential for AVP. Ultra-wideband (UWB) is one of the most widely adopted techniques. However, the base station placement significantly influences the system’s positioning accuracy, especially for the irregular architecture of underground parking lots. This article proposes a three-stage practical and economical layout planning approach for UWB base stations, including determining the deployment strategy and layout parameters and comprehensive adjustment and scheme verification. The approach considers regional differentiation accuracy requirements for AVP, such as ramp area, surface fluctuation area, and narrow area. The adopted positioning method of a UWB system is the time difference of arrival (TDOA), and the evaluation index of positioning accuracy is the horizontal dilution of precision (HDOP). Through experimental tests in an actual parking lot, the proposed approach is confirmed to ensure stability and economy with fewer UWB base stations and can meet the positioning accuracy requirements of AVP.

Toward Performing Image Classification and Object Detection With Convolutional Neural Networks in Autonomous Driving Systems: A Survey

Nowadays Convolutional Neural Networks (CNNs) are being employed in a wide range of industrial technologies for a variety of sectors, such as medical, automotive, aviation, agriculture, space, etc. This paper reviews the state-of-the-art in both the field of CNNs for image classification and object detection and Autonomous Driving Systems (ADSs) in a synergetic way. Layer-based details of CNNs along with parameter and floating-point operation number calculations are outlined. Using an evolutionary approach, the majority of the outstanding image classification CNNs, published in the open literature, is introduced with a focus on their accuracy performance, parameter number, model size, and inference speed, highlighting the progressive developments in convolutional operations. Results of a novel investigation of the convolution types and operations commonly used in CNNs are presented, including a timing analysis aimed at assessing their impact on CNN performance. This extensive experimental study provides new insight into the behaviour of each convolution type in terms of training time, inference time, and layer level decomposition. Building blocks for CNN-based object detection are also discussed, such as backbone networks and baseline types, and then representative state-of-the-art designs are outlined. Experimental results from the literature are summarised for each of the reviewed models. This is followed by an overview of recent ADSs related works and current industry activities, aiming to bridge academic research and industry practice on CNNs and ADSs. Design approaches targeted at solving problems of automakers in achieving real-time implementations are also proposed based on a discussion of design constraints, human vs. machine evaluations and trade-off analysis of accuracy vs. size. Current technologies, promising directions, and expectations from the literature on ADSs are introduced including a comprehensive trade-off analysis from a human-machine perspective.