Safety Of Autonomous Vehicles Research Articles

Trusted Perception Method for Traffic Signs That Are Physically Attacked

Traffic sign recognition is a crucial method by which autonomous driving systems acquire road information, and is predominantly based on deep neural networks (DNNs). However, the recognition results of DNNs are not always trustworthy for traffic signs subject to abnormal disturbance. Recently, the phenomenon of adversarial examples successfully deceiving DNNs has garnered considerable attention. Because DNN-based computer vision techniques are becoming increasingly prevalent in traffic scenarios, the misclassification of attacked traffic signs by DNN classifiers poses serious safety hazards. Although numerous methods have been proposed for crafting physical adversarial examples that are robust in the real world, most existing defense approaches focus on digital attacks, which necessitate the adversary infiltrating the embedded system; thus, it becomes challenging to obtain results. A reliable approach for defending against physical adversarial traffic signs enables autonomous vehicles to achieve trusted perception of traffic signs. In this paper, we present a deep image prior-based pipeline to defend against robust adversarial traffic signs in the real world, an approach that circumvents the need for prior data sets during training. Our approach protects the safety of autonomous vehicles by performing image reconstruction of captured traffic sign images. The genuine traffic sign class can be inferred by leveraging the consistency of the victim classifier’s decision results for reconstructed images at different stages. Additionally, we evaluate the efficacy of our defense pipeline for detecting other potential types of physical adversarial traffic signs that may exist in the real world, thus demonstrating the generalizability of our approach.

An in-depth evaluation of deep learning-enabled adaptive approaches for detecting obstacles using sensor-fused data in autonomous vehicles

This paper delivers an exhaustive analysis of the fusion of multi-sensor technologies, including traditional sensors such as cameras, Light Detection and Ranging(LiDAR), Radio Detection and Ranging(RADAR), and ultrasonic sensors, with Artificial Intelligence(AI) powered methodologies in obstacle detection for Autonomous Vehicles(AVs). With the growing momentum in AVs adoption, a heightened need exists for versatile and resilient obstacle detection systems. Our research delves into study of literatures, where proposed approaches assimilate data from this diverse sensor suite, integrated through Deep Learning(DL) techniques, to refine AV performance. Recent advancements and prevailing challenges within the domain are thoroughly examined, with particular focus on the integration of sensor fusion techniques, the facilitation of real-time processing via edge and fog computing, and the implementation of advanced artificial intelligence architectures, including Convolutional Neural Networks(CNNs), Recurrent Neural Networks(RNNs), and Generative Adversarial Networks(GANs), to enhance data interpretation efficacy. In conclusion, the paper underscores the critical contribution of multi-sensor arrays and deep learning in enhancing the safety and reliability of autonomous vehicles, offering significant perspectives for future research and technological progress.

Trustworthy autonomous driving via defense-aware robust reinforcement learning against worst-case observational perturbations

Despite the substantial advancements in reinforcement learning (RL) in recent years, ensuring trustworthiness remains a formidable challenge when applying this technology to safety-critical autonomous driving domains. One pivotal bottleneck is that well-trained driving policy models may be particularly vulnerable to observational perturbations or perceptual uncertainties, potentially leading to severe failures. In view of this, we present a novel defense-aware robust RL approach tailored for ensuring the robustness and safety of autonomous vehicles in the face of worst-case attacks on observations. The proposed paradigm primarily comprises two crucial modules: an adversarial attacker and a robust defender. Specifically, the adversarial attacker is devised to approximate the worst-case observational perturbations that attempt to induce safety violations (e.g., collisions) in the RL-driven autonomous vehicle. Additionally, the robust defender is developed to facilitate the safe RL agent to learn robust optimal policies that maximize the return while constraining the policy and cost perturbed by the adversarial attacker within specified bounds. Finally, the proposed technique is assessed across three distinct traffic scenarios: highway, on-ramp, and intersection. The simulation and experimental results indicate that our scheme enables the agent to execute trustworthy driving policies, even in the presence of the worst-case observational perturbations.

Enhancing the Safety of Autonomous Vehicles in Adverse Weather by Deep Learning-Based Object Detection

Recognizing and categorizing items in weather-adverse environments poses significant challenges for autonomous vehicles. To improve the robustness of object-detection systems, this paper introduces an innovative approach for detecting objects at different levels by leveraging sensors and deep learning-based solutions within a traffic circle. The suggested approach improves the effectiveness of single-stage object detectors, aiming to advance the performance in perceiving autonomous racing environments and minimizing instances of false detection and low recognition rates. The improved framework is based on the one-stage object-detection model, incorporating multiple lightweight backbones. Additionally, attention mechanisms are integrated to refine the object-detection process further. Our proposed model demonstrates superior performance compared to the state-of-the-art method on the DAWN dataset, achieving a mean average precision (mAP) of 99.1%, surpassing the previous result of 84.7%.

Performance Evaluation of YOLOv5-based Custom Object Detection Model for Campus-Specific Scenario

This study evaluates the performance of a custom object detection model based on the YOLOv5 architecture, specifically tailored for autonomous electric vehicles. The model undergoes pre-processing using the Roboflow computer vision platform, which offers a wide range of tools for data pre-processing and model training. The experiments were conducted on a diverse dataset comprising various objects encountered in campus-specific driving scenarios, such as pedestrians, vehicles, buildings, and obstacles. The performance of the custom object detection model is assessed using standard metrics, including precision, recall, mean average precision (mAP), and intersection-over-union (IoU) at different thresholds. The training process was conducted in a controlled environment, resulting in a Precision of 0.851, a Recall of 0.831, and a mAP of 0.843. These metrics were analyzed to evaluate the YOLOv5-based custom object detection model's ability to detect and categorize objects accurately, its precision in predicting bounding boxes, and its capability to handle various object categories. We also examined the effects of different hyperparameters and data augmentation techniques on the model's performance, including variations in learning rate, batch size, and optimizer algorithms to determine their impact on accuracy and convergence. This analysis provided valuable insights into the model's strengths and weaknesses, highlighting areas for improvement and optimization. These findings are instrumental in developing and deploying advanced object detection systems to enhance the safety and reliability of autonomous electric vehicles.

Research on Road Sign Recognition Method for Autonomous Vehicles Based on Convolutional Neural Networks

With the rapid development of autonomous driving technology, road sign recognition has become an important link to ensure the safety of autonomous vehicles. This paper proposes a road sign recognition method for autonomous vehicles based on Convolutional Neural Networks (CNN), aiming to improve the accuracy and efficiency of road sign recognition by using CNN. Firstly, the background and significance of this issue were introduced, including the development of autonomous vehicles and their demand for road sign recognition technology. The focus was on the application of convolutional neural networks in road sign recognition. This article provides a detailed introduction to the dataset used, the designed convolutional neural network model, and data preprocessing methods. In order to further improve accuracy, methods such as data augmentation and image enhancement were adopted. The research results of this paper indicate that the road sign recognition method for autonomous vehicles based on convolutional neural networks has high accuracy and efficiency, providing important support for the practical application of autonomous driving technology.

How would autonomous vehicles behave in real-world crash scenarios?

Autonomous Vehicles (AVs) have the potential to revolutionize transportation systems by enhancing traffic safety. Safety testing is undoubtedly a critical step for enabling large-scale deployment of AVs. High-risk scenarios are particularly important as they pose significant challenges and provide valuable insights into the driving capabilities of AVs. This study presents a novel approach to assess the safety of AVs using in-depth crash data, with a particular focus on real-world crash scenarios. First, based on the high-definition video recording of the whole process prior to the crash occurrences, 453 real-world crashes involving 596 passenger cars from China In-depth Mobility Safety Study-Traffic Accident (CIMSS-TA) database were reconstructed. Pertinent static and dynamic elements needed for the construction of the testing scenarios were extracted. Subsequently, 596 testing scenarios were created via each passenger car’s perspective within the simulation platform. Following this, each of the crash-involved passenger cars was replaced with Baidu Apollo, a famous automated driving system (ADS), for counterfactual simulation. Lastly, the safety performance of the AV was assessed using the simulation results. A logit model was utilized to identify the fifteen crucial scenario elements that have significant impacts on the test results. The findings demonstrated that the AV could avoid 363 real-world crashes, accounting for approximately 60.91% of the total, and effectively mitigated injuries in the remaining 233 unavoidable scenarios compared to a human driver. Moreover, the AV maintain a smoother speed in most of the scenarios. The common feature of these unavoidable scenarios is that the AV is in a passive state, and the crashes are not caused by the AV violating traffic rules, but rather caused by abnormal behavior exhibited by the human drivers. Additionally, seven specific scenarios have been identified wherein AVs are unable to avoid a crash. These findings demonstrate that, compared to human drivers, AVs can avoid crashes that are difficult for humans to avoid, thereby enhancing traffic safety.

BIPOOLNET: An advanced UNet architecture for enhanced lane detection in autonomous vehicles

In the rapidly evolving landscape of autonomous vehicle technology, the imperative to bolster safety within smart urban ecosystems has never been more critical. This endeavor requires the deployment of advanced detection systems capable of navigating the intricacies of pedestrian, near-vehicle, and lane detection challenges, with a particular focus on the nuanced requirements of curved lane navigation – a domain where traditional AI models exhibit notable deficiencies. This paper introduces BIPOOLNET, an innovative encoder-decoder neural architecture, ingeniously augmented with a feature pyramid to facilitate the precise delineation of curved lane geometries. BIPOOLNET integrates max pooling and average pooling to extract critical features and mitigate the complexity of the feature map, redefining the benchmarks for lane detection technology. Rigorous evaluation using the TuSimple dataset underscores BIPOOLNET’s exemplary performance, evidenced by an unprecedented accuracy rate of 98.45%, an F1-score of 98.17%, and notably minimal false positive (1.84%) and false negative (1.09%) rates. These findings not only affirm BIPOOLNET’s supremacy over extant models but also signal a paradigm shift in enhancing the safety and navigational precision of autonomous vehicles, offering a scalable, robust solution to the multifaceted challenges posed by real-world driving dynamics.

Pedestrian Perception Tracking in Complex Environment of Unmanned Vehicles Based on Deep Neural Networks

INTRODUCTION: In recent years, machine learning and deep learning have emerged as pivotal technologies with transformative potential across various industries. Among these, the automobile industry stands out as a significant arena for the application of these technologies, particularly in the development of smart cars with unmanned driving systems. This article delves into the extensive research conducted on the detection technology employed by autonomous vehicles to navigate road conditions, a critical aspect of driverless car technology.
 OBJECTIVES: The primary aim of this research is to explore and highlight the intricacies of road condition detection for autonomous vehicles. Emphasizing the importance of this key component in the development of driverless cars, we aim to provide insights into cutting-edge algorithms that enhance the capabilities of these vehicles, ultimately contributing to their widespread adoption.
 METHODS: In addressing the challenge of road condition detection, we introduce the TidyYOLOv4 algorithm. This algorithm, deemed more advantageous than YOLOv4, particularly excels in pedestrian recognition within urban traffic environments. Its real-time capabilities make it a suitable choice for detecting pedestrians on the road under dynamic conditions.
 RESULTS: The application of the TidyYOLOv4 algorithm in autonomous vehicles has yielded promising results, especially in enhancing pedestrian recognition in urban traffic settings. The algorithm's real-time functionality proves crucial in ensuring the timely detection of pedestrians on the road, thereby improving the overall safety and efficiency of autonomous vehicles.
 CONCLUSION: In conclusion, the detection of road conditions is a critical aspect of autonomous vehicle technology, with implications for safety and efficiency. The TidyYOLOv4 algorithm emerges as a noteworthy advancement, outperforming its predecessor YOLOv4 in pedestrian recognition within urban traffic environments. As companies continue to invest in driverless technology, leveraging such advanced algorithms becomes imperative for the successful deployment of autonomous vehicles in real-world scenarios.

Hierarchical Lateral Planning With Minimum Curvature Peak for Autonomous Vehicles

Abstract This study focuses on the lateral safety of autonomous vehicles. The aim is to provide greater safety margins for the motion controller under hazardous conditions by improving the planning methods. The proposed lateral planning method employs a hierarchical structure. First, a graph search method that considers road curvature is presented. This method benefits from the design of jump nodes and can generate a safe corridor in real-time by backtracking. Next, a concise and effective optimization problem is constructed in the Frenet frame. It is designed to optimize the path by incorporating the peak, integral, and differential of curvature into the objective function. By solving this problem using CasADi, the path within the safe corridor is obtained. Finally, the proposed planning method is combined with a model predictive motion controller and tested through simulation and experimentation on a real vehicle platform. This planning method has good scalability, adaptability, and real-time performance. Simulation and experimental results show that this method can help avoid exceeding lateral limits of road surface friction, improve path-following accuracy, and enhance the safety of autonomous vehicles in dangerous conditions.

Machine Learning Approaches for In-Vehicle Failure Prognosis in Automobiles: A Review

The automobile industry has a growing need for reliable and safe health monitoring systems equipped with low-cost sensor networks and intelligent algorithms. This paper provides an overview of approaches already exist, used in on-board health monitoring systems for vehicles. It focuses on the methodologies, theories, and applications employed in the data measurement and data analysis systems of vehicle (cars) on-board health monitoring systems. A fault detection and diagnosis system, which is accurate, plays a vital role in ensuring the safety of autonomous vehicles by preventing potentially dangerous situations. This study focuses on emphasizing a fault diagnosis system that utilizes hybrid methods. Among the various options considered in this analysis, internal sensors emerge as the preferred choice due to their numerous benefits, including affordability, durability, widespread availability, ease of access, and low energy consumption. Model-based methods require various techniques that may introduce errors to estimation results, while signal-based methods necessitate a time-consuming process of including all possible conditions in a pre-built database. Based on this review, future development trends in designing new low-cost health monitoring systems for vehicles are also discussed.

Fault detection and fault-tolerant control of dual-motor autonomous steering system

In order to improve the safety of autonomous ground vehicle (AGV) and the accuracy of trajectory tracking, a fault detection (FD) scheme of steering system and a fault-tolerant control (FTC) method are proposed. First, this paper analyzes the causes of steering motor failure and designs a passive fault-tolerant controller to ensure that the torque motor under coordinated control can still follow the angle when a slight fault occurs in the angle motor. Second, the corresponding FD schemes are designed for different motor faults, and the unknown input observer (UIO) of the steering resistance moment is designed considering the uncertainty of the steering system caused by the fault motor. The active fault-tolerant controller composed of backstepping control and global sliding mode control (SMC) is further combined to control the fault-free motor. Finally, simulation and HiL test results show the effectiveness of the proposed FD scheme and FTC strategy.

Research on Behavioral Decision at an Unsignalized Roundabout for Automatic Driving Based on Proximal Policy Optimization Algorithm

Unsignalized roundabouts have a significant impact on traffic flow and vehicle safety. To address the challenge of autonomous vehicles passing through roundabouts with low penetration, improve their efficiency, and ensure safety and stability, we propose the proximal policy optimization (PPO) algorithm to enhance decision-making behavior. We develop an optimization-based behavioral choice model for autonomous vehicles that incorporates gap acceptance theory and deep reinforcement learning using the PPO algorithm. Additionally, we employ the CoordConv network to establish an aerial view for spatial perception information gathering. Furthermore, a dynamic multi-objective reward mechanism is introduced to maximize the PPO algorithm’s reward pool function while quantifying environmental factors. Through simulation experiments, we demonstrate that our optimized PPO algorithm significantly improves training efficiency by enhancing the reward value function by 2.85%, 7.17%, and 19.58% in scenarios with 20, 100, and 200 social vehicles, respectively, compared to the PPO+CCMR algorithm. The effectiveness of simulation training also increases by 11.1%, 13.8%, and 7.4%. Moreover, there is a reduction in crossing time by 2.37%, 2.62%, and 13.96%. Our optimized PPO algorithm enhances path selection during autonomous vehicle simulation training as they tend to drive in the inner ring over time; however, the influence of social vehicles on path selection diminishes as their quantity increases. The safety of autonomous vehicles remains largely unaffected by our optimized PPO algorithm.

NeXtFusion: Attention-Based Camera-Radar Fusion Network for Improved Three-Dimensional Object Detection and Tracking

Accurate perception is crucial for autonomous vehicles (AVs) to navigate safely, especially in adverse weather and lighting conditions where single-sensor networks (e.g., cameras or radar) struggle with reduced maneuverability and unrecognizable targets. Deep Camera-Radar fusion neural networks offer a promising solution for reliable AV perception under any weather and lighting conditions. Cameras provide rich semantic information, while radars act like an X-ray vision, piercing through fog and darkness. This work proposes a novel, efficient Camera-Radar fusion network called NeXtFusion for robust AV perception with an improvement in object detection accuracy and tracking. Our proposed approach of utilizing an attention module enhances crucial feature representation for object detection while minimizing information loss from multi-modal data. Extensive experiments on the challenging nuScenes dataset demonstrate NeXtFusion’s superior performance in detecting small and distant objects compared to other methods. Notably, NeXtFusion achieves the highest mAP score (0.473) on the nuScenes validation set, outperforming competitors like OFT (35.1% improvement) and MonoDIS (9.5% improvement). Additionally, NeXtFusion demonstrates strong performance in other metrics like mATE (0.449) and mAOE (0.534), highlighting its overall effectiveness in 3D object detection. Furthermore, visualizations of nuScenes data processed by NeXtFusion further demonstrate its capability to handle diverse real-world scenarios. These results suggest that NeXtFusion is a promising deep fusion network for improving AV perception and safety for autonomous driving.

A Tube Linear Model Predictive Control Approach for Autonomous Vehicles Subjected to Disturbances

The path tracking performance of autonomous vehicles is degraded by common disturbances, especially those that affect the safety of autonomous vehicles (AVs) in obstacle avoidance conditions. To improve autonomous vehicle tracking performances and their computational efficiency when subjected to common disturbances, this paper proposes a tube linear model predictive controller (MPC) framework for autonomous vehicles. A bicycle vehicle dynamics model is developed and employed in the tube MPC control design in the proposed framework. A robust invariant set is calculated with an efficient linear programming (LP) method, and it is used to guarantee that the constraints are satisfied under common disturbance conditions. The results show that the computational cost of robust positively invariant sets that are constructed by the LP method is much less than that obtained by the traditional method. In addition, all the trajectories of the tube linear MPC successfully avoided obstacles when under disturbance conditions, but only about 80% of the trajectories obtained with the traditional MPC successfully avoided obstacles under disturbance conditions. The proposed framework is effective.

A quantitative blind area risks assessment method for safe driving assistance

The task of driving in complex and occluded environments presents significant challenges, as it requires the perception and prediction of one’s surrounding environment. This is especially important as blind spots, or areas that are difficult to directly detect due to occlusion by surrounding vehicles or obstacles, which can pose a serious risk of accidents. To address this issue, this study proposes a method called “Uncertainty Shadow Safety”, which aims to evaluate the potential risk of invisible areas surrounding the ego vehicle based on the combination of static road information and the presence of traffic participants. The proposed system calculates the “shadow uncertainty” of a non-visible road area based on its physical properties and the probabilities of various traffic participants being present in that area. It then performs a quantitative assessment of the potential risk from various directions, taking into account the surrounding anomalous movement states. Simulation tests were conducted in various scenarios to demonstrate the feasibility and effectiveness of the Uncertainty Shadow Safety system. The advanced warning provided by the system to potential risks in invisible areas can help to reduce the likelihood of collisions and improve driving safety for both human drivers and autonomous vehicles.

Event‐triggered adaptive secure lateral stabilization for autonomous vehicles under actuator attacks

SummaryFalse data injection attacks can disrupt the steering control actions and make a real threat to both the security and safety of autonomous vehicles. In this paper, a secure event‐triggered lateral control approach of autonomous vehicles subject to actuator attacks is investigated. Firstly, an arbitrary unknown actuator attacks is considered in the secure lateral steering control of autonomous vehicles. Thus, to save communication resources for the limited bandwidth CAN bus, the periodic event‐triggered transmission scheme is utilized, transforming the established lateral steering control into a time‐delay system through the consideration of periodic event‐triggered sampling. Then, an adaptive control compensation scheme is developed to mitigate the malicious effects caused by actuator attacks. The proposed secure control approach is skilled in compensating the unknown attacked steering control actions in an adaptive way. The stabilization criteria under the adaptive secure control law is well derived by Lyapunov–Krasovskii method and some linear inequality matrices operations. At last, the effectiveness of the proposed secure control scheme is verified by some numerical experiments borrowed from a practical vehicle.

Goal-Guided Graph Attention Network with Interactive State Refinement for Multi-Agent Trajectory Prediction.

The accurate prediction of the future trajectories of traffic participants is crucial for enhancing the safety and decision-making capabilities of autonomous vehicles. Modeling social interactions among agents and revealing the inherent relationships is crucial for accurate trajectory prediction. In this context, we propose a goal-guided and interaction-aware state refinement graph attention network (SRGAT) for multi-agent trajectory prediction. This model effectively integrates high-precision map data and dynamic traffic states and captures long-term temporal dependencies through the Transformer network. Based on these dependencies, it generates multiple potential goals and Points of Interest (POIs). Through its dual-branch, multimodal prediction approach, the model not only proposes various plausible future trajectories associated with these POIs, but also rigorously assesses the confidence levels of each trajectory. This goal-oriented strategy enables SRGAT to accurately predict the future movement trajectories of other vehicles in complex traffic scenarios. Tested on the Argoverse and nuScenes datasets, SRGAT surpasses existing algorithms in key performance metrics by adeptly integrating past trajectories and current context. This goal-guided approach not only enhances long-term prediction accuracy, but also ensures its reliability, demonstrating a significant advancement in trajectory forecasting.

A Review of Crowdsourcing Update Methods for High-Definition Maps

High-definition (HD) maps serve as crucial infrastructure for autonomous driving technology, facilitating vehicles in positioning, environmental perception, and motion planning without being affected by weather changes or sensor-visibility limitations. Maintaining precision and freshness in HD maps is paramount, as delayed or inaccurate information can significantly impact the safety of autonomous vehicles. Utilizing crowdsourced data for HD map updating is widely recognized as a superior method for preserving map accuracy and freshness. Although it has garnered considerable attention from researchers, there remains a lack of comprehensive exploration into the entire process of updating HD maps through crowdsourcing. For this reason, it is imperative to review and discuss crowdsourcing techniques. This paper aims to provide an overview of the overall process of crowdsourced updates, followed by a detailed examination and comparison of existing methodologies concerning the key techniques of data collection, information extraction, and change detection. Finally, this paper addresses the challenges encountered in crowdsourced updates for HD maps.

Autonomous Vehicles for All?

The traditional build-and-expand approach is not a viable solution to keep roadway traffic rolling safely, so technological solutions, such as autonomous vehicles (AVs), are favored. AVs have considerable potential to increase the carrying capacity of roads, ameliorate the chore of driving, improve safety, provide mobility for those who cannot drive, and help the environment. However, they also raise concerns over whether they are socially responsible, taking into account issues such as fairness, equity, and transparency. Regulatory bodies have focused on AV safety, cybersecurity, privacy, and legal liability issues, but have failed to adequately address social responsibility. Thus, existing AV developers do not have to embed social responsibility factors in their proprietary technology. Adverse bias may therefore occur in the development and deployment of AV technology. For instance, an artificial intelligence-based pedestrian detection application used in an AV may, in limited lighting conditions, be biased to detect pedestrians who belong to a particular racial demographic more efficiently compared to pedestrians from other racial demographics. Also, AV technologies tend to be costly, with a unique hardware and software setup which may be beyond the reach of lower-income people. In addition, data generated by AVs about their users may be misused by third parties such as corporations, criminals, or even foreign governments. AVs promise to dramatically impact labor markets, as many jobs that involve driving will be made redundant. We argue that the academic institutions, industry firms, and government agencies overseeing AV development and deployment must act proactively to ensure that AVs serve all and do not increase the digital divide in our society.