Autonomous Driving (AD) has garnered significant attention in recent years across multiple domains. Despite notable advancements in algorithms and hardware, large-scale deployment of autonomous vehicles remains constrained by the lack of adequate real-time guarantees. This article comprehensively investigates real-time assurance challenges in AD systems from theoretical and practical perspectives. First, we introduce foundational concepts of real-time systems and analyze common modeling approaches, including the multi-rate DAG and processing chain DAG models. We then delve into the task scheduling and communication mechanisms of three representative middleware systems–ROS2, Cyber, and ERDOS–and the seL4 operating system. Our analysis reveals their underlying design philosophies and optimization strategies for real-time performance. The findings highlight the critical difficulty of meeting stringent real-time requirements in autonomous systems. By offering insights into current limitations and opportunities for improvement, this article aims to establish a deeper understanding of real-time assurance issues and encourage greater focus on system-level guarantees within the AD community.

Autonomous driving systems (ADS) have been an active area of research, with the potential to deliver significant benefits to society. However, before large-scale deployment on public roads, extensive testing is necessary to validate their functionality and safety under diverse driving conditions. Therefore, different testing approaches are required, and achieving effective and efficient testing of ADS remains an open challenge. Recently, generative AI has emerged as a powerful tool across many domains and is increasingly applied to ADS testing due to its ability to interpret context, reason about complex tasks, and generate diverse outputs. To understand its role in ADS testing, we systematically analyzed 92 relevant studies and synthesized their findings into six major application categories, primarily centered on scenario-based testing. We further evaluated the effectiveness of these approaches and compiled a comprehensive set of resources, including 37 datasets, 17 simulators, 61 ADS systems, 141 metrics, and 100 benchmarks used for evaluation. In addition, we identified 27 limitations in existing research and outlined six major directions for future work. This survey provides an overview and practical insights into the use of generative AI for testing ADS, highlights existing challenges, and outlines future research opportunities in this rapidly evolving field.

Simulation-based testing is essential for evaluating the safety of Autonomous Driving Systems (ADSs). Comprehensive evaluation requires testing across diverse scenarios that can trigger various types of violations under different conditions. While existing methods typically focus on individual diversity metrics, such as input scenarios, ADS-generated motion commands, and system violations, they often fail to capture the complex interrelationships among these elements. For instance, identical motion commands can produce different collision risks in varying scenes, and the same collision may result from different commands under different scenarios. This oversight leads to gaps in testing coverage, potentially missing critical issues in the ADS under evaluation. In this paper, we propose <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Causal-Fuzzer</monospace>, the first causality-aware fuzzing technique that enables efficient and comprehensive testing of ADSs by constructing causal graphs to model the interrelationships among scenarios, actions, and violations. Unlike existing methods that treat diversity metrics independently, we recognize these elements are causally interconnected and use their relationships to identify more diverse violations triggered by fundamentally different causal mechanisms. Specifically, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Causal-Fuzzer</monospace> proposes (1) a causality-based feedback mechanism that quantifies the combined diversity of test scenarios by assessing whether they activate new causal relationships, and (2) a causality-driven mutation strategy that prioritizes mutations on input scenario elements with higher causal impact on ego action changes and violation occurrence to enable interpretable and efficient test generation. We evaluated <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Causal-Fuzzer</monospace> on an industry-grade ADS Apollo, with a high-fidelity simulator LGSVL. Our empirical results demonstrate that <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Causal-Fuzzer</monospace> significantly outperforms existing methods in (1) identifying a greater diversity of violations (96.5 violations on average, compared to 66.9 for the best baseline method), (2) providing enhanced testing sufficiency with improved coverage of causal relationships (13.6 unique sceneaction- violation patterns on average, compared to 8.6 for the best baseline method), and (3) achieving greater efficiency in detecting critical scenarios, strong robustness under noise conditions, and good generalizability across varying scenario complexities and violation types. Our source code and experimental results are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://sites.google.com/view/causal-fuzzer</uri>.

The generation of motion behaviors plays a pivotal role in constructing effective simulated scenarios for testing autonomous driving systems (ADSs). The controllability (i.e., the ability to synthesize specific motion patterns) and multimodality (i.e., the capacity to represent multiple motion intentions) of generated motion behaviors are essential for the purposeful and comprehensive evaluation of ADS. Although recent studies have made progress in either multimodal or controllable motion behavior generation, it remains a major challenge to simultaneously generate multimodal motion behaviors in a controllable manner. In this work, we propose a unified framework, CoMoGen, to generate multimodal motion behaviors in a controllable manner under open-loop evaluation assumption. The proposed framework consists of three core components: i) a learning-based vehicle placer, responsible for positioning generated vehicles in non-conflicting initial locations; ii) a robust model-based trajectory candidate generator, capable of synthesizing controllable and multimodal trajectory candidates. iii) a learning-based trajectory selector, developed to evaluate and select multimodal trajectories for the placed vehicles. Experiments on the INTERACTION dataset demonstrate strong controllability and multimodality of CoMoGen. Further experiments on three additional real-world datasets, that are unseen during training, as well as on diverse synthesized high-definition maps, validate the remarkable generalization capability of CoMoGen.

Autonomous Driving Systems (ADS) are safety-critical. Abundant and various driving scenarios are required to train accurate and robust models, and comprehensively test each module of autonomous driving systems (i.e., perception, tracking, prediction, planning, and control modules). However, collecting driving scenario data from the real physical world is expensive and inefficient. Most existing works generate simulated driving scenarios by varying the behaviors of dynamic objects on simple road networks (e.g., highways), while the influence of roadside structures and scenarios with complex road networks are not considered. This paper proposes a novel driving scenario generation approach, Automated Scenario Crafting (AutoSceCraft), to automatically produce abundant driving scenarios containing various road networks, traffic rules, roadside structures, and dynamic objects at low cost. To validate the effectiveness and efficiency of our proposed framework, AutoSceCraft is integrated into three popular driving simulators, including SMARTS, esmini, and CARLA. Numerical experiments and scenario visualization results show that AutoSceCraft can generate effectively and efficiently various driving scenarios from scratch for testing and training various modules (including perception, prediction, and planning modules) within autonomous driving systems.

Despite the growing importance of autonomous driving (AD) education, educators face substantial challenges in providing undergraduate students with accessible, hands-on learning experiences that connect theory and practice. AD education requires students to master complex, interdisciplinary systems, yet traditional teaching methods often fail to fully integrate theory and practice. Various strategies, such as simulation environments and small-scale autonomous vehicle platforms, have been explored to address these challenges. However, these approaches often lack the realism or scalability required to provide students with a comprehensive understanding of AD systems. To address these challenges, we developed and evaluated a competition-based learning framework designed to teach AD software technology. In a three-year iterative design study, we implemented a framework combining a cost-effective 1/5-scaled autonomous vehicle platform with team-based competitions. This framework was progressively improved through a mixed-methods analysis of data from 445 students across five competitions and two capstone design courses. The quantitative results showed statistically significant improvements in the students’ self-assessed skills with large effect sizes, demonstrating that the framework significantly boosted students’ technical abilities and motivation. The qualitative feedback confirmed the educational value of the platform’s realism and peer-based observations of the competitions. This article presents the complete educational framework, outlines the iterative design process and key design decisions, and discusses the lessons learned. It also provides a validated model and practical guidelines for educators seeking to incorporate effective, competition-based AD education into their curricula.

With self-driving technology, safety under intricate conditions for vehicles is indispensable. Image dehazing as a key element of visual sensing for self-driving vehicles is indispensable under poor environmental conditions. In complex traffic scenarios, visual perception systems must accurately recognize obstacles, traffic signs, and lane structures to ensure driving safety. However, under adverse weather conditions such as fog, haze, or sandstorms, image quality can severely degrade, leading to diminished detection performance and potential safety risks. The paper reviews the progression of image dehazing for self-driving vehicles with classical methods, deep learning-based methods, and unsupervised learning-based methods. The classical methods show efficiency but cannot cope with instability under intricate conditions. Deep learning-based methods, particularly CNNs and Transformers, have registered remarkable progress with issues of computational costs and data adaptability. Unsupervised learning-based methods reduce reliance on labeled data with issues of training instability and incomplete reconstruction. The paper also explores image dehazing as an integration with object detection and multi-sensor fusion, with future directions on lightweight design, data generalization, and multi-modal fusion.

Ensuring the safety of Autonomous Driving Systems (ADS) at urban intersections remains challenging due to complex interactions between vehicles and traffic management infrastructure. This study validates an ADS equipped with connected perception using Infrastructure-to-Vehicle (I2V) communication within a combined virtual and hybrid testing approach. The validation follows the overall structure and methodology of the SUNRISE Safety Assurance Framework (SAF), which is applied in detail where required by the scope of the study. Five representative urban intersection scenarios, covering both nominal driving conditions and safety-critical edge cases, are evaluated using virtual simulations in MATLAB/Simulink (2014b) and hybrid experiments integrating OMNeT++ (5.7.1)/Veins (5.2)/SUMO (1.12.0) with real-world components. Key Performance Indicators (KPIs) related to safety, decision-making, longitudinal control, passenger comfort, and V2X communication performance are analyzed. The results show strong consistency between virtual and hybrid testing, with ego vehicle speed deviations below 2 km/h and trigger distance differences under 3 m. V2X communication achieves a near-perfect Cooperative Awareness Message (CAM) delivery ratio, with an average latency of approximately 142 ms. While this latency remains within the tolerance of the deployed ADS, the overall end-to-end delay highlights opportunities for further optimization. The study demonstrates how the SUNRISE SAF can effectively structure ADS validation, identifies critical scenarios such as right-of-way violations by non-priority obstacles, and provides insights into improving connectivity handling and low-speed braking behavior for Cooperative, Connected, and Automated Mobility (CCAM) systems in urban environments.

Ensuring safe operation of autonomous vehicle: A comprehensive survey of Operational Design Condition.

Autonomous driving (AD) has emerged as a transformative technology that holds the potential to free humans from the need for manual driving and provide a safer, more comfortable and efficient driving experience. However, most AD systems make decisions solely based on vehicle dynamics and environmental factors such as road conditions and surrounding vehicles, while the occupant’s mental states, such as subjective feelings and experience, are neglected. As a result, autonomous vehicles (AVs) often fail to meet the occupant’s physical and mental demands, ultimately leading to a compromised driving experience. In this study, we propose an occupant-aware decision-making paradigm (ODP) for AD systems. ODP first perceives the occupant’s physical and physiological states that are closely related to mental states, such as facial expressions and physiological signals, through the occupant monitoring system (OMS). Then, a large vision-language model (VLM) processes the occupant’s physical and physiological states via the chain of thought (CoT) technique to analyze the occupant’s mental states and infer the occupant’s needs. Finally, the VLM makes driving decisions that match the occupant’s demands and preferences. Experimental results show that ODP can make decisions that are significantly better aligned with the occupant’s actual needs than existing methods.

Autonomous driving systems (ADSs) must be sufficiently tested to ensure their safety. Though various ADS testing methods have shown promising results, they are limited to a fixed vehicle characteristics setting (VCS). The impact of variations in vehicle characteristics (e.g., mass, tire friction) on the safety of ADSs has not been sufficiently and systematically studied. Such variations are often due to wear and tear, production errors and so on, which may lead to unexpected driving behaviours of ADSs. To this end, in this article, we propose a method, named SafeVar , to systematically find minimum variations to the original vehicle characteristics setting, which affect the safety of the ADS deployed on the vehicle. To evaluate the effectiveness of SafeVar , we employed two ADSs and conducted experiments with two driving scenarios. Results show that SafeVar , equipped with NSGA-II, generates more critical settings that put the vehicle into unsafe situations, as compared with the baseline algorithm. We also identified critical vehicle characteristics and reported to which extent varying their settings put the ADS vehicle into unsafe situations.

This study investigated how people respond to driving errors committed by human drivers versus algorithm-controlled autonomous driving systems, focusing on how the driving agent and error severity shape error tolerance and trust. Drawing on Social Identity Theory and the Perfect Automation Schema, we proposed that autonomous driving systems operated by algorithms are perceived as out-group entities held to extremely high performance standards; consequently, even minor errors caused by autonomous vehicle algorithms may elicit disproportionately negative reactions. We conducted an online experiment employing a 2 (driving agent: algorithm vs. human) × 2 (error severity: fatal vs. minor) between-subjects design. Participants (N = 800) were randomly assigned to read one of four driving error scenarios, and then evaluated the error as well as the driving agent's trustworthiness. Results revealed a significant interaction between driving agent and error severity. When the error was fatal, both driving agents received highly negative evaluations on both error tolerance and trust. By contrast, when the error was minor, participants were more tolerant of the human error and expressed greater trust in the human driver than in the algorithm. These findings suggest that even minor, noncritical malfunctions in autonomous driving systems can undermine users' confidence and exacerbate negative evaluations. Overall, this study highlights social and cognitive biases in how people perceive and judge autonomous driving systems, deepening our understanding of the algorithm aversion phenomenon.

Simulation models for perception sensors are integral components of automotive simulators used for the virtual validation of Autonomous Driving Systems (ADS). They also function as powerful tools for generating synthetic datasets to train deep learning-based perception models. LiDAR is a widely adopted perception sensor in ADS due to its high precision in 3D environment scanning. However, developing realistic LiDAR simulation models is a significant technical challenge. Unrealistic simulations can result in a large gap between the synthesised and real-world point clouds, limiting their effectiveness in perception and planning modules. Recently, deep generative models have emerged as promising solutions to synthesise realistic sensory data. However, for LiDAR simulation, deep generative models have been primarily hybridised with conventional algorithms, leaving unified generative approaches largely unexplored in the literature. Addressing this gap, we propose a unified generative framework to enhance LiDAR simulation fidelity. Our proposed framework projects LiDAR point clouds into depth-reflectance images via a lossless transformation, and employs our novel Controllable Lidar point cloud Generative model, CoLiGen, to translate the images. We extensively evaluate our CoLiGen model, comparing it with the state-of-the-art image-to-image translation models using various metrics to assess the realness, faithfulness, and performance of a downstream perception model. Our results show that CoLiGen exhibits superior performance across most metrics. The dataset and source code for this research are available at https://github.com/hamedhaghighi/CoLiGen.git.

Facial Trauma Prevention: Harnessing the Power of Autonomous Driving Systems.

Safety testing serves as the fundamental pillar for the development of autonomous driving systems (ADSs), and decision-making plays a key role in ADSs. To ensure the safety of ADSs, it is paramount to generate a range of critical test scenarios to test the safety of decision-making in ADSs. While existing researches primarily focus on reproducing real-world traffic accidents in simulation environments to create test scenarios, it is essential to highlight that many of these accidents do not result in safety violations of decision-making in ADSs due to the differences between human driving and autonomous driving. More importantly, we observe that some accident-free real-world scenarios can lead to misbehaviors of ADSs. Therefore, orthogonally to existing work, it is equally important to discover safety violations of ADSs from routine traffic scenarios (i.e., accident-free scenarios) to ensure the safety of Autonomous Vehicles (AVs). We introduce CRISER , a novel methodology to achieve the above goal. It automatically generates abstract and concrete scenarios from real-traffic videos where human-driving worked safely. Based on them, CRISER discovers safety violations of the ADS's decision-making in semantic equivalent scenarios (i.e., the test scenarios with the same semantics as the original accident-free traffic videos). Specifically, CRISER enhances the ability of Large Multimodal Models (LMMs) to accurately extract scenario semantics from accident-free traffic videos and generate test scenarios by multi-modal few-shot Chain of Thought (CoT). Based on them, CRISER explores the behavior differences between the ego vehicle (i.e., the vehicle connected to the ADS under test) and human-driving in semantic equivalent scenarios. During the exploration search, CRISER keeps the semantic consistency of test scenarios with accident-free traffic videos, and explores the universality of discovered safety violations of the ADS. We implement and evaluate CRISER on the industrial-grade Level-4 ADS, Apollo. The experimental results demonstrate that CRISER can accurately extract scenario semantics and generate test scenarios from traffic videos, and effectively discover distinct types of safety violations of Apollo’s decision-making in accident-free traffic scenarios.

Interest in the adoption of autonomous vehicles (AVs) continues to grow. It is essential to design new software architectures that meet stringent real-time, safety, and scalability requirements while integrating heterogeneous hardware and software solutions from different vendors and developers. This paper presents a lightweight, modular, and scalable architecture grounded in Service-Oriented Architecture (SOA) principles and implemented in ROS 2 (Robot Operating System 2). The proposed design leverages ROS 2’s Data Distribution System-based Quality-of-Service model to provide reliable communication, structured lifecycle management, and fault containment across distributed compute nodes. The architecture is organized into Perception, Planning, and Control layers with decoupled sensor access paths to satisfy heterogeneous frequency and hardware constraints. The decision-making core follows an event-driven policy that prioritizes fresh updates without enforcing global synchronization, applying zero-order hold where inputs are not refreshed. The architecture was validated on a 1:10-scale autonomous vehicle operating on a city-like track. The test environment covered canonical urban scenarios (lane-keeping, obstacle avoidance, traffic-sign recognition, intersections, overtaking, parking, and pedestrian interaction), with absolute positioning provided by an indoor GPS (Global Positioning System) localization setup. This work shows that the end-to-end Perception–Planning pipeline consistently met worst-case deadlines, yielding deterministic behaviour even under stress. The proposed architecture can be deemed compliant with real-time application standards for our use case on the 1:10 test vehicle, providing a robust foundation for deployment and further refinement.

The rapid advancement of Large Language Models (LLMs) has opened new avenues for their integration into autonomous driving systems (ADS). This paper provides a comprehensive literature review of the emerging role of LLMs in ADS, focusing on high-level strategic functions such as route planning, scene understanding, human-vehicle interaction, and safety analysis. We analyse over 40 recent studies and categorise the application of LLMs into five key domains: strategic planning and reasoning, multimodal environment perception, human-LLM interaction and personalisation, risk prediction and safety enhancement, and technical limitations of real-time LLM integration. Our findings suggest that while LLMs excel in semantic interpretation and explainable decision-making, they face significant challenges in real-time control due to latency, inconsistency, and unverifiable outputs. As such, their use should be restricted to strategic-level components where human oversight and higher response tolerances are acceptable. Based on these insights, we outline four significant research gaps related to real-world validation, interface standardisation, multimodal fusion, and risk reasoning. We recommend hybrid architectures and inference optimisation as promising directions for the future development of safe and scalable LLM-enhanced autonomous transport systems.

With the transformation of artificial intelligence from a "rule-driven" to a "data-driven" paradigm, the end-to-end architecture has become a frontier direction in the development of intelligent driving systems. However, the "black-box" nature of its decision-making poses significant challenges to the current admission management framework based on modular design, resulting in insufficient regulatory foundations and incomplete policy coverage. To address these risks, a scenario-based dynamic risk quantification model is developed to provide a scientific basis for accurate assessment and hierarchical supervision. A full-chain safety management mechanism covering four dimensions is further established to systematically control risks throughout the entire lifecycle of the technology. Finally, three core policy optimization measures are formulated to facilitate the transition from "static access" to "dynamic access + continuous safety assessment", to strengthen the supporting standards and technical systems, to define the mechanisms for hierarchical accountability and social risk sharing, and to establish a governance framework for the safe and orderly deployment of end-to-end intelligent driving systems.

As the core system in autonomous vehicles, Autonomous Driving Systems (ADSs) are highly configurable, where misconfigurations can significantly impact control safety, reliability, and overall performance. Although several testing methods have been proposed to detect misconfiguration-induced violations, most primarily focus on identifying the presence of incorrect configurations rather than pinpointing the specific configuration parameters responsible for these violations. However, identifying and understanding the root causes of misconfiguration-induced violations is essential for effective debugging and rapid system recovery. In this paper, we propose a CounterFactual-based Root Cause Analysis (RCA) method, CF-RCA, to identify the root causes of misconfiguration-induced violations by performing counterfactual attribution. Specifically, CF-RCA first formalizes the relationships between various configuration parameters and violations by learning a structural causal model. Then, based on the causal model, CF-RCA employs counterfactual attribution to estimate the impact of each configuration parameter on violations and identifies the most impactful parameter as the RCA result. We evaluate CF-RCA on the MetaDrive simulator with 12,926 driving scenarios, and the results show CF-RCA can efficiently identify violation-causing parameters, achieving 98.3% accuracy. Finally, the experimental comparisons with existing methods and tests across different ADSs further demonstrate the superiority and generalizability of CF-RCA.

Make full use of testing information: An integrated accelerated testing and evaluation method for autonomous driving systems.