Autonomous Driving Research Articles

Deep Learning-Enhanced Environment Perception for Autonomous Driving: MDNet with CSP-DarkNet53

Deep Learning-Enhanced Environment Perception for Autonomous Driving: MDNet with CSP-DarkNet53

Edge-Cloud Continuum: Integrating Edge Computing and Cloud Computing for IOT Applications

Abstract: This research will use a mixed-method approach with qualitative understanding and the quantitative analysis of data to examine how edge and cloud computing blend into the frame of an edge-cloud continuum for IoT applications. It adopts an exploratory and descriptive approach to research in carrying out a holistic assessment of the edge-cloud continuum in efficiency, scalability, and performance. The data required for collection was obtained through simulations using the IoTSim-Osmosis framework and also through the use of other case studies and literature published elsewhere before preparing this document. The data is usual to what has been used in several IoT deployments, including smart cities, the health sector, and industrial automation. Based on those KPIs - latency, 70.83%; bandwidth utilization, 40% reduced; energy consumption, 25% reduced; and task completion rate improved by 18.75% - the edge-cloud continuum significantly outperforms traditional cloud-only systems. Especially a very big reduction in latency is significantly important for real-time applications as it would drive the potential ability to improve responsiveness in those applications, like autonomous cars and smart healthcare. The current study demonstrates that the edge-cloud continuum can efficiently enhance resource allocation and enhance the effectiveness of complex IoT systems. This has wide implications for designing and implementing IoT solutions across industry domains

Research and Analysis of Deep Learning in Environmental Perception for Autonomous Driving

Autonomous driving technology, as a revolutionary development in the transportation field, relies on efficient and reliable environmental perception systems. Deep learning plays a key role in the environmental perception of autonomous driving, with its capabilities widely applied in areas such as signal and sign recognition, semantic segmentation, instance segmentation, and SLAM (Simultaneous Localization and Mapping). This study conducts research and analysis on the aforementioned key technologies. It concludes that while deep learning technology has shown excellent performance in the perceptual systems of autonomous driving, it still faces challenges that require further research and resolution, such as generalization performance, eal-time requirements, and safety issues.

Integrated automatic parking path planning and trajectory tracking optimization method

Current automatic parking systems often separate path planning and trajectory tracking, increasing complexity and failing to meet the parking needs of autonomous cars. Consequently, this study presents an approach based on Multi-stage Nonlinear Model Predictive Control (MSNMPC) for integrated autonomous parking path planning and trajectory tracking optimization. The method represents the evolution of uncertainty parameters over time through a scenario tree, and the MSNMPC controller with a predictive horizon P defines P + 1 phases, each with specific cost and constraint functions that depend only on the vehicle state and control inputs of that phase to satisfy the constraints of all phases of the parking process. In addition, the method integrates path planning and tracking control into one optimization problem, which is solved online to achieve integrated parking control. Simulation confirms that, in comparison to MPC with RRT* hierarchical control, the integrated parking scheme of MSNMPC has less lateral error and offers superior flexibility and tracking performance. To verify the generality of the scheme, it was validated in the diagonal parking scenario and the vertical parking scenario, respectively. The results show that, compared to the control experiments, the parking elapsed time has been reduced by 10.76% and 9.02%, respectively, enhancing parking efficiency. In addition, the parking error has decreased by 30.77% and 38.46%, respectively, thus improving parking accuracy. Moreover, the minimum safe distance in the control scheme for addressing uncertainty factors is 0.7 m greater than that of the control experiments, meeting the driving requirements for driverless vehicles in parking scenarios.

Investigating Electromagnetic Interference Induced Risks for Autonomous Driving

Abstract. The results of immunity tests on components of self-driving automobiles to radiated high power electromagnetic (HPEM) pulses are presented in this work. It is of particular interest to investigate such automobiles' resilience towards deliberate attacks with electromagnetic interference (EMI). Different types of HPEM capabilities at the Bundeswehr Research Institute for Protective Technologies and CBRN Protection (WIS) in Munster are applied for this investigation. Two types of automotive components have been tested, a two-axis acceleration sensor and an electronic power steering unit. Statistics of errors from the devices under test (DUTs), as well as a characterization of the applied pulses are presented. A correlation can be drawn between the pulse repetition rate or the amplitude of the applied pulses and the severity of the effect on the DUTs.

Development and Validation of a Driving Simulator for Comfort Assessment

AbstractThe passenger’s comfort is of vital importance in today’s car, even more so in the future autonomous car. Its thorough analysis requires the use of objective measures, but the subjectivity of the passenger must also be considered. The relationship between both aspects is an uncommon research avenue, and could greatly benefit from the use of a vehicle simulator. This study use a simulator to replicate a journey with diverse characteristics that numerous passengers may experience (including engaging in different activities). The appropriate variables for a proper comfort analysis are determined in this study, resulting in a comprehensive database for the same purpose. Additionally, a passenger comfort based questionnaire is developed and applied to obtain the subjective assessment of different passengers. The results of these questionnaires help not only in the identification and study of the self perception of comfort by the passengers but also in the use of the simulator for comfort experiments. The detailed comparison between both the objective approach of signal analysis and the subjective approach of questionnaires that is performed thanks to the simulator creates a foundation for future research and narrows the existing research gap.

Behavioral Cloning Strategies in Steering Angle Prediction: Applications in Mobile Robotics and Autonomous Driving

Artificial neural networks (ANNs) are artificial intelligence techniques that have made autonomous driving more efficient and accurate; however, autonomous driving faces ongoing challenges in the accuracy of decision making based on the analysis of the vehicle environment. A critical task of ANNs is steering angle prediction, which is essential for safe and effective navigation of mobile robots and autonomous vehicles. In this study, to optimize steering angle prediction, NVIDIA’s architecture was adapted and modified along with the implementation of the Swish activation function to train convolutional neural networks (CNNs) by behavioral cloning. The CNN used human driving data obtained from the UDACITY beta simulator and tests in real scenarios, achieving a significant improvement in the loss function during training, indicating a higher efficiency in replicating human driving behavior. The proposed neural network was validated through implementation on a differential drive mobile robot prototype, by means of a comparative analysis of trajectories in autonomous and manual driving modes. This work not only advances the accuracy of steering angle predictions but also provides valuable information for future research and applications in mobile robotics and autonomous driving. The performance results of the model trained with the proposed CNN show improved accuracy in various operational contexts.

A Better Data Foundation for Highly Automated and Autonomous Driving Functions

A Better Data Foundation for Highly Automated and Autonomous Driving Functions

A Design Method for Road Vehicles with Autonomous Driving Control

The past three decades have witnessed extensive studies on motion-planning and tracking-control for autonomous vehicles (AVs). There is, however, a lack of studies on effective design methods for AVs, which consist of the subsystems of the mechanical vehicle, tracking-control, motion-planning, etc. To tackle this problem, this paper proposes a design approach for AVs. The proposed method features a design framework with two layers: at the upper layer, a particle swarm optimization (PSO) algorithm serves as a solver to a multi-objective optimization problem for desired AV trajectory-tracking performance; at the lower layer, a coupled dynamic analysis is conducted among the three subsystems, i.e., a nonlinear model for the mechanical vehicle, a motion-planning module, and a controller based on nonlinear model predictive control (NLMPC) for direction control. The simulation results demonstrate that the proposed method can effectively determine the desired design variables for the NLMPC controller and the mechanical vehicle to achieve optimal trajectory-tracking performance. The research findings from this work provide guidelines for designing AVs.

Designing LiDAR‐Detectable Dark‐Tone Materials with High Near‐Infrared Reflectivity for Autonomous Driving: A Comprehensive Review

AbstractAutonomous driving relies on the precise recognition of objects using light detection and ranging (LiDAR) technology, that operates at a specific wavelength of 905 nm. Black objects, such as carbon black used in vehicle coating, tend to absorb this specific wavelength significantly, which limits the performance of LiDAR sensors. To address this issue, researchers have explored creating dark‐toned materials that can be detected by LiDAR with high NIR reflectivity while maintaining a true blackness (L* < 20 based on the CIE color coordinates). These materials fall into two categories: organic and inorganic pigments. Organic pigments can be synthetically adjusted to achieve true blackness by manipulating their functional groups, but achieving high NIR reflectivity remains challenging, often requiring a bilayer structure with NIR‐reflective white base and an upper layer of organic black pigments. Additionally, the need for hydrophobic additives and resistance to degradation from sunlight further restricts their use. In the case of inorganic pigments, the desired LiDAR‐detectable properties can be obtained through careful control of their composition, structure, and morphology, allowing for single‐layer coatings with appropriate design. This review highlights recent advancements in developing organic and inorganic LiDAR‐detectable black pigments and outlines future material design strategies for autonomous vehicle systems.

Neural Approach to Coordinate Transformation for LiDAR–Camera Data Fusion in Coastal Observation

The paper presents research related to coastal observation using a camera and LiDAR (Light Detection and Ranging) mounted on an unmanned surface vehicle (USV). Fusion of data from these two sensors can provide wider and more accurate information about shore features, utilizing the synergy effect and combining the advantages of both systems. Fusion is used in autonomous cars and robots, despite many challenges related to spatiotemporal alignment or sensor calibration. Measurements from various sensors with different timestamps have to be aligned, and the measurement systems need to be calibrated to avoid errors related to offsets. When using data from unstable, moving platforms, such as surface vehicles, it is more difficult to match sensors in time and space, and thus, data acquired from different devices will be subject to some misalignment. In this article, we try to overcome these problems by proposing the use of a point matching algorithm for coordinate transformation for data from both systems. The essence of the paper is to verify algorithms based on selected basic neural networks, namely the multilayer perceptron (MLP), the radial basis function network (RBF), and the general regression neural network (GRNN) for the alignment process. They are tested with real recorded data from the USV and verified against numerical methods commonly used for coordinate transformation. The results show that the proposed approach can be an effective solution as an alternative to numerical calculations, due to process improvement. The image data can provide information for identifying characteristic objects, and the obtained accuracies for platform dynamics in the water environment are satisfactory (root mean square error—RMSE—smaller than 1 m in many cases). The networks provided outstanding results for the training set; however, they did not perform as well as expected, in terms of the generalization capability of the model. This leads to the conclusion that processing algorithms cannot overcome the limitations of matching point accuracy. Further research will extend the approach to include information on the position and direction of the vessel.

Bidirectional Planning for Autonomous Driving Framework with Large Language Model.

Autonomous navigation systems often struggle in dynamic, complex environments due to challenges in safety, intent prediction, and strategic planning. Traditional methods are limited by rigid architectures and inadequate safety mechanisms, reducing adaptability to unpredictable scenarios. We propose SafeMod, a novel framework enhancing safety in autonomous driving by improving decision-making and scenario management. SafeMod features a bidirectional planning structure with two components: forward planning and backward planning. Forward planning predicts surrounding agents' behavior using text-based environment descriptions and reasoning via large language models, generating action predictions. These are embedded into a transformer-based planner that integrates text and image data to produce feasible driving trajectories. Backward planning refines these trajectories using policy and value functions learned through Actor-Critic-based reinforcement learning, selecting optimal actions based on probability distributions. Experiments on CARLA and nuScenes benchmarks demonstrate that SafeMod outperforms recent planning systems in both real-world and simulation testing, significantly improving safety and decision-making. This underscores SafeMod's potential to effectively integrate safety considerations and decision-making in autonomous driving.

Responses of Vehicular Occupants During Emergency Braking and Aggressive Lane-Change Maneuvers.

To validate active human body models for investigating occupant safety in autonomous cars, it is crucial to comprehend the responses of vehicle occupants during evasive maneuvers. This study sought to quantify the behavior of midsize male and small female passenger seat occupants in both upright and reclined postures during three types of vehicle maneuvers. Volunteer tests were conducted using a minivan, where vehicle kinematics were measured with a DGPS sensor and occupant kinematics were captured with a stereo-vision motion capture system. Seatbelt loads, belt pull-out, and footrest reaction forces were also documented. The interior of the vehicle was 3D-scanned for modeling purposes. Results indicated that seatback angles significantly affected occupant kinematics, with small female volunteers displaying reduced head and torso movements, except during emergency braking with a upright posture seatback. Lane-change maneuvers revealed that maximum lateral head excursions varied depending on the maneuver's direction. The study concluded that seatback angles were crucial in determining the extent of occupant movement, with notable variations in head and torso excursions observed. The collected data assist in understanding occupant behavior during evasive maneuvers and contribute to the validation of human body models, offering essential insights for enhancing safety systems in autonomous vehicles.

Exploring Deep Q-Network for Autonomous Driving Simulation Across Different Driving Modes

The rapid growth in vehicle ownership has led to increased traffic congestion, making the need for autonomous driving solutions more urgent. Autonomous Vehicles (AVs) offer a promising solution to improve road safety and reduce traffic accidents by adapting to various driving conditions without human intervention. This research focuses on implementing Deep Q-Network (DQN) to enhance AV performance in different driving modes: safe, normal, and aggressive. DQN was selected for its ability to handle complex, dynamic environments through experience replay, asynchronous training, and epsilon-greedy exploration. We designed a simulation environment using the Highway-env platform and evaluated the DQN model under varying traffic densities. The performance of the AV was assessed based on two key metrics: success rate and total reward. Our findings show that the DQN model achieved a success rate of 90.75%, 94.625%, and 95.875% in safe, normal, and aggressive modes, respectively. Although the success rate increased with traffic intensity, the total reward remained lower in aggressive driving scenarios, indicating room for optimization in decision-making processes under highly dynamic conditions. This study demonstrates that DQN can adapt effectively to different driving needs, but further optimization is needed to enhance performance in more challenging environments. Future work will focus on improving the DQN algorithm to maximize both success rate and reward in high-traffic scenarios and testing the model in more diverse and complex environments.

Axes-aligned non-linear optimized PnP algorithm

Pose estimation is an important component of many real-world computer vision systems. Most existing pose estimation algorithms need a large number of point correspondences to accurately determine the pose of an object. Since the number of point correspondences depends on the object’s appearance, lighting and other external conditions, detecting many points may not be feasible. In many real-world applications, the movement of objects is limited, e.g. due to gravity. Hence, detecting objects with only three degrees of freedom is usually sufficient. This allows us to improve the accuracy of pose estimation by changing the underlying equations of the perspective-n-point problem to three variables instead of six. By using the simplified equations, our algorithm is more robust against detection errors with limited point correspondences. In this article, we study three scenarios where such constraints apply. The first one is about parking a vehicle on a specific spot. Here, a stationary camera is detecting the vehicle to assist the driver. The second scenario describes the perspective of a moving camera detecting objects in its environment. This scenario is common for driver assistance systems, autonomous cars or mobile robots. Third, we describe a camera observing objects from a birds-eye view, which occurs in industrial applications. In all three scenarios, observed objects can only move in the ground plane and rotate around the vertical axis. Hence, three degrees of freedom are sufficient to estimate the pose. Experiments with synthetic data and real-world photographs have shown that our algorithm outperforms state-of-the-art pose estimation algorithms. Depending on the scenario, our algorithm is able to achieve 50% better accuracy, while being equally fast.

Enhancing the Heat Dissipation Efficiency of Computing Units Within Autonomous Driving Systems and Electric Vehicles

With the rapid development of autonomous driving technology and electric vehicles, the demand for computing units in vehicles has surged, particularly for high-density computing related to artificial intelligence, sensor fusion, and real-time data processing. This increase has posed significant heat dissipation challenges for automotive electronic systems. Effective heat dissipation is essential for ensuring system stability, safety, and extending the lifespan of these systems. This study explores the thermal design of computing units in current autonomous driving systems and electric vehicles. The electronic components within the computing units generate significant heat, but due to environmental constraints, fans can not be used for internal cooling, as their operation can draw in dust from the environment, contaminating the internal electronic components. Therefore, computing units are often designed with a sealed structure, utilizing the thermal conductivity of metal casings to quickly dissipate internal heat sources. The results of this study show that comparing embedded cooling modules within the casing to commonly used thermal pads reveals that changing the materials of the cooling module can significantly enhance cooling performance, noticeably reducing the temperature of the internal electronic components and minimizing the risk of overheating. Additionally, this can improve the overall reliability and performance of the system, providing new ideas and application prospects for future automotive thermal design.

Want a ride? Attitudes towards autonomous driving and behavior in autonomous vehicles

SummaryResearch conducted previously has focused on either attitudes toward or behaviors associated with autonomous driving. In this paper, we bridge these two dimensions by exploring how attitudes towards autonomous driving influence behavior in an autonomous car. We conducted a field experiment with twelve participants engaged in non-driving related tasks. Our findings indicate that attitudes towards autonomous driving do not affect participants’ driving interventions in vehicle control and eye glance behavior. Therefore, studies on autonomous driving technology lacking field tests might be unreliable for assessing the potential behaviors, attitudes, and acceptance of autonomous vehicles.

Assessing Unknown Hazards for SOTIF Based on Twin Scenarios Empowered Autonomous Driving

Assessing Unknown Hazards for SOTIF Based on Twin Scenarios Empowered Autonomous Driving

The Square-Root Unscented and the Square-Root Cubature Kalman Filters on Manifolds.

Estimating the state of a system by fusing sensor data is a major prerequisite in many applications. When the state is time-variant, derivatives of the Kalman filter are a popular choice for solving that task. Two variants are the square-root unscented Kalman filter (SRUKF) and the square-root cubature Kalman filter (SCKF). In contrast to the unscented Kalman filter (UKF) and the cubature Kalman filter (CKF), they do not operate on the covariance matrix but on its square root. In this work, we modify the SRUKF and the SCKF for use on manifolds. This is particularly relevant for many state estimation problems when, for example, an orientation is part of a state or a measurement. In contrast to other approaches, our solution is both generic and mathematically coherent. It has the same theoretical complexity as the UKF and CKF on manifolds, but we show that the practical implementation can be faster. Furthermore, it gains the improved numerical properties of the classical SRUKF and SCKF. We compare the SRUKF and the SCKF on manifolds to the UKF and the CKF on manifolds, using the example of odometry estimation for an autonomous car. It is demonstrated that all algorithms have the same localization performance, but our SRUKF and SCKF have lower computational demands.

4Seasons: Benchmarking Visual SLAM and Long-Term Localization for Autonomous Driving in Challenging Conditions

AbstractIn this paper, we present a novel visual SLAM and long-term localization benchmark for autonomous driving in challenging conditions based on the large-scale 4Seasons dataset. The proposed benchmark provides drastic appearance variations caused by seasonal changes and diverse weather and illumination conditions. While significant progress has been made in advancing visual SLAM on small-scale datasets with similar conditions, there is still a lack of unified benchmarks representative of real-world scenarios for autonomous driving. We introduce a new unified benchmark for jointly evaluating visual odometry, global place recognition, and map-based visual localization performance which is crucial to successfully enable autonomous driving in any condition. The data has been collected for more than one year, resulting in more than 300 km of recordings in nine different environments ranging from a multi-level parking garage to urban (including tunnels) to countryside and highway. We provide globally consistent reference poses with up to centimeter-level accuracy obtained from the fusion of direct stereo-inertial odometry with RTK GNSS. We evaluate the performance of several state-of-the-art visual odometry and visual localization baseline approaches on the benchmark and analyze their properties. The experimental results provide new insights into current approaches and show promising potential for future research. Our benchmark and evaluation protocols will be available at https://go.vision.in.tum.de/4seasons.