Autonomous Vehicle Perception Research Articles

Improving Autonomous Vehicle Perception through Evaluating LiDAR Capabilities and Handheld Retroreflectivity Assessments

Improving Autonomous Vehicle Perception through Evaluating LiDAR Capabilities and Handheld Retroreflectivity Assessments

Impact of risk perception and trust in autonomous vehicles on pedestrian crossing decision: Navigating the social-technological intersection with the ICLV model

In the rapidly evolving realm of transportation technology, the dynamic relationship between pedestrians and technological innovations has attained unprecedented importance. The complex social-technological intersection surrounding pedestrian road crossings has emerged as an attention for traffic safety. What distinguishes the contemporary urban environment is the rapid assimilation of intelligent transportation systems (ITS) into the transportation infrastructure, including technological elements such as autonomous vehicles, advanced surveillance systems, and smart infrastructure. To investigate how pedestrians perceive risks, trust technology, and make decisions in this era of technological progress, we designed a video-based questionnaire utilizing the stated preference (SP) methodology. We collected SP data from 589 Chinese pedestrians and employed an integrated choice and latent variable (ICLV) model to quantify the influence of risk perception and trust in autonomous vehicle (trust in AV), treated as latent variables, on their crossing decisions. Our findings indicate that the presence of autonomous vehicles significantly affects pedestrian crossing decisions. Specifically, an increase in the approaching vehicle speed and a decrease in the approaching vehicle distance increase the pedestrians’ tendency to choose not to cross the road, and the latent variables of risk perception and trust in AV strongly predict this phenomenon. The results of the scenario analysis show that, compared with overall pedestrians, middle-aged pedestrians and high-risk perception-level pedestrians are more conservative in their crossing decisions, but high levels of trust in AV improve pedestrians’ willingness to cross the street. Additionally, the pedestrian-related findings of this study at the social-technological intersection provide better understanding of the decision process and contribute to the planning and development of urban intelligent transportation systems.

Sensor and sensor fusion technology in autonomous vehicles

The perception and navigation of autonomous vehicles heavily rely on the utilization of sensor technology and the integration of sensor fusion techniques, which play an essential role in ensuring a secure and proficient understanding of the vehicle's environment.This paper highlights the significance of sensors in autonomous vehicles and how sensor fusion techniques enhance their capabilities. Firstly, the paper introduces the different types of sensors commonly used in autonomous vehicles and explains their principles of operation, strengths, and limitations in capturing essential information about the vehicles environment. Next, the paper discusses various sensor fusion algorithms, such as Kalman filters and particle filters. Furthermore, the paper explores the challenges associated with sensor fusion and addresses the issue of handling sensor failures or uncertainties. The benefits of sensor fusion technology in autonomous vehicles are also presented. These include improved perception of the environment, enhanced object recognition and tracking, better trajectory planning, and enhanced safety through redundancy and fault tolerance. Lastly, the paper discusses the advancements and highlights the integration of artificial intelligence and machine learning techniques to optimize sensor fusion algorithms and improve the overall autonomy of the vehicle. Following thorough analysis, the deduction can be made that sensor and sensor fusion technology assume a critical function in facilitating efficient and secure autonomous vehicle navigation within intricate surroundings.

A Vehicle-Edge-Cloud Framework for Computational Analysis of a Fine-Tuned Deep Learning Model.

The cooperative, connected, and automated mobility (CCAM) infrastructure plays a key role in understanding and enhancing the environmental perception of autonomous vehicles (AVs) driving in complex urban settings. However, the deployment of CCAM infrastructure necessitates the efficient selection of the computational processing layer and deployment of machine learning (ML) and deep learning (DL) models to achieve greater performance of AVs in complex urban environments. In this paper, we propose a computational framework and analyze the effectiveness of a custom-trained DL model (YOLOv8) when deployed in diverse devices and settings at the vehicle-edge-cloud-layered architecture. Our main focus is to understand the interplay and relationship between the DL model's accuracy and execution time during deployment at the layered framework. Therefore, we investigate the trade-offs between accuracy and time by the deployment process of the YOLOv8 model over each layer of the computational framework. We consider the CCAM infrastructures, i.e., sensory devices, computation, and communication at each layer. The findings reveal that the performance metrics results (e.g., 0.842 mAP@0.5) of deployed DL models remain consistent regardless of the device type across any layer of the framework. However, we observe that inference times for object detection tasks tend to decrease when the DL model is subjected to different environmental conditions. For instance, the Jetson AGX (non-GPU) outperforms the Raspberry Pi (non-GPU) by reducing inference time by 72%, whereas the Jetson AGX Xavier (GPU) outperforms the Jetson AGX ARMv8 (non-GPU) by reducing inference time by 90%. A complete average time comparison analysis for the transfer time, preprocess time, and total time of devices Apple M2 Max, Intel Xeon, Tesla T4, NVIDIA A100, Tesla V100, etc., is provided in the paper. Our findings direct the researchers and practitioners to select the most appropriate device type and environment for the deployment of DL models required for production.

Public perception of autonomous vehicles acceptance in Hungary

AbstractThe integration of Autonomous Vehicles (AVs) into our modern society hinges on gaining widespread acceptance from potential road users. To indicate the preparedness of these road users and elucidate their perspectives regarding the use of AVs in future, it is imperative to conduct surveys gauging public acceptance and satisfaction with this emerging mode of transportation. This paper reports the results of a comprehensive questionnaire study involving 1,000 individuals in Gyor City. The survey's primary objective was to assess participants' attitudes and willingness to embrace autonomous vehicles within the city's road networks. The study delved into various socio-demographic factors, such as age, gender, and employment status, while also exploring participants' prior knowledge and opinions regarding the advantages and limitations of AVs. The findings reveal a generally favorable disposition among the public toward the inclusion of AVs in urban traffic, paving the way for the acceptance of mixed traffic patterns. Notably, respondents in younger age groups exhibit greater enthusiasm for incorporating AVs into their daily transportation, whereas individuals aged 65 and above express more reservations, displaying a conservative outlook. Furthermore, participants with prior knowledge and a deeper understanding of AVs exhibit a markedly more positive inclination toward this emerging technology compared to those lacking such familiarity.

Geometric information constraint 3D object detection from LiDAR point cloud for autonomous vehicles under adverse weather

3D object detection, as the core of the autonomous vehicle perception module, is essential for efficient transportation and comfortable experiences. However, the challenge of 3D object detection under adverse weather conditions hinders the advancement of autonomous vehicles to higher levels. Hence, achieving accurate 3D object detection under adverse weather conditions is increasingly crucial as it forms the foundation for trajectory planning and driving strategy making in autonomous vehicles, thereby revolutionizing transportation modes for both goods and passengers. Advances in Light Detection and Ranging (LiDAR) technology have facilitated the development of 3D object detection in the past few years. Adverse weather, which inevitably occurs in real-world driving scenarios, could degrade measurement accuracy and point density of LiDAR and lead to particle interference. Detecting accurate 3D bounding boxes from sparse, incomplete point clouds with particle interference is difficult. Therefore, this research presents a novel geometric information constraint network for 3D object detection tasks from LiDAR point clouds under adverse weather (GIC-Net). In this study, we focus on how to incorporate geometric location information and line geometric feature information into the network against adverse weather. Further, we propose a geometric location constrained backbone module (GLC) to reduce rain and snow particle interference and ensure sufficient receptive fields. Then, we design a line geometric feature constraint module (LGFC) to add line constraints of 3D bounding boxes into the training process. Finally, a line loss function is designed, and features from the GLC and LGFC modules are fed into the multi-task detection head for accurate 3D bounding box prediction. Experiments on the Canadian Adverse Driving Conditions (CADC) autonomous vehicle dataset demonstrate the superiority of our method over six other state-of-the-art methods under adverse weather, which is at least 13.32 %, 4.67 %, and 10.44 % mAP higher than the other compared methods in the car, truck, and pedestrian classes respectively. Also, we further verify the better generalization ability of our network compared to other methods.

A multi-level privacy-preserving scheme for extracting traffic images

Traffic images are constantly used as a stick to assess traffic conditions. Traffic flow statistical analysis, road condition safety monitoring, vehicle violation detection, accident surveillance, and the driving environment perception of autonomous vehicles are all functionalities that depend on the processing of traffic images. However, traffic images contain privacy-sensitive information related to users, such as license plate numbers, drivers and passengers. Extracting, analyzing and sharing such privacy-sensitive information without any security measures may raise concerns among users about potential privacy violations. This paper proposes a multi-level privacy protection scheme for traffic image extraction based on compressive sensing, which has the advantages of data undersampling, privacy protection and data access control. Detailed compression rate analysis demonstrates that the proposed solution can effectively reduce the transmission load of image information. Security analysis demonstrates that the proposed approach achieves multi-level reconstruction qualities and high-security strength for users with different authorities. Thus, it has a good application prospect in an image-based intelligent traffic management system.

Autonomous Vehicles: Applications of Deep Reinforcement Learning

This research paper offers a concise exploration into the pivotal role played by deep reinforcement learning in propelling advancements in both perception and decision-making aspects of autonomous vehicles. Through the fusion of deep learning and reinforcement learning techniques, we delve into how these methodologies synergistically contribute to augmenting the navigational proficiency of autonomous vehicles amidst intricate and dynamically changing environments. With a focus on a streamlined presentation, this paper provides a succinct yet insightful overview of various perception algorithms utilized by autonomous vehicles, emphasizing object detection, semantic segmentation, and LiDAR-based techniques.

Object Detection in Autonomous Vehicles under Adverse Weather: A Review of Traditional and Deep Learning Approaches

Enhancing the environmental perception of autonomous vehicles (AVs) in intelligent transportation systems requires computer vision technology to be effective in detecting objects and obstacles, particularly in adverse weather conditions. Adverse weather circumstances present serious difficulties for object-detecting systems, which are essential to contemporary safety procedures, infrastructure for monitoring, and intelligent transportation. AVs primarily depend on image processing algorithms that utilize a wide range of onboard visual sensors for guidance and decisionmaking. Ensuring the consistent identification of critical elements such as vehicles, pedestrians, and road lanes, even in adverse weather, is a paramount objective. This paper not only provides a comprehensive review of the literature on object detection (OD) under adverse weather conditions but also delves into the ever-evolving realm of the architecture of AVs, challenges for automated vehicles in adverse weather, the basic structure of OD, and explores the landscape of traditional and deep learning (DL) approaches for OD within the realm of AVs. These approaches are essential for advancing the capabilities of AVs in recognizing and responding to objects in their surroundings. This paper further investigates previous research that has employed both traditional and DL methodologies for the detection of vehicles, pedestrians, and road lanes, effectively linking these approaches with the evolving field of AVs. Moreover, this paper offers an in-depth analysis of the datasets commonly employed in AV research, with a specific focus on the detection of key elements in various environmental conditions, and then summarizes the evaluation matrix. We expect that this review paper will help scholars to gain a better understanding of this area of research.

Feature Aware Re-weighting (FAR) in Bird’s Eye View for LiDAR-based 3D object detection in autonomous driving applications

3D object detection is a key element for the perception of autonomous vehicles. LiDAR sensors are commonly used to perceive the surrounding area, producing a sparse representation of the scene in the form of a point cloud. The current trend is to use deep learning neural network architectures that predict 3D bounding boxes. The vast majority of architectures process the LiDAR point cloud directly but, due to computation and memory constraints, at some point they compress the input to a 2D Bird’s Eye View (BEV) representation. In this work, we propose a novel 2D neural network architecture, namely the Feature Aware Re-weighting Network, for feature extraction in BEV using local context via an attention mechanism, to improve the 3D detection performance of LiDAR-based detectors. Extensive experiments on five state-of-the-art detectors and three benchmarking datasets, namely KITTI, Waymo and nuScenes, demonstrate the effectiveness of the proposed method in terms of both detection performance and minimal added computational burden. We release our code at https://github.com/grgzam/FAR.

Deep learning based visual perception and decision-making technology for autonomous vehicles

Along with the social fast developing and the scientific and technological advancement day by day, the automatic driving system has been widely used in many areas, so it has been a hotspot. Because of the constant progress of the deep learning technique, the vision environmental sensor technique and the behavior decision technique have obtained some inspiration. The paper introduces a new method for detecting target with human body, which is based on recurrent convolution neural networks. In recent years, we have seen a number of developments in the field of decision making, such as reinforcement learning, and the development of deep neural networks. This thesis mainly focuses on the research of VMI and decision algorithms, and gives a brief introduction to their further application in ATS. In this paper, some key techniques of vision sensing and behavior decision are discussed in detail. This paper analyses the merits and deficiencies of current study on environment sensing technique and behavior decision technique. Finally, this paper summarizes and forecasts the vision environmental awareness and behavior decision techniques.

Enhanced Perception for Autonomous Vehicles at Obstructed Intersections: An Implementation of Vehicle to Infrastructure (V2I) Collaboration

In urban intersections, the sensory capabilities of autonomous vehicles (AVs) are often hindered by visual obstructions, posing significant challenges to their robust and safe operation. This paper presents an implementation study focused on enhancing the safety and robustness of Connected Automated Vehicles (CAVs) in scenarios with occluded visibility at urban intersections. A novel LiDAR Infrastructure System is established for roadside sensing, combined with Baidu Apollo’s Automated Driving System (ADS) and Cohda Wireless V2X communication hardware, and an integrated platform is established for roadside perception enhancement in autonomous driving. The field tests were conducted at the Singapore CETRAN (Centre of Excellence for Testing & Research of Autonomous Vehicles—NTU) autonomous vehicle test track, with the communication protocol adhering to SAE J2735 V2X communication standards. Communication latency and packet delivery ratio were analyzed as the evaluation metrics. The test results showed that the system can help CAV detect obstacles in advance under urban occluded scenarios.

Perception and Acceptance of Autonomous Vehicles: Influencing Factors and the Relevance of Subjective Knowledge

Autonomous vehicle technology (AV) is rapidly evolving, and numerous studies have already dealt with influencing factors of acceptance of AVs. Subjective knowledge (SK) has been highlighted as a relevant factor in predicting trust and acceptance. However, it has not been integrated at the model level, which is the aim of the present study. In addition, the present study investigated the increase of SK using an explainer video. In an online study, participants (N = 435) watched a four-minute explainer video about AVs. Before and after the video, participants were asked about their SK about AVs and then completed questionnaires on perceived risk, perceived ease of use, perceived usefulness, trust, intention to use, and personal innovativeness. The results indicate that SK is a crucial factor in improving perception and, indirectly, trust and acceptance of AVs. In addition, the increase in SK depended on the initial SK.

Time-aware and task-transferable adversarial attack for perception of autonomous vehicles

With rapid development of self-driving vehicles, recent work in adversarial machine learning started to study adversarial examples (AEs) for perception of autonomous driving (AD). However, generating practical AEs for the perception module remains a significant challenge. Traditional adversarial attacks tend to focus on a single computer vision task, making it difficult to compromise multiple perception tasks such as object detection and segmentation simultaneously. Additionally, the limited computational resources available on-board and the necessity for online operation pose further obstacles to deploying adversarial attacks on real autonomous driving platforms. To address the aforementioned issues, we propose the Time-aware Perception Attack (TPA), which is a real-time cross-task adversarial attack for perception of autonomous driving. In particular, we propose a novel backbone based adversarial attack method to modify input images to approach Lipschitz Constant Point (LCP), which results in erroneous inferences for all the sub-models in perception module. The novel part of this work is proposing an efficient yet effective LCP approaching algorithm. Comparing to conventional LCP based attacks, which consume significant amount of computation resources and can be only applied on small DNNs, TPA generates AEs on an intermediate layer of surrogate backbone, significantly enhancing the cross-task transferability and accelerates the attack process. Evaluation results on Berkeley Driving Dataset 100k (BDD100k) show that, comparing to the state-of-the-art baselines, the proposed TPA achieves higher attack effectiveness and faster processing speed and outperforms the baselines by a large margin.

Emerging Trends in Autonomous Vehicle Perception: Multimodal Fusion for 3D Object Detection

The pursuit of autonomous driving relies on developing perception systems capable of making accurate, robust, and rapid decisions to interpret the driving environment effectively. Object detection is crucial for understanding the environment at these systems’ core. While 2D object detection and classification have advanced significantly with the advent of deep learning (DL) in computer vision (CV) applications, they fall short in providing essential depth information, a key element in comprehending driving environments. Consequently, 3D object detection becomes a cornerstone for autonomous driving and robotics, offering precise estimations of object locations and enhancing environmental comprehension. The CV community’s growing interest in 3D object detection is fueled by the evolution of DL models, including Convolutional Neural Networks (CNNs) and Transformer networks. Despite these advancements, challenges such as varying object scales, limited 3D sensor data, and occlusions persist in 3D object detection. To address these challenges, researchers are exploring multimodal techniques that combine information from multiple sensors, such as cameras, radar, and LiDAR, to enhance the performance of perception systems. This survey provides an exhaustive review of multimodal fusion-based 3D object detection methods, focusing on CNN and Transformer-based models. It underscores the necessity of equipping fully autonomous vehicles with diverse sensors to ensure robust and reliable operation. The survey explores the advantages and drawbacks of cameras, LiDAR, and radar sensors. Additionally, it summarizes autonomy datasets and examines the latest advancements in multimodal fusion-based methods. The survey concludes by highlighting the ongoing challenges, open issues, and potential directions for future research.

Automatic Seabed Target Segmentation of AUV via Multilevel Adversarial Network and Marginal Distribution Adaptation

The real-time segmentation results of side-scan sonar (SSS) images can realize the intelligent perception of autonomous underwater vehicles (AUVs). However, there remains a compelling challenge that the models trained with training data may not generalize well in practical applications due to the various marine conditions/working devices. This article proposes a real-time cross-domain segmentation adaptation scheme for SSS images based on 1) adversarial training via min-max training to distribute SSS images in the output space and 2) marginal distribution adaption to improve the segmentation performance by minimizing the measured distribution distance. To further enhance the adapted model, the multi-layer feature discriminators are established to realize the fine-grained alignment. Experiments on the real-world SSS datasets demonstrate that our proposed model outperforms state-of-the-art methods and achieves the actual industrial application for AUV.

Experimental Validation: Perception and Localization Systems for Autonomous Vehicles using the Extended Kalman Filter Algorithm

Abstract This study validates EKF-SLAM for indoor autonomous vehicles by experimentally integrating the MPU6050 sensor and encoder data using an extended Kalman filter. Real-world tests show significant improvements, achieving high accuracy with just 1% and 3% errors in the X and Y axes. RPLiDAR A1M8 is utilized for mapping, producing accurate maps visualized through RViz-ROS. The research demonstrates the novelty and practical utility of EKF-SLAM in real-world scenarios, showcasing unprecedented effectiveness and precision.

Glare Mitigation for Enhanced Autonomous Vehicle Perception

Glare Mitigation for Enhanced Autonomous Vehicle Perception

Public Perception of the Introduction of Autonomous Vehicles

Autonomous vehicles (AVs) will transform transport, but public opinion will play a key role in decisions on how widely and quickly they are adopted. The purpose of the study presented here was to investigate community’s views on that transition. As a method for primary data collection on public awareness, attitudes, and readiness to use autonomous cars, survey was conducted in Saudi Arabia. Following that, we used statistical tools to analyse responses. Our findings indicate that the participants are largely receptive to using new technologies and had favourable attitudes towards the transition. Ordinal logistic regression model showed a wide variation in public opinion regarding the expected benefits that may accompany the transition. Our findings reveal that awareness of AVs’ benefits is positively correlated with the age of participants. Perceived costs on one side, and convenience and safety on the other, were found to have had a substantial impact on the opinions of the participants. Investigation presented here shows a sample of the public’s perception of AVs in Saudi Arabia. This can guide the development of AVs and their deployment in that region as well as worldwide.

Psychological factors shaping public acceptance of the adoption of autonomous vehicles in Indonesia

IntroductionThe rapid development of autonomous vehicles (AVs) has the potential to revolutionize transportation systems and address societal challenges. However, understanding the psychological factors influencing public acceptance and readiness for AV adoption is crucial for successful implementation. This study provides a comprehensive analysis of these psychological factors, specifically focusing on the context of health issues in Indonesia. MethodologyA mixed-methods approach analyzed quantitative data from 500 respondents to a questionnaire addressing key perceptions of AVs using regression analysis, coupled with 30 qualitative interviews. ResultsThe results revealed that trust in AV technology mediates the relationship between control perception, perceived safety, human-machine interaction, educational status, cultural factors, and public acceptance of AV adoption. Cultural factors were also identified as moderators influencing this relationship. Qualitative analysis of interview data further highlighted the significance of trust, reliability, and safety concerns in shaping public acceptance. Participants emphasized the need for effective human-machine interaction and educational awareness to enhance understanding and acceptance. DiscussionThis research contributes to the existing literature by explicitly examining psychological factors within the context of health issues in Indonesia, providing valuable insights for policymakers aiming to promote AV adoption. Ultimately, understanding and addressing these psychological factors will pave the way for a successful and sustainable AV future.