Autonomous Vehicle Perception Research Articles

A multi-level privacy-preserving scheme for extracting traffic images

Traffic images are constantly used as a stick to assess traffic conditions. Traffic flow statistical analysis, road condition safety monitoring, vehicle violation detection, accident surveillance, and the driving environment perception of autonomous vehicles are all functionalities that depend on the processing of traffic images. However, traffic images contain privacy-sensitive information related to users, such as license plate numbers, drivers and passengers. Extracting, analyzing and sharing such privacy-sensitive information without any security measures may raise concerns among users about potential privacy violations. This paper proposes a multi-level privacy protection scheme for traffic image extraction based on compressive sensing, which has the advantages of data undersampling, privacy protection and data access control. Detailed compression rate analysis demonstrates that the proposed solution can effectively reduce the transmission load of image information. Security analysis demonstrates that the proposed approach achieves multi-level reconstruction qualities and high-security strength for users with different authorities. Thus, it has a good application prospect in an image-based intelligent traffic management system.

Autonomous Vehicles: Applications of Deep Reinforcement Learning

This research paper offers a concise exploration into the pivotal role played by deep reinforcement learning in propelling advancements in both perception and decision-making aspects of autonomous vehicles. Through the fusion of deep learning and reinforcement learning techniques, we delve into how these methodologies synergistically contribute to augmenting the navigational proficiency of autonomous vehicles amidst intricate and dynamically changing environments. With a focus on a streamlined presentation, this paper provides a succinct yet insightful overview of various perception algorithms utilized by autonomous vehicles, emphasizing object detection, semantic segmentation, and LiDAR-based techniques.

Object Detection in Autonomous Vehicles under Adverse Weather: A Review of Traditional and Deep Learning Approaches

Enhancing the environmental perception of autonomous vehicles (AVs) in intelligent transportation systems requires computer vision technology to be effective in detecting objects and obstacles, particularly in adverse weather conditions. Adverse weather circumstances present serious difficulties for object-detecting systems, which are essential to contemporary safety procedures, infrastructure for monitoring, and intelligent transportation. AVs primarily depend on image processing algorithms that utilize a wide range of onboard visual sensors for guidance and decisionmaking. Ensuring the consistent identification of critical elements such as vehicles, pedestrians, and road lanes, even in adverse weather, is a paramount objective. This paper not only provides a comprehensive review of the literature on object detection (OD) under adverse weather conditions but also delves into the ever-evolving realm of the architecture of AVs, challenges for automated vehicles in adverse weather, the basic structure of OD, and explores the landscape of traditional and deep learning (DL) approaches for OD within the realm of AVs. These approaches are essential for advancing the capabilities of AVs in recognizing and responding to objects in their surroundings. This paper further investigates previous research that has employed both traditional and DL methodologies for the detection of vehicles, pedestrians, and road lanes, effectively linking these approaches with the evolving field of AVs. Moreover, this paper offers an in-depth analysis of the datasets commonly employed in AV research, with a specific focus on the detection of key elements in various environmental conditions, and then summarizes the evaluation matrix. We expect that this review paper will help scholars to gain a better understanding of this area of research.

Feature Aware Re-weighting (FAR) in Bird’s Eye View for LiDAR-based 3D object detection in autonomous driving applications

3D object detection is a key element for the perception of autonomous vehicles. LiDAR sensors are commonly used to perceive the surrounding area, producing a sparse representation of the scene in the form of a point cloud. The current trend is to use deep learning neural network architectures that predict 3D bounding boxes. The vast majority of architectures process the LiDAR point cloud directly but, due to computation and memory constraints, at some point they compress the input to a 2D Bird’s Eye View (BEV) representation. In this work, we propose a novel 2D neural network architecture, namely the Feature Aware Re-weighting Network, for feature extraction in BEV using local context via an attention mechanism, to improve the 3D detection performance of LiDAR-based detectors. Extensive experiments on five state-of-the-art detectors and three benchmarking datasets, namely KITTI, Waymo and nuScenes, demonstrate the effectiveness of the proposed method in terms of both detection performance and minimal added computational burden. We release our code at https://github.com/grgzam/FAR.

Deep learning based visual perception and decision-making technology for autonomous vehicles

Along with the social fast developing and the scientific and technological advancement day by day, the automatic driving system has been widely used in many areas, so it has been a hotspot. Because of the constant progress of the deep learning technique, the vision environmental sensor technique and the behavior decision technique have obtained some inspiration. The paper introduces a new method for detecting target with human body, which is based on recurrent convolution neural networks. In recent years, we have seen a number of developments in the field of decision making, such as reinforcement learning, and the development of deep neural networks. This thesis mainly focuses on the research of VMI and decision algorithms, and gives a brief introduction to their further application in ATS. In this paper, some key techniques of vision sensing and behavior decision are discussed in detail. This paper analyses the merits and deficiencies of current study on environment sensing technique and behavior decision technique. Finally, this paper summarizes and forecasts the vision environmental awareness and behavior decision techniques.

Enhanced Perception for Autonomous Vehicles at Obstructed Intersections: An Implementation of Vehicle to Infrastructure (V2I) Collaboration

In urban intersections, the sensory capabilities of autonomous vehicles (AVs) are often hindered by visual obstructions, posing significant challenges to their robust and safe operation. This paper presents an implementation study focused on enhancing the safety and robustness of Connected Automated Vehicles (CAVs) in scenarios with occluded visibility at urban intersections. A novel LiDAR Infrastructure System is established for roadside sensing, combined with Baidu Apollo’s Automated Driving System (ADS) and Cohda Wireless V2X communication hardware, and an integrated platform is established for roadside perception enhancement in autonomous driving. The field tests were conducted at the Singapore CETRAN (Centre of Excellence for Testing & Research of Autonomous Vehicles—NTU) autonomous vehicle test track, with the communication protocol adhering to SAE J2735 V2X communication standards. Communication latency and packet delivery ratio were analyzed as the evaluation metrics. The test results showed that the system can help CAV detect obstacles in advance under urban occluded scenarios.

Perception and Acceptance of Autonomous Vehicles: Influencing Factors and the Relevance of Subjective Knowledge

Autonomous vehicle technology (AV) is rapidly evolving, and numerous studies have already dealt with influencing factors of acceptance of AVs. Subjective knowledge (SK) has been highlighted as a relevant factor in predicting trust and acceptance. However, it has not been integrated at the model level, which is the aim of the present study. In addition, the present study investigated the increase of SK using an explainer video. In an online study, participants (N = 435) watched a four-minute explainer video about AVs. Before and after the video, participants were asked about their SK about AVs and then completed questionnaires on perceived risk, perceived ease of use, perceived usefulness, trust, intention to use, and personal innovativeness. The results indicate that SK is a crucial factor in improving perception and, indirectly, trust and acceptance of AVs. In addition, the increase in SK depended on the initial SK.

Time-aware and task-transferable adversarial attack for perception of autonomous vehicles

With rapid development of self-driving vehicles, recent work in adversarial machine learning started to study adversarial examples (AEs) for perception of autonomous driving (AD). However, generating practical AEs for the perception module remains a significant challenge. Traditional adversarial attacks tend to focus on a single computer vision task, making it difficult to compromise multiple perception tasks such as object detection and segmentation simultaneously. Additionally, the limited computational resources available on-board and the necessity for online operation pose further obstacles to deploying adversarial attacks on real autonomous driving platforms. To address the aforementioned issues, we propose the Time-aware Perception Attack (TPA), which is a real-time cross-task adversarial attack for perception of autonomous driving. In particular, we propose a novel backbone based adversarial attack method to modify input images to approach Lipschitz Constant Point (LCP), which results in erroneous inferences for all the sub-models in perception module. The novel part of this work is proposing an efficient yet effective LCP approaching algorithm. Comparing to conventional LCP based attacks, which consume significant amount of computation resources and can be only applied on small DNNs, TPA generates AEs on an intermediate layer of surrogate backbone, significantly enhancing the cross-task transferability and accelerates the attack process. Evaluation results on Berkeley Driving Dataset 100k (BDD100k) show that, comparing to the state-of-the-art baselines, the proposed TPA achieves higher attack effectiveness and faster processing speed and outperforms the baselines by a large margin.

Emerging Trends in Autonomous Vehicle Perception: Multimodal Fusion for 3D Object Detection

The pursuit of autonomous driving relies on developing perception systems capable of making accurate, robust, and rapid decisions to interpret the driving environment effectively. Object detection is crucial for understanding the environment at these systems’ core. While 2D object detection and classification have advanced significantly with the advent of deep learning (DL) in computer vision (CV) applications, they fall short in providing essential depth information, a key element in comprehending driving environments. Consequently, 3D object detection becomes a cornerstone for autonomous driving and robotics, offering precise estimations of object locations and enhancing environmental comprehension. The CV community’s growing interest in 3D object detection is fueled by the evolution of DL models, including Convolutional Neural Networks (CNNs) and Transformer networks. Despite these advancements, challenges such as varying object scales, limited 3D sensor data, and occlusions persist in 3D object detection. To address these challenges, researchers are exploring multimodal techniques that combine information from multiple sensors, such as cameras, radar, and LiDAR, to enhance the performance of perception systems. This survey provides an exhaustive review of multimodal fusion-based 3D object detection methods, focusing on CNN and Transformer-based models. It underscores the necessity of equipping fully autonomous vehicles with diverse sensors to ensure robust and reliable operation. The survey explores the advantages and drawbacks of cameras, LiDAR, and radar sensors. Additionally, it summarizes autonomy datasets and examines the latest advancements in multimodal fusion-based methods. The survey concludes by highlighting the ongoing challenges, open issues, and potential directions for future research.

Automatic Seabed Target Segmentation of AUV via Multilevel Adversarial Network and Marginal Distribution Adaptation

The real-time segmentation results of side-scan sonar (SSS) images can realize the intelligent perception of autonomous underwater vehicles (AUVs). However, there remains a compelling challenge that the models trained with training data may not generalize well in practical applications due to the various marine conditions/working devices. This article proposes a real-time cross-domain segmentation adaptation scheme for SSS images based on 1) adversarial training via min-max training to distribute SSS images in the output space and 2) marginal distribution adaption to improve the segmentation performance by minimizing the measured distribution distance. To further enhance the adapted model, the multi-layer feature discriminators are established to realize the fine-grained alignment. Experiments on the real-world SSS datasets demonstrate that our proposed model outperforms state-of-the-art methods and achieves the actual industrial application for AUV.

Experimental Validation: Perception and Localization Systems for Autonomous Vehicles using the Extended Kalman Filter Algorithm

Abstract This study validates EKF-SLAM for indoor autonomous vehicles by experimentally integrating the MPU6050 sensor and encoder data using an extended Kalman filter. Real-world tests show significant improvements, achieving high accuracy with just 1% and 3% errors in the X and Y axes. RPLiDAR A1M8 is utilized for mapping, producing accurate maps visualized through RViz-ROS. The research demonstrates the novelty and practical utility of EKF-SLAM in real-world scenarios, showcasing unprecedented effectiveness and precision.

Fast-BEV: A Fast and Strong Bird's-Eye View Perception Baseline.

Recently, perception task based on Bird's-Eye View (BEV) representation has drawn more and more attention, and BEV representation is promising as the foundation for next-generation Autonomous Vehicle (AV) perception. However, most existing BEV solutions either require considerable resources to execute on-vehicle inference or suffer from modest performance. This paper proposes a simple yet effective framework, termed Fast-BEV, which is capable of performing faster BEV perception on the on-vehicle chips. Towards this goal, we first empirically find that the BEV representation can be sufficiently powerful without expensive transformer based transformation nor depth representation. Our Fast-BEV consists of five parts, We innovatively propose (1) a lightweight deploymentfriendly view transformation which fast transfers 2D image feature to 3D voxel space, (2) an multi-scale image encoder which leverages multi-scale information for better performance, (3) an efficient BEV encoder which is particularly designed to speed up on-vehicle inference. We further introduce (4) a strong data augmentation strategy for both image and BEV space to avoid over-fitting, (5) a multiframe feature fusion mechanism to leverage the temporal information. Among them, (1) and (3) enable Fast-BEV to be fast inference and deployment friendly on the on-vehicle chips, (2), (4) and (5) ensure that Fast-BEV has competitive performance. All these make Fast-BEV a solution with high performance, fast inference speed, and deployment-friendly on the on-vehicle chips of autonomous driving. Through experiments, on 2080Ti platform, our R50 model can run 52.6 FPS with 47.3% NDS on the nuScenes validation set, exceeding the 41.3 FPS and 47.5% NDS of the BEVDepth-R50 model [1] and 30.2 FPS and 45.7% NDS of the BEVDet4D-R50 model [2]. Our largest model (R101@900x1600) establishes a competitive 53.5% NDS on the nuScenes validation set. We further develop a benchmark with considerable accuracy and efficiency on current popular on-vehicle chips. The code is released at: https://github.com/Sense-GVT/FastBEV.

Toward Robust 3D Perception for Autonomous Vehicles: A Review of Adversarial Attacks and Countermeasures

Toward Robust 3D Perception for Autonomous Vehicles: A Review of Adversarial Attacks and Countermeasures

LIDAROC: Realistic LiDAR Cover Contamination Dataset for Enhancing Autonomous Vehicle Perception Reliability

LIDAROC: Realistic LiDAR Cover Contamination Dataset for Enhancing Autonomous Vehicle Perception Reliability

Glare Mitigation for Enhanced Autonomous Vehicle Perception

Glare Mitigation for Enhanced Autonomous Vehicle Perception

Public Perception of the Introduction of Autonomous Vehicles

Autonomous vehicles (AVs) will transform transport, but public opinion will play a key role in decisions on how widely and quickly they are adopted. The purpose of the study presented here was to investigate community’s views on that transition. As a method for primary data collection on public awareness, attitudes, and readiness to use autonomous cars, survey was conducted in Saudi Arabia. Following that, we used statistical tools to analyse responses. Our findings indicate that the participants are largely receptive to using new technologies and had favourable attitudes towards the transition. Ordinal logistic regression model showed a wide variation in public opinion regarding the expected benefits that may accompany the transition. Our findings reveal that awareness of AVs’ benefits is positively correlated with the age of participants. Perceived costs on one side, and convenience and safety on the other, were found to have had a substantial impact on the opinions of the participants. Investigation presented here shows a sample of the public’s perception of AVs in Saudi Arabia. This can guide the development of AVs and their deployment in that region as well as worldwide.

Psychological factors shaping public acceptance of the adoption of autonomous vehicles in Indonesia

IntroductionThe rapid development of autonomous vehicles (AVs) has the potential to revolutionize transportation systems and address societal challenges. However, understanding the psychological factors influencing public acceptance and readiness for AV adoption is crucial for successful implementation. This study provides a comprehensive analysis of these psychological factors, specifically focusing on the context of health issues in Indonesia. MethodologyA mixed-methods approach analyzed quantitative data from 500 respondents to a questionnaire addressing key perceptions of AVs using regression analysis, coupled with 30 qualitative interviews. ResultsThe results revealed that trust in AV technology mediates the relationship between control perception, perceived safety, human-machine interaction, educational status, cultural factors, and public acceptance of AV adoption. Cultural factors were also identified as moderators influencing this relationship. Qualitative analysis of interview data further highlighted the significance of trust, reliability, and safety concerns in shaping public acceptance. Participants emphasized the need for effective human-machine interaction and educational awareness to enhance understanding and acceptance. DiscussionThis research contributes to the existing literature by explicitly examining psychological factors within the context of health issues in Indonesia, providing valuable insights for policymakers aiming to promote AV adoption. Ultimately, understanding and addressing these psychological factors will pave the way for a successful and sustainable AV future.

Leveraging the edge and cloud for V2X-based real-time object detection in autonomous driving

Environmental perception is a key element of autonomous driving because the information received from the perception module influences core driving decisions. An outstanding challenge in real-time perception for autonomous driving lies in finding the best trade-off between detection quality and latency. Major constraints on both computation and power must be taken into account for real-time perception in autonomous vehicles. Larger detection models tend to produce the best results but are also slower at runtime. Since the most accurate detectors may not run in real-time locally, we investigate the possibility of offloading computation to edge and cloud platforms, which are less resource-constrained. We create a synthetic dataset to train object detection models and evaluate different offloading strategies. We measure inference and processing times for object detection on real hardware, and we rely on a network simulation framework to estimate data transmission latency. Our study compares different trade-offs between prediction quality and end-to-end delay. Following the existing literature, we aim to perform object detection at a rate of 20Hz. Since sending raw frames over the network implies additional transmission delays, we also explore the use of JPEG and H.265 compression at varying qualities and measure their impact on prediction. We show that models with adequate compression can be run in real-time on the edge/cloud while outperforming local detection performance.

Phased Feature Extraction Network for Vehicle Search Tasks Based on Cross-Camera for Vehicle-Road Collaborative Perception.

The objective of vehicle search is to locate and identify vehicles in uncropped, real-world images, which is the combination of two tasks: vehicle detection and re-identification (Re-ID). As an emerging research topic, vehicle search plays a significant role in the perception of cooperative autonomous vehicles and road driving in the distant future and has become a trend in the future development of intelligent driving. However, there is no suitable dataset for this study. The Tsinghua University DAIR-V2X dataset is utilized to create the first cross-camera vehicle search dataset, DAIR-V2XSearch, which combines the cameras at both ends of the vehicle and the road in real-world scenes. The primary purpose of the current search network is to address the pedestrian issue. Due to varying task scenarios, it is necessary to re-establish the network in order to resolve the problem of vast differences in different perspectives caused by vehicle searches. A phased feature extraction network (PFE-Net) is proposed as a solution to the cross-camera vehicle search problem. Initially, the anchor-free YOLOX framework is selected as the backbone network, which not only improves the network's performance but also eliminates the fuzzy situation in which multiple anchor boxes correspond to a single vehicle ID in the Re-ID branch. Second, for the vehicle Re-ID branch, a camera grouping module is proposed to effectively address issues such as sudden changes in perspective and disparities in shooting under different cameras. Finally, a cross-level feature fusion module is designed to enhance the model's ability to extract subtle vehicle features and the Re-ID's precision. Experiments demonstrate that our proposed PFE-Net achieves the highest precision in the DAIR-V2XSearch dataset.

CenterPoint-SE: A Single-Stage Anchor-Free 3-D Object Detection Algorithm With Spatial Awareness Enhancement

Real-time and accurate 3-D object detection is one of the foundational technologies for environmental perception in autonomous vehicles. However, the existing second-stage anchor-based 3-D object detection algorithms have high accuracy, but they are challenging in terms of computation complexity and latency. Due to poor perception of spatial features, the accuracy of the existing single-stage anchor-free detection algorithms with low latency are difficult to be implemented into autonomous vehicles. Therefore, we focus on enhancing the spatial perception ability of the anchor-free detection network based on CenterPoints. In this paper, we propose a single-stage anchor-free 3-D object detector CenterPoint-Space-Enhancement (CenterPoint-SE) algorithm and construct an efficient 3-D backbone network to extract fine-grained spatial geometric features by introducing a spatial attention mechanism and residual structure. At the same time, a powerful spatial semantic feature fusion module, the enhancement of feature fusion (EF-Fusion), is designed. In addition, we add a lightweight IoU prediction branch to improve the algorithm’s perception of various object sizes. Finally, we add a foreground point segmentation auxiliary training branch to enable the 3-D backbone to obtain object boundary features. We use the ONCE dataset to train and validate the proposed model, and the results showed that the proposed CenterPoint-SE achieves 70.33 mAP and an inference speed of 17.15 FPS, outperforming other methods.