Autonomous Vehicle Perception Research Articles

Low-Latency Edge Video Analytics for On-Road Perception of Autonomous Ground Vehicles

To improve the transportation efficiency of advanced manufacturing, cameras have been extensively deployed to enhance the on-road perception of autonomous ground vehicles in smart industrial parks. Considering the informative yet substantial data volume of contents generated by those cameras, we employ vehicle-to-everything links to deliver the captured video frames to neighboring vehicles, road-side units, or base stations, in order to respond to vehicle-control-related video queries. To help vehicles obtain low-latency and high-accuracy on-road information for autonomous driving, an optimization problem is formulated, taking into account the impact of vehicle mobility and diverse resource demands of different video queries. Then, a two-stage algorithm is proposed to determine the frame rate of video cameras, as well as the destination of the associated video frames based on matching theory. Extensive simulation results show that this approach can improve the accuracy by up to 18% and reduce the response delay by 13.2%.

CCPAV: Centralized cooperative perception for autonomous vehicles using CV2X

Cooperative perception improves awareness for various traffic situations by connecting autonomous vehicles to each other as well as to the surrounding environment. Overcoming the line-of-sight challenge due to the limitations of vehicle's local sensors is of great importance. However, inefficient solutions can quickly lead to depletion of the limited communication resources. This is critical, especially that the dominant CV2X (Cellular Vehicle to Everything) technology witnesses unprecedented growth in the number and density of connected smart devices. To this end, this paper provides a new centralized cooperative perception approach using CV2X. The system uses vehicle trajectories to prioritize messages communicated messages. Through the basestation (BS), the system prioritizes message requests while being constrained by the available network resources. This system is then implemented and evaluated using 1000 different traffic scenarios. These scenarios are generated using both SUMO (Simulation of Urban MObility) traffic simulator and a new implemented camera simulator used to represent the vehicle's sensors’ abilities to perceive the surrounding environment. Results show that the system, on average, can execute at least 95% of the total message values using only 100 physical resource blocks for CV2X regardless of the autonomous vehicle densities. This is very robust especially for congested networks as the number of messages requests executed varies between 65% to 95% given the different autonomous vehicle densities. To the best of our knowledge, this work is the first to use vehicle trajectories to jointly select messages for transmission and allocate RBs (Resource Blocks).

Enhanced performance of bionic ciliary piezoelectric microsensor for hydrodynamic perception

This work presents a bionic ciliary piezoelectric microsensor in d33 mode for enhancing performance of self-powered hydrodynamic perception. The sensor is composed of a cylindrical pillar emulating the fish cilium and a piezoelectric sensing diaphragm imitating the hair cell. Interdigital electrodes are designed on both sides of the sensing diaphragm to realize the radial polarization, whose pattern consists of a few semi-circular rings connected to each other. Based on the receptance method and Kirchhoff plate theory, a mathematical model is established to evaluate the resonance frequency of the sensor. The output characteristics of the sensor are numerically obtained by coupling the drag force acquired by computational fluid dynamics into the finite element model. The effects of key structural parameters on the sensor response are further investigated for the purpose of optimizing the device design. Prototypes are fabricated by the microelectromechanical systems technology and stereolithography, characterized using the impedance spectrum and tested employing a dipole stimulus. The experimental values of the sensor response to hydrodynamic flow velocities are in fine agreement with the numerical simulation. The results indicate a high sensitivity of 2.483 V/(m/s) and a relatively flat frequency response of the sensor. This work provides important guidance for designing the more efficient ciliary microsensor that can conduct flow field perception for underwater autonomous vehicles.

The Elderly Acceptance of Autonomous Vehicle Services in Beijing, China

China's population has been ageing fast at an accelerated pace since the 21st century, while the mobility and convenience of travel for the elderly were weakened by the limited transportation options. Autonomous vehicles, an emerging mode of transport, provide flexibility for the elderly, provided that the rapid and widespread this new mode of transportation can be accepted by the elderly population. This paper plans to gain insights from the attitudes of the Beijing elderly using autonomous vehicles, perceiving the importance of different types of services. The stated preference survey was conducted over 376 respondents from various generations, which examined their perception of autonomous vehicles after their retirement. An ordered logistics model used in this paper shows that people who use emerging transportation modes (e.g., car-sharing and ride-hailing) are more likely to use autonomous vehicle services. Accessing shopping activities and medical care is the two crucial purposes for the elderly autonomous vehicle users. Additionally, the clustering and correlation analysis indicate that people with a higher willingness to use autonomous vehicles are the earliest target users of these services. However, living with children and running errands in short distances by bicycle and walk, the elderly in the early generation hardly adopts autonomous vehicle services after their retirement.

基于扩展计划行为理论的驾驶员跟驰意向分析

Road traffic flow will show the phenomenon of mixed driving of autonomous vehicles and traditional vehicles in the future. In order to explore psychological factors that affect traditional vehicle drivers' intention to follow autonomous vehicles in the human-machine hybrid driving environment, three variables of driver's understanding, risk perception, and acceptance of autonomous vehicles are introduced to construct an extended theoretical framework of planning behavior. Three hundred thirty-one subjective evaluation data are obtained through questionnaire survey. The internal consistency and reliability of the data are tested by SPSS and AMOS softwares. Finally, the path analysis and mediation effect analysis are carried out by using structural equation model to test relationship among the influencing factors. The results show that the structural equation model of driver's car-following intention based on the theory of extended planning behavior has good explanatory power for the driver's car-following intention in the mixed driving environment. Driver's behavioral attitude, subjective norm and perceived behavioral control of autonomous vehicles have a significant positive and direct effect on driver's car-following intention. Driver's risk perception and acceptance of autonomous vehicles have a significant indirect effect on driver's car-following intention through intermediary variables, in which the effect of risk perception is negative, and the effect of acceptance is positive. The degree of understanding has both a significant positive direct effect and a significant positive indirect effect on driver's car-following intention. The research results can be used as the basis for vehicle interaction behavior analysis in the human-machine hybrid driving environment.

A Markovian Error Model for False Negatives in DNN-based Perception-Driven Control Systems

This paper presents an improved Markovian error model for Deep Neural Network (DNN) based perception in autonomous vehicles and other perception-driven control systems. Many modern autonomous systems rely on DNN-driven perception-based control/planning methodologies such as autonomous navigation, where the perception errors significantly affect the control/planning performance and the systems’ safety. The traditional independent, identically-distributed (IID) perception error model is inadequate for perception-based control/planning applications because image sequences supplied to a DNN-based perception module are not independent in the real world. Based on this observation, we develop a novel Markov model to describe the error behavior of a DNN perception model—an error in one frame is likely to signal errors in successive frames, effectively reducing sample rate for the control command. We evaluate the effect of Markovian perception errors on a drone-control simulator and show that the Markovian error model provides a better estimate of control performance than does a traditional independent, identically-distributed (IID) model.

Advances in Visual Simultaneous Localisation and Mapping Techniques for Autonomous Vehicles: A Review.

The recent advancements in Information and Communication Technology (ICT) as well as increasing demand for vehicular safety has led to significant progressions in Autonomous Vehicle (AV) technology. Perception and Localisation are major operations that determine the success of AV development and usage. Therefore, significant research has been carried out to provide AVs with the capabilities to not only sense and understand their surroundings efficiently, but also provide detailed information of the environment in the form of 3D maps. Visual Simultaneous Localisation and Mapping (V-SLAM) has been utilised to enable a vehicle understand its surroundings, map the environment, and identify its position within the area. This paper presents a detailed review of V-SLAM techniques implemented for AV perception and localisation. An overview of SLAM techniques is presented. In addition, an in-depth review is conducted to highlight various V-SLAM schemes, their strengths, and limitations. Challenges associated with V-SLAM deployment and future research directions are also provided in this paper.

Understanding acceptance of autonomous vehicles in Japan, UK, and Germany

ABSTRACT This paper investigates the acceptance of Autonomous Vehicles (AVs) in Japan, the UK and Germany and speculates on the implications for policy and practice. Three on-line surveys of 3,000 members of the public in total, which were conducted in January 2017 (Japan), March 2018 (UK) and November/December 2018 (Germany) were analysed using Principal Component Analysis and then with an Ordered Logit Model. It finds that acceptance of AVs was higher amongst people with higher expectations of the benefits of AVs, those with less knowledge of AVs, and those with lower perceptions of risk. It also finds frequent drivers and car passengers to be more accepting, but that socio-economic factors were mostly insignificant. Finally, there were significant cultural differences between the levels of acceptance between Japan (broadly positive), the UK (broadly neutral) and Germany (broadly negative). These findings suggest that AV promoters should raise (or at least maintain) expectations of AVs among the public; engage with the public to reverse the negative perception of AVs; address AV-generated fears; not bother targeting people by socio-economic group; target frequent car drivers and passengers with information about what AVs could do for them; and target countries where AVs already enjoy a positive image.

Curb Detection and Compensation Method for Autonomous Driving via a 3-D-LiDAR Sensor

Effective detection of curbs is crucial for the perception and localization of autonomous vehicles, but, in practical applications, complex road scenes and the occlusion of curbs by various obstacles are two difficult problems to deal with. Therefore, this article proposes a fast curb detection and compensation method. First, the multiblock RANSAC method is used for ground segmentation to distinguish between ground points and nonground points. Second, the candidate curb points are obtained by processing the ground points with the jumping-window algorithm associated with multispatial features. The jumping-window algorithm only processes the region of interest, which greatly improves the operation efficiency of the method. Third, a joint filtering method is used to refine the candidate curb points, mainly to remove obstacles and other interfering points contained in the candidate curb points. Finally, the missing part of the curb caused by the obstruction is interpolated by a piecewise cubic spline to compensate for the curb points and improve the continuity. We use the KITTI dataset to evaluate the proposed method, and the obtained precision, recall, and harmonic mean in various road scenarios are all above 80%, which is sufficient to demonstrate the robustness of our method. At the same time, the average processing time of each frame is only 47 ms, which can meet the real-time requirements of automatic driving.

Modeling Perception in Autonomous Vehicles via 3D Convolutional Representations on LiDAR

This paper proposes an algorithm to model and process streams of LiDAR data under an autonomous vehicle framework. LiDAR is assumed to be an exteroceptive sensor that allows the vehicle to have dynamic 3D scene perception of its surroundings. We employ an encoder-decoder architecture based on 3D-Convolutional layers called 3D Convolution Encoder-Decoder (3D-CED), together with a transfer learning strategy to extract a set of features from point clouds, which are relevant in the context of autonomous driving. The resulting features allow to make inferences of the future point cloud data and detect multiple abstraction level anomalies in controlled scenarios by utilizing a probabilistic switching dynamic model called High Dimensional Markov Jump Particle Filter (HD-MJPF). Moreover, a comparison is provided between piecewise linear, piecewise nonlinear, and nonlinear predictive models for anomaly detection at multiple abstraction levels. Our approach is evaluated with data collected from the LiDAR sensors of the autonomous vehicle while performing certain tasks in a controlled environment.

Feasibility of Hyperspectral Single Photon Lidar for Robust Autonomous Vehicle Perception.

Autonomous vehicle perception systems typically rely on single-wavelength lidar sensors to obtain three-dimensional information about the road environment. In contrast to cameras, lidars are unaffected by challenging illumination conditions, such as low light during night-time and various bidirectional effects changing the return reflectance. However, as many commercial lidars operate on a monochromatic basis, the ability to distinguish objects based on material spectral properties is limited. In this work, we describe the prototype hardware for a hyperspectral single photon lidar and demonstrate the feasibility of its use in an autonomous-driving-related object classification task. We also introduce a simple statistical model for estimating the reflectance measurement accuracy of single photon sensitive lidar devices. The single photon receiver frame was used to receive 30 12.3 nm spectral channels in the spectral band 1200–1570 nm, with a maximum channel-wise intensity of 32 photons. A varying number of frames were used to accumulate the signal photon count. Multiple objects covering 10 different categories of road environment, such as car, dry asphalt, gravel road, snowy asphalt, wet asphalt, wall, granite, grass, moss, and spruce tree, were included in the experiments. We test the influence of the number of spectral channels and the number of frames on the classification accuracy with random forest classifier and find that the spectral information increases the classification accuracy in the high-photon flux regime from 50% to 94% with 2 channels and 30 channels, respectively. In the low-photon flux regime, the classification accuracy increases from 30% to 38% with 2 channels and 6 channels, respectively. Additionally, we visualize the data with the t-SNE algorithm and show that the photon shot noise in the single photon sensitive hyperspectral data contributes the most to the separability of material specific spectral signatures. The results of this study provide support for the use of hyperspectral single photon lidar data on more advanced object detection and classification methods, and motivates the development of advanced single photon sensitive hyperspectral lidar devices for use in autonomous vehicles and in robotics.

Perceived usefulness and intentions to adopt autonomous vehicles

Understanding the mental process of public acceptance of autonomous vehicles (AVs) is important to the prediction and change of adoption behavior. We present a conceptual model to incorporate background factors such as demographic variables and travel behaviors attributes to the understanding of AV perceived usefulness and intention to adopt AVs. Using data from the 2019 California Vehicle Survey (CVS), we investigate the relationships between observed and latent variables with regard to AV acceptance via structural equation modeling (SEM) techniques. The results show that perceived usefulness is an important determinant of behavioral intention. Householders who are young, well-educated, and males perceive higher usefulness of AVs than other population segments. Households that have telecommuters, transit riders, transportation network company (TNC; e.g., Uber & Lyft) riders, and electric vehicles (EVs) owners, and households that own or plan to install photovoltaic cell (solar) panels also anticipate high benefits of AVs. Living or working at places with access to infrastructure such as EV charging stations and hydrogen fueling stations also add to positive perception of AVs’ advantages. Controlling for perceived usefulness, households having higher annual income and EVs express a stronger interest in buying an AV but not in ridesharing. Young educated households with more TNC riders show a greater propensity to AV sharing services but not for owning AVs. The proposed conceptual model can help pinpoint how background factors such as socioeconomic status affects behavioral intention via its antecedent cognitive construct more accurately to represent the mental process of intention formation. The practical discoveries can assist policymakers identifying population segments that will be the first adopters of this technology.

Deep Reinforcement Learning for Perception and Control of Autonomous Vehicles

Deep reinforcement learning is an effective combination of the decision-making ability of reinforcement learning and the perceptual ability of deep learning, which has achieved a series of milestones in the field of artificial intelligence in recent years. This method has also obtained many beneficial attempts in the field of intelligent driving, which has strongly promoted the development and progress of key technologies of autonomous vehicles. This report briefly reviews the development of deep reinforcement learning methods and specifically introduces the research progress of my team in perception and decision making of autonomous vehicles based on deep reinforcement learning methods.

Fast and Robust Registration of Partially Overlapping Point Clouds

Real-time registration of partially overlapping point clouds has emerging applications in cooperative perception for autonomous vehicles and multi-agent SLAM. The relative translation between point clouds in these applications is higher than in traditional SLAM and odometry applications, which challenges the identification of correspondences and a successful registration. In this paper, we propose a novel registration method for partially overlapping point clouds where correspondences are learned using an efficient point-wise feature encoder, and refined using a graph-based attention network. This attention network exploits geometrical relationships between key points to improve the matching in point clouds with low overlap. At inference time, the relative pose transformation is obtained by robustly fitting the correspondences through sample consensus. The evaluation is performed on the KITTI dataset and a novel synthetic dataset including low-overlapping point clouds with displacements of up to 30 m. The proposed method achieves on-par performance with state-of-the-art methods on the KITTI dataset, and outperforms existing methods for low overlapping point clouds. Additionally, the proposed method achieves significantly faster inference times, as low as 410 ms, between 5 and 35 times faster than competing methods. Our code and dataset are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/eduardohenriquearnold/fastreg</uri> .

Transformer point net: cost-efficient classification of on-road objects captured by light ranging sensors on low-resolution conditions

The three-dimensional perception applications have been growing since Light Detection and Ranging devices have become more affordable. On those applications, the navigation and collision avoidance systems stand out for their importance in autonomous vehicles, which are drawing an appreciable amount of attention these days. The on-road object classification task on three-dimensional information is a solid base for an autonomous vehicle perception system, where the analysis of the captured information has some factors that make this task challenging. On these applications, objects are represented only on one side, its shapes are highly variable and occlusions are commonly presented. But the highest challenge comes with the low resolution, which leads to a significant performance dropping on classification methods. While most of the classification architectures tend to get bigger to obtain deeper features, we explore the opposite side contributing to the implementation of low-cost mobile platforms that could use low-resolution detection and ranging devices. In this paper, we propose an approach for on-road objects classification on extremely low-resolution conditions. It uses directly three-dimensional point clouds as sequences on a transformer-convolutional architecture that could be useful on embedded devices. Our proposal shows an accuracy that reaches the 89.74 % tested on objects represented with only 16 points extracted from the Waymo, Lyft's level 5 and Kitti datasets. It reaches a real time implementation (22 Hz) in a single core processor of 2.3 Ghz.

At the Convergence of the Mobility Trends: Eco-Friendly Connected Automated Vehicles (CAV) and the Energy Efficiency of their Perception System

&lt;div class="section abstract"&gt;&lt;div class="htmlview paragraph"&gt;Increasing financial commitment to accelerating and expanding electrified autonomous vehicle deployment promises vast societal and environmental benefits. The positive effects of these technological breakthroughs will be attained and maximized not only because of the improved energy conversion and optimized driving strategies of autonomous vehicles, but also through manufacturers’ intentional selection of energy efficient componentry. After reviewing the energy conversion advantages of electrified vehicles and the potential to optimize autonomous vehicle operation by optimizing their driving strategies in accordance with advanced driver assistance system performance, this paper highlights the broad implications of selecting energy efficient lidar sensors as the central components in autonomous vehicle perception stacks. Energy efficient components have significant impact on electrified autonomous vehicles’ overall range, energy consumption, and required service and maintenance infrastructure. In evaluating the relative efficiency of various lidar sensors, this paper proposes a new metric: the ratio of sensor performance specifications (field of view, range, resolution) compared to sensor characteristics that directly impact electrified autonomous vehicle efficiency (power consumption, weight, volume).&lt;/div&gt;&lt;/div&gt;

A Scalable and Accurate De-Snowing Algorithm for LiDAR Point Clouds in Winter

Accurate and efficient environmental awareness is a fundamental capability of autonomous driving technology and the real-time data collected by sensors offer autonomous vehicles an intuitive impression of their environment. Unfortunately, the ambient noise caused by varying weather conditions immediately affects the ability of autonomous vehicles to accurately understand their environment and its expected impact. In recent years, researchers have improved the environmental perception capabilities of simultaneous localization and mapping (SLAM), object detection and tracking, semantic segmentation and panoptic segmentation, but relatively few studies have focused on enhancing environmental perception capabilities in adverse weather conditions, such as rain, snow and fog. To enhance the environmental perception of autonomous vehicles in adverse weather, we developed a dynamic filtering method called Dynamic Distance–Intensity Outlier Removal (DDIOR), which integrates the distance and intensity of points based on the systematic and accurate analysis of LiDAR point cloud data characteristics in snowy weather. Experiments on the publicly available WADS dataset (Winter Adverse Driving dataSet) showed that our method can efficiently remove snow noise while fully preserving the detailed features of the environment.

A Review of IoT Technology for the Connected Autonomous Vehicles Ecosystem

Connected Autonomous Vehicles Ecosystem (CAVE) is fast becoming a reality. This paper reviews various architectures that leverage IoT technology to progressively create a system of interconnected Autonomous Vehicles (AV) that is effective, robust and secure. Accurate and fast perception of the external environment is key to movement of AVs in CAVE. IoT technology through sensors and internetworking can help an AV to do reliable perception. State of the art methods for perception of AVs in CAVE are reviewed. Self-awareness is another important aspect. IoT and Artificial Intelligence (AI) can come together to assist an AV in self-diagnosing anything abnormal. One such use case is presented for onboard diagnostics in an AV. Accurate AI is key to making effective strategies and to take control actions with error feedbacks that carefully maneuver an AV from its source to destination. The transition from legacy vehicles to CAVE requires AI methods that combat threats and have effective overall movement in mixed mode traffic systems (MMTS). Strategies for effective movement in MMTS are presented.&#x0D; HIGHLIGHTS&#x0D; &#x0D; Role of IoT technology is identified in different areas of Connected Autonomous Vehicle Ecosystem (CAVE)&#x0D; A comparison of different technologies in various domains of CAVE is presented and their pros and cons are discussed&#x0D; Use case for onboard diagnostics of an autonomous vehicle in CAVE is presented to emphasize the role of IoT Technology in [automated] preventive maintenance&#x0D; Key studies are reviewed that emphasize important issues like robust architectures for CAVE, threats in CAVE and reliable functioning in mixed mode traffic systems&#x0D; &#x0D; GRAPHICAL ABSTRACT

What influences vulnerable road users’ perceptions of autonomous vehicles? A comparative analysis of the 2017 and 2019 Pittsburgh surveys

• Vulnerable road users’ perceptions of autonomous vehicles (AVs) in 2017 and 2019 in Pittsburgh were compared. • Interactive experiences with AVs increased from 2017 to 2019. • Perceived safety of AVs increased from 2017 to 2019, while AVs' receptivity did not change significantly. • Interactive experiences with AVs positively affected vulnerable road users’ perceptions. • We recommend policymakers consider the benefits and safety of vulnerable road users in AV testing. Autonomous vehicles (AVs) are expected to eliminate many driver errors to save thousands of lives. As vulnerable road users, pedestrians and bicyclists are more likely to be injured or killed in accidents involving cars. Therefore, understanding the perceptions of vulnerable road users is crucial for developing and deploying AVs and solving public concerns because the acceptance of AVs relies on both AV users and other road users, such as vulnerable road users. This paper analyzed two surveys collected in 2017 and 2019 by BikePGH in Pittsburgh to understand vulnerable road users’ perceptions of AVs in different years. The analysis showed that vulnerable road users’ interactive experiences with AVs increased from 2017 to 2019 in Pittsburgh, and the interactive experiences with AVs positively affected vulnerable road users’ perceived safety and receptivity toward AVs in 2017 and 2019. Specifically, vulnerable road users’ perceived safety of AVs significantly increased from 2017 to 2019, while their receptivity toward AVs did not change significantly during that period. Additionally, autonomous driving accidents negatively affected vulnerable road users’ perceptions of AVs. Therefore, we recommend that policymakers provide opportunities for the public to interact with AVs and guarantee vulnerable road users’ safety and benefits in AV testing.

Application of the Theory of Planned Behavior in Autonomous Vehicle-Pedestrian Interaction

Automobile manufacturers, alongside technology providers, researchers, and public agencies, are conducting extensive testing to design autonomous vehicles (AVs) algorithms that will provide a complete understanding of road users, specifically pedestrians. Pedestrian behavior and actions determination are highly unpredictable depending on behavioral beliefs, context, and socio-demographic variables. Context includes everything that potentially affects one’s behavior; in AVs–pedestrian interaction, context may consist of weather conditions, road structure, social factors norms, and traffic volume. These influencing elements, therefore, need to be focused on during the development of pedestrian interaction algorithms. For this purpose, the pedestrian behavior questionnaire for FAVs (PBQF) is designed based on the theory of planned behavior (TPB). A total of almost 1000 voluntary participants completed this multilingual survey. As socio-demographic values and physiological perception varies with local norms, regions, and ethnicity, participants from 27 countries were therefore chosen to account for this variation. One of the key findings of this study is the influence of pedestrian attributes and the context on pedestrian behavior. Pedestrian action cannot be understood without visual observation of the pedestrian themselves and their context. The findings showed that pedestrians build communication with vehicles based on their driving styles. The vehicle’s driving style leads pedestrians to think that the vehicle is human-driven or autonomous. The results also revealed that pedestrians use several cues to show their intention. The general perception of AVs was also analyzed, and the communication between AVs and pedestrians with different displaying options was investigated.